[03:24:35] Now I got `409 Client Error: Conflict for url: https://k8s.tools.eqiad1.wikimedia.cloud:6443/apis/apps/v1/namespaces/tool-campwiz-backend/deployments` [03:25:05] T422538 [03:32:58] adding `.tools.local` worked successfully. (re @wmtelegram_bot: @nokibsarkar: I have something similar to your cross-namespace backend link in wikibugs and the hostname that works is "...) [04:11:28] After push-to-deploy is used, deployment is created, job is running, do I need to turn on `toolforge webservice buildservice start|restart --mount all`? [06:32:26] Hi [06:33:01] Why is my content not published [07:00:37] !log admin test shutting cloudrabbit1001 network interface - T417393 [07:00:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [07:00:45] T417393: Carry out controlled network switch down tests in cloud - https://phabricator.wikimedia.org/T417393 [07:26:52] !log admin unshut cloudrabbit1001 network interface - T417393 [07:27:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [07:27:01] T417393: Carry out controlled network switch down tests in cloud - https://phabricator.wikimedia.org/T417393 [07:36:47] !log admin shut cloudcontrol1011 network interface - T417393 [07:36:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [07:36:58] T417393: Carry out controlled network switch down tests in cloud - https://phabricator.wikimedia.org/T417393 [07:57:42] !log admin unshut cloudcontrol1011 network interface - T417393 [07:57:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [07:57:50] T417393: Carry out controlled network switch down tests in cloud - https://phabricator.wikimedia.org/T417393 [08:00:54] @nokibsarkar it should not be needed, push-to-deploy (toolforge components deploy) already detects if the image changed and restart the job if it did [08:04:59] the hostname http://campwiz-backend-beta.tool-campwiz-backend-beta.svc should have worked without .tools.local, as that is added to the /etc/resolve.conf automatically by k8s, ex: [08:05:03] https://www.irccloud.com/pastebin/124CN389/ [08:05:38] anyhow, glad it's working now [08:14:47] !log admin perform more network tests on cloudrabbit1001 - T417393 [08:14:55] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [08:14:56] T417393: Carry out controlled network switch down tests in cloud - https://phabricator.wikimedia.org/T417393 [09:45:26] !log admin bounce designate on cloudcontrol [09:45:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [13:34:58] !log admin leave designate processes up only on cloudcontrol1006 to ease debugging - T422646 [13:35:06] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [13:35:06] T422646: Designate API timing out - https://phabricator.wikimedia.org/T422646 [13:48:29] !log admin stop designate and memcached on all cloudcontrol1* [13:48:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [14:03:13] !log admin bounce nova on cloudcontrol1006 [14:03:20] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:56:49] !log difronzo@tools-bastion-15 tools.wikivisage deployed v0.7.7 (by DiFronzo on GitHub) [15:56:51] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikivisage/SAL [16:06:24] !log devtools - upgrading openssl, libssl3 on gitlab-1002 [16:06:26] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Devtools/SAL [18:17:55] !log admin restarting openstack services in eqiad1 before digging into a tofu failure [18:17:58] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL