[02:09:54] !log codesearch forcing manual run of codesearch-write-config for T303641 [02:09:57] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Codesearch/SAL [02:09:57] T303641: Add Codex to Codesearch - https://phabricator.wikimedia.org/T303641 [08:00:43] andrewbogott: ^ fyi, see st.w's message above [09:33:32] wm-bot is not playing [09:33:40] 3/15/2022 9:28:15 AM [09:33:40] @rc-ping [09:33:51] not getting any RC feed [09:34:57] anything known to be an issue with RC feeds that would cause this issue for WM-BOT ? [09:45:50] @addshore is xmlrcs functional? I have restarted WM-BOT and no change evident and the rc-ping is still a static point [09:47:22] ^^^ @andrewbogott @bd808 if you know anything to test that service as you are shown on the nove resource;huggle [09:48:40] https://nagf.toolforge.org/?project=huggle <= death on Monday? [09:49:53] https://grafana-labs.wikimedia.org/dashboard/db/labs-project-board?var-project=huggle&var-server=All&orgId=1 <= not ound [09:50:03] *not found [09:59:59] sDrewth: is the middle of the night for them... [10:11:06] sure arturo [10:11:30] I have a small window when I can speak, and hope that they can fix it when they see it [10:14:50] Time zones and windows can suck [10:19:49] ok [10:30:43] !log admin Setting cloudvirt 'cloudvirt1022.eqiad.wmnet' maintenance. - cookbook ran by arturo@nostromo [10:30:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:32:00] !log admin Set cloudvirt 'cloudvirt1022.eqiad.wmnet' maintenance. - cookbook ran by arturo@nostromo [10:32:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:41:54] taavi, andrewbogott: I've moved everything completely off accounts-mwoauth2 (to accounts-mwoauth3) so feel free to do whatever you want with accounts-mwoauth2 otherwise I'll kill it in a few days. [11:24:00] !log tools cleared error state on queue continuous@tools-sgeexec-0939.tools.eqiad.wmflabs (a job took a very long time to be scheduled...) [11:24:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:14:13] !log admin Unsetting cloudvirt 'cloudvirt1022.eqiad.wmnet' maintenance. - cookbook ran by arturo@nostromo [13:14:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [13:14:34] !log admin Unset cloudvirt 'cloudvirt1022.eqiad.wmnet' maintenance. - cookbook ran by arturo@nostromo [13:14:36] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [15:05:05] !log huggle Manually started xmlrcs using instructions from https://wikitech.wikimedia.org/wiki/XmlRcs#Maintainer_info [15:05:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Huggle/SAL [15:06:33] sDrewth: ^ I think I got the service running. For whatever reason petan made this not start by itself, so any time the instance gets rebooted someone has to manually ssh in and start things up. [15:07:06] It could really use a tiny amount of automation love to make things start automatically. [16:32:31] !log toolhub Update demo server to 75d050 [16:32:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolhub/SAL [16:57:08] !log tools build & push docker image docker-registry.tools.wmflabs.org/toolforge-jobs-framework-emailer:latest from https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer (084ee51) - cookbook ran by arturo@nostromo [16:57:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [17:55:48] !log toolsbeta deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer (084ee51) - cookbook ran by arturo@nostromo [17:55:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [20:36:13] !log admin Draining 'cloudvirt1026.eqiad.wmnet'. (T281276) - cookbook ran by andrew@buster [20:36:13] !log admin Setting cloudvirt 'cloudvirt1026.eqiad.wmnet' maintenance. (T281276) - cookbook ran by andrew@buster [20:36:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [20:36:18] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [20:36:57] !log admin Set cloudvirt 'cloudvirt1026.eqiad.wmnet' maintenance. (T281276) - cookbook ran by andrew@buster [20:36:59] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [20:58:17] !log admin Drained 'cloudvirt1026.eqiad.wmnet'. (T281276) - cookbook ran by andrew@buster [20:58:21] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL