[01:47:27] * bd808 off [11:47:17] https://grid-deprecation.toolforge.org/ now also lists all disabled tools [11:47:46] anyhow, I'm not seeing any issues after the crontab one was fixed yesterday, so I think komla can go ahead and continue disabling tools [11:50:38] awesome, I was currently checking if the jobs listed as "disabled" in phab are showing any activity in https://grid-deprecation.toolforge.org/ [11:50:53] is the enable/disable script logging somewhere when it's run? [11:53:08] found it: /var/log/disable-tool.log [11:54:38] that and sudo logs I guess. https://gitlab.wikimedia.org/repos/cloud/toolforge/disable-tool/-/merge_requests/10 makes the first one much less noisy [11:58:32] "glamify" is marked as disabled in https://grid-deprecation.toolforge.org/ but has some cron jobs from this morning, and I don't find it in the disable log [12:04:33] it seems like andrewbogott used that while testing the grid disabling tools and it was auto-enabled due to the cron bug. https://phabricator.wikimedia.org/T319772#9408487 makes me unsure if it should be actually disabled or not [12:04:54] I removed the disabled marker file to make grid-deprecation accurate again when it next updates [12:08:14] yeah the phab comment makes me think it should _not_ be disabled for now [13:52:41] dhinus: the maintainer has requested for more time but says it is ok if the tools go offline until he is ready. so his tools are good candidates for testing purposes when testing the disable_tools. but the tools should be in the 'backlog' column. [13:53:20] komla: thanks for clarifying! [15:16:06] going through the alerts before the break, cloudvirt1063 has been alerting as down for 5 days: https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=1&host=cloudvirt1063 [15:16:18] if it's expected, could someone ack/downtime it? [15:22:16] XioNoX: that host is down due to T353408. it's already downtimed via alertmanager, but I'll downtime it in icinga too [15:22:18] T353408: Cloudvirt1063.eqiad.wmnet overheating - https://phabricator.wikimedia.org/T353408 [15:22:21] * taavi can't wait to only have one alerting system [15:24:31] yeahh... [15:24:32] thx :) [15:29:45] are archived tools in /srv/tools/archivedtools deleted after some time? I've been able to confirm, other deleted tools can be found there but not the devlibrarycard tool. [15:29:54] https://www.irccloud.com/pastebin/F2GQtD5m/ [15:31:25] oh I didn't add the zipped extension [15:31:44] we are good [15:31:50] https://www.irccloud.com/pastebin/Wt8FViuo/ [17:12:20] * bd808 is kind of excited for Coren to have shown up to work on tools [17:21:06] I copied the toolforge meeting notes from gdoc to https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Workgroup/2023-12-19 [17:21:59] feel free to improve the notes if something is wrong or missing :) [18:55:01] * bd808 lunch