[04:32:54] http://hoo.toolforge.org/ is 504ing [05:28:43] !log tools.coverage Restart stuck webserver and move from php7.3 to php7.4 while at it. [05:28:44] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.coverage/SAL [05:29:24] https://coverage.toolforge.org/charts/mediawiki.png is back up [08:41:11] there was a k8s worker stuck, might have affected several tools, let me know if you see any more issues as it might be a wider issue than just one worker [08:48:47] !log tools.wlh webservice restart [08:48:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wlh/SAL [08:49:42] yep, there's a few more k8s nodes having trouble, seems related to the NFS hiccup we had a couple days ago (T346123) [08:49:43] T346123: [Ceph] [alert] toolschecker: NFS read/writeable on Toolforge - https://phabricator.wikimedia.org/T346123 [08:49:50] !log tools.templatecount webservice restart [08:49:50] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.templatecount/SAL [08:50:32] !log tools.signatures webservice restart [08:50:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.signatures/SAL [08:51:30] (my restarts above, responding to https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Multiple_toolforge_tools_still_not_working) [09:05:04] Hey TheresNoTime [09:28:31] !log tools rebooting tools-sge-cron-2 (T346123) [09:28:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [09:28:34] T346123: [Ceph] [alert] toolschecker: NFS read/writeable on Toolforge - https://phabricator.wikimedia.org/T346123 [09:58:37] RhinosF1: o/ [10:17:27] !log admin faiolver cloudgw1001 into cloudgw1002, investigating a nftables syntax error [10:21:42] Hello, can I ask questions about connecting to the replica databases here? [10:22:21] yes [10:28:22] thanks! Currently my problem is the following: I created an toolforge user account, created a project and now im trying to connect to the replica database using the file replica.my.cnf in the tools home directory. Unfortunately in every way i tried (command line, putty, mysql workbench) theres the error "access denied for user X@Y (using [10:28:23] password:YES). Do you know where the problem might be? [10:29:05] X beeing the user name in myreplica [10:31:03] !log admin faiolver cloudgw1001 into cloudgw1002, investigating a nftables syntax error (T346432) [10:32:57] !log admin faiolver cloudgw1001 into cloudgw1002, investigating a nftables syntax error (T346432) [10:33:02] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:33:02] T346432: cloudgw: syntax error in nftables config if using sets with intervals and some element overlaps an interval - https://phabricator.wikimedia.org/T346432 [10:37:45] Guest23: (can't devote any time to troubleshooting atm, sorry, but re. MySQL Workbench, did you follow https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database#Connecting_with... ?) [10:39:17] which tool? can you share an example command you tried? [10:46:12] TheresNoTime yes, these are the instructions i followed [10:50:11] taavi its a tool i created myself, "wikistatistics". In my home directory there is no replica.my.cnf, so i used the one from the tools directory. I used the commands provided in https://wikitech.wikimedia.org/wiki/Help:Toolforge/Database. I read in an old issue that there are problems with newly created accounts, but the issue was old (from before [10:50:12] 2020) so i figured this was fixed [10:52:30] https://phabricator.wikimedia.org/T71679 this is the issue I was writing about [10:52:50] !log tools.stewardbots Restarted SULWatcher stucked on IRC [10:52:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [11:07:03] Guest23: looks like this is a different issue with the account setup system. I've filed T346433 to track that [11:07:04] T346433: maintain-dbusers stuck on ToolsDB account creation failures - https://phabricator.wikimedia.org/T346433 [11:08:47] taavi thank you! So there is nothing i can do about it at the moment? [11:09:18] that's correct, unfortunately [11:10:36] Okay, so im going to track the issue and try to login regularly to see if it works. Nevertheless thank you for helping me! [11:43:36] !log merging NAT change for T346426 in cloudgw [11:43:37] arturo: Unknown project "merging" [11:43:37] T346426: Some VPS instances still using ns-recursor0 - https://phabricator.wikimedia.org/T346426 [11:43:46] !log admin merging NAT change for T346426 in cloudgw [11:43:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [12:16:28] Does anyone know how/where to update the "Source code" value shown on https://toolsadmin.wikimedia.org/tools/id/admin (see https://phabricator.wikimedia.org/T345418 )? [12:28:23] andre: I think it's just there, in toolsadmin, if you are logged in isn't there an 'edit' button on the top right? [12:28:44] it's only there if you're a maintainer of that tool :P [12:28:54] oh, I can change it then [12:30:05] done :) [13:08:02] uh, thank you! [16:30:27] I face a DNS name resolution failure with "www.spacecom.mil" on my cloud vps VM. As per Arturo's e-mail I checked that my resolv.conf points to 172.20.255.1 and that's the case. I don't know if this name resolution worked before the maintenance. [19:04:50] don-vip, it seems to be just that one domain that doesn't resolve. I don't think it's related to the maintenance because it fails on my test cloud as well. [19:05:14] If you open a ticket I can look at it more later; you might want to just skip ahead and put an entry in your hosts file though.