[02:36:57] !log admin rebooting cloudnet2005-dev from mgmt -- ssh is failing and the console shows a user prompt but not a password prompt. [02:37:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [05:47:33] https://translate.wmcloud.org/ is down. I can login to instance via ssh and service is running there without any issue. What can be wrong? [05:47:48] I soft rebooted instance but it didn't help. [07:45:28] kart_: seemingly that already fixed itself? I'm seeing successful logins from you in the auth log [07:46:04] login is fine. Service isn't loading the interface. [07:49:05] toolforge jobs: I've many of them, and on one of them I have to add the --timeout - best way to do it? dump+load? (but this will interrupt other unrelevant jobs) thanks [07:49:55] kart_: your security group rules seem to use source hosts that don't match https://wikitech.wikimedia.org/wiki/Help:Using_a_web_proxy_to_reach_Cloud_VPS_servers_from_the_internet#Security_groups (or it's previous iterations, so they were not automatically migrated to include some new IP space) [07:50:09] bozzy: `load` will not affect running jobs with no changes [07:57:46] as usual, taavi saves my a** [08:00:58] Ah and btw there is the --job parameter for toolforge jobs load --job file.yaml to limit what would be imported. I love this [08:07:06] !log valeriobozzolan@tools-bastion-13 tools.itwiki toolforge jobs: itwiki-orphanizerbot, itwiki-deletionbot: adopt timeout 7200 seconds for [[w:it:Special:PermaLink/144874982#Bot_Fermo_3]] [08:07:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.itwiki/SAL [08:14:37] taavi: Thanks! [08:17:30] !log admin powercycle clouservices2005-dev.codfw.wmnet [08:17:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [10:44:13] !log metricsinfra add generic TargetDown rule for better detection of issues like T392889 [10:44:14] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Metricsinfra/SAL [10:44:15] T392889: replication broken on cloudinfra-db04 - https://phabricator.wikimedia.org/T392889 [10:44:30] !log toolsbeta fix security groups for frontproxy-nginx metricsinfra job [10:44:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [13:04:04] !log tools add container image to docker registry docker-registry.tools.wmflabs.org/tofu-provisioning:20250512 (T393686) [13:04:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [13:04:08] T393686: tofu-provisioning: factorize gitlab pipeline logic - https://phabricator.wikimedia.org/T393686 [17:42:30] !log lucaswerkmeister@tools-bastion-13 tools.quickcategories deployed d949e8ee4a (Python 3.13 using --use-latest-versions via T381923) [17:42:33] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.quickcategories/SAL [18:08:05] !log lucaswerkmeister@tools-bastion-13 tools.pagepile kubectl rollout restart deployment pagepile # tool was unresponsive, `webservice` seemed confused about whether the service (on deprecated php7.3) was up or not [18:08:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile/SAL [18:36:28] Whats the login host for toolsbeta? :D [18:38:57] login.toolsbeta.org [18:39:43] !log lucaswerkmeister@tools-bastion-13 tools.wdactle deployed 93a2898396 (fix EntitySchema crash) [18:39:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wdactle/SAL [18:44:13] Thats the one! ty! [18:49:30] !log tools.bs-map-editor rebuild with new builder [18:49:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bs-map-editor/SAL [18:54:11] !log lucaswerkmeister@tools-bastion-13 tools.wdactle deployed 9f541762ae (another mobile design attempt, this time with position: sticky) [18:54:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wdactle/SAL