[00:01:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [00:02:36] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30032 bytes in 0.355 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [00:02:54] (03update) 10sopzy: Implemented a link to the SPARQL query for the author's list of works at the Wikidata Query Service [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/11 [00:07:38] 10Tool-paulina: At an author's list of works, show a link to the SPARQL query at the Wikidata Query Service - https://phabricator.wikimedia.org/T397554#11300996 (10Sopzy) >>! In T397554#11288085, @Sopzy wrote: >>>! In T397554#11269589, @Nat_WDU wrote: >> Hello @Sopzy, @Oluwatumininu.m and @Dipanshu1223 , the thr... [00:40:28] FIRING: WidespreadPuppetAgentFailure: Widespread puppet agent failures in project metricsinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DWidespreadPuppetAgentFailure [00:55:28] RESOLVED: WidespreadPuppetAgentFailure: Widespread puppet agent failures in project metricsinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DWidespreadPuppetAgentFailure [02:34:30] (03update) 10afanyulionel: Add new-page [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/106 [02:36:28] 10Tool-paulina: Create new generic page - https://phabricator.wikimedia.org/T407385#11301114 (10Afanyulionel) >>! In T407385#11300929, @Pepe_piton wrote: > @Oluwatumininu.m @Dipanshu1223 @Nurah_Wakili Perfect! > > @Chumzine Very good! And very nice that you added the link on the site menu. One detail: if you se... [02:37:31] 10Tool-paulina: Prefill nationality of author and country of origin of work at the country page - https://phabricator.wikimedia.org/T405604#11301115 (10Afanyulionel) >>! In T405604#11300256, @Pepe_piton wrote: > @Aradhya05 Nice work! It works really well. I also liked that you used code like `request.args.get('n... [03:04:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [03:09:40] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30033 bytes in 3.389 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [03:12:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [03:15:44] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30033 bytes in 7.612 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [03:21:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [03:26:38] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30031 bytes in 0.206 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [03:43:48] 10Tool-wsindex, 10Developer-Outreach, 10Wikisource Reader App, 10Outreachy (Round 31): Outreachy 31: Improve the Wikisource Reader App - https://phabricator.wikimedia.org/T405593#11301158 (10Bodhisattwa) [04:52:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [04:54:40] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30031 bytes in 3.950 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [04:57:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [04:58:36] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30031 bytes in 0.200 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [05:18:32] 06cloud-services-team (FY2025/26-Q1), 10Toolforge (Toolforge iteration 24), 07Epic: [KR] WE6.3 Introduce a sustainability scoring system for the Toolforge platform - https://phabricator.wikimedia.org/T368600#11301256 (10komla) I followed up on this. [05:42:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [05:45:40] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30033 bytes in 3.100 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [07:05:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [07:06:46] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30039 bytes in 9.456 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [07:08:05] !log filippo@cloudcumin1001 admin START - Cookbook wmcs.ceph.osd.undrain_node (T405478) [07:08:12] T405478: Experiment with cloudcephosd1050 and cloudcephosd1051 in single-nic configuration - https://phabricator.wikimedia.org/T405478 [07:09:46] PROBLEM - Wikitech-static main page has content on wikitech-static.wikimedia.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Wikitech-static [07:12:46] RECOVERY - Wikitech-static main page has content on wikitech-static.wikimedia.org is OK: HTTP OK: HTTP/1.1 200 OK - 30051 bytes in 8.985 second response time https://wikitech.wikimedia.org/wiki/Wikitech-static [07:13:57] (03PS1) 10Filippo Giunchedi: ceph: bump timeout for drain/undrain [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1198209 (https://phabricator.wikimedia.org/T405478) [07:19:16] (03update) 10sopzy: Resolve T406989 "Proof of concept for adding new statements" [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/42 [07:21:34] (03update) 10sopzy: Resolve T406989 "Proof of concept for adding new statements" [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/42 [07:25:36] 10Tool-paulina: Proof of concept for adding new statements - https://phabricator.wikimedia.org/T406989#11301444 (10Sopzy) >>! In T406989#11283609, @Pepe_piton wrote: > Hi! > > @Oluwatumininu.m Great work, I really like your solution! It works just as expected. I like that you also included code to hide the butt... [07:29:33] 06cloud-services-team, 10Toolforge: toolforge jobs dump prints "WARNING: unexpected key 'job_type' in job, omitting it from dump" - https://phabricator.wikimedia.org/T408033#11301449 (10taavi) →14Duplicate dup:03T405786 [07:29:35] 06cloud-services-team, 10Toolforge: [jobs-cli] `job_type` not handled in `dump`, generates warning per job - https://phabricator.wikimedia.org/T405786#11301451 (10taavi) [07:30:34] (03update) 10sopzy: Implemented a link to the SPARQL query for the author's list of works at the Wikidata Query Service [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/11 [07:31:06] 10Tool-paulina: Create new generic page - https://phabricator.wikimedia.org/T407385#11301453 (10Oluwatumininu.m) >>! In T407385#11300929, @Pepe_piton wrote: > @Oluwatumininu.m @Dipanshu1223 @Nurah_Wakili Perfect! > Thank you very much for the feedback! [07:35:33] 10Tool-wsindex, 10Wikisource Reader App: Add an option to sort list of books alphabetically - https://phabricator.wikimedia.org/T407311#11301461 (10Saiphani02) Hi @System625, I've changed the title field to title_native_language as a temporary fix since title fields are empty for most non-English language work... [07:49:09] 10Tool-paulina: Results pagination - https://phabricator.wikimedia.org/T399974#11301479 (10System625) > @System625, pagination logic looks solid. Can you give a few details about the workaround to keep the "in-public-domain" filter? > I did find the pagination buttons on the bottom instead of on top a bit harde... [07:52:37] (03update) 10bahago: Feature/add work [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/130 [07:57:47] (03merge) 10dcaro: _do_run - retry runtime errors [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/139 (https://phabricator.wikimedia.org/T403175) (owner: 10damian) [08:00:33] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.171-20251023075757-a5105f4d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1008 (https://phabricator.wikimedia.org/T403175) [08:00:39] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: components-api: bump to 0.0.171-20251023075757-a5105f4d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1008 (https://phabricator.wikimedia.org/T403175) [08:10:54] (03update) 10bahago: Feature/add work [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/130 [08:11:08] 06cloud-services-team, 10Toolforge: functional tests are optimistic about retries/timeouts - https://phabricator.wikimedia.org/T408002#11301517 (10dcaro) >>! In T408002#11299332, @DamianZaremba wrote: > Somewhat related but for example `config generate with some existing jobs works, is loadable and creates the... [08:11:25] 06cloud-services-team, 10Toolforge: [functional tests,toolforge-deploy] functional tests are optimistic about retries/timeouts - https://phabricator.wikimedia.org/T408002#11301518 (10dcaro) [08:45:26] 10Tool-wsindex, 10Wikisource Reader App: Add an option to sort list of books alphabetically - https://phabricator.wikimedia.org/T407311#11301684 (10System625) Hello @Saiphani02 , I've created a PR concerning the logic update: https://codeberg.org/ph4ni/wsindex/pulls/6 please kindly review [09:13:48] (03update) 10dcaro: logs: use logs-api for logs [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/121 [09:56:29] !log dcaro@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component components-api [10:01:24] !log dcaro@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [10:01:39] !log dcaro@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component components-api [10:06:46] !log dcaro@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api [10:11:24] (03update) 10taavi: Remove old front proxy instances [repos/cloud/toolforge/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/98 (https://phabricator.wikimedia.org/T283948) [10:11:35] (03merge) 10taavi: Remove old front proxy instances [repos/cloud/toolforge/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/98 (https://phabricator.wikimedia.org/T283948) [10:15:28] FIRING: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on toolsbeta-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [10:16:30] FIRING: [4x] ProbeDown: Service toolsbeta-proxy-7:443 has failed probes (http_beta_toolforge_org_ip4) - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:18:02] FIRING: [4x] ProbeDown: Service tools-proxy-10:443 has failed probes (http_toolforge_org_ip4) - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:21:30] RESOLVED: [4x] ProbeDown: Service toolsbeta-proxy-7:443 has failed probes (http_beta_toolforge_org_ip4) - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:22:53] (03approved) 10dcaro: components-api: bump to 0.0.171-20251023075757-a5105f4d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1008 (https://phabricator.wikimedia.org/T403175) (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [10:22:57] (03merge) 10dcaro: components-api: bump to 0.0.171-20251023075757-a5105f4d [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1008 (https://phabricator.wikimedia.org/T403175) (owner: 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620) [10:23:51] 06cloud-services-team, 10Toolforge (Toolforge iteration 24), 13Patch-For-Review: [components-api] Intermittent internal API failures / retry internal requests - https://phabricator.wikimedia.org/T403175#11302064 (10dcaro) 05Open→03Resolved a:03dcaro Deployed \o/, @DamianZaremba thanks! [10:25:28] RESOLVED: PuppetStaleCertificates: Found non-revoked Puppet certificates for 1 deleted instances on toolsbeta-puppetserver-1 - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetStaleCertificates - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetStaleCertificates [10:29:40] 06cloud-services-team (FY2025/26-Q1), 10Toolforge (Toolforge iteration 24), 13Patch-For-Review: [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#11302074 (10dcaro) Moved the draft to etherpad for easy collaboration editing it: http... [10:31:32] (03open) 10system625: Feat: Add author creation form page [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/146 [10:33:15] 10Tool-paulina: Proof of concept: Add author form - https://phabricator.wikimedia.org/T407551#11302118 (10System625) Hello @Nat_WDU @Pepe_piton I made a merge request concerning this task: https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/146 Please kindly review [10:40:32] RESOLVED: [4x] ProbeDown: Service tools-proxy-10:443 has failed probes (http_toolforge_org_ip4) - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/custom&var-module=All - https://prometheus-alerts.wmcloud.org/?q=alertname%3DProbeDown [10:41:44] (03open) 10aradhya: implemented pagination for search results (T399974) [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 [10:44:19] 10Tool-paulina: Results pagination - https://phabricator.wikimedia.org/T399974#11302164 (10Aradhya05) Hi @Pepe_piton @Nat_WDU @DidiCoronel, I have completed the task and submitted the merge request: https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 Kindly review! Thank you [11:00:45] (03update) 10nurahwakili: T407558- Proof of concept add work form [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/139 [11:50:49] (03CR) 10David Caro: [C:03+1] "LGTM" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1198209 (https://phabricator.wikimedia.org/T405478) (owner: 10Filippo Giunchedi) [12:04:20] (03merge) 10dcaro: Fail with friendly errors on missing pipelines [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/288 (https://phabricator.wikimedia.org/T407982) (owner: 10damian) [12:15:35] 06cloud-services-team, 10Toolforge: [build service] Python pack is outdated, does not support stable release - https://phabricator.wikimedia.org/T408108 (10DamianZaremba) 03NEW [12:16:16] 06cloud-services-team, 10Toolforge: [build service] Python pack is outdated, does not support stable release - https://phabricator.wikimedia.org/T408108#11302549 (10DamianZaremba) Similar to https://phabricator.wikimedia.org/T401875 Related to https://phabricator.wikimedia.org/T380127 [12:16:34] 06cloud-services-team (FY2025/26-Q1), 10Toolforge (Toolforge iteration 24), 13Patch-For-Review: [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#11302555 (10dcaro) [12:17:04] (03CR) 10Filippo Giunchedi: [C:03+2] ceph: bump timeout for drain/undrain [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1198209 (https://phabricator.wikimedia.org/T405478) (owner: 10Filippo Giunchedi) [12:23:32] (03merge) 10dcaro: getBuild - add digest to image [repos/cloud/toolforge/builds-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-api/-/merge_requests/144 (https://phabricator.wikimedia.org/T403322) (owner: 10damian) [12:29:36] !log tools.cluebotng Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18748267293 (https://github.com/cluebotng/component-configs/commits/bc8f1b883d0d53edf08bea5e5319ee7ee0b4fb82) [12:29:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng/SAL [12:30:58] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/weapon-of-mass-description] - 10https://gerrit.wikimedia.org/r/1198317 (owner: 10L10n-bot) [12:30:59] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [labs/tools/commons-mass-description] - 10https://gerrit.wikimedia.org/r/1198315 (owner: 10L10n-bot) [12:32:06] 06cloud-services-team, 10Toolforge: [build service] Python pack is outdated, does not support stable release - https://phabricator.wikimedia.org/T408108#11302634 (10DamianZaremba) https://wikitech.wikimedia.org/wiki/Help:Toolforge/Building_container_images states > Compared to the current images, you can use n... [12:32:33] !log tools.cluebotng-review Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/18748267282 (https://github.com/cluebotng/component-configs/commits/bc8f1b883d0d53edf08bea5e5319ee7ee0b4fb82) [12:32:35] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.cluebotng-review/SAL [12:34:20] (03update) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-api: bump to 0.0.202-20251023122338-895f8556 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1009 (https://phabricator.wikimedia.org/T403322) [12:34:21] (03open) 10group_203_bot_f4d95069bb2675e4ce1fff090c1c1620: builds-api: bump to 0.0.202-20251023122338-895f8556 [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1009 (https://phabricator.wikimedia.org/T403322) [12:39:34] 06cloud-services-team, 10Toolforge (Toolforge iteration 24): [lima-kilo] toolforge_deploy_mr.py doesn't handle missing pipelines - https://phabricator.wikimedia.org/T407982#11302653 (10dcaro) 05Open→03Resolved a:03dcaro Merged! [13:12:07] (03open) 10dcaro: toolforge_deploy_mr: support deploying local code [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/290 [13:14:11] !log dcaro@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-api [13:19:39] !log dcaro@cloudcumin1001 toolsbeta END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [13:25:43] !log dcaro@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component builds-api [13:34:20] !log dcaro@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [13:39:30] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: metricsinfra-thanos-fe-2 Puppet failure - https://phabricator.wikimedia.org/T407837#11302900 (10taavi) 05Open→03Resolved a:03taavi [13:53:55] 10Toolforge, 06Infrastructure-Foundations, 10netops, 06tools-infrastructure-team: Plan networking for Toolforge-on-Metal experiment - https://phabricator.wikimedia.org/T407140#11302980 (10Andrew) If we want to attempt the everything-on-containers plan that Alex is promoting, the next step is probably to ma... [13:59:56] (03update) 10dcaro: toolforge_deploy_mr: support deploying local code [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/290 [14:01:58] RESOLVED: PuppetAgentNoResources: No Puppet resources found on instance metricsinfra-thanos-fe-2 on project metricsinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [14:02:18] 10Toolforge, 06Infrastructure-Foundations, 10netops, 06tools-infrastructure-team: Plan networking for Toolforge-on-Metal experiment - https://phabricator.wikimedia.org/T407140#11303033 (10taavi) The initial networking questions in the task description still need deciding. It seems like we are leaning towar... [14:03:41] !log dcaro@cloudcumin1001 tools START - Cookbook wmcs.toolforge.component.deploy for component builds-api [14:08:51] !log dcaro@cloudcumin1001 tools END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api [14:28:07] 10Tool-paulina: Create new generic page - https://phabricator.wikimedia.org/T407385#11303118 (10Dipanshu1223) >>! In T407385#11300929, @Pepe_piton wrote: > @Oluwatumininu.m @Dipanshu1223 @Nurah_Wakili Perfect! > > @Chumzine Very good! And very nice that you added the link on the site menu. One detail: if you se... [14:29:27] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125 (10dcaro) 03NEW [14:30:52] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303140 (10dcaro) It might be that the taskruns are to complex to parse by the version converting hooks, as they might be doing so on every run: ` root@tools-k... [14:31:11] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303143 (10DamianZaremba) 14s (filtered) is not much longer than 17s (all entries) which makes me think there is a lot going on before the filtering. [14:33:05] 06cloud-services-team, 10Data-Services, 06Data-Persistence, 06Data-Platform-SRE: Decide how to use the new clouddb hosts (clouddb102[2-5]) - https://phabricator.wikimedia.org/T401295#11303153 (10fnegri) Are you leaving s1 alone based on its size, or expected traffic? Are some sections growing in size much... [14:37:58] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303159 (10dcaro) If it's doing the conversion, it has to convert before filtering I think [14:42:19] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303183 (10dcaro) Looking at https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/, I think we might be ab... [14:46:56] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303192 (10dcaro) In lima-kilo we only have the latest versions though: ` dcaro@toolslocal$ kubectl get crds pipelineruns.tekton.dev -o json | jq '.status.stor... [14:48:05] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303202 (10dcaro) I knew this sounded familiar: {T376710} [14:49:09] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303210 (10dcaro) But back then we only upgraded the pipelineruns, it seems we will need to do something similar with the taskruns too [14:50:48] 10Toolforge (Toolforge iteration 24): [bulids-builder,tekton] upgrade taskuns to v1 from v1beta1 in storage and delete v1beta1 from stored versions - https://phabricator.wikimedia.org/T408127 (10dcaro) 03NEW [14:52:37] 10Toolforge (Toolforge iteration 24): [bulids-builder,tekton] upgrade taskuns to v1 from v1beta1 in storage and delete v1beta1 from stored versions - https://phabricator.wikimedia.org/T408127#11303235 (10dcaro) p:05Triage→03High [14:52:48] 10Toolforge, 06tools-infrastructure-team, 13Patch-For-Review: Merge Toolforge Nginx front proxy into the existing K8s HAProxy setup - https://phabricator.wikimedia.org/T283948#11303237 (10taavi) 05Open→03Resolved [14:53:11] 10Toolforge (Toolforge iteration 24): [builds-api] listing deployments gets very slow when querying taskruns - https://phabricator.wikimedia.org/T408125#11303240 (10dcaro) p:05Triage→03High [14:53:27] 06cloud-services-team, 10Data-Services, 06Data-Persistence, 06Data-Platform-SRE: Decide how to use the new clouddb hosts (clouddb102[2-5]) - https://phabricator.wikimedia.org/T401295#11303242 (10fnegri) 05Stalled→03In progress [14:54:13] 06cloud-services-team (FY2025/26-Q1), 10Data-Services, 06Data-Persistence, 06Data-Platform-SRE: Decide how to use the new clouddb hosts (clouddb102[2-5]) - https://phabricator.wikimedia.org/T401295#11303246 (10fnegri) [15:08:25] !log filippo@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=99) (T405478) [15:08:33] T405478: Experiment with cloudcephosd1050 and cloudcephosd1051 in single-nic configuration - https://phabricator.wikimedia.org/T405478 [15:16:23] 10Cloud Services Proposals, 06cloud-services-team, 10Cloud-VPS: Decision Request - How openstack projects relate to tofu-infra - https://phabricator.wikimedia.org/T385604#11303365 (10fnegri) a:03Andrew [15:56:15] 10Cloud Services Proposals, 06cloud-services-team, 10Cloud-VPS: Decision Request - How openstack projects relate to tofu-infra - https://phabricator.wikimedia.org/T385604#11303559 (10dcaro) >>! In T385604#11299337, @Andrew wrote: > Having reviewed and reconsidered this, I now vote for a modified version of... [16:02:53] !log filippo@cloudcumin1001 admin START - Cookbook wmcs.ceph.osd.undrain_node (T405478) [16:03:01] T405478: Experiment with cloudcephosd1050 and cloudcephosd1051 in single-nic configuration - https://phabricator.wikimedia.org/T405478 [16:06:26] (03update) 10aradhya: implemented pagination for search results (T399974) [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 [16:11:36] (03update) 10dipanshu1223: T399974:"Pagination feature" [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/145 [16:19:02] 06cloud-services-team (FY2025/26-Q1), 10Data-Services, 06Data-Persistence, 06Data-Platform-SRE: Decide how to use the new clouddb hosts (clouddb102[2-5]) - https://phabricator.wikimedia.org/T401295#11303665 (10Ladsgroup) >>! In T401295#11303153, @fnegri wrote: > Are you leaving s1 alone based on its size,... [16:19:39] (03update) 10miiswom: Proof of concept: Add author form [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/131 [16:27:17] (03PS1) 10BCornwall: varnish: Stub browser-detection for tests [labs/private] - 10https://gerrit.wikimedia.org/r/1198369 (https://phabricator.wikimedia.org/T404826) [16:29:36] 10Wikibugs: #translatewiki still getting notifs weeks after merging config change - https://phabricator.wikimedia.org/T408139 (10bd808) 03NEW [16:30:54] (03open) 10dcaro: builds-builder: add script to upgrade taskruns [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1010 (https://phabricator.wikimedia.org/T408127) [16:31:19] 10Wikibugs, 07affects-translatewiki.net: #translatewiki still getting notifs weeks after merging config change - https://phabricator.wikimedia.org/T408139#11303731 (10bd808) [16:34:39] !log dcaro@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-api [16:40:59] (03update) 10aradhya: implemented pagination for search results (T399974) [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 [16:41:16] (03update) 10dcaro: builds-builder: add script to upgrade taskruns [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1010 (https://phabricator.wikimedia.org/T408127) [16:41:22] (03update) 10dcaro: builds-builder: add script to upgrade taskruns [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1010 (https://phabricator.wikimedia.org/T408127) [16:42:55] !log dcaro@cloudcumin1001 toolsbeta END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api [16:46:03] (03update) 10aradhya: implemented pagination for search results (T399974) [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 [16:46:07] 10Wikibugs, 07affects-translatewiki.net: #translatewiki still getting notifs weeks after merging config change - https://phabricator.wikimedia.org/T408139#11303757 (10bd808) I'm not seeing any message posted by the bot to https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL that would match the... [16:46:37] 10Tool-paulina: Results pagination - https://phabricator.wikimedia.org/T399974#11303758 (10Aradhya05) Hi @Pepe_piton @Nat_WDU @DidiCoronel , I have completed this task and submitted the merge request : https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/147 Kindly review! Thank you [16:52:52] 06cloud-services-team, 06SRE, 13Patch-For-Review: latest Trixie image (as of 2025-10-16) grub failure - https://phabricator.wikimedia.org/T407586#11303781 (10Andrew) I can reproduce this on a second server (cloudcontrol1008-dev) which is also a config b R450. I could /not/ reproduce this on mapstest1002-dev... [16:53:39] (03update) 10fnegri: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) (owner: 10taavi) [16:54:17] (03approved) 10fnegri: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) (owner: 10taavi) [16:54:24] 06cloud-services-team, 06SRE, 13Patch-For-Review: latest Trixie image (as of 2025-10-16) grub failure - https://phabricator.wikimedia.org/T407586#11303790 (10Andrew) On my last attempt on 2010-dev (raid5) I'm intrigued by this out of memory error: ` Booting from Hard drive C: GRUB loading.. Welcome to GRUB... [16:55:10] 06cloud-services-team, 06SRE, 13Patch-For-Review: latest Trixie image (as of 2025-10-16) grub failure on R540 hardware - https://phabricator.wikimedia.org/T407586#11303796 (10Andrew) [16:56:03] 06cloud-services-team, 06SRE, 13Patch-For-Review: latest Trixie image (as of 2025-10-16) grub failure on R450 hardware - https://phabricator.wikimedia.org/T407586#11303803 (10Andrew) [16:56:15] 06cloud-services-team, 06SRE, 13Patch-For-Review: latest Trixie image (as of 2025-10-16) grub failure on R450 hardware - https://phabricator.wikimedia.org/T407586#11303807 (10Andrew) [17:02:10] !log dcaro@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component builds-api [17:06:25] 10Toolforge (Toolforge iteration 24): [builds-builder] Host all the build-time dependencies on gitlab - https://phabricator.wikimedia.org/T408140 (10dcaro) 03NEW [17:08:41] !log dcaro@cloudcumin1001 toolsbeta END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api [17:14:46] (03open) 10dcaro: inject_buildpacks: use gitlab-hosted cached files [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/76 (https://phabricator.wikimedia.org/T408140) [17:19:18] (03update) 10dcaro: toolforge_deploy_mr: support deploying local code [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/290 [17:22:27] (03update) 10dcaro: inject_buildpacks: use gitlab-hosted cached files [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/76 (https://phabricator.wikimedia.org/T408140) [17:22:50] (03update) 10dcaro: inject_buildpacks: use gitlab-hosted cached files [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/76 (https://phabricator.wikimedia.org/T408140) [17:26:17] (03update) 10taavi: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) [17:26:34] (03update) 10dcaro: inject_buildpacks: use gitlab-hosted cached files [repos/cloud/toolforge/builds-builder] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/76 (https://phabricator.wikimedia.org/T408140) [17:26:59] (03update) 10taavi: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) [17:27:43] (03merge) 10taavi: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) [17:28:39] !log taavi@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component ingress-nginx [17:30:47] !log taavi@cloudcumin1001 toolsbeta END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-nginx [17:31:49] !log taavi@cloudcumin1001 toolsbeta START - Cookbook wmcs.toolforge.component.deploy for component ingress-nginx [17:32:04] (03update) 10dcaro: builds-builder: add script to upgrade taskruns [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1010 (https://phabricator.wikimedia.org/T408127) [17:33:58] !log taavi@cloudcumin1001 toolsbeta END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-nginx [17:42:07] (03update) 10miiswom: Proof of concept: Add author form [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/131 [17:53:51] (03update) 10fnegri: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) (owner: 10taavi) [18:08:23] (03update) 10fnegri: ingress-nginx: Update to chart-4.13.3 on lima/toolsbeta [repos/cloud/toolforge/toolforge-deploy] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/1007 (https://phabricator.wikimedia.org/T383516) (owner: 10taavi) [20:11:20] (03update) 10miiswom: Proof of concept: Add author form [toolforge-repos/paulina] - 10https://gitlab.wikimedia.org/toolforge-repos/paulina/-/merge_requests/131 [20:12:16] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt2004-dev.codfw.wmnet' [20:12:39] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=99) on host 'cloudvirt2004-dev.codfw.wmnet' [20:41:22] 10PAWS, 06tools-platform-team: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T408157#11304506 (10LibUp-bot) [20:41:23] 10Toolforge, 06tools-platform-team: New upstream release for Pywikibot - https://phabricator.wikimedia.org/T408158#11304508 (10LibUp-bot) [20:43:21] 10Toolforge (Toolforge iteration 24), 10Luthor, 06tools-infrastructure-team, 06tools-platform-team: Can't log in to Luthor - https://phabricator.wikimedia.org/T407048#11304525 (10taavi) 05Open→03Resolved a:03taavi This seems to have been fixed as a side effect of {T283948}. [21:16:49] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:16:54] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:20:17] FIRING: JobUnavailable: Reduced availability for job openstack in cloud@codfw - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [21:21:20] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:21:22] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:24:19] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:24:22] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:25:43] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:25:45] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:26:51] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:26:53] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:29:47] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:29:53] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:34:23] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:34:26] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) on deployment codfw1dev for all services [21:35:25] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [21:40:17] RESOLVED: JobUnavailable: Reduced availability for job openstack in cloud@codfw - https://wikitech.wikimedia.org/wiki/Prometheus#Prometheus_job_unavailable - https://grafana.wikimedia.org/d/NEJu05xZz/prometheus-targets - https://alerts.wikimedia.org/?q=alertname%3DJobUnavailable [21:40:39] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services [21:42:14] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt2004-dev.codfw.wmnet' [21:42:39] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt2004-dev.codfw.wmnet' [21:52:34] FIRING: TooManyCloudvirtsDown: Reduced availability for CloudVPS codfw - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TooManyCloudvirtsDown - TODO - https://alerts.wikimedia.org/?q=alertname%3DTooManyCloudvirtsDown [22:19:04] RESOLVED: TooManyCloudvirtsDown: Reduced availability for CloudVPS codfw - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/TooManyCloudvirtsDown - TODO - https://alerts.wikimedia.org/?q=alertname%3DTooManyCloudvirtsDown [22:50:35] !log andrew@cloudcumin1001 cloudvirt-canary START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on codfw1dev, with recreate False, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [22:52:08] !log andrew@cloudcumin1001 cloudvirt-canary END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) on codfw1dev, with recreate False, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [22:54:20] !log andrew@cloudcumin1001 cloudvirt-canary START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on codfw1dev, with recreate True, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [22:55:46] !log andrew@cloudcumin1001 cloudvirt-canary END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) on codfw1dev, with recreate True, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [23:02:38] !log andrew@cloudcumin1001 cloudvirt-canary START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on codfw1dev, with recreate False, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [23:03:07] !log andrew@cloudcumin1001 cloudvirt-canary END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) on codfw1dev, with recreate False, for hosts list: ['cloudvirt2004-dev', 'cloudvirt2005-dev', 'cloudvirt2006-dev'] [23:12:47] (03update) 10raymond-ndibe: Use the image name as provided by builds-api [repos/cloud/toolforge/components-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/components-api/-/merge_requests/143 (https://phabricator.wikimedia.org/T403322) (owner: 10damian)