[00:19:13] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#10737010 (10Raymond_Ndibe) [00:27:49] 10Toolforge (Toolforge iteration 19): [jobs-api] rare Bug where jobs for certain tools can't be listed - https://phabricator.wikimedia.org/T391786 (10Raymond_Ndibe) 03NEW [00:28:18] 10Toolforge (Toolforge iteration 19): [jobs-api] can't list jobs for certain old jobs - https://phabricator.wikimedia.org/T391786#10737022 (10Raymond_Ndibe) p:05Triage→03High [00:28:56] 10Toolforge (Toolforge iteration 19): [jobs-api] can't list jobs for certain old jobs - https://phabricator.wikimedia.org/T391786#10737024 (10Raymond_Ndibe) a:03Raymond_Ndibe [00:29:58] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api] can't list jobs for certain old jobs - https://phabricator.wikimedia.org/T391786#10737025 (10Raymond_Ndibe) [00:34:58] (03open) 10raymond-ndibe: [jobs-api] fix cron parsing bug [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/155 (https://phabricator.wikimedia.org/T391786) [00:35:36] 06cloud-services-team, 10Toolforge (Toolforge iteration 19), 13Patch-For-Review: [jobs-api] can't list jobs for certain old jobs - https://phabricator.wikimedia.org/T391786#10737029 (10Raymond_Ndibe) 05Open→03In progress [01:56:50] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#10737039 (10Raymond_Ndibe) The most recent changes made to `toolforge jobs load *` makes this a bit easier to do. Basically... [01:59:57] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#10737040 (10Raymond_Ndibe) **Step 1** create **create_tools_migrations_list.sh**. This file get's the name of the tools t... [02:02:10] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#10737042 (10Raymond_Ndibe) **Step 2** Create **create_tools_jobs_dump.sh**. This script will create a dump of the jobs of a... [02:05:56] 06cloud-services-team, 10Toolforge (Toolforge iteration 19): [jobs-api,infra] upgrade all the existing toolforge jobs to the latest job version - https://phabricator.wikimedia.org/T359649#10737043 (10Raymond_Ndibe) **Step 3** Create **migrate_tool_jobs_to_latest_version.sh** script. This script takes the dumps... [05:03:41] FIRING: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [05:10:19] FIRING: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [05:13:41] RESOLVED: CloudVPSDesignateLeaks: Detected 1 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [05:15:19] RESOLVED: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [07:44:19] FIRING: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [07:54:41] RESOLVED: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [08:14:17] (03PS1) 10Slyngshede: IDP: Add dummy secret for Phabricator (test) [labs/private] - 10https://gerrit.wikimedia.org/r/1136327 [08:16:30] (03PS2) 10Slyngshede: IDP: Add dummy secret for Phabricator (test) [labs/private] - 10https://gerrit.wikimedia.org/r/1136327 (https://phabricator.wikimedia.org/T377061) [08:23:34] (03CR) 10Slyngshede: [V:03+2 C:03+1] IDP: Add dummy secret for Phabricator (test) [labs/private] - 10https://gerrit.wikimedia.org/r/1136327 (https://phabricator.wikimedia.org/T377061) (owner: 10Slyngshede) [08:23:36] (03CR) 10Slyngshede: [V:03+2 C:03+2] IDP: Add dummy secret for Phabricator (test) [labs/private] - 10https://gerrit.wikimedia.org/r/1136327 (https://phabricator.wikimedia.org/T377061) (owner: 10Slyngshede) [08:26:19] FIRING: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [08:46:19] RESOLVED: HighIOWaitStalling: High iowait detected on clouddumps1002:9100. - https://wikitech.wikimedia.org/wiki/Portal:Data_Services/Admin/Shared_storage#Dumps - https://grafana.wikimedia.org/d/000000568/wmcs-dumps-general-view - https://alerts.wikimedia.org/?q=alertname%3DHighIOWaitStalling [09:11:51] 10wikitech.wikimedia.org, 06serviceops-radar, 06SRE, 13Patch-For-Review, 07SRE-Unowned: Redesign wikitech-static - https://phabricator.wikimedia.org/T376400#10737627 (10Volans) Ack, I can confirm the pages I was having trouble with are now found in search (at the cost of a larger index, I think is around... [09:14:12] (03approved) 10dcaro: [jobs-api] fix cron parsing bug [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/155 (https://phabricator.wikimedia.org/T391786) (owner: 10raymond-ndibe) [09:14:28] (03update) 10dcaro: [jobs-api] fix cron parsing bug [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/155 (https://phabricator.wikimedia.org/T391786) (owner: 10raymond-ndibe) [10:24:23] (03PS1) 10Jelto: ceph: move apus_keys to ceph folder [labs/private] - 10https://gerrit.wikimedia.org/r/1136344 (https://phabricator.wikimedia.org/T378922) [10:31:17] (03CR) 10MVernon: "check experimental" [labs/private] - 10https://gerrit.wikimedia.org/r/1136344 (https://phabricator.wikimedia.org/T378922) (owner: 10Jelto) [11:26:41] (03CR) 10Jelto: [V:03+2 C:03+2] ceph: move apus_keys to ceph folder [labs/private] - 10https://gerrit.wikimedia.org/r/1136344 (https://phabricator.wikimedia.org/T378922) (owner: 10Jelto) [11:34:06] (03CR) 10Jelto: [V:03+2 C:03+2] "this did not solve the issue, `Function lookup() did not find a value for the name 'profile::ceph::s3::client::apus_keys'`" [labs/private] - 10https://gerrit.wikimedia.org/r/1136344 (https://phabricator.wikimedia.org/T378922) (owner: 10Jelto) [12:23:10] (03open) 10l10n-bot: Localisation updates from https://translatewiki.net. [toolforge-repos/wd-image-positions] - 10https://gitlab.wikimedia.org/toolforge-repos/wd-image-positions/-/merge_requests/33 [12:54:18] !log taavi@cloudcumin1001 bastion START - Cookbook wmcs.vps.add_user_to_project for user 'viktoriahillerudwmse' in role 'reader' [12:55:26] !log taavi@cloudcumin1001 bastion END (PASS) - Cookbook wmcs.vps.add_user_to_project (exit_code=0) for user 'viktoriahillerudwmse' in role 'reader' [13:09:53] (03PS1) 10Ssingh: hiera: durum: add dummy ECH private key [labs/private] - 10https://gerrit.wikimedia.org/r/1136376 (https://phabricator.wikimedia.org/T205378) [13:11:03] (03CR) 10Ssingh: [V:03+2 C:03+2] hiera: durum: add dummy ECH private key [labs/private] - 10https://gerrit.wikimedia.org/r/1136376 (https://phabricator.wikimedia.org/T205378) (owner: 10Ssingh) [13:41:56] 10cloud-services-team (FY2024/2025-Q3-Q4), 10Cloud-VPS, 13Patch-For-Review: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283#10738923 (10dcaro) 05Resolved→03Open This broke the project creation cookbook, reopening to track that work [13:46:17] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [13:46:18] wmbot~dcaro@acme: Unknown project "dcaro-test1" [13:46:19] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [13:46:47] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [13:46:47] wmbot~dcaro@acme: Unknown project "dcaro-test1" [13:48:20] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [13:48:21] wmbot~dcaro@acme: Unknown project "dcaro-test1" [13:48:48] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [13:48:49] wmbot~dcaro@acme: Unknown project "dcaro-test1" [13:56:22] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [13:56:24] wmbot~dcaro@acme: Unknown project "dcaro-test1" [13:56:24] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [13:56:53] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [13:56:53] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:01:36] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [14:01:37] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:01:38] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [14:02:10] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/183 (https://phabricator.wikimedia.org/T375283) [14:05:04] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [14:05:04] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:06:20] (03PS1) 10Ssingh: modules: move durum.yaml to secret snake oil [labs/private] - 10https://gerrit.wikimedia.org/r/1136389 [14:11:21] (03PS2) 10Ssingh: modules: move durum.yaml to secret snake oil [labs/private] - 10https://gerrit.wikimedia.org/r/1136389 [14:12:41] 06cloud-services-team, 10Cloud-VPS: wmcs-cookbooks: write a cookbook to delete an openstack project - https://phabricator.wikimedia.org/T391836 (10Andrew) 03NEW [14:13:53] !log andrew@cloudcumin1001 tfinfratests START - Cookbook wmcs.vps.create_project for project tfinfratests in codfw1dev [14:13:54] andrew@cloudcumin1001: Unknown project "tfinfratests" [14:13:59] !log andrew@cloudcumin1001 tfinfratests END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project tfinfratests in codfw1dev [14:14:00] andrew@cloudcumin1001: Unknown project "tfinfratests" [14:15:06] (03PS1) 10Jelto: gitlab: use a wmflib::expand_path compatible path for apus keys [labs/private] - 10https://gerrit.wikimedia.org/r/1136391 (https://phabricator.wikimedia.org/T378922) [14:15:52] (03CR) 10Ssingh: [V:03+2 C:03+2] modules: move durum.yaml to secret snake oil [labs/private] - 10https://gerrit.wikimedia.org/r/1136389 (owner: 10Ssingh) [14:23:09] (03update) 10raymond-ndibe: [jobs-api] split job models to oneoff, scheduled and continuous [repos/cloud/toolforge/jobs-api] (use_pydantic_for_core_job_model) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/154 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [14:34:03] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/183 (https://phabricator.wikimedia.org/T375283) [14:34:50] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [14:34:51] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:34:52] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [14:35:23] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [14:35:23] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:35:33] (03update) 10raymond-ndibe: [jobs-api] move custom validations out of api models [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/150 (https://phabricator.wikimedia.org/T389118) [14:36:22] (03update) 10raymond-ndibe: [jobs-api] use pydantic for all models [repos/cloud/toolforge/jobs-api] (move_most_custom_validations_out_of_api_models) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/139 (https://phabricator.wikimedia.org/T389118) [14:36:58] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [14:36:59] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:37:28] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [14:37:28] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:37:44] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [14:37:45] wmbot~dcaro@acme: Unknown project "dcaro-test1" [14:38:20] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/184 (https://phabricator.wikimedia.org/T375283) [14:39:22] (03update) 10raymond-ndibe: [jobs-api] split job models to oneoff, scheduled and continuous [repos/cloud/toolforge/jobs-api] (use_pydantic_for_core_job_model) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/154 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [14:41:48] (03update) 10raymond-ndibe: [jobs-api] split job models to oneoff, scheduled and continuous [repos/cloud/toolforge/jobs-api] (use_pydantic_for_core_job_model) - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/154 (https://phabricator.wikimedia.org/T389118 https://phabricator.wikimedia.org/T390136) [14:50:23] (03open) 10malakatshy: index.html [toolforge-repos/multilingual-missing-articles] - 10https://gitlab.wikimedia.org/toolforge-repos/multilingual-missing-articles/-/merge_requests/3 [14:50:36] (03merge) 10malakatshy: index.html [toolforge-repos/multilingual-missing-articles] - 10https://gitlab.wikimedia.org/toolforge-repos/multilingual-missing-articles/-/merge_requests/3 [15:27:12] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [15:27:14] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:27:14] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [15:27:26] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/184 (https://phabricator.wikimedia.org/T375283) [15:27:28] (03open) 10andrew: Add new codfw1dev project, 'tfinfradev' [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/185 [15:28:27] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [15:28:27] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:29:04] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/186 (https://phabricator.wikimedia.org/T375283) [15:29:56] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [15:29:56] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:31:24] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/186 (https://phabricator.wikimedia.org/T375283) [15:31:46] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [15:31:46] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:32:20] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/187 (https://phabricator.wikimedia.org/T375283) [15:34:07] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [15:34:09] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:34:09] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [15:36:03] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/187 (https://phabricator.wikimedia.org/T375283) [15:36:29] (03PS1) 10David Caro: vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) [15:36:35] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [15:36:36] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:37:08] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/188 (https://phabricator.wikimedia.org/T375283) [15:37:11] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/188 (https://phabricator.wikimedia.org/T375283) [15:37:13] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [15:37:13] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:38:19] (03PS2) 10David Caro: vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) [15:38:28] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/188 (https://phabricator.wikimedia.org/T375283) [15:38:42] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [15:38:43] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:39:21] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/189 (https://phabricator.wikimedia.org/T375283) [15:40:49] (03update) 10raymond-ndibe: [jobs-api] move custom validations out of api models [repos/cloud/toolforge/jobs-api] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-api/-/merge_requests/150 (https://phabricator.wikimedia.org/T389118) [15:43:14] (03CR) 10CI reject: [V:04-1] vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) (owner: 10David Caro) [15:43:44] (03update) 10raymond-ndibe: [jobs-cli] schedule timeout default None [repos/cloud/toolforge/jobs-cli] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/merge_requests/96 (https://phabricator.wikimedia.org/T389118) [15:47:19] !log andrew@cloudcumin1001 tfinfratests START - Cookbook wmcs.vps.create_project for project tfinfratests in codfw1dev [15:47:21] andrew@cloudcumin1001: Unknown project "tfinfratests" [15:47:31] (03open) 10group_199_bot_333a6c67971a471aeb1cf0b14ccf9f49: projects: added project tfinfratests [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/190 [15:50:15] !log andrew@cloudcumin1001 tfinfratests END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project tfinfratests in codfw1dev [15:50:15] andrew@cloudcumin1001: Unknown project "tfinfratests" [15:51:08] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [15:51:09] wmbot~dcaro@acme: Unknown project "dcaro-test1" [15:51:09] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [15:57:30] 10VPS-project-Wikistats: Add madwikisource to wikistats - https://phabricator.wikimedia.org/T391772#10739762 (10Dzahn) waiting for T391747 [16:01:06] 10VPS-project-Wikistats: Add nupwiki to wikistats - https://phabricator.wikimedia.org/T391856 (10Pppery) 03NEW [16:12:01] (03merge) 10andrew: Add new codfw1dev project, 'tfinfradev' [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/185 [16:12:13] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [16:13:04] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [16:16:13] (03open) 10andrew: tfinfratest.codfw1dev: manage default security group [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/191 [16:16:26] (03merge) 10andrew: tfinfratest.codfw1dev: manage default security group [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/191 [16:16:33] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [16:17:28] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan+apply for main branch [16:19:15] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [16:19:53] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan+apply for main branch [16:20:59] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [16:21:49] !log andrew@cloudcumin1001 admin END (FAIL) - Cookbook wmcs.openstack.tofu (exit_code=99) running tofu plan+apply for main branch [16:21:52] (03PS3) 10David Caro: vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) [16:22:40] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/189 (https://phabricator.wikimedia.org/T375283) [16:23:11] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [16:23:12] wmbot~dcaro@acme: Unknown project "dcaro-test1" [16:23:13] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [16:23:48] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/192 (https://phabricator.wikimedia.org/T375283) [16:25:02] (03CR) 10CI reject: [V:04-1] vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) (owner: 10David Caro) [16:26:03] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [16:26:03] wmbot~dcaro@acme: Unknown project "dcaro-test1" [16:26:06] !log dcaro@acme dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in eqiad1 (T375283) [16:26:07] wmbot~dcaro@acme: Unknown project "dcaro-test1" [16:29:54] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/192 (https://phabricator.wikimedia.org/T375283) [16:30:16] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/193 (https://phabricator.wikimedia.org/T375283) [16:30:54] !log dcaro@acme dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in eqiad1 (T375283) [16:30:55] wmbot~dcaro@acme: Unknown project "dcaro-test1" [16:30:56] T375283: tofu-infra: refactor repo structure - https://phabricator.wikimedia.org/T375283 [16:31:01] (03PS4) 10David Caro: vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) [16:32:53] (03close) 10dcaro: projects: added project tfinfratests [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/190 (owner: 10group_199_bot_333a6c67971a471aeb1cf0b14ccf9f49) [16:34:17] (03PS5) 10David Caro: vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) [16:34:21] (03CR) 10CI reject: [V:04-1] vps.create_project: update the project file created [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) (owner: 10David Caro) [16:38:37] (03CR) 10David Caro: [V:03+1] "Example of the generated MR:" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) (owner: 10David Caro) [16:40:14] (03update) 10dcaro: DONOTMERGE projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/193 (https://phabricator.wikimedia.org/T375283) [16:50:08] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [16:50:45] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [16:56:46] (03CR) 10Andrew Bogott: [C:03+1] "Diff looks good" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1136402 (https://phabricator.wikimedia.org/T375283) (owner: 10David Caro) [17:08:58] (03open) 10andrew: Update codfw1dev setup to use a new project, tfinfradev [repos/cloud/cloud-vps/tf-infra-test] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tf-infra-test/-/merge_requests/1 [17:09:34] (03merge) 10andrew: Update codfw1dev setup to use a new project, tfinfradev [repos/cloud/cloud-vps/tf-infra-test] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tf-infra-test/-/merge_requests/1 [17:17:59] !log andrew@cloudcumin1001 admin START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [17:19:55] !log andrew@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services [18:29:01] (03approved) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/wd-image-positions] - 10https://gitlab.wikimedia.org/toolforge-repos/wd-image-positions/-/merge_requests/33 (owner: 10l10n-bot) [18:29:04] (03merge) 10lucaswerkmeister: Localisation updates from https://translatewiki.net. [toolforge-repos/wd-image-positions] - 10https://gitlab.wikimedia.org/toolforge-repos/wd-image-positions/-/merge_requests/33 (owner: 10l10n-bot) [18:48:23] 10Tool-documentation, 10Adiutor, 06Wiki-Mentor-Africa, 07good first task: [Documentation] Update and improve docs for Adiutor tool - https://phabricator.wikimedia.org/T390392#10740835 (10TBurmeister) Hi @CBID2, thanks for wanting to work on this task! It's great that you created a User page and a subpage f... [18:58:24] 10Tool-documentation, 10Adiutor, 06Wiki-Mentor-Africa, 07good first task: [Documentation] Update and improve docs for Adiutor tool - https://phabricator.wikimedia.org/T390392#10740871 (10TBurmeister) One other thing I should've mentioned: If you'd prefer to work in Markdown, that is also a welcome form of... [20:43:09] 06cloud-services-team, 10Cloud-VPS: tf-infra-test misbehavior in codfw1dev - https://phabricator.wikimedia.org/T391718#10741333 (10Andrew) >>! In T391718#10734677, @Andrew wrote: > First, I'm going to see how this behavior changes if I run things from within the tf-infra-dev project. Seems like it should be th... [23:31:25] (03update) 10chuckonwumelu: Draft: Start [repos/cloud/toolforge/tofu-provisioning] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/tofu-provisioning/-/merge_requests/1