[00:01:22] (03CR) 10BryanDavis: [C:03+1] "Untested, but the logic seems correct. I can make time later to test if that is desired." [labs/striker] - 10https://gerrit.wikimedia.org/r/1083302 (owner: 10Majavah) [00:02:00] (03CR) 10BryanDavis: [C:03+1] "Untested, but the logic seems correct. I can make time to test later if that is desired." [labs/striker] - 10https://gerrit.wikimedia.org/r/1083303 (owner: 10Majavah) [00:08:26] 10wikitech.wikimedia.org: ☂ Wikitech account linking and SUL error reporting - https://phabricator.wikimedia.org/T376267#10270792 (10bd808) >>! In T376267#10264842, @Robertsky wrote: > @Ladsgroup I can't log into the wikitech account which makes it a nonstarter (as somehow the password for wikitech wiki was los... [00:38:17] 10wikitech.wikimedia.org: ☂ Wikitech account linking and SUL error reporting - https://phabricator.wikimedia.org/T376267#10270799 (10Robertsky) @bd808 well, the wikitech account does not have an email address attached to it as found out by @Ladsgroup. imo, a longshot to tie the accounts together is my request f... [01:15:23] FIRING: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [01:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [03:02:43] 10VPS-project-Codesearch: Include dependencies of WMF deployed extensions in the WMF-deployed index - https://phabricator.wikimedia.org/T361002#10270921 (10Krinkle) @taavi Are you suggesting we index UserMerge, which is not deployed at WMF, under the "MediaWiki deployed at WMF" index? Task T360991 mentions Dis... [03:02:53] 10VPS-project-Codesearch: Include dependencies of WMF deployed extensions in the WMF-deployed index - https://phabricator.wikimedia.org/T361002#10270923 (10Krinkle) p:05Triage→03Medium [05:15:23] FIRING: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [05:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [07:08:44] 06cloud-services-team, 10Toolforge: I keep getting Toolforge messages about rustup - https://phabricator.wikimedia.org/T378437 (10Magnus) 03NEW [07:09:22] 06cloud-services-team, 10Toolforge: I keep getting Toolforge messages about rustup - https://phabricator.wikimedia.org/T378437#10271050 (10Magnus) Or put the tool name in there so I can see it's mine... [08:08:28] 06cloud-services-team, 10Toolforge: I keep getting Toolforge messages about rustup - https://phabricator.wikimedia.org/T378437#10271096 (10dcaro) > Or put the tool name in there so I can see it's mine... The tool name itself is `rustup` (it's the second block in the subject, and the recipient of the email is... [08:51:58] 06cloud-services-team, 10Toolforge: Timeouts when pulling artifacts from github with toolforge build - https://phabricator.wikimedia.org/T377255#10271245 (10dcaro) 05Open→03Resolved a:03dcaro Ack, please reference this task it if it happens again (and you remember xd). [09:07:46] !log dcaro@urcuchillay TEST1 START - Cookbook wmcs.vps.create_project for project TEST1 in eqiad1 [09:07:48] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:07:53] !log dcaro@urcuchillay TEST1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project TEST1 in eqiad1 [09:07:54] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:31] !log dcaro@urcuchillay TEST1 START - Cookbook wmcs.vps.create_project for project TEST1 in eqiad1 [09:08:31] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:39] !log dcaro@urcuchillay TEST1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project TEST1 in eqiad1 [09:08:39] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:45] !log dcaro@urcuchillay TEST1 START - Cookbook wmcs.vps.create_project for project TEST1 in eqiad1 [09:08:46] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:50] !log dcaro@urcuchillay TEST1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project TEST1 in eqiad1 [09:08:50] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:56] !log dcaro@urcuchillay TEST1 START - Cookbook wmcs.vps.create_project for project TEST1 in eqiad1 [09:08:56] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:08:58] !log dcaro@urcuchillay TEST1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project TEST1 in eqiad1 [09:08:58] wmbot~dcaro@urcuchillay: Unknown project "TEST1" [09:09:21] !log dcaro@urcuchillay toolsbeta START - Cookbook wmcs.vps.create_project for project toolsbeta in eqiad1 [09:09:23] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [09:09:26] !log dcaro@urcuchillay toolsbeta END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project toolsbeta in eqiad1 [09:09:28] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolsbeta/SAL [09:15:23] RESOLVED: OpenstackAPIResponse: Openstack API average response time is too high. - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/OpenstackAPIResponse - https://grafana.wikimedia.org/d/UUmLqqX4k - https://alerts.wikimedia.org/?q=alertname%3DOpenstackAPIResponse [09:19:06] !log dcaro@urcuchillay wp1 START - Cookbook wmcs.vps.create_project for project wp1 in eqiad1 [09:19:08] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:19:10] !log dcaro@urcuchillay wp1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project wp1 in eqiad1 [09:19:10] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:19:54] !log dcaro@urcuchillay wp1 START - Cookbook wmcs.vps.create_project for project wp1 in eqiad1 [09:19:55] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:20:07] !log dcaro@urcuchillay wp1 END (ERROR) - Cookbook wmcs.vps.create_project (exit_code=97) for project wp1 in eqiad1 [09:20:10] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:20:46] !log dcaro@urcuchillay wp1 START - Cookbook wmcs.vps.create_project for project wp1 in eqiad1 [09:20:47] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:20:49] !log dcaro@urcuchillay wp1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project wp1 in eqiad1 [09:20:50] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:21:03] !log dcaro@urcuchillay wp1 START - Cookbook wmcs.vps.create_project for project wp1 in eqiad1 [09:21:03] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:21:06] !log dcaro@urcuchillay wp1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project wp1 in eqiad1 [09:21:06] wmbot~dcaro@urcuchillay: Unknown project "wp1" [09:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [09:22:11] !log dcaro@urcuchillay sillytest1 START - Cookbook wmcs.vps.create_project for project sillytest1 in eqiad1 [09:22:12] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:22:46] !log dcaro@urcuchillay sillytest1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project sillytest1 in eqiad1 [09:22:46] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:25:17] !log dcaro@urcuchillay sillytest1 START - Cookbook wmcs.vps.create_project for project sillytest1 in eqiad1 [09:25:18] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:25:51] !log dcaro@urcuchillay sillytest1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project sillytest1 in eqiad1 [09:25:51] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:26:18] !log dcaro@urcuchillay sillytest1 START - Cookbook wmcs.vps.create_project for project sillytest1 in eqiad1 [09:26:18] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:26:28] !log dcaro@urcuchillay sillytest1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project sillytest1 in eqiad1 [09:26:29] wmbot~dcaro@urcuchillay: Unknown project "sillytest1" [09:31:38] (03PS1) 10David Caro: vps.create_project: add the checks mentioned in the wiki [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 [09:47:08] (03PS1) 10David Caro: vps.create_project: add users and quotas [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084044 [09:49:23] 06cloud-services-team, 10Cloud-VPS: [cloud-vps] creating a new project can override existing DNS entries - https://phabricator.wikimedia.org/T360294#10271460 (10dcaro) Also added to the cookbook: https://gerrit.wikimedia.org/r/c/cloud/wmcs-cookbooks/+/1084042 [09:53:42] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services: [wikireplicas] Automated tests for views - https://phabricator.wikimedia.org/T368050#10271477 (10fnegri) [10:13:55] 10Cloud Services Proposals, 10cloud-services-team (FY2024/2025-Q1-Q2), 10Cloud-VPS: Decision Request - How to do the Cloud VPS VXLAN/IPv6 migration - https://phabricator.wikimedia.org/T377467#10271532 (10aborrero) >>! In T377467#10270652, @bd808 wrote: >>>! In T377467#10267530, @aborrero wrote: >> * we can... [10:33:10] 10wikitech.wikimedia.org, 13Patch-For-Review, 10Wiki-Setup (Delete / Redirect): Retire labtestwiki - https://phabricator.wikimedia.org/T378260#10271586 (10Ladsgroup) Let me know if I can help on anything on this side [10:38:28] FIRING: PuppetAgentNoResources: No Puppet resources found on instance cloudinfra-idp-1 on project cloudinfra - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [10:46:13] 06cloud-services-team, 10wikitech.wikimedia.org, 10MW-on-K8s, 06serviceops: Review/update wikitech-static syncing after wikitech moves to Kubernetes - https://phabricator.wikimedia.org/T374114#10271594 (10aborrero) I noticed this alert today: {F57655150} I'm not sure if it is related to the work in this... [10:55:52] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: openstack: wmf sink: extend it to support IPv6 - https://phabricator.wikimedia.org/T378192#10271660 (10aborrero) the native upstream designate notification handler code does a good chunk of the things we do in our custom code. In particular: * [[ http... [11:03:10] (03PS1) 10David Caro: vps.create_project: small refactor [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084053 [11:03:10] (03PS1) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [11:04:38] (03CR) 10FNegri: [C:03+1] "I think all of these checks can also be implemented in OpenTofu, but it's not trivial so I'm fine with keeping them in the cookbook for no" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:05:40] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:05:45] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:05:53] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:05:54] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:08:19] (03PS2) 10David Caro: vps.create_project: add users and quotas [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084044 [11:08:19] (03PS2) 10David Caro: vps.create_project: small refactor [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084053 [11:08:19] (03PS2) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [11:08:20] (03PS1) 10David Caro: vps.create_project: skip codfw1dev from recordset checks [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084056 [11:08:31] (03PS2) 10David Caro: vps.create_project: add the checks mentioned in the wiki [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 [11:08:31] (03PS3) 10David Caro: vps.create_project: add users and quotas [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084044 [11:08:32] (03PS3) 10David Caro: vps.create_project: small refactor [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084053 [11:08:36] (03PS3) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [11:09:03] (03CR) 10David Caro: vps.create_project: add the checks mentioned in the wiki (031 comment) [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:09:53] (03CR) 10David Caro: "Ack. The checks for users need us to have the users in tofu, and the recordsets the same, I don't think we are going to have the old wmclo" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:10:48] (03PS3) 10David Caro: vps.create_project: add the checks mentioned in the wiki [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 [11:10:48] (03PS4) 10David Caro: vps.create_project: add users and quotas [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084044 [11:10:48] (03PS4) 10David Caro: vps.create_project: small refactor [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084053 [11:10:49] (03PS4) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [11:11:58] (03CR) 10David Caro: "Oh, the users come from LDAP too, so not sure how to put that into tofu either, it would have to call out to openstack or ldap somehow." [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:12:13] (03Abandoned) 10David Caro: vps.create_project: skip codfw1dev from recordset checks [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084056 (owner: 10David Caro) [11:12:47] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:12:48] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:12:51] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:12:52] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:13:34] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:13:34] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:13:37] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:13:37] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:15:06] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:15:06] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:15:10] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:15:10] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:15:41] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:15:41] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:15:48] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:15:49] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:19:46] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [11:19:48] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:19:50] (03PS1) 10Muehlenhoff: Remove obsolete stub cert for config-master [labs/private] - 10https://gerrit.wikimedia.org/r/1084058 (https://phabricator.wikimedia.org/T357750) [11:19:55] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/104 [11:23:59] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [11:24:00] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [11:24:06] (03CR) 10FNegri: [C:03+1] "Recordsets, we might import everything, but let's discuss separately." [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:25:06] (03CR) 10FNegri: [C:03+1] "s/I don't it/I don't think it/" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:26:40] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services: [wikireplicas] Automated tests for views - https://phabricator.wikimedia.org/T368050#10271754 (10fnegri) a:03fnegri [11:30:14] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/104 [11:30:22] (03open) 10dcaro: IGNORE: dcaro test [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/105 [11:32:26] (03CR) 10Arturo Borrero Gonzalez: [C:04-1] "please implement this in the tofu-infra repository. It should be easy to do." [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [11:59:06] 06cloud-services-team, 10Data-Services: [wikireplicas] Automatically check for missing tables - https://phabricator.wikimedia.org/T378470 (10fnegri) 03NEW [11:59:33] 06cloud-services-team, 10Data-Services: [wikireplicas] Automatically check for missing tables - https://phabricator.wikimedia.org/T378470#10271967 (10fnegri) [12:22:29] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete stub cert for config-master [labs/private] - 10https://gerrit.wikimedia.org/r/1084058 (https://phabricator.wikimedia.org/T357750) (owner: 10Muehlenhoff) [12:47:26] (03PS1) 10Muehlenhoff: Remove obsolete stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084101 (https://phabricator.wikimedia.org/T360636) [12:51:48] (03PS1) 10Muehlenhoff: Remove wikitech stub cert [labs/private] - 10https://gerrit.wikimedia.org/r/1084104 (https://phabricator.wikimedia.org/T371878) [12:58:05] 06cloud-services-team, 10wikitech.wikimedia.org, 10MW-on-K8s, 06serviceops: Review/update wikitech-static syncing after wikitech moves to Kubernetes - https://phabricator.wikimedia.org/T374114#10272227 (10Clement_Goubert) The import process seems to stall out after a few hundred pages, ~~I'll try to upgrad... [13:09:32] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Toolforge (Toolforge iteration 16), 07Epic: [Hypotesis] 6.3.5 Develop the sustainability score - https://phabricator.wikimedia.org/T376896#10272342 (10dcaro) 05In progress→03Stalled [13:10:15] (03update) 10dcaro: IGNORE: dcaro test [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/105 [13:11:51] (03close) 10dcaro: IGNORE: dcaro test [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/105 [13:17:11] (03PS5) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [13:17:28] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [13:17:29] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:17:36] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [13:17:37] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:17:40] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/106 [13:18:28] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [13:18:29] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:18:33] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [13:18:33] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:19:07] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/106 [13:19:17] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [13:19:17] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:19:26] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/107 [13:21:01] (03CR) 10CI reject: [V:04-1] vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 (owner: 10David Caro) [13:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [13:21:31] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/107 [13:35:56] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/107 [13:37:00] (03open) 10aborrero: Revert "codfw1dev: use neutron extension to publish DNS records for ports" [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/108 (https://phabricator.wikimedia.org/T377740) [13:39:07] (03merge) 10aborrero: Revert "codfw1dev: use neutron extension to publish DNS records for ports" [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/108 (https://phabricator.wikimedia.org/T377740) [13:39:31] !log aborrero@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [13:40:11] !log aborrero@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [13:43:41] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/107 [13:49:05] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/107 [13:49:25] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [13:49:26] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:49:29] (03approved) 10fnegri: tofu-infra: delete test projects [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/92 (owner: 10aborrero) [13:49:39] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [13:49:39] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:49:40] (03update) 10fnegri: tofu-infra: delete test projects [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/92 (owner: 10aborrero) [13:49:50] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/109 [13:54:59] !log dcaro@urcuchillay dcaro-test1 END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project dcaro-test1 in codfw1dev [13:55:01] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:55:22] (03close) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/109 [13:55:38] !log dcaro@urcuchillay dcaro-test1 START - Cookbook wmcs.vps.create_project for project dcaro-test1 in codfw1dev [13:55:38] wmbot~dcaro@urcuchillay: Unknown project "dcaro-test1" [13:55:50] (03open) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/110 [14:10:14] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Toolforge (Toolforge iteration 16), 05Goal: [toolsdb] Upgrade to MariaDB 10.6 - https://phabricator.wikimedia.org/T352206#10272783 (10fnegri) [14:10:56] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Toolforge (Toolforge iteration 16), 05Goal: [toolsdb] test failover procedure - https://phabricator.wikimedia.org/T344719#10272785 (10fnegri) [14:18:44] 10Toolforge (Toolforge iteration 16): [components-api] Add feature flag to disable user endpoints for deployment in tools - https://phabricator.wikimedia.org/T378500 (10dcaro) 03NEW [14:18:44] 10Toolforge (Toolforge iteration 16): [components-api] Add feature flag to disable user endpoints for deployment in tools - https://phabricator.wikimedia.org/T378500#10272853 (10dcaro) p:05Triage→03High [14:20:17] 10Toolforge (Toolforge iteration 16), 13Patch-For-Review: [lima-kilo] support caching of container images using a cache disk - https://phabricator.wikimedia.org/T378180#10272856 (10dcaro) p:05Triage→03Medium [14:20:25] 06cloud-services-team, 10Toolforge (Toolforge iteration 16), 13Patch-For-Review: [harbor] Do not clean up images currently running in production - https://phabricator.wikimedia.org/T377854#10272857 (10dcaro) p:05Triage→03High [14:31:06] 10Toolforge (Toolforge iteration 16), 13Patch-For-Review: restarting a continuous jobs causes for some seconds two jobs are running side by side - https://phabricator.wikimedia.org/T375366#10272899 (10dcaro) a:03aborrero [14:34:58] 06cloud-services-team, 10Toolforge: [maintain-dbusers] When it stops working (ex. nfs got stuck), it still replies as ok to prometheus - https://phabricator.wikimedia.org/T375224#10272913 (10dcaro) [14:35:00] 10Toolforge (Toolforge iteration 16): lima-kilo installation giving inconsistent result. Sometimes it works, sometimes it doesn't - https://phabricator.wikimedia.org/T375163#10272915 (10dcaro) a:03Raymond_Ndibe [14:35:30] 06cloud-services-team, 10Toolforge: [k8s,infra,cookbook] change the hiera under the -k8s-control prefix whet adding/removing an etcd node - https://phabricator.wikimedia.org/T371370#10272921 (10dcaro) [14:37:56] 06cloud-services-team, 10Toolforge, 13Patch-For-Review: [toolforge,storage] Provide per-tool access to cloud-vps object storage - https://phabricator.wikimedia.org/T358496#10272930 (10dcaro) [14:39:11] 10Toolforge (Toolforge iteration 16), 07Documentation: [harbor,docs] Improve Harbor quota handling and docs - https://phabricator.wikimedia.org/T351092#10272933 (10dcaro) a:03Raymond_Ndibe [14:39:43] 06cloud-services-team, 10Toolforge (Toolforge iteration 16): [api-gateway] add alert for uptime - https://phabricator.wikimedia.org/T348633#10272934 (10dcaro) a:03dcaro [14:46:26] 06cloud-services-team, 10Toolforge (Toolforge iteration 16), 13Patch-For-Review: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.28 - https://phabricator.wikimedia.org/T362867#10272985 (10fnegri) [14:46:29] 10Toolforge (Toolforge iteration 16): [components-api] Add feature flag to disable user endpoints for deployment in tools - https://phabricator.wikimedia.org/T378500#10272986 (10fnegri) [14:57:42] 10VPS-project-Wikistats: Add tcywikisource to wikistats - https://phabricator.wikimedia.org/T378475#10273019 (10Dzahn) 05Open→03Stalled blocked on T377919 [14:58:18] 10VPS-project-Wikistats: Add tcywiktionary to wikistats - https://phabricator.wikimedia.org/T378467#10273025 (10Dzahn) 05Open→03Stalled blocked on T377922 [15:10:50] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static is out of disk - https://phabricator.wikimedia.org/T348503#10273093 (10Clement_Goubert) 05Resolved→03Open a:05taavi→03Clement_Goubert For documentation purpose, the maintenance script `deleteArchivedFiles.php` does not work and return 0 f... [15:12:14] (03CR) 10Majavah: [C:03+2] tools: membership: Post note to SUL user talk page [labs/striker] - 10https://gerrit.wikimedia.org/r/1083302 (owner: 10Majavah) [15:12:20] (03CR) 10Majavah: [C:03+2] tools: membership: Link to ldap tool instead of wikitech [labs/striker] - 10https://gerrit.wikimedia.org/r/1083303 (owner: 10Majavah) [15:12:27] 06cloud-services-team, 10wikitech.wikimedia.org, 10MW-on-K8s, 06serviceops: Review/update wikitech-static syncing after wikitech moves to Kubernetes - https://phabricator.wikimedia.org/T374114#10273089 (10Clement_Goubert) This is {T348503} again, reopening this task. [15:13:06] (03CR) 10Clément Goubert: [C:03+1] Remove obsolete stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084101 (https://phabricator.wikimedia.org/T360636) (owner: 10Muehlenhoff) [15:14:30] (03Merged) 10jenkins-bot: tools: membership: Post note to SUL user talk page [labs/striker] - 10https://gerrit.wikimedia.org/r/1083302 (owner: 10Majavah) [15:14:30] (03Merged) 10jenkins-bot: tools: membership: Link to ldap tool instead of wikitech [labs/striker] - 10https://gerrit.wikimedia.org/r/1083303 (owner: 10Majavah) [15:26:45] 06cloud-services-team, 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273184 (10sbassett) 05Open→03Resolved a:03Dreamy_Jazz >>! In T375751#1027125... [15:30:15] (03open) 10aborrero: codfw1dev: drop dedicated DNS zone cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/111 (https://phabricator.wikimedia.org/T377740) [15:31:16] 06cloud-services-team, 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273261 (10Dreamy_Jazz) Are we sure that the replicas have been fully updated? My l... [15:32:46] (03open) 10dcaro: yaml: pre-parse yamls with python [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/112 [15:33:19] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] (parse_yamls_with_python) - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/110 [15:33:22] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] (parse_yamls_with_python) - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/110 [15:34:02] (03approved) 10taavi: codfw1dev: drop dedicated DNS zone cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/111 (https://phabricator.wikimedia.org/T377740) (owner: 10aborrero) [15:34:16] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273204 (10sbassett) [15:35:22] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] (parse_yamls_with_python) - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/110 [15:35:49] (03update) 10dcaro: projects: added project dcaro-test1 [repos/cloud/cloud-vps/tofu-infra] (parse_yamls_with_python) - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/110 [15:35:58] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273332 (10fnegri) [15:37:13] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273352 (10sbassett) If it's just the analytics replicas that were... [15:38:02] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084101 (https://phabricator.wikimedia.org/T360636) (owner: 10Muehlenhoff) [15:38:28] (03merge) 10aborrero: codfw1dev: drop dedicated DNS zone cloudinfra-codfw1dev.codfw1dev.wikimedia.cloud [repos/cloud/cloud-vps/tofu-infra] - 10https://gitlab.wikimedia.org/repos/cloud/cloud-vps/tofu-infra/-/merge_requests/111 (https://phabricator.wikimedia.org/T377740) [15:38:31] !log aborrero@cloudcumin1001 admin START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [15:38:44] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273359 (10Dreamy_Jazz) >>! In T375751#10273311, @fnegri wrote: > @... [15:38:58] !log aborrero@cloudcumin1001 admin END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [15:39:01] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove wikitech stub cert [labs/private] - 10https://gerrit.wikimedia.org/r/1084104 (https://phabricator.wikimedia.org/T371878) (owner: 10Muehlenhoff) [15:39:39] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static is out of disk - https://phabricator.wikimedia.org/T348503#10273346 (10Clement_Goubert) 05Open→03Resolved Resync done, alert cleared. cc @aborrero [15:42:09] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static: clarify alert destinations - https://phabricator.wikimedia.org/T378512 (10aborrero) 03NEW [15:42:31] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273311 (10fnegri) 05Resolved→03In progress a:05Dreamy_Jazz→... [15:44:59] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10273354 (10fnegri) p:05High→03Medium [15:45:15] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static is out of disk - https://phabricator.wikimedia.org/T348503#10273385 (10aborrero) >>! In T348503#10273346, @Clement_Goubert wrote: > Resync done, alert cleared. cc @aborrero Thanks! as a follow up: {T378512} [15:48:38] (03PS1) 10Muehlenhoff: Remove obsolete Icinga stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084148 [15:48:38] (03PS1) 10Muehlenhoff: Remove obsolete rendering stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084149 (https://phabricator.wikimedia.org/T357750) [15:48:39] (03PS1) 10Muehlenhoff: Remove stub certs for ms-fe [labs/private] - 10https://gerrit.wikimedia.org/r/1084150 (https://phabricator.wikimedia.org/T357750) [15:51:40] 06Toolforge-standards-committee: Reset members and owners for toolforge-standards-committee@lists.wikimedia.org - https://phabricator.wikimedia.org/T375134#10273424 (10bd808) [15:54:08] (03PS6) 10David Caro: vps.create_project: create the tofu patch [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084054 [15:55:13] 06Toolforge-standards-committee, 07User-notice: Refresh membership of Toolforge standards committee - https://phabricator.wikimedia.org/T370474#10273439 (10bd808) [15:55:15] 06Toolforge-standards-committee: Reset members and owners for toolforge-standards-committee@lists.wikimedia.org - https://phabricator.wikimedia.org/T375134#10273434 (10bd808) 05Open→03Resolved @JJMC89 has volunteered to co-admin the list. More folks are also welcome, but this at least gives us 2 folks fo... [15:57:53] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete Icinga stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084148 (owner: 10Muehlenhoff) [15:58:12] (03CR) 10Muehlenhoff: [V:03+2 C:03+2] Remove obsolete rendering stub certs [labs/private] - 10https://gerrit.wikimedia.org/r/1084149 (https://phabricator.wikimedia.org/T357750) (owner: 10Muehlenhoff) [15:59:22] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static: clarify alert destinations - https://phabricator.wikimedia.org/T378512#10273471 (10aborrero) p:05Triage→03Medium [16:03:20] 06cloud-services-team, 10Toolforge: Add --timeout to toolforge jobs - https://phabricator.wikimedia.org/T377782#10273494 (10dcaro) p:05Triage→03Low [16:07:24] 06cloud-services-team, 10wikitech.wikimedia.org: wikitech-static: clarify alert destinations - https://phabricator.wikimedia.org/T378512#10273520 (10bd808) > I guess this is because wikitech was hosted on cloudweb hardware servers. I suppose that was part of it, but I think really WMCS "ownership" is an artif... [16:12:01] 06cloud-services-team, 10Cloud-VPS, 13Patch-For-Review: Create mechanism to allow the use of vanity domains by projects behind the Cloud VPS shared HTTP proxy - https://phabricator.wikimedia.org/T342398#10273557 (10taavi) Note to self: we might want to add CAA records for any aliases that people will point C... [16:34:50] 06cloud-services-team, 10Toolforge, 10Temporary accounts: Check impact of Temporary Accounts on Toolforge tools - https://phabricator.wikimedia.org/T378516 (10fnegri) 03NEW [16:35:05] 06cloud-services-team, 10Toolforge, 10Temporary accounts: Check impact of Temporary Accounts on Toolforge tools - https://phabricator.wikimedia.org/T378516#10273689 (10fnegri) p:05Triage→03Medium [16:38:46] (03open) 10dcaro: restish: add autocompletion config [repos/cloud/toolforge/lima-kilo] - 10https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/205 [16:41:31] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10Temporary accounts: Verify if Temporary Accounts require any changes to Wiki Replicas - https://phabricator.wikimedia.org/T377879#10273720 (10fnegri) 05In progress→03Resolved Looks like there is nothing we need to do for this one. I also c... [16:41:43] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10Temporary accounts: Verify if Temporary Accounts require any changes to Wiki Replicas - https://phabricator.wikimedia.org/T377879#10273725 (10fnegri) [16:41:44] 06cloud-services-team, 10Toolforge, 10Temporary accounts: Check impact of Temporary Accounts on Toolforge tools - https://phabricator.wikimedia.org/T378516#10273726 (10fnegri) [16:42:03] (03PS1) 10Majavah: Allow creating apex domains [openstack/horizon/wmf-proxy-dashboard] - 10https://gerrit.wikimedia.org/r/1084172 (https://phabricator.wikimedia.org/T342398) [16:52:31] (03CR) 10Majavah: [C:03+2] Allow creating apex domains [openstack/horizon/wmf-proxy-dashboard] - 10https://gerrit.wikimedia.org/r/1084172 (https://phabricator.wikimedia.org/T342398) (owner: 10Majavah) [17:00:18] 06cloud-services-team, 10Cloud-VPS: openstack: wmf sink: extend it to support IPv6 - https://phabricator.wikimedia.org/T378192#10273814 (10aborrero) I created this repository to host the new code: https://gitlab.wikimedia.org/repos/cloud/cloud-vps/designate-sink-nova-fixed-ptr [17:07:05] (03PS1) 10Majavah: views: Fix variable confusion [openstack/horizon/wmf-proxy-dashboard] - 10https://gerrit.wikimedia.org/r/1084176 [17:07:27] (03CR) 10Majavah: [C:03+2] views: Fix variable confusion [openstack/horizon/wmf-proxy-dashboard] - 10https://gerrit.wikimedia.org/r/1084176 (owner: 10Majavah) [17:08:03] (03Merged) 10jenkins-bot: views: Fix variable confusion [openstack/horizon/wmf-proxy-dashboard] - 10https://gerrit.wikimedia.org/r/1084176 (owner: 10Majavah) [17:15:37] (03CR) 10David Caro: "Adding it on tofu-infra gives no feedback to the cookbook user until it's too late, I'm not saying it should not be added there (that prob" [cloud/wmcs-cookbooks] - 10https://gerrit.wikimedia.org/r/1084042 (owner: 10David Caro) [17:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [17:41:17] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10Temporary accounts: Verify if Temporary Accounts require any changes to Wiki Replicas - https://phabricator.wikimedia.org/T377879#10274014 (10kostajh) One issue is/was {T375751} [17:42:42] 10cloud-services-team (FY2024/2025-Q1-Q2), 10Data-Services, 10AbuseFilter, 06Data Products, and 8 others: Public wiki replicas contain abuse filter logs for filters that are private or protected - https://phabricator.wikimedia.org/T375751#10274025 (10kostajh) >>! In T375751#10273261, @Dreamy_Jazz wrote: >... [17:59:14] 10wikitech.wikimedia.org: ☂ Wikitech account linking and SUL error reporting - https://phabricator.wikimedia.org/T376267#10274128 (10bd808) >>! In T376267#10270799, @Robertsky wrote: > @bd808 well, the wikitech account does not have an email address attached to it as found out by @Ladsgroup. @Robertsky The mis... [18:18:34] FIRING: DiskSpace: Disk space cloudbackup1004:9100:/srv 5.972% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup1004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [19:49:33] (03CR) 10Brian Wolff: "Ultimately this is a fairly low risk domain since its on toolforge and isolated from everything else. I think defacement (or some sort of " [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1080836 (https://phabricator.wikimedia.org/T377168) (owner: 10Brian Wolff) [21:06:53] (03CR) 10Jforrester: [C:03+1] frontend/composer.json: Add missing ext- [labs/codesearch] - 10https://gerrit.wikimedia.org/r/1080102 (owner: 10Reedy) [21:18:11] FIRING: PuppetAgentNoResources: No Puppet resources found on instance toolsbeta-harbor-1 on project toolsbeta - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetAgentNoResources [21:21:31] FIRING: CloudVPSDesignateLeaks: Detected 4 stray dns records - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/Designate_record_leaks - https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-openstack-eqiad-nova-fullstack - https://alerts.wikimedia.org/?q=alertname%3DCloudVPSDesignateLeaks [22:18:34] FIRING: DiskSpace: Disk space cloudbackup1004:9100:/srv 5.63% free - https://wikitech.wikimedia.org/wiki/Monitoring/Disk_space - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&viewPanel=12&var-server=cloudbackup1004 - https://alerts.wikimedia.org/?q=alertname%3DDiskSpace [22:47:38] FIRING: PuppetCertificateAboutToExpire: Puppet CA certificate mwv-builder-03.mediawiki-vagrant.eqiad.wmflabs is about to expire in -4d 0h 1m 25s - https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/PuppetCertificateAboutToExpire - https://prometheus-alerts.wmcloud.org/?q=alertname%3DPuppetCertificateAboutToExpire [23:30:04] 10Tool-Pageviews, 10Tool-wikistatistics2-0, 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549 (10MusikAnimal) 03NEW [23:31:38] 10Tool-Pageviews, 10Tool-wikistatistics2-0, 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275187 (10MusikAnimal) I've tagged the components that this app may potentially be part of. If any maintainers o... [23:34:19] 10Tool-Pageviews, 10Tool-wikistatistics2-0, 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275188 (10MusikAnimal) [23:37:42] 10Tool-Pageviews, 10Tool-wikistatistics2-0, 06Data-Engineering, 10Data-Engineering-Wikistats, 10PageViewInfo: Pageviews Analysis 3.0 (Vue + Codex) - https://phabricator.wikimedia.org/T378549#10275191 (10MusikAnimal) [23:51:53] 06cloud-services-team, 10Data-Services, 06DBA: Prepare and check storage layer for tcywiktionary - https://phabricator.wikimedia.org/T378462#10275215 (10Zabe) wiki has been created