[01:52:45] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10378154 [07:10:39] 10Phabricator: Fix commit identity of Valerio Bozz. - https://phabricator.wikimedia.org/T381461 (10valerio.bozzolan) 03NEW [07:11:03] 10Phabricator: Fix commit identity of Valerio Bozz. - https://phabricator.wikimedia.org/T381461#10378366 (10valerio.bozzolan) [07:42:11] 10GitLab (Pipeline Services Migration🐤), 06collaboration-services, 10Wikidata, 10Wikidata Query UI, and 2 others: move commons-query.wikimedia.org and query.wikidata.org to kubernetes - https://phabricator.wikimedia.org/T350793#10378435 (10Jelto) Yesterday, there were reports of `commons-query.wikimedia.or... [08:44:05] 10Phabricator: Fix commit identity of Valerio Bozz. - https://phabricator.wikimedia.org/T381461#10378554 (10Aklapper) I'm afraid I cannot follow... > And on the "Provenance" section please click "Change Identity" There is no "Change Identity" thingie shown for me... > - remove @ValerioBoz-WMCH What makes you... [08:59:43] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10378577 [09:10:22] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Quibble: Speed up git clone operation in wmf-quibble job - https://phabricator.wikimedia.org/T374717#10378586 (10hashar) 05Open→03Resolved A build of wmf-quibble-core-vendor-mysql-php74 from this morning show the zuul clone took 3... [09:35:43] nicely done hashar! [09:37:48] kostajh: yeah that one has been a long tail :b [09:38:09] I eventually gave up trying to understand why git takes so long processing WikimediaMessages [09:38:12] (13 seconds) [09:39:01] I also can't see the effect in the dashboards [09:39:11] cause the parallelism got enabled roughly during the same period [09:47:31] amazing - thank you! :) Now we just need a plan for those pesky Selenium jobs [09:55:04] dancy, ryankemper: really sorry for the mess with the Scap bootstrapping, got a fix here: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100407 [09:57:40] 06Release-Engineering-Team, 06Data Products, 06Data-Platform, 06Data-Platform-SRE, and 4 others: Generate a dumps-enabled mediawiki image - https://phabricator.wikimedia.org/T381473 (10Joe) 03NEW [10:00:30] 10Scap, 13Patch-For-Review: scap installer: allow for non-deployment server installs - https://phabricator.wikimedia.org/T380772#10378729 (10jnuche) Sorry for the problem with the host bootstrapping, I didn't remember the bootstrapping code was a separate script from `install_local_version.sh`. Patch in the co... [10:11:23] (03open) 10jnuche: install-world: follow-up to recent rework [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/609 (https://phabricator.wikimedia.org/T380772) [10:12:13] (03update) 10jnuche: install-world: follow-up to recent rework [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/609 (https://phabricator.wikimedia.org/T380772) [10:22:38] (03merge) 10jnuche: install-world: follow-up to recent rework [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/609 (https://phabricator.wikimedia.org/T380772) [10:28:06] 06Release-Engineering-Team, 06Data Products, 06Data-Platform-SRE, 10Dumps-Generation, and 3 others: Migrate current-generation dumps to run on kubernetes - https://phabricator.wikimedia.org/T352650#10378803 (10Joe) [11:36:10] 10Phabricator, 06collaboration-services, 06Trust-and-Safety, 10Znuny: Sunset / Retire Legalpad Phabricator application? - https://phabricator.wikimedia.org/T363009#10379050 (10Aklapper) [11:53:59] 10Continuous-Integration-Config, 10MediaWiki-extensions-WikibaseClient, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 2 others: [REPO][CLIENT][SW] Re-start Wikibase test coverage reporting - https://phabricator.wikimedia.org/T288396#10379144 (10ItamarWMDE) [12:45:12] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10379441 [13:36:59] (03CR) 10Hashar: [C:03+2] zuul: Add CampaignEvents and WikimediaCampaignEvents to the gate [integration/config] - 10https://gerrit.wikimedia.org/r/1094459 (https://phabricator.wikimedia.org/T380595) (owner: 10Daimona Eaytoy) [13:38:16] (03Merged) 10jenkins-bot: zuul: Add CampaignEvents and WikimediaCampaignEvents to the gate [integration/config] - 10https://gerrit.wikimedia.org/r/1094459 (https://phabricator.wikimedia.org/T380595) (owner: 10Daimona Eaytoy) [13:39:00] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/c/integration/config/+/1094459 : Add CampaignEvents and WikimediaCampaignEvents to the gate - T380595 [13:39:03] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:39:03] T380595: Add CampaignEvents and WikimediaCampaignEvents to the list of gated extensions - https://phabricator.wikimedia.org/T380595 [14:31:18] 10Continuous-Integration-Config, 10CampaignEvents, 10WikimediaCampaignEvents, 10Campaigns-Product-Team (Campaign-Tools-Current-Sprint): Add CampaignEvents and WikimediaCampaignEvents to the list of gated extensions - https://phabricator.wikimedia.org/T380595#10379886 (10Daimona) 05Open→03Resolved T... [14:55:49] 10Gerrit (Gerrit 3.11): Add support for Vue language in Gerrit code editor - https://phabricator.wikimedia.org/T380668#10379979 (10Paladox) It's properly fixed with https://gerrit-review.googlesource.com/c/gerrit/+/444763 Fixed in the CM plugin here: https://gerrit-review.googlesource.com/c/plugins/codemirror-e... [15:00:51] dancy: fyi - scap's offline scripts don't play well with statsd and logstash. Left some details at https://phabricator.wikimedia.org/T380609#10379995. Could use a second pair of eyes from someone more familiar with how and where Scap runs these scripts. Is it docker-build? What other env vars are set? Do we have a solution for logging already? [15:02:07] 10Scap, 10MediaWiki-libs-Stats, 10MediaWiki-Maintenance-system, 10Observability-Metrics, 10MW-1.44-notes (1.44.0-wmf.6; 2024-12-03): Maintenance scripts do not emit StatsLib metrics - https://phabricator.wikimedia.org/T380609#10379995 (10Krinkle) >>! From `#wikimedia-operations`: > mergeMes... [15:52:19] Krinkle: Thanks for the ticket. I'll check it out later. [16:33:04] 10Scap: scap installer: allow for non-deployment server installs - https://phabricator.wikimedia.org/T380772#10380367 (10jnuche) 05Open→03Resolved New Puppet bootstrapping script merged. Tested on `releases2003` after the merge. Closing again [16:35:18] 10Scap: scap installer: allow for non-deployment server installs - https://phabricator.wikimedia.org/T380772#10380375 (10jnuche) @dduvall You may be interested in this change. You can now deploy new releases of Scap to a particular target without modifying Scap on the deployment server. [17:43:47] 06Release-Engineering-Team, 06Data Products, 06Data-Platform, 06Data-Platform-SRE, and 4 others: Generate a dumps-enabled mediawiki image - https://phabricator.wikimedia.org/T381473#10380690 (10BTullis) Thanks for creating this ticket. I noticed a few other things that our image will need, based on: [[htt... [18:17:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10380834 [18:47:24] 06Release-Engineering-Team, 06Data Products, 06Data-Platform, 06Data-Platform-SRE, and 4 others: Generate a dumps-enabled mediawiki image - https://phabricator.wikimedia.org/T381473#10380942 (10hashar) You can retrieve a tarball directly from Gerrit using eg: Direct commit: https://gerrit.wikimedia.org/g/... [19:20:48] (03CR) 10Hashar: [C:03+2] Zuul: Add RV1971 to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1099765 (owner: 10Pppery) [19:22:35] (03Merged) 10jenkins-bot: Zuul: Add RV1971 to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/1099765 (owner: 10Pppery) [19:22:50] ¡deployed [19:39:38] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) - https://grafana.wikimedia.org/alerting/grafana/b9a8470a-ebab-46f7-9be2-22b5e74a528b/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [19:44:38] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) - https://grafana.wikimedia.org/alerting/grafana/b9a8470a-ebab-46f7-9be2-22b5e74a528b/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [19:58:18] (03open) 10dancy: JobCard.vue: Fix statusChipMessage logic [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/610 [19:58:20] (03update) 10dancy: JobCard.vue: Fix statusChipMessage logic [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/610 [19:58:53] (03update) 10dancy: JobCard.vue: Fix statusChipMessage logic [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/610 [20:15:57] i'm going to try to do this: https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/How_code_is_updated#Cherry-picking_a_patch_from_gerrit i hope i don't break everything [20:17:07] (03open) 10dancy: fakeApiServer/fakeBackport: Make job signalling work. [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/611 [20:17:10] (03update) 10dancy: fakeApiServer/fakeBackport: Make job signalling work. [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/611 [20:19:29] (03update) 10dancy: fakeApiServer/fakeBackport: Make job signalling work. [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/611 [20:19:55] (03update) 10dancy: Draft: web: improve JobViewerPage UI [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/608 (https://phabricator.wikimedia.org/T379413) (owner: 10lwatson) [20:22:17] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100530 [20:22:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:26:04] i just don't understand how YAML with random templating engines became the devops standard [20:26:18] it is just beyond my understanding. how did we bring this upon ourselves. [20:26:54] anyway, what i'm saying is that my thing didn't work because Apache expression syntax and the random templating engine interpolation syntax conflict [20:27:08] and i needed to insert a %{} inside another %{} [20:27:17] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100530 patchset 2 [20:27:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:31:29] that, and the markdown link syntax. is it () or [] first? is is the label or the target first? has anyone ever gotten that right at first try? so many questions, so few answers. [20:34:41] 10GitLab (Account Approval), 06Release-Engineering-Team: Requesting GitLab account activation for ranjithraj - https://phabricator.wikimedia.org/T381531 (10Ranjithraj) 03NEW [20:40:48] puppet errors are something else [20:41:25] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100531 [20:41:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:44:05] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100532 [20:44:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:45:45] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100533 [20:45:46] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:48:36] !log Cherry-picked https://gerrit.wikimedia.org/r/c/operations/puppet/+/1100534 [20:48:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:00:21] (03open) 10dancy: spiderpig/api.py: Return interaction info in GET /api/jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/612 [21:00:24] (03update) 10dancy: spiderpig/api.py: Return interaction info in GET /api/jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/612 [21:02:10] (03merge) 10dancy: spiderpig/api.py: Return interaction info in GET /api/jobs [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/612 [21:10:35] FIRING: DatasourceError: Queue (Jenkins jobs + Zuul functions) - https://grafana.wikimedia.org/alerting/grafana/b9a8470a-ebab-46f7-9be2-22b5e74a528b/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [21:20:35] RESOLVED: DatasourceError: Queue (Jenkins jobs + Zuul functions) - https://grafana.wikimedia.org/alerting/grafana/b9a8470a-ebab-46f7-9be2-22b5e74a528b/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [21:28:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 06cloud-services-team, 10Cloud-VPS, 10ci-test-error (WMF-deployed Build Failure): Various CI jobs running in the integration Cloud VPS project failing due to transient DNS lookup... - https://phabricator.wikimedia.org/T374830#10381549 [22:03:01] (03PS2) 10Thcipriani: Reinstate the banner for the developer survey [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1100163 (owner: 10Hashar) [22:03:01] (03CR) 10Thcipriani: "Got you the privacy link, I'll get the survey link Soon™ Thank you for this ❤️" [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1100163 (owner: 10Hashar) [22:05:23] (03Abandoned) 10Thcipriani: Add a banner for the 2024 developer survey [software/gerrit] (deploy/wmf/stable-3.10) - 10https://gerrit.wikimedia.org/r/1100162 (https://phabricator.wikimedia.org/T351109) (owner: 10Thcipriani) [22:17:30] (03update) 10dancy: fakeApiServer/fakeBackport: Make job signalling work. [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/611 [22:18:53] (03merge) 10dancy: fakeApiServer/fakeBackport: Make job signalling work. [repos/releng/scap] - 10https://gitlab.wikimedia.org/repos/releng/scap/-/merge_requests/611