[03:53:33] (03PS2) 10Subramanya Sastry: Add a div.mw-parser-output wrapper as a child of Parsoid's body [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 [03:53:35] (03PS1) 10Subramanya Sastry: Additional fixes to improve visualdiff results [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712848 [03:53:53] (03CR) 10Subramanya Sastry: "Good to go." [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 (owner: 10Subramanya Sastry) [03:54:16] (03CR) 10Subramanya Sastry: "After this, I'll go back to direct push" [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712848 (owner: 10Subramanya Sastry) [08:40:23] 10phabricator maintenance bot, 10SRE Observability (FY2021/2022-Q1): update legacy project tags - https://phabricator.wikimedia.org/T288756 (10Ladsgroup) (Operator of the bot here). I don't know the details of this case but if it's a one-time thing, a silent mass editing is your way to go, if it's something th... [09:25:45] 10Release-Engineering-Team (Doing), 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10Qgil) Thank you so much @Aklapper for this investigation and these queries! @jrbs can we take these queries and run with them or do we need anything... [12:29:29] ACKNOWLEDGEMENT - SSH on contint2001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds daniel_zahn https://phabricator.wikimedia.org/T283582 https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [13:13:05] !log `mwscript extensions/CentralAuth/maintenance/importMissingLocalNames.php --wiki metawiki` on the beta cluster [13:13:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:24:39] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [14:29:17] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [16:00:43] 10Beta-Cluster-Infrastructure, 10User-Zabe: COPYING and CREDITS give 403 on beta cluster - https://phabricator.wikimedia.org/T288838 (10Zabe) [17:23:52] 10Release-Engineering-Team (Doing), 10Elections: Create list of developers eligible to vote on the 2021 board vote - https://phabricator.wikimedia.org/T288455 (10jrbs) >>! In T288455#7280942, @Qgil wrote: > Thank you so much @Aklapper for this investigation and these queries! > > @jrbs can we take these queri... [18:10:04] 10Release-Engineering-Team (Radar), 10MW-on-K8s, 10SRE, 10serviceops: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10dancy) Note: There is always a delay of 3 seconds before the 500 response is returned. [18:29:30] 10Release-Engineering-Team (Radar), 10MW-on-K8s, 10SRE, 10serviceops: The restricted/mediawiki-webserver image should include skins and resources - https://phabricator.wikimedia.org/T285232 (10dancy) favicon.ico issue is an example of T288848 [18:51:33] (03PS1) 10BBlack: operations-dnslint: create version 0.0.12 [integration/config] - 10https://gerrit.wikimedia.org/r/712990 (https://phabricator.wikimedia.org/T252132) [18:51:35] (03PS1) 10BBlack: operations-dnslist: bump to 0.0.12 [integration/config] - 10https://gerrit.wikimedia.org/r/712991 (https://phabricator.wikimedia.org/T252132) [19:14:44] It seems creation of this child task was not logged on the parent. Is that a regression? https://phabricator.wikimedia.org/T288851 https://phabricator.wikimedia.org/T283056#7255333 [19:30:44] (03CR) 10Arlolra: Add a div.mw-parser-output wrapper as a child of Parsoid's body (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 (owner: 10Subramanya Sastry) [19:53:30] It seems like beta cluster hasn't updated to master for over 36 hours now. [19:59:39] Last was https://github.com/wikimedia/mediawiki/commit/63da9a14300c212c678b147f5703ffa356a452c7 [20:01:50] zabe: it's stuck https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-sync-world/ [20:02:23] urbanecm: you've unstuck it before ^ [20:02:43] (4 days ago) [20:02:47] RhinosF1: let me kick it again :/ [20:02:58] Ty [20:03:06] But ye :( [20:03:37] Project beta-update-databases-eqiad build #52472: 04FAILURE in 6.6 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/52472/ [20:03:42] :/ [20:03:51] !log Kill beta-scap-sync-world job for the usual reason [20:03:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:04:23] should run automatically soon [20:04:26] It's moving now zabe [20:04:37] https://integration.wikimedia.org/ci/view/Beta/job/beta-code-update-eqiad/357294/ [20:05:27] urbanecm: that database fail is an actual exception [20:05:40] yeah, looking into it [20:06:54] !log deployment-prep: sudo -u jenkins-deploy /usr/local/bin/wmf-beta-update-databases.py [20:06:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:07:07] manually running the script sounds to do something [20:09:50] !log Manually start `beta-update-databases-eqiad` CI job [20:09:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:09:54] maybe a fluke? [20:11:17] Maybe [20:11:20] Yippee, build fixed! [20:11:21] Project beta-update-databases-eqiad build #52473: 09FIXED in 1 min 45 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/52473/ [20:11:24] Yeah [20:11:32] It'll run in 10 minutes again [20:11:33] the manual job finished successfully [20:11:36] i know [20:11:40] * urbanecm didn't want to wait [20:11:45] Might be grumpy at not running for over a day [20:11:50] possible [20:12:06] or just mad at us for no reason at all :) [20:12:21] It's Friday evening so :) [20:12:38] And for my I'm on holiday next week [20:12:44] enjoy :) [20:13:19] Hoping it'll be good [20:14:08] Probably going to ignore the world for 3 days so no one can bother me [20:22:17] zabe: 1.37.0-alpha (dc32117) [20:22:17] 18:39, 13 August 2021 [20:23:43] yes, thanks [20:25:55] urbanecm: looks good now, thanks for kicking it [20:26:18] any time :) [20:45:35] (03CR) 10Subramanya Sastry: Add a div.mw-parser-output wrapper as a child of Parsoid's body (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 (owner: 10Subramanya Sastry) [21:37:05] (03CR) 10Arlolra: Add a div.mw-parser-output wrapper as a child of Parsoid's body (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 (owner: 10Subramanya Sastry) [22:03:19] (03CR) 10Subramanya Sastry: Add a div.mw-parser-output wrapper as a child of Parsoid's body (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712575 (owner: 10Subramanya Sastry) [22:09:55] (03CR) 10Arlolra: Additional fixes to improve visualdiff results (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712848 (owner: 10Subramanya Sastry) [22:14:07] (03CR) 10Subramanya Sastry: Additional fixes to improve visualdiff results (031 comment) [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/712848 (owner: 10Subramanya Sastry)