[06:15:15] 10Release-Engineering-Team (Seen), 10DBA, 10Epic: Implement a system to automatically deploy schema changes without needing DBA intervention - https://phabricator.wikimedia.org/T121857 (10Ladsgroup) Isn't this a duplicate of {T288235}? Or that would be a part before moving this one? [06:16:18] 10Release-Engineering-Team (Seen), 10DBA, 10Epic: Implement a system to automatically deploy schema changes without needing DBA intervention - https://phabricator.wikimedia.org/T121857 (10Marostegui) It is essentially the same, I would merge both. [06:26:19] 10Release-Engineering-Team (Seen), 10DBA, 10Epic: Implement a system to automatically deploy schema changes without needing DBA intervention - https://phabricator.wikimedia.org/T121857 (10Ladsgroup) [06:26:38] 10Release-Engineering-Team (Seen), 10DBA, 10Epic: Implement a system to automatically deploy schema changes without needing DBA intervention - https://phabricator.wikimedia.org/T121857 (10Ladsgroup) Given that the other had more concrete information and was better scoped. I merged this to the other. [08:15:28] Good morning [09:06:02] 10Phabricator, 10DBA, 10User-Ladsgroup, 10Vuln-DoS, 10Wikimedia-Incident: Phabricator can not connect to MySQL - https://phabricator.wikimedia.org/T298369 (10mmodell) >>! In T298369#7591308, @Marostegui wrote: > I am not sure if this is entirely related, but I have found quite a few runs of the following... [10:20:00] 10Release-Engineering-Team (Radar), 10Gerrit (Gerrit 3.3): Gerrit errors on change #650396, missing blob - https://phabricator.wikimedia.org/T270472 (10hashar) [11:50:51] * taavi waits patiently for the 0 running PCC jobs to complete before the one job in the queue starts [11:56:06] PROBLEM - SSH on contint1001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [12:57:02] RECOVERY - SSH on contint1001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [13:17:20] (03CR) 10Hashar: [C: 03+2] add Sanjay to allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/749867 (owner: 10RhinosF1) [13:17:22] (03CR) 10Hashar: [C: 03+2] Add Chiefwei to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/750256 (owner: 10Zabe) [13:20:00] (03Merged) 10jenkins-bot: add Sanjay to allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/749867 (owner: 10RhinosF1) [13:20:02] (03Merged) 10jenkins-bot: Add Chiefwei to the CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/750256 (owner: 10Zabe) [13:20:16] ty hashar [13:20:40] (03CR) 10Hashar: [C: 03+2] "I have updated the job:" [integration/config] - 10https://gerrit.wikimedia.org/r/749859 (owner: 10Majavah) [13:21:19] (03CR) 10Hashar: [C: 03+2] zuul: [cloud/toolforge/disable-tool] add tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/749853 (owner: 10Majavah) [13:22:34] (03Merged) 10jenkins-bot: jjb: beta-publish-deb: build on bullseye too [integration/config] - 10https://gerrit.wikimedia.org/r/749859 (owner: 10Majavah) [13:23:09] (03Merged) 10jenkins-bot: zuul: [cloud/toolforge/disable-tool] add tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/749853 (owner: 10Majavah) [13:36:41] (03PS13) 10Hashar: jjb: play with Jinja2 [integration/config] - 10https://gerrit.wikimedia.org/r/739282 [14:00:27] (03CR) 10Hashar: [C: 03+2] "no changes in jjb ;)" [integration/config] - 10https://gerrit.wikimedia.org/r/739282 (owner: 10Hashar) [14:02:29] (03Merged) 10jenkins-bot: jjb: play with Jinja2 [integration/config] - 10https://gerrit.wikimedia.org/r/739282 (owner: 10Hashar) [14:33:17] (03PS7) 10Hashar: jjb: migrate docker --volume to "volumes" yaml map [integration/config] - 10https://gerrit.wikimedia.org/r/739583 [14:35:18] !log Upgraded Java 11 on releases1002 && releases2002 [14:35:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:37:44] !log Upgraded Java 11 on contint2001 && contint1001. Restarted CI Jenkins. [14:37:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [14:42:13] (03PS8) 10Hashar: jjb: migrate docker --volume to "volumes" yaml map [integration/config] - 10https://gerrit.wikimedia.org/r/739583 [14:53:57] (03CR) 10Hashar: [C: 03+2] "The diff is all about "--volume /srv/git:/srv/git:ro" being moved around, it is however a noop finally :]" [integration/config] - 10https://gerrit.wikimedia.org/r/739583 (owner: 10Hashar) [14:55:59] (03Merged) 10jenkins-bot: jjb: migrate docker --volume to "volumes" yaml map [integration/config] - 10https://gerrit.wikimedia.org/r/739583 (owner: 10Hashar) [14:57:07] I am going to restart gerrit [15:00:18] Project mwcore-phpunit-coverage-master build #1867: 04FAILURE in 18 sec: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1867/ [15:00:30] 10Release-Engineering-Team, 10MW-on-K8s, 10serviceops: Build MediaWiki images for kubernetes on the deployment servers - https://phabricator.wikimedia.org/T297673 (10Joe) [15:03:07] the job failed cause Gerrit was restarting [15:03:17] I triggered it again manually [15:15:16] Yippee, build fixed! [15:15:17] Project mwcore-phpunit-coverage-master build #1868: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/1868/ [15:23:19] easy job fix [16:44:11] 10Continuous-Integration-Infrastructure: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10Xqt) [16:51:46] 10Continuous-Integration-Infrastructure: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10Xqt) See also T270389. 20% of Pywikibot pypi downloads are made for Python 3.9 (T266984) and it would be appropriate to have tests wit this Python version then. [17:30:04] hashar: hi! could you reply to https://phabricator.wikimedia.org/T247653#7553885 when you have a moment? [19:02:08] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Cloud-VPS, 10Wikidata, and 2 others: Wikibase selenium tests timeout, seemingly due to "memory compaction" events on CI VMs - https://phabricator.wikimedia.org/T281122 (10kostajh) >>! In T281122#7584992, @Ladsgroup wrote: >>>! In... [20:58:59] 10Release-Engineering-Team (Doing), 10MediaWiki-Core-Tests, 10MediaWiki-ResourceLoader, 10MW-1.38-notes (1.38.0-wmf.13; 2021-12-13), 10Performance-Team (Radar): Move bundlesize test from npm script to MediaWikiIntegrationTest - https://phabricator.wikimedia.org/T255149 (10kostajh) 05Open→03Resolved I... [21:05:56] PROBLEM - SSH on contint1001.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [23:08:18] RECOVERY - SSH on contint1001.mgmt is OK: SSH OK - OpenSSH_6.6 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook