[06:13:11] dancy: I think I will get rid of Zuul. I am tired of all those bugs [06:13:26] Thanks for the fix! [07:16:40] 10Continuous-Integration-Config: Please add mediawiki-i18n-check-docker to repository mediawiki/extensions/WikiVideos - https://phabricator.wikimedia.org/T321769 (10Nikerabbit) [07:17:53] 10Continuous-Integration-Config: Please add mediawiki-i18n-check-docker to repository mediawiki/extensions/WikiVideos - https://phabricator.wikimedia.org/T321769 (10Nikerabbit) In the previous task I asked: > Per https://www.mediawiki.org/wiki/Gerrit/L10n-bot this should have happened automatically, but it hasn'... [07:21:50] (03CR) 10Hashar: [C: 03+2] "Thank you for the patch James. I am updating all the Jenkins jobs." [integration/config] - 10https://gerrit.wikimedia.org/r/849647 (https://phabricator.wikimedia.org/T289222) (owner: 10Jforrester) [07:24:20] (03Merged) 10jenkins-bot: jjb: Update Python jobs to ones with py39 [integration/config] - 10https://gerrit.wikimedia.org/r/849647 (https://phabricator.wikimedia.org/T289222) (owner: 10Jforrester) [07:27:29] 10Continuous-Integration-Config: Please add mediawiki-i18n-check-docker to repository mediawiki/extensions/WikiVideos - https://phabricator.wikimedia.org/T321769 (10hashar) In integration/config zuul/layout.yaml we define a template of jobs such as: ` lang=yaml templates: - name: extension-quibble gate-and-s... [07:29:58] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10hashar) 05Open→03Resolved a:03jbond Thanks @jbond !!!!!!! [08:01:35] 10Continuous-Integration-Infrastructure: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10hashar) 05Resolved→03Open When adding `py39` to a `tox.ini` CI fails due to lack of `distutils.cmd` (example: https://gerrit.wikimedia.org/r/850002). `distutils` is deprecated and will be r... [08:21:13] 10Phabricator, 10Release-Engineering-Team, 10translatewiki.net, 10Language-Team (Language-2022-October-December), and 2 others: Phabricator translations can no more be received due to decommissioning of git-ssh - https://phabricator.wikimedia.org/T321350 (10Nikerabbit) p:05Triage→03Medium [08:30:38] (03CR) 10Hashar: "We had curl installed in other composer-php* images as part of T235221 at about the same time the composer-php74 got created. I guess that" [integration/config] - 10https://gerrit.wikimedia.org/r/849712 (https://phabricator.wikimedia.org/T321753) (owner: 10Dwisehaupt) [08:36:16] (03PS2) 10Hashar: docker: add curl for composer-security-check [integration/config] - 10https://gerrit.wikimedia.org/r/849712 (https://phabricator.wikimedia.org/T321753) (owner: 10Dwisehaupt) [08:37:32] (03CR) 10Hashar: [C: 03+2] "I have amended your change to also rebuild the child images doing:" [integration/config] - 10https://gerrit.wikimedia.org/r/849712 (https://phabricator.wikimedia.org/T321753) (owner: 10Dwisehaupt) [08:39:13] (03Merged) 10jenkins-bot: docker: add curl for composer-security-check [integration/config] - 10https://gerrit.wikimedia.org/r/849712 (https://phabricator.wikimedia.org/T321753) (owner: 10Dwisehaupt) [08:47:16] (03PS1) 10Hashar: jjb: update jobs for composer-php74 (add curl) [integration/config] - 10https://gerrit.wikimedia.org/r/850034 (https://phabricator.wikimedia.org/T321753) [08:47:39] (03CR) 10Hashar: [C: 03+2] jjb: update jobs for composer-php74 (add curl) [integration/config] - 10https://gerrit.wikimedia.org/r/850034 (https://phabricator.wikimedia.org/T321753) (owner: 10Hashar) [08:48:47] (03PS1) 10Jbond: tox-buster: add python39-distutils to image [integration/config] - 10https://gerrit.wikimedia.org/r/850035 (https://phabricator.wikimedia.org/T289222) [08:49:41] (03Merged) 10jenkins-bot: jjb: update jobs for composer-php74 (add curl) [integration/config] - 10https://gerrit.wikimedia.org/r/850034 (https://phabricator.wikimedia.org/T321753) (owner: 10Hashar) [08:55:29] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10mfossati) Just attended without luck: no patches scheduled. Also, I'll try to get shell access to `deployment` and `debug` machines before the next training, see {T32... [09:20:08] jbond: sorry for the distutils mess :( [09:22:03] (03CR) 10Hashar: [C: 03+2] tox-buster: add python39-distutils to image [integration/config] - 10https://gerrit.wikimedia.org/r/850035 (https://phabricator.wikimedia.org/T289222) (owner: 10Jbond) [09:22:32] hashar: no problem it was failry easy to backport [09:23:19] python3.10 looks like it is also building fairly easily so i should be able to add that as well. but will probablky be tomorrow as the tests take about 6 hours to run [09:23:51] (03Merged) 10jenkins-bot: tox-buster: add python39-distutils to image [integration/config] - 10https://gerrit.wikimedia.org/r/850035 (https://phabricator.wikimedia.org/T289222) (owner: 10Jbond) [09:25:51] jbond: ouch [09:26:04] happy to see you are taking the opportunity to ship 3.10 as well [09:26:53] could we have added them to the pyall components instead of component/py39 ? [09:27:30] hashar: yes ill rename py39 to pyall [09:28:17] (03PS1) 10Hashar: jjb: update Python jobs for Python 3.9 distutils [integration/config] - 10https://gerrit.wikimedia.org/r/850044 (https://phabricator.wikimedia.org/T289222) [09:28:40] (03CR) 10Hashar: [C: 04-2] "Images failed to build" [integration/config] - 10https://gerrit.wikimedia.org/r/850044 (https://phabricator.wikimedia.org/T289222) (owner: 10Hashar) [09:30:08] jbond: tox-buster failed to build :-\ https://phabricator.wikimedia.org/P36695 [09:30:20] python3-dev : Depends: python3-distutils (>= 3.7.2-1~) but it is not going to be installed [09:30:21] Depends: dh-python but it is not going to be installed [09:30:21] python3-pip : Depends: python3-distutils but it is not going to be installed (image.py:210) [09:30:21] bah [09:31:53] looking [09:32:18] I tried `docker run --rm -it --user=root --entrypoint=bash docker-registry.wikimedia.org/releng/tox-buster:0.6.0` [09:32:29] then `apt update && apt install -y python39-distutils` and that one works [09:33:00] but there is also `python3-distutils/buster-wikimedia 3.9.2-1~deb10u1 all [upgradable from: 3.7.3-1]` [09:34:16] so I think what happens is python3-distutils ends up installed from component/py39 [09:36:28] hashar: can you try again [09:36:43] doing [09:41:52] jbond: looks like the build is processing nicely [09:42:33] \o/ [09:46:34] uploading those images is soooo slow [09:46:54] (03CR) 10Hashar: [C: 03+2] "It initially failed https://phabricator.wikimedia.org/P36695 but got fixed:" [integration/config] - 10https://gerrit.wikimedia.org/r/850035 (https://phabricator.wikimedia.org/T289222) (owner: 10Jbond) [09:47:05] (03CR) 10Hashar: [C: 03+2] "Images got build :)" [integration/config] - 10https://gerrit.wikimedia.org/r/850044 (https://phabricator.wikimedia.org/T289222) (owner: 10Hashar) [09:48:50] (03Merged) 10jenkins-bot: jjb: update Python jobs for Python 3.9 distutils [integration/config] - 10https://gerrit.wikimedia.org/r/850044 (https://phabricator.wikimedia.org/T289222) (owner: 10Hashar) [09:57:48] jbond: it works. I have added py39 to a tox file ( https://gerrit.wikimedia.org/r/c/integration/quibble/+/850002/1/tox.ini ) and the build worked https://integration.wikimedia.org/ci/job/integration-quibble-tox-docker/965/console [10:02:53] hashar: also confirmed here https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/850038 [10:13:47] \o/ [10:41:37] 10Beta-Cluster-Infrastructure, 10SRE, 10Beta-Cluster-reproducible: Thumbnails on beta cluster return 503 Service Unavailable - https://phabricator.wikimedia.org/T321654 (10TheresNoTime) Just noting the things I've tried (unsuccessfully): - Purging the varnish cache for this file - Deleting some generated t... [10:43:50] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Deploy mediawiki kubernetes services - https://phabricator.wikimedia.org/T321786 (10Clement_Goubert) [10:44:53] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Deploy mediawiki kubernetes services - https://phabricator.wikimedia.org/T321786 (10Clement_Goubert) 05Open→03In progress p:05Triage→03High [10:45:03] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [11:25:33] 10Phabricator: Allow tools to use phabricator webhooks - https://phabricator.wikimedia.org/T321790 (10kostajh) [11:26:02] 10Phabricator, 10Release-Engineering-Team: Allow tools to use phabricator webhooks - https://phabricator.wikimedia.org/T321790 (10kostajh) [12:29:01] Project beta-update-databases-eqiad build #62518: 04FAILURE in 8 min 59 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/62518/ [12:38:05] Amir1, ^ meh, looks related to your reorg patch [12:38:14] oh no [12:38:19] I will fix it [12:38:44] oh fun stuff [12:38:46] centralauth causing issues? /me acts surprised [12:40:11] I'm making patches now [12:46:08] taavi: zabe https://gerrit.wikimedia.org/r/849578 ? [12:47:26] I guess you need to bump the required mw version [12:48:16] did you test it? last time I touched a namespace in the session provider it broke the ability to be logged in [12:49:17] this really shouldn't affect anything [12:57:23] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10ArielGlenn) I verify that mfossati did complete an earlier training, see T302204#7735291 [13:05:04] (03PS1) 10Jforrester: [WIP] Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 [13:06:57] (03CR) 10CI reject: [V: 04-1] [WIP] Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 (owner: 10Jforrester) [13:07:21] (03PS2) 10Jforrester: [WIP] Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 [13:12:15] (03PS3) 10Jforrester: Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 [13:12:59] Amir1: (sorry for the ping in a random channel) please hold off merging anything related to the WebRequest rename [13:13:14] https://phabricator.wikimedia.org/T166010#8349431 [13:13:34] Just realized that taint-check does not support aliases for hardcoded methods [13:16:03] (03PS4) 10Jforrester: Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 [13:18:03] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 (owner: 10Jforrester) [13:29:12] Yippee, build fixed! [13:29:12] Project beta-update-databases-eqiad build #62519: 09FIXED in 9 min 11 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/62519/ [13:32:24] (03PS3) 10Subramanya Sastry: Tweak es/hu CSS based on site messages / Common.css [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/849662 [13:54:37] you all had so much fun while I had lunch [14:24:24] "fun" :-P [14:47:58] kostajh: do you have a system to inject LocalSettings.php when running Selenium tests? [14:48:13] I thought something like that got added at some point but I can't find the history so maybe I made it up [14:48:50] hashar: yes it is called in the wdio.conf.js GrowthExperiments [14:50:23] kostajh: I am asking cause Ezekiel from QTE apparently has a similar use case ;) [14:50:34] I will link them to your code and asked for a phabricator task [14:51:43] maybe those bits can be upstreamed to wdio-mediawiki [14:51:50] that might be useful to other repos [14:53:43] kostajh: thank you! [16:00:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10serviceops, 10Patch-For-Review, 10Test-Coverage: Upgrade our php-xdebug package for php7.2 - https://phabricator.wikimedia.org/T234418 (10Tgr) [16:15:49] (03PS1) 10Umherirrender: Zuul: [mediawiki/extensions/WikiVideos] Add [integration/config] - 10https://gerrit.wikimedia.org/r/850198 (https://phabricator.wikimedia.org/T321769) [16:39:34] (03PS1) 10Ahmon Dancy: make-container-image Makefile: Fix build-and-push-debug-image target [tools/release] - 10https://gerrit.wikimedia.org/r/850205 [16:40:35] (03CR) 10Ahmon Dancy: [C: 03+2] make-container-image Makefile: Fix build-and-push-debug-image target [tools/release] - 10https://gerrit.wikimedia.org/r/850205 (owner: 10Ahmon Dancy) [16:41:21] (03Merged) 10jenkins-bot: make-container-image Makefile: Fix build-and-push-debug-image target [tools/release] - 10https://gerrit.wikimedia.org/r/850205 (owner: 10Ahmon Dancy) [17:24:15] 10Release-Engineering-Team (GitLab II: Wrath of Kahn 👾), 10Scap: Update scap documentation and other references for new GitLab location - https://phabricator.wikimedia.org/T321847 (10dancy) [17:35:33] (03PS4) 10Subramanya Sastry: Tweak es/fr/hu CSS based on site messages / Common.css [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/849662 [18:04:35] 10Release-Engineering-Team (GitLab II: Wrath of Kahn 👾), 10Scap: Update scap documentation and other references for new GitLab location - https://phabricator.wikimedia.org/T321847 (10dancy) p:05Triage→03Medium [18:05:03] 10Release-Engineering-Team (GitLab II: Wrath of Kahn 👾), 10Scap: Update scap documentation and other references for new GitLab location - https://phabricator.wikimedia.org/T321847 (10dancy) I ran this on deploy1002 in /srv/deployment/scap: ` git remote rm origin git remote add origin https://gitlab.wikimedia.o... [19:04:17] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for mfossati - https://phabricator.wikimedia.org/T313812 (10thcipriani) 05Open→03Resolved a:03ArielGlenn ## {icon check-circle-o} Training completed! Hey @mfossati thanks for attending deployment training Here's a list... [20:21:31] PROBLEM - Check systemd state on doc1002 is CRITICAL: CRITICAL - degraded: The following units failed: rsync-doc-doc2001.codfw.wmnet.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [20:37:23] (03PS1) 10Stef Dunlap: Add Stef Dunlap (kindrobot) as UTC late backport deployer [tools/release] - 10https://gerrit.wikimedia.org/r/850271 [20:38:30] Hey, that's me! [20:46:24] (03CR) 10Thcipriani: [C: 03+2] "🎉" [tools/release] - 10https://gerrit.wikimedia.org/r/850271 (owner: 10Stef Dunlap) [20:47:08] (03Merged) 10jenkins-bot: Add Stef Dunlap (kindrobot) as UTC late backport deployer [tools/release] - 10https://gerrit.wikimedia.org/r/850271 (owner: 10Stef Dunlap) [20:55:49] (03CR) 10Hashar: [C: 03+2] Zuul: [mediawiki/extensions/WikiVideos] Add [integration/config] - 10https://gerrit.wikimedia.org/r/850198 (https://phabricator.wikimedia.org/T321769) (owner: 10Umherirrender) [20:57:42] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/WikiVideos] Add [integration/config] - 10https://gerrit.wikimedia.org/r/850198 (https://phabricator.wikimedia.org/T321769) (owner: 10Umherirrender) [20:58:24] !log Reloaded Zuul for https://gerrit.wikimedia.org/r/850198 # T321769 [20:58:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:58:26] T321769: Please add mediawiki-i18n-check-docker to repository mediawiki/extensions/WikiVideos - https://phabricator.wikimedia.org/T321769 [21:12:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Epic: Have dependencies of gated extensions in the gate - https://phabricator.wikimedia.org/T204252 (10Dreamy_Jazz) [21:16:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Epic: Have dependencies of gated extensions in the gate - https://phabricator.wikimedia.org/T204252 (10Dreamy_Jazz) [21:16:57] RECOVERY - Check systemd state on doc1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [21:22:42] (03PS1) 10Dreamy Jazz: Add CentralAuth to the gated extensions list [integration/config] - 10https://gerrit.wikimedia.org/r/850278 (https://phabricator.wikimedia.org/T321864) [21:24:38] (03PS2) 10Dreamy Jazz: Add CentralAuth to the gated extensions list [integration/config] - 10https://gerrit.wikimedia.org/r/850278 (https://phabricator.wikimedia.org/T321864) [21:29:08] 10Phabricator, 10Release-Engineering-Team: Add EChetty to acl_phabricator group on phab - https://phabricator.wikimedia.org/T321197 (10thcipriani) 05Open→03Resolved a:03thcipriani Sorry for the delay, done now. [21:29:20] 10Phabricator, 10Release-Engineering-Team: Add JArguello to acl_phabricator group on phab - https://phabricator.wikimedia.org/T321198 (10thcipriani) 05Open→03Resolved a:03thcipriani Sorry for the delay! Done now. [21:30:16] (03CR) 10Jforrester: [C: 04-2] "This will eradicate all testing of core's user auth system, and so can't possibly happen, sorry." [integration/config] - 10https://gerrit.wikimedia.org/r/850278 (https://phabricator.wikimedia.org/T321864) (owner: 10Dreamy Jazz) [21:35:29] (03PS5) 10Jforrester: Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 (https://phabricator.wikimedia.org/T203694) [21:35:34] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 (https://phabricator.wikimedia.org/T203694) (owner: 10Jforrester) [21:36:26] Krinkle: Castor only works for composer and npm fetches, not random curls, right? [21:37:22] (03Abandoned) 10Dreamy Jazz: Add CentralAuth to the gated extensions list [integration/config] - 10https://gerrit.wikimedia.org/r/850278 (https://phabricator.wikimedia.org/T321864) (owner: 10Dreamy Jazz) [21:37:24] (03Merged) 10jenkins-bot: Zuul: [mediawiki/core] Run standalone jobs [integration/config] - 10https://gerrit.wikimedia.org/r/850151 (https://phabricator.wikimedia.org/T203694) (owner: 10Jforrester) [21:38:31] James_F: correct, but see ForeignResourceManager ctor. [21:38:48] !log Zuul: [mediawiki/core] Run standalone jobs T203694 [21:38:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:38:51] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10Epic: Have dependencies of gated extensions in the gate - https://phabricator.wikimedia.org/T204252 (10Dreamy_Jazz) [21:38:51] T203694: Run ForeignResourceManager verification on MediaWiki core commits - https://phabricator.wikimedia.org/T203694 [21:38:57] more specifically, the Docker image for composer and npm jobs contains ENV vars that point hteir caches to the castor/XDG dir. [21:39:22] 10Phabricator, 10Release-Engineering-Team, 10Data Pipelines: Custom task form permissions for Data Platform Team PMs - https://phabricator.wikimedia.org/T320648 (10thcipriani) 05Open→03Resolved a:03thcipriani 👋 added you all to #acl*phabricator so you should have access to do the needful now. Let me kn... [21:39:35] ForeignResourceManager supports it out of the box. [21:39:44] Hmm. [21:39:52] Indeed. [22:14:05] (03PS5) 10Subramanya Sastry: Tweak es/fr/hu CSS based on site messages / Common.css [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/849662 [22:14:07] (03PS1) 10Subramanya Sastry: Remove default styles that are now shipped by Cite [integration/visualdiff] - 10https://gerrit.wikimedia.org/r/850284 [22:23:00] 10Continuous-Integration-Config: Please add mediawiki-i18n-check-docker to repository mediawiki/extensions/WikiVideos - https://phabricator.wikimedia.org/T321769 (10Umherirrender) 05Open→03Resolved a:03Umherirrender [22:24:48] 10GitLab (Project Migration), 10Release-Engineering-Team: Create new GitLab project group: 10nm - https://phabricator.wikimedia.org/T321022 (10thcipriani) 05Open→03Resolved a:03thcipriani 🦊 https://gitlab.wikimedia.org/repos/10nm I made these folks owners of this group: * @Berrely * @Chlod * @Remagoxer...