[00:57:20] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Growth-Team (Current Sprint), 10Patch-For-Review: Reopen beta eswiki - https://phabricator.wikimedia.org/T306833 (10Etonkovidova) >>! In T306833#7882689, @Tgr wrote: > This is done. @Etonkovidova let me know if you'd like anything to be configured... [02:00:50] 10Continuous-Integration-Config, 10Codex, 10Design-Systems-Team, 10Test-Coverage, 10User-DannyS712: Public codex code coverage report - https://phabricator.wikimedia.org/T303899 (10DannyS712) >>! In T303899#7816452, @DannyS712 wrote: > Hmm, doesn't appear to be visible at https://doc.wikimedia.org/cover/... [02:06:09] 10Beta-Cluster-Infrastructure, 10Wikimedia-production-error: Unable to edit any page on beta cluster - https://phabricator.wikimedia.org/T306946 (10Stang) [02:06:39] 10Beta-Cluster-Infrastructure, 10Wikimedia-production-error: Unable to edit any page on beta cluster - https://phabricator.wikimedia.org/T306946 (10Stang) [02:07:04] 10Beta-Cluster-Infrastructure, 10Wikimedia-production-error: Unable to edit any page on beta cluster - https://phabricator.wikimedia.org/T306946 (10DannyS712) [02:28:03] (03CR) 10Stang: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/786410 (owner: 10Stang) [02:28:22] (03PS2) 10Stang: Add WhitePhosphorus to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/786410 [02:56:37] 10Beta-Cluster-Infrastructure: Request for admin permission on wikidata beta cluster - https://phabricator.wikimedia.org/T306952 (10Stang) [02:57:31] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on wikidata beta cluster - https://phabricator.wikimedia.org/T306952 (10DannyS712) 05Open→03Resolved a:03DannyS712 Done [02:59:10] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on wikidata beta cluster - https://phabricator.wikimedia.org/T306952 (10Stang) @DannyS712 Hmm, is wikidata, not meta... [03:02:25] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on wikidata beta cluster - https://phabricator.wikimedia.org/T306952 (10DannyS712) >>! In T306952#7883227, @Stang wrote: > @DannyS712 Hmm, is wikidata, not meta... I have no idea how I ended up on meta, but fixed - removed from meta... [03:28:25] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on {wikidata, meta} beta cluster - https://phabricator.wikimedia.org/T306952 (10Stang) 05Resolved→03Open [03:28:37] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on {wikidata, meta} beta cluster - https://phabricator.wikimedia.org/T306952 (10Stang) @DannyS712 ^ [05:35:49] 10Continuous-Integration-Infrastructure: Add Python 3.9 to Wikimedia CI - https://phabricator.wikimedia.org/T289222 (10Xqt) > Or drop support for python 3.4, 3.5 and 3.6 entirely and get the new minimum supported version to be 3.7 Why not? All these versions have reached their EOL. There is no syntax change ex... [07:17:52] 10Continuous-Integration-Infrastructure, 10docker-pkg: docker-pkg 3.0.1 fails with: 404 Client Error: Not Found ("pull access denied for wikimedia-bullseye, - https://phabricator.wikimedia.org/T297619 (10hashar) 05Open→03Declined The docker-pkg config file requires a `seed_image` variable to be defined aka... [07:19:09] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on {wikidata, meta} beta cluster - https://phabricator.wikimedia.org/T306952 (10Zabe) 05Open→03Resolved +sysop on meta [07:31:06] 10Beta-Cluster-Infrastructure, 10User-DannyS712: Request for admin permission on {wikidata, meta} beta cluster - https://phabricator.wikimedia.org/T306952 (10Stang) Thanks! [07:33:01] (03CR) 10Hashar: [C: 03+2] Add WhitePhosphorus to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/786410 (owner: 10Stang) [07:35:04] (03Merged) 10jenkins-bot: Add WhitePhosphorus to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/786410 (owner: 10Stang) [07:39:04] (03CR) 10Hashar: [C: 03+2] "I have updated the https://integration.wikimedia.org/ci/job/selenium-daily-beta-Vector/ job." [integration/config] - 10https://gerrit.wikimedia.org/r/777884 (https://phabricator.wikimedia.org/T301184) (owner: 10Jdlrobson) [07:40:13] 10Continuous-Integration-Infrastructure, 10MinervaNeue, 10Vector, 10Accessibility, and 3 others: Add automated accessibility tests in CI to generate accessibility benchmarks for Skins - https://phabricator.wikimedia.org/T301184 (10hashar) The [[ https://integration.wikimedia.org/ci/job/selenium-daily-beta-... [07:40:26] (03CR) 10Elukey: "Thanks a lot for the merge! I am reading https://wikitech.wikimedia.org/wiki/Scap/Release, would it be ok to cut a new minor release with " [tools/scap] - 10https://gerrit.wikimedia.org/r/785154 (https://phabricator.wikimedia.org/T303801) (owner: 10Elukey) [07:40:52] (03Merged) 10jenkins-bot: Add selenium daily job to Vector [integration/config] - 10https://gerrit.wikimedia.org/r/777884 (https://phabricator.wikimedia.org/T301184) (owner: 10Jdlrobson) [07:41:46] (03CR) 10Hashar: [C: 03+2] Switch to `-composer` tests for Tuleap* repos [integration/config] - 10https://gerrit.wikimedia.org/r/775786 (owner: 10Robert Vogel) [07:43:37] (03Merged) 10jenkins-bot: Switch to `-composer` tests for Tuleap* repos [integration/config] - 10https://gerrit.wikimedia.org/r/775786 (owner: 10Robert Vogel) [07:44:53] Jenkins code updates on beta appear to be blocked. I wonder if someone like hashar can fix them? [07:45:30] dwalden: poor jobs :) [07:45:39] I will kick the jobs from https://integration.wikimedia.org/ci/view/Beta/ [07:45:55] Thanks [07:46:25] one job started [07:46:27] i will monitor [07:55:14] dwalden: looks like it is working [07:58:27] That is good to hear, thanks. [07:58:36] yeah that fixed it :-] [08:01:05] beta is up to date now, thanks for sorting that out [09:22:27] (03CR) 10Hashar: [C: 04-1] "There are a few things pending for Quibble, notably moving jobs to use Apache instead of the PHP built-in server and have a memcached inst" [integration/config] - 10https://gerrit.wikimedia.org/r/777030 (https://phabricator.wikimedia.org/T303867) (owner: 10Jforrester) [09:43:19] (03CR) 10Hashar: [C: 03+1] jjb: Update non-Quibble jobs to composer 2.3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/777029 (https://phabricator.wikimedia.org/T303867) (owner: 10Jforrester) [09:51:04] 10Phabricator: I'm unable to edit Phabricator project descriptions - https://phabricator.wikimedia.org/T306968 (10awight) [10:12:28] 10Phabricator: I'm unable to edit Phabricator project descriptions - https://phabricator.wikimedia.org/T306968 (10Peachey88) I think Form 85 [Sprint Projects] needs to have Trusted-Contribs added it to (https://phabricator.wikimedia.org/transactions/editengine/projects.project/view/85/#1913) similar to how form... [10:29:17] 10Phabricator: I'm unable to edit Phabricator project descriptions - https://phabricator.wikimedia.org/T306968 (10Aklapper) Docs: https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects#Project_policies [10:39:53] 10Phabricator: Unable to edit Phabricator Sprint project descriptions (form 85) - https://phabricator.wikimedia.org/T306968 (10Aklapper) [10:40:04] 10Phabricator: Unable to edit Phabricator Sprint project descriptions (form 85) - https://phabricator.wikimedia.org/T306968 (10Aklapper) 05Open→03Resolved a:03Aklapper Thanks for finding this. Fixed in https://phabricator.wikimedia.org/transactions/editengine/projects.project/view/85/#2298 [10:43:04] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Growth-Team (Current Sprint), 10Patch-For-Review: Reopen beta eswiki - https://phabricator.wikimedia.org/T306833 (10Tgr) a:03Tgr [10:43:14] 10Release-Engineering-Team, 10Scap, 10serviceops, 10User-brennen: Deploy Scap version 4.7.0 - https://phabricator.wikimedia.org/T306827 (10JMeybohm) 05Open→03Resolved >>! In T306827#7881534, @dancy wrote: >>>! In T306827#7879622, @JMeybohm wrote: >> Rolled out to canaries + deploy1002, still super slow... [11:07:47] 10Project-Admins: Archive Kaltura Player Phab project and decline all its open tasks - https://phabricator.wikimedia.org/T306971 (10Aklapper) 05Open→03Stalled p:05Triage→03Low [11:08:05] 10Project-Admins: Archive Kaltura Player Phab project and decline all its open tasks - https://phabricator.wikimedia.org/T306971 (10Aklapper) [11:18:30] 10Phabricator: Unable to edit Phabricator Sprint project descriptions (form 85) - https://phabricator.wikimedia.org/T306968 (10awight) Confirmed fixed, thank you @Peachey88 and @Aklapper! [11:25:13] 10Project-Admins: Create project tag for WMDE technical needs from WMF - https://phabricator.wikimedia.org/T305148 (10Aklapper) [11:27:18] 10Project-Admins: Create project tag for WMDE technical needs from WMF - https://phabricator.wikimedia.org/T305148 (10Aklapper) 05Stalled→03Resolved a:03Aklapper @AOdit_WMF: Requested public project #WMDE-technical-needs-from-WMF has been created. (In case you need to edit the project or project workboard... [11:30:29] 10Project-Admins: Create Project Tag for LOD Network_WMDE (Linked Open Data Network) - https://phabricator.wikimedia.org/T305146 (10Aklapper) 05Stalled→03Resolved a:03Aklapper Requested public project has been created: #linked-open-data-network-program (In case you need to edit the project or project wor... [11:32:54] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Growth-Team (Current Sprint), 10Patch-For-Review: Reopen beta eswiki - https://phabricator.wikimedia.org/T306833 (10Tgr) >>! In T306833#7883033, @Etonkovidova wrote: > Should `eswiki betalabs` have a fully functioned Special:Homepage? GLAM campai... [11:47:08] (03PS1) 10AikoChou: inference-services: articlequality cleanup [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) [13:32:34] (03CR) 10Elukey: [C: 03+1] "Left a nit for the commit msg but LGTM" [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [13:50:26] hashar: Can you deploy https://gerrit.wikimedia.org/r/c/integration/config/+/777029 ? I can't deploy jjb things any more (python 2.7). :-( [13:59:28] 10Beta-Cluster-Infrastructure, 10SRE, 10Traffic, 10Beta-Cluster-reproducible: Beta cluster down: Error: 502, Next Hop Connection Failed (Feb 2022) - https://phabricator.wikimedia.org/T302699 (10dom_walden) Happening again: ` dwalden@deployment-mediawiki12:~$ sudo tail /var/log/apache2.log Apr 27 13:58:00 d... [14:01:56] (03CR) 10Kevin Bazira: [C: 03+1] "LGMT!" [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [14:24:53] 10Release-Engineering-Team (Doing), 10Scap: Puppet config for scap-over-scap - https://phabricator.wikimedia.org/T306991 (10jnuche) [14:27:25] 10Release-Engineering-Team (Doing), 10Scap: Puppet config for scap-over-scap - https://phabricator.wikimedia.org/T306991 (10jnuche) [14:34:19] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10User-zeljkofilipin: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS - https://phabricator.wikimedia.org/T267890 (10Esanders) It there a task/timelines for dropping Node 12? It is EOL and security updates stop on 3... [14:39:31] 10Project-Admins: Archive Kaltura Player Phab project and decline all its open tasks - https://phabricator.wikimedia.org/T306971 (10TheDJ) A note that some kaltura tickets are also valid videojs or other timedmediahandler tickets. I try to keep an eye on this , but there is some crossover at times. [14:50:18] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10User-zeljkofilipin: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS - https://phabricator.wikimedia.org/T267890 (10Jdforrester-WMF) >>! In T267890#7884480, @Esanders wrote: > It there a task/timelines for dropping... [14:50:33] 10Release-Engineering-Team, 10Scap, 10serviceops, 10User-brennen: Deploy Scap version 4.7.0 - https://phabricator.wikimedia.org/T306827 (10dancy) Thanks! [14:50:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10User-zeljkofilipin: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS - https://phabricator.wikimedia.org/T267890 (10Jdforrester-WMF) [14:50:44] 10Continuous-Integration-Config, 10Release-Engineering-Team (Seen), 10User-zeljkofilipin: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS - https://phabricator.wikimedia.org/T267890 (10Jdforrester-WMF) [14:53:34] (03PS1) 10Ahmon Dancy: Release 4.7.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/786979 [14:53:36] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.7.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/786979 (owner: 10Ahmon Dancy) [14:55:16] (03Merged) 10jenkins-bot: Release 4.7.1-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/786979 (owner: 10Ahmon Dancy) [14:57:29] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.7.1 - https://phabricator.wikimedia.org/T306998 (10dancy) [14:58:14] (03CR) 10Ahmon Dancy: Remove the --global option from git lfs calls (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/785154 (https://phabricator.wikimedia.org/T303801) (owner: 10Elukey) [14:59:00] (03CR) 10Elukey: Remove the --global option from git lfs calls (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/785154 (https://phabricator.wikimedia.org/T303801) (owner: 10Elukey) [15:03:27] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.7.1 - https://phabricator.wikimedia.org/T306998 (10elukey) This is due to a new feature for `git-lfs` on Buster, so probably only relevant to the ORES nodes (we are going to upgrade them to Buster). I can take care of the new package, an... [15:11:43] 10Release-Engineering-Team (Next), 10Gerrit (Gerrit 3.4), 10Patch-For-Review: Upgrade to Gerrit 3.4 - https://phabricator.wikimedia.org/T292759 (10hashar) a:03hashar [15:18:56] (03PS1) 10Hashar: tox: default to python3 [integration/config] - 10https://gerrit.wikimedia.org/r/786990 [15:19:06] James_F: ^ that should switch jjb to use python3 :) [15:19:27] hashar: Does it work? ;-) [15:19:40] well yeah I think :] [15:19:52] * James_F pulls to find out. [15:20:32] !log Updating non-quibble jobs to composer 2.3.3 | T303867 | https://gerrit.wikimedia.org/r/c/integration/config/+/777029 [15:20:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:20:38] T303867: Update bundled composer to 2.2.x+ - https://phabricator.wikimedia.org/T303867 [15:20:53] James_F: I am updating the jobs meanwhile [15:20:58] <3 [15:21:02] and I think composer had another CVE after 2.3.3 [15:21:22] note that it is a 2.1 > 2.3 upgrade so there might be some effect :] I will drop a mail to wikitech-l [15:21:34] 10Release-Engineering-Team (Doing), 10Scap: Puppet config for scap-over-scap - https://phabricator.wikimedia.org/T306991 (10dancy) [15:21:43] Yeah. :-( [15:22:32] Yup, that works (did a no-op push on the brand new selenium-daily-beta-Vector job). [15:22:45] (03CR) 10Jforrester: [C: 03+2] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/786990 (owner: 10Hashar) [15:24:16] magic [15:24:38] (03Merged) 10jenkins-bot: tox: default to python3 [integration/config] - 10https://gerrit.wikimedia.org/r/786990 (owner: 10Hashar) [15:26:16] 10Continuous-Integration-Config, 10Composer, 10Patch-For-Review: Update bundled composer to 2.2.x+ - https://phabricator.wikimedia.org/T303867 (10hashar) I have [[ https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/QEJ7MMN3WKOOSJ7D2RCSESPFUKVEF3O7/ | posted to wikitech-l ]]: >... [15:26:18] requests.exceptions.HTTPError: 503 Server Error: Service Unavailable for url: https://integration.wikimedia.org/ci/job/parsoidsvc-composer-package-php74-docker/config.xml [15:26:20] bah [15:27:06] (03CR) 10Hashar: [C: 03+2] "Jobs are being updated." [integration/config] - 10https://gerrit.wikimedia.org/r/777029 (https://phabricator.wikimedia.org/T303867) (owner: 10Jforrester) [15:29:09] (03Merged) 10jenkins-bot: jjb: Update non-Quibble jobs to composer 2.3.3 [integration/config] - 10https://gerrit.wikimedia.org/r/777029 (https://phabricator.wikimedia.org/T303867) (owner: 10Jforrester) [15:30:03] ^ deployed [15:30:55] hashar: If quibble is undeployable right now can we revert it so we can bump composer for them too? The quibble jobs (for vendor.git) were the main reason for moving it… [15:31:08] 10Continuous-Integration-Config, 10Composer, 10Patch-For-Review: Update bundled composer to 2.2.x+ - https://phabricator.wikimedia.org/T303867 (10Jdforrester-WMF) [15:43:03] James_F: it is half deployed and relies on changes in the source repo iirc [15:43:36] but I definitely have the goal to flip it this week :] [15:48:13] (03PS1) 10Ahmon Dancy: README.md updates [releng/phatality] - 10https://gerrit.wikimedia.org/r/786996 [15:49:02] (03CR) 10jerkins-bot: [V: 04-1] README.md updates [releng/phatality] - 10https://gerrit.wikimedia.org/r/786996 (owner: 10Ahmon Dancy) [15:49:37] (03CR) 10Hashar: [C: 03+2] [WMF] register our plugins as submodules [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/756109 (owner: 10Hashar) [15:50:05] (03CR) 10Ahmon Dancy: [C: 03+2] test: Introduce unit tests, covering task formatting helper [releng/phatality] - 10https://gerrit.wikimedia.org/r/772916 (https://phabricator.wikimedia.org/T304736) (owner: 10Krinkle) [15:56:14] (03Abandoned) 10Ahmon Dancy: README.md updates [releng/phatality] - 10https://gerrit.wikimedia.org/r/786996 (owner: 10Ahmon Dancy) [15:56:19] 10Release-Engineering-Team (Doing), 10Scap: Puppet config for scap-over-scap - https://phabricator.wikimedia.org/T306991 (10jnuche) [15:56:45] (03CR) 10Ahmon Dancy: [C: 03+2] helpers: Simplify makeTitle by delegating formatting to MediaWiki core [releng/phatality] - 10https://gerrit.wikimedia.org/r/772918 (owner: 10Krinkle) [15:57:00] (03Merged) 10jenkins-bot: [WMF] register our plugins as submodules [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/756109 (owner: 10Hashar) [15:57:40] (03PS1) 10Hnowlan: Review access change [generated-data-platform/datasets/image-suggestions] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/786417 [15:58:38] (03CR) 10Ahmon Dancy: [C: 03+2] Remove phatalityId concept, search for task title instead [releng/phatality] - 10https://gerrit.wikimedia.org/r/773515 (owner: 10Krinkle) [16:00:54] (03PS1) 10Hashar: Gerrit v3.4.3 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) [16:01:13] (03CR) 10jerkins-bot: [V: 04-1] Gerrit v3.4.3 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [16:02:17] (03PS1) 10Ahmon Dancy: Add Dockerfile and script to build phatality zip [releng/phatality] - 10https://gerrit.wikimedia.org/r/787002 [16:05:13] 10Release-Engineering-Team (Priority Backlog 📥), 10Scap, 10Infrastructure-Foundations, 10serviceops, 10Patch-For-Review: Use scap to deploy itself to scap targets - https://phabricator.wikimedia.org/T303559 (10jnuche) [16:06:48] (03CR) 10Hashar: "recheck after ./deploy_artifacts.py --version=3.4.3 gerrit.war plugins/*" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [16:10:40] (03CR) 10Hashar: "recheck" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [16:24:49] (03PS2) 10AikoChou: inference-services: articlequality cleanup [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) [16:26:15] (03CR) 10AikoChou: inference-services: articlequality cleanup (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/786934 (https://phabricator.wikimedia.org/T301766) (owner: 10AikoChou) [16:31:15] 10Release-Engineering-Team (Radar), 10Scap, 10serviceops: Deploy Scap version 4.7.1 - https://phabricator.wikimedia.org/T306998 (10thcipriani) [16:32:51] 10Release-Engineering-Team (Doing), 10MediaWiki-Docker, 10Performance-Team (Radar): Composer version in mediawiki-docker unsupported by mediawiki-vendor - https://phabricator.wikimedia.org/T306802 (10thcipriani) a:03brennen @brennen says that JamesF has a patch over on GitLab [16:34:16] 10Release-Engineering-Team (Priority Backlog 📥): Provide way to enhance Logstash logs with information from the Train Log Triage process - https://phabricator.wikimedia.org/T302041 (10thcipriani) [16:37:11] 10Continuous-Integration-Infrastructure, 10Cloud-VPS, 10Wikidata, 10wdwb-tech, 10cloud-services-team (Kanban): Wikibase selenium tests timeout, seemingly due to "memory compaction" events on CI VMs - https://phabricator.wikimedia.org/T281122 (10thcipriani) >>! In T281122#7870525, @hashar wrote: > I am no... [16:40:19] (03PS2) 10Ahmon Dancy: Add Dockerfile and script to build phatality zip [releng/phatality] - 10https://gerrit.wikimedia.org/r/787002 [16:54:42] (03PS1) 10Majavah: calendar: Temporarily remove myself [tools/release] - 10https://gerrit.wikimedia.org/r/787014 [16:55:29] (03CR) 10Majavah: [C: 03+2] calendar: Temporarily remove myself [tools/release] - 10https://gerrit.wikimedia.org/r/787014 (owner: 10Majavah) [16:56:44] (03Merged) 10jenkins-bot: calendar: Temporarily remove myself [tools/release] - 10https://gerrit.wikimedia.org/r/787014 (owner: 10Majavah) [16:57:17] (03CR) 10Eevans: [V: 03+2 C: 03+2] Review access change [generated-data-platform/datasets/image-suggestions] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/786417 (owner: 10Hnowlan) [17:29:25] Project beta-code-update-eqiad build #389766: 15ABORTED in 58 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/389766/ [17:30:18] Project beta-code-update-eqiad build #389767: 15ABORTED in 44 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/389767/ [18:15:34] 10Continuous-Integration-Config, 10Composer, 10Patch-For-Review: Update bundled composer to 2.2.x+ - https://phabricator.wikimedia.org/T303867 (10Krinkle) See also: {T306802} [18:15:42] (03CR) 10Ahmon Dancy: "I haven't tested yet. Just some comments based on static analysis." [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [18:25:54] (03CR) 10Ahmon Dancy: backport: compare fetched commits with backports (033 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [18:38:42] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Dzahn) cc: @Jelto @Papaul has asked which partman recipe to use. SInce these are the first physical servers that might not... [18:45:30] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Dzahn) The current install_server config is: ` gitlab*) echo partman/flat.cfg virtual.cfg ;; \ ` so we can't ke... [18:45:59] (03CR) 10Thcipriani: [C: 03+2] "nice use of CACHEBUSTER" [releng/phatality] - 10https://gerrit.wikimedia.org/r/787002 (owner: 10Ahmon Dancy) [18:54:19] (03PS20) 10Jeena Huneidi: backport: compare fetched commits with backports [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) [18:54:29] (03CR) 10Jeena Huneidi: backport: compare fetched commits with backports (039 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [18:56:12] 10Beta-Cluster-Infrastructure, 10Wikimedia-Site-requests, 10Growth-Team (Current Sprint), 10Patch-For-Review: Reopen beta eswiki - https://phabricator.wikimedia.org/T306833 (10Etonkovidova) 05Open→03Resolved >>! In T306833#7883935, @Tgr wrote: >>>! In T306833#7883033, @Etonkovidova wrote: >> Should `es... [18:57:43] (03PS21) 10Jeena Huneidi: backport: compare fetched commits with backports [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) [19:03:49] (03PS22) 10Jeena Huneidi: backport: compare fetched commits with backports [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) [19:04:06] (03CR) 10Jeena Huneidi: backport: compare fetched commits with backports (035 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [19:06:30] !log Updating operations/software/gerrit branches and tags from upstream # T292759 [19:06:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:06:33] T292759: Upgrade to Gerrit 3.4 - https://phabricator.wikimedia.org/T292759 [19:09:01] (03CR) 10Hashar: [C: 04-1] "Since I prepared the Gerrit 3.4 upgrade a new release has been cut: 3.4.4 which has a security fix. So we should aim at that one instead o" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [19:14:01] 10Beta-Cluster-Infrastructure, 10Inuka-Team, 10Wikimedia-Extension-setup, 10Wikimedia-extension-review-queue, 10Wikistories (Beta): Deploy Wikistories extension to beta cluster - https://phabricator.wikimedia.org/T303004 (10SBisson) 05Open→03Resolved a:03SBisson [19:22:33] (03PS1) 10Hashar: Merge tag 'v3.4.4' into wmf/stable-3.4 [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787055 (https://phabricator.wikimedia.org/T292759) [19:26:04] (03CR) 10Ahmon Dancy: backport: compare fetched commits with backports (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) (owner: 10Jeena Huneidi) [19:29:50] (03PS2) 10Hashar: Merge tag 'v3.4.4' into wmf/stable-3.4 [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787055 (https://phabricator.wikimedia.org/T292759) [19:32:27] (03PS23) 10Jeena Huneidi: backport: compare fetched commits with backports [tools/scap] - 10https://gerrit.wikimedia.org/r/773897 (https://phabricator.wikimedia.org/T295495) [19:38:36] hashar: Do you have some time to help me with failing tests for https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/787058 ? [19:38:50] I did https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/787054 earlier and it didn't have test problems. [19:39:43] Good run is https://integration.wikimedia.org/ci/job/helm-lint/7251/console and bad run is https://integration.wikimedia.org/ci/job/helm-lint/7255/console [19:41:30] (03CR) 10Hashar: [C: 03+2] Merge tag 'v3.4.4' into wmf/stable-3.4 [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787055 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [19:42:49] I'm testing a straight revert commit now. Taking a break now too [19:45:18] (03PS2) 10Hashar: Gerrit v3.4.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) [19:45:36] dancy: yeah over meet if you don't mind ;) [19:46:07] (03CR) 10jerkins-bot: [V: 04-1] Gerrit v3.4.4 and rebuild plugins [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [19:48:36] (03Merged) 10jenkins-bot: Merge tag 'v3.4.4' into wmf/stable-3.4 [software/gerrit] (wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787055 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [19:56:54] dancy: I don't know what is going on. Looks like helm lint fails just before the " REPORTING ON DEPLOYMENTS" stage [19:56:57] :-\ [19:57:20] and I know absolutely nothing about that [19:59:06] (03CR) 10Hashar: "recheck" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [20:01:41] (03CR) 10Hashar: "Updated from 3.4.3 to 3.4.4 :)" [software/gerrit] (deploy/wmf/stable-3.4) - 10https://gerrit.wikimedia.org/r/787000 (https://phabricator.wikimedia.org/T292759) (owner: 10Hashar) [20:02:28] I am off :) [20:25:30] 10Release-Engineering-Team (Next), 10Gerrit (Gerrit 3.4), 10Patch-For-Review: Upgrade to Gerrit 3.4 - https://phabricator.wikimedia.org/T292759 (10hashar) I have scheduled the upgrade tomorrow at 8:00 UTC (10:00 CEST) https://wikitech.wikimedia.org/wiki/Deployments#Thursday,_April_28 I have send the announc... [20:44:00] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, and 2 others: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Dzahn) You should be unblocked to install OS. partman recipe set to raid1-dev. [20:45:11] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Dzahn) @Papaul You should be unblocked to install OS. partman recipe set to raid1-dev. [20:48:44] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Papaul) @Dzahn thanks [21:05:50] 10GitLab (Project Migration), 10cloud-services-team (Kanban): GitLab: separate trees for WMCS infrastructure and tenants - https://phabricator.wikimedia.org/T305301 (10brennen) @bd808 and I discussed naming for toolforge tools on GitLab not that long ago, in the context of {T191182} From my notes there: * Na... [21:07:07] 10GitLab (Project Migration), 10Release-Engineering-Team (🌱 Spring Cleaning — April 2022), 10User-brennen, 10cloud-services-team (Kanban): GitLab: separate trees for WMCS infrastructure and tenants - https://phabricator.wikimedia.org/T305301 (10brennen) 05Open→03In progress a:03brennen [21:40:28] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Dzahn) confirming that the "gitlab" hosts should use a public IP and the "gitlab-runner" hosts should use a private IP. [21:42:28] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-eqiad, 10serviceops: Q3:(Need By: TBD) rack/setup/install gitlab100[3|4] and gitlab-runner100[2|3|4] - https://phabricator.wikimedia.org/T301177 (10Dzahn) confirming that the "gitlab" hosts should use a public IP and the "gitlab-runner" hosts should use a... [21:42:43] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work): Deploy new bullseye elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10bking) Elasticsearch is installed on our new Bullseye VM, but fails with error: `Unr... [21:46:07] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Radar), 10Discovery, 10Discovery-Search (Current work): Deploy new bullseye elastic cluster nodes on deployment-prep - https://phabricator.wikimedia.org/T299797 (10RKemper) >>! In T299797#7886111, @bking wrote: > Elasticsearch is installed on our n... [21:49:14] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Dzahn) 05Open→03In progress [21:55:42] 10Release-Engineering-Team, 10serviceops: helm-linter started failing on operations/deployment-charts today - https://phabricator.wikimedia.org/T307043 (10dancy) [22:03:17] 10Release-Engineering-Team, 10serviceops: helm-linter started failing on operations/deployment-charts today - https://phabricator.wikimedia.org/T307043 (10dancy) [22:05:16] 10Release-Engineering-Team, 10serviceops: helm-linter started failing on operations/deployment-charts today - https://phabricator.wikimedia.org/T307043 (10dancy) p:05Triage→03High [22:24:36] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host git... [22:25:48] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Papaul) [22:26:21] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw, 10Patch-For-Review: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Papaul) [22:39:08] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host gitlab2002.wikimedia.org with... [22:53:50] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host gitlab2002.wikimedia.org... [22:58:41] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host gitlab2002.wikimedia.org with... [22:59:52] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host gitlab-runner2002.codfw.w... [23:04:13] (03PS1) 10SBassett: Add new permissions for supplemental security tasks [tools/release] - 10https://gerrit.wikimedia.org/r/787110 (https://phabricator.wikimedia.org/T307036) [23:04:54] (03CR) 10jerkins-bot: [V: 04-1] Add new permissions for supplemental security tasks [tools/release] - 10https://gerrit.wikimedia.org/r/787110 (https://phabricator.wikimedia.org/T307036) (owner: 10SBassett) [23:08:57] (03PS2) 10SBassett: Add new permissions for supplemental security tasks [tools/release] - 10https://gerrit.wikimedia.org/r/787110 (https://phabricator.wikimedia.org/T307036) [23:17:41] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10Papaul) @Dzahn OS install failed on gitlab-runner2002 because of partitioning. I think is becasue you have: partman/raid1-2... [23:31:20] 10GitLab (Infrastructure), 10DC-Ops, 10SRE, 10ops-codfw: Q3:(Need By: TBD) rack/setup/install gitlab200[2|3] and gitlab-runner200[2|3|4] - https://phabricator.wikimedia.org/T301183 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host gitlab-runner2002.codfw.wmnet... [23:31:49] (03CR) 10DannyS712: Add new permissions for supplemental security tasks (031 comment) [tools/release] - 10https://gerrit.wikimedia.org/r/787110 (https://phabricator.wikimedia.org/T307036) (owner: 10SBassett)