[00:00:34] (DatasourceError) firing: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [00:05:34] (DatasourceError) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://grafana.wikimedia.org/alerting/grafana/iS0FSjJ4z/view - https://wikitech.wikimedia.org/wiki/Monitoring/DatasourceError - https://alerts.wikimedia.org/?q=alertname%3DDatasourceError [01:38:31] 10Release-Engineering-Team (Priority Backlog 📥), 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10DLynch) [01:39:06] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), and 3 others: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10DLynch) 05Open→03Resolved a:03DLynch [01:40:42] 10Release-Engineering-Team (Priority Backlog 📥), 10Content-Transform-Team, 10MediaWiki-Platform-Team, 10MW-1.42-notes (1.42.0-wmf.4; 2023-11-07), and 3 others: No longer possible to collapse sections on mobile - https://phabricator.wikimedia.org/T350627 (10DLynch) ssastry did a code search and noticed that... [02:39:43] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/DonationInterface] Add phan as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/972077 (https://phabricator.wikimedia.org/T224763) [02:40:00] (03CR) 10Jforrester: [C: 03+2] "Oops, deployed this four hours ago." [integration/config] - 10https://gerrit.wikimedia.org/r/972056 (owner: 10Jforrester) [02:40:04] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/DonationInterface] Add phan as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/972077 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:41:17] (03Merged) 10jenkins-bot: jjb: Update Doxygen-based jobs to 1.9.8 [integration/config] - 10https://gerrit.wikimedia.org/r/972056 (owner: 10Jforrester) [02:41:19] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/DonationInterface] Add phan as experimental [integration/config] - 10https://gerrit.wikimedia.org/r/972077 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:41:53] !log Zuul: [mediawiki/extensions/DonationInterface] Add phan as experimental for T224763 [02:41:56] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:41:57] T224763: Add phan to DonationInterface extension - https://phabricator.wikimedia.org/T224763 [02:45:58] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/DonationInterface] Add phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/972078 (https://phabricator.wikimedia.org/T224763) [02:47:16] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/DonationInterface] Add phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/972078 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:48:27] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/DonationInterface] Add phan deps [integration/config] - 10https://gerrit.wikimedia.org/r/972078 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:48:44] !log Zuul: [mediawiki/extensions/DonationInterface] Add phan deps for T224763 [02:48:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:48:48] T224763: Add phan to DonationInterface extension - https://phabricator.wikimedia.org/T224763 [02:54:33] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/DonationInterface] Promote phan to voting [integration/config] - 10https://gerrit.wikimedia.org/r/972079 (https://phabricator.wikimedia.org/T224763) [02:54:58] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/DonationInterface] Promote phan to voting [integration/config] - 10https://gerrit.wikimedia.org/r/972079 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:55:37] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/extensions/DonationInterface] Promote phan to voting [integration/config] - 10https://gerrit.wikimedia.org/r/972079 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [02:58:08] (03PS2) 10Jforrester: Zuul: [mediawiki/extensions/DonationInterface] Promote phan to voting [integration/config] - 10https://gerrit.wikimedia.org/r/972079 (https://phabricator.wikimedia.org/T224763) [02:59:17] (03CR) 10CI reject: [V: 04-1] Zuul: [mediawiki/extensions/DonationInterface] Promote phan to voting [integration/config] - 10https://gerrit.wikimedia.org/r/972079 (https://phabricator.wikimedia.org/T224763) (owner: 10Jforrester) [03:02:00] James_F: thanks for rolling out the doxygen change! [03:47:04] Happy to help. [06:53:43] (03PS6) 10VolkerE: Replace WikimediaUI Base with Codex design tokens [integration/docroot] - 10https://gerrit.wikimedia.org/r/969496 (https://phabricator.wikimedia.org/T331403) [08:36:21] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services, 10Patch-For-Review: Migrate to using new GitLab CI runner authentication scheme - https://phabricator.wikimedia.org/T344951 (10Jelto) [09:19:44] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Technical-Debt: Remove custom WMF Phab downstream hacks superseded by upstream Phorge changes - https://phabricator.wikimedia.org/T341799 (10Aklapper) [09:58:39] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Remove Java 8 images from integration/config - https://phabricator.wikimedia.org/T350587 (10Antoine_Quhen) [11:14:20] 10Phabricator (Upstream), 10Upstream: Searching shorter terms ("gerrit", "cors") in global Phabricator search leads to upstream request timeout (too many results?) - https://phabricator.wikimedia.org/T258803 (10Aklapper) Still an issue on 2023-11-07; I don't think this is fixable but I'd say the error message... [11:15:26] 10Phabricator, 10Wikibase Cloud, 10Wikibase Product Platform Team WPP, 10Wikidata, and 2 others: Update Herald rule to tag Wikidata tech tasks - https://phabricator.wikimedia.org/T349104 (10Aklapper) +#Phabricator [11:43:08] 10Phabricator: 2FA Reset Request - https://phabricator.wikimedia.org/T350576 (10Aklapper) @Deniz_WMDE: Here is the paste: P53145. (The other option would have been setting up a video call with another person who knows both you and me.) [11:55:37] 10Phabricator: 2FA Reset Request - https://phabricator.wikimedia.org/T350576 (10Deniz_WMDE) Thanks, I provided the text phrase. Happy to set up a video call too if needed. [12:08:42] 10Phabricator: 2FA Reset Request - https://phabricator.wikimedia.org/T350576 (10Aklapper) 05Open→03Resolved p:05Triage→03Medium a:03Aklapper Thanks! I followed https://wikitech.wikimedia.org/wiki/Phabricator#Removing_Two_Factor_Authentication and confirmed that https://meta.wikimedia.org/w/index.php?ti... [12:10:53] 10Phabricator: 2FA Reset Request - https://phabricator.wikimedia.org/T350576 (10Deniz_WMDE) Can confirm it's stripped, thank you very much! [12:46:35] (Queue (Jenkins jobs + Zuul functions) alert) firing: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [12:54:48] 10GitLab (CI & Job Runners), 10collaboration-services: GitLab Trusted Runner configuration fails: 404 Project Not Found - https://phabricator.wikimedia.org/T350679 (10Jelto) [12:54:59] 10GitLab (CI & Job Runners), 10collaboration-services: GitLab Trusted Runner configuration fails: 404 Project Not Found - https://phabricator.wikimedia.org/T350679 (10Jelto) p:05Triage→03Medium [12:56:05] PROBLEM - Work requests waiting in Zuul Gearman server on contint2002 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [12:56:35] 10GitLab (CI & Job Runners), 10collaboration-services: GitLab Trusted Runner configuration fails: 404 Project Not Found - https://phabricator.wikimedia.org/T350679 (10Jelto) [13:06:35] (Queue (Jenkins jobs + Zuul functions) alert) resolved: - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [13:12:55] RECOVERY - Work requests waiting in Zuul Gearman server on contint2002 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [13:31:18] o/ the coverage and codehealth jobs on Zuul seem to not be running for the past 45min or so (every job is queued), this feels a bit suspicious, is that known, am I paranoid, something something? [13:41:57] ah, it's restarted. yay! :) [14:34:19] 10GitLab (CI & Job Runners), 10collaboration-services, 10serviceops: Standardize Debian package builds on GitLab CI - https://phabricator.wikimedia.org/T304491 (10MatthewVernon) Tutorial now moved to main namespace [[ https://wikitech.wikimedia.org/wiki/Debian_packaging/Tutorial | Debian_packaging/Tutorial ]... [15:05:53] 10GitLab (CI & Job Runners), 10collaboration-services: GitLab Trusted Runner configuration fails: 404 Project Not Found - https://phabricator.wikimedia.org/T350679 (10Jelto) The script fails when trying to access project https://gitlab.wikimedia.org/repos/qte/catalyst/prototype-api with the ID `1658`. Locally... [15:22:26] (03PS1) 10Hashar: Remap serving plugins under /r/plugins/ [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/972396 [15:31:30] 10GitLab (Misc), 10Release-Engineering-Team (Priority Backlog 📥): Investigate and document "Depends-On" in GitLab - https://phabricator.wikimedia.org/T349872 (10Jelto) Is this premium feature [Merge request dependencies ](https://docs.gitlab.com/ee/user/project/merge_requests/dependencies.html) similar to the... [15:35:30] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10User-brennen: Clean up Phabricator production error logs - https://phabricator.wikimedia.org/T337500 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/extensions/-/merge_requests/17 Security... [15:36:33] 10Phabricator, 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services, 10Patch-For-Review: Stop logging Bugzilla to Phabricator redirector misses - https://phabricator.wikimedia.org/T344884 (10CodeReviewBot) brennen merged https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge... [16:01:28] (03PS1) 10Zoranzoki21: Zuul: [mediawiki/extensions/JWTAuth] Enable CI [integration/config] - 10https://gerrit.wikimedia.org/r/972407 [16:02:22] 10GitLab (Misc), 10Release-Engineering-Team (Priority Backlog 📥): Investigate and document "Depends-On" in GitLab - https://phabricator.wikimedia.org/T349872 (10thcipriani) >>! In T349872#9312963, @Jelto wrote: > Is this premium feature [Merge request dependencies ](https://docs.gitlab.com/ee/user/project/merg... [16:25:35] (03PS2) 10Zoranzoki21: Zuul: [mediawiki/extensions/JWTAuth] Enable CI [integration/config] - 10https://gerrit.wikimedia.org/r/972407 [16:28:09] (03PS1) 10Zoranzoki21: parameter_functions.py: Remove archived Jade extension [integration/config] - 10https://gerrit.wikimedia.org/r/972414 (https://phabricator.wikimedia.org/T281430) [16:43:16] 10GitLab, 10Wikidata Dev Team, 10wmde-wikidata-tech: [QB] Investigate moving QG from Gerrit to GitLab - https://phabricator.wikimedia.org/T350705 (10ItamarWMDE) [16:43:47] 10GitLab, 10Wikidata Dev Team, 10wmde-wikidata-tech: [QB] Investigate moving QB from Gerrit to GitLab - https://phabricator.wikimedia.org/T350705 (10ItamarWMDE) [17:25:39] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 10collaboration-services: Stop logging Bugzilla to Phabricator redirector misses - https://phabricator.wikimedia.org/T344884 (10Aklapper) [19:27:11] 10Beta-Cluster-Infrastructure, 10Commons, 10Structured-Data-Backlog: [commons-beta] main page displays Internal error - https://phabricator.wikimedia.org/T350726 (10Etonkovidova) [19:59:32] (03PS1) 10Hashar: Make serve:plugins emit a 404 for missing files [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/972460 [20:18:42] (03PS2) 10Hashar: Remap serving plugins under /r/plugins/ [software/gerrit] (deploy/wmf/stable-3.5) - 10https://gerrit.wikimedia.org/r/972396 [21:28:41] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), 10ci-test-error (WMF-deployed Build Failure): `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/v... - https://phabricator.wikimedia.org/T287419 [21:35:13] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), 10ci-test-error (WMF-deployed Build Failure): `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/v... - https://phabricator.wikimedia.org/T287419 [21:53:35] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), 10ci-test-error (WMF-deployed Build Failure): `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/v... - https://phabricator.wikimedia.org/T287419 [21:58:47] 10Release-Engineering-Team (Priority Backlog 📥), 10Patch-For-Review, 10Release, 10Train Deployments: 1.42.0-wmf.4 deployment blockers - https://phabricator.wikimedia.org/T350080 (10Tgr) [22:15:01] 10Continuous-Integration-Config, 10Release-Engineering-Team (Radar), 10MediaWiki-Vendor, 10Parsoid (Tracking), and 2 others: `mediawiki-core-php72-phan-docker` job runs `composer install` instead of using packages from mediawiki/vendor - https://phabricator.wikimedia.org/T287419 (10ssastry) I pushed that... [22:22:27] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/JWTAuth] Enable CI [integration/config] - 10https://gerrit.wikimedia.org/r/972407 (owner: 10Zoranzoki21) [22:22:33] (03CR) 10Jforrester: [C: 03+2] parameter_functions.py: Remove archived Jade extension [integration/config] - 10https://gerrit.wikimedia.org/r/972414 (https://phabricator.wikimedia.org/T281430) (owner: 10Zoranzoki21) [22:23:55] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/JWTAuth] Enable CI [integration/config] - 10https://gerrit.wikimedia.org/r/972407 (owner: 10Zoranzoki21) [22:24:05] (03Merged) 10jenkins-bot: parameter_functions.py: Remove archived Jade extension [integration/config] - 10https://gerrit.wikimedia.org/r/972414 (https://phabricator.wikimedia.org/T281430) (owner: 10Zoranzoki21) [22:24:55] !log Zuul: [mediawiki/extensions/JWTAuth] Enable CI [22:24:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:25:46] 10Beta-Cluster-Infrastructure, 10Thumbor, 10serviceops, 10Beta-Cluster-reproducible: deployment-prep needs a Thumbor instance - https://phabricator.wikimedia.org/T344605 (10Tgr) Well that didn't work. The error actually comes from [[https://gerrit.wikimedia.org/g/operations/puppet/+/b86fe11b85d3b27b1fcedcc... [22:56:18] (03PS1) 10Jforrester: Zuul: [mediawiki/extensions/JWTAuth] Disable browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/972487 [22:56:27] (03CR) 10Jforrester: [C: 03+2] Zuul: [mediawiki/extensions/JWTAuth] Disable browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/972487 (owner: 10Jforrester) [22:57:43] (03Merged) 10jenkins-bot: Zuul: [mediawiki/extensions/JWTAuth] Disable browser tests [integration/config] - 10https://gerrit.wikimedia.org/r/972487 (owner: 10Jforrester) [22:58:09] !log Zuul: [mediawiki/extensions/JWTAuth] Disable browser tests [22:58:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:58:38] (03PS1) 10Zoranzoki21: Zuul: [mediawiki/extensions/JWTAuth] Disable Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/972489 [22:59:09] (03PS2) 10Zoranzoki21: Zuul: [mediawiki/extensions/JWTAuth] Disable Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/972489 [22:59:55] (03Abandoned) 10Zoranzoki21: Zuul: [mediawiki/extensions/JWTAuth] Disable Selenium tests [integration/config] - 10https://gerrit.wikimedia.org/r/972489 (owner: 10Zoranzoki21)