[13:15:29] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new project and update herald - https://phabricator.wikimedia.org/T315160 (10Aklapper) [13:15:45] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new affects-Miraheze project and update herald - https://phabricator.wikimedia.org/T315160 (10Aklapper) p:05Triage→03Low a:03Aklapper [13:18:09] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new affects-Miraheze project and update herald - https://phabricator.wikimedia.org/T315160 (10Aklapper) Requested public project tag #affects-miraheze has been created: https://phabricator.wikimedia.org/projec... [13:22:00] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new affects-Miraheze project and update herald - https://phabricator.wikimedia.org/T315160 (10Aklapper) Removed #User-Rhinosf1 from && added #affects-miraheze to 98 tasks. [13:24:32] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new affects-Miraheze project and update herald - https://phabricator.wikimedia.org/T315160 (10Aklapper) I changed Herald rule H342 to be only about "Add #user-RhinosF1 project tag when task is assigned to Rhin... [13:31:18] 10Phabricator, 10Project-Admins, 10User-RhinosF1: Split Miraheze-Linked column of #user-rhinosf1 to new affects-Miraheze project and update herald - https://phabricator.wikimedia.org/T315160 (10RhinosF1) >>! In T315160#8153807, @Aklapper wrote: > I changed Herald rule H342 to be only about "Add #user-RhinosF... [15:18:22] (03CR) 10Ahmon Dancy: [C: 03+2] "Tested in production." [tools/scap] - 10https://gerrit.wikimedia.org/r/822687 (owner: 10Jeena Huneidi) [15:22:49] (03Merged) 10jenkins-bot: scap backport: Don't allow WIP [tools/scap] - 10https://gerrit.wikimedia.org/r/822687 (owner: 10Jeena Huneidi) [15:27:45] (03CR) 10Ahmon Dancy: [C: 03+2] "Tested in prod." [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [15:32:19] (03Merged) 10jenkins-bot: Scap backport --list: Add mergeable column [tools/scap] - 10https://gerrit.wikimedia.org/r/810078 (https://phabricator.wikimedia.org/T303967) (owner: 10Jeena Huneidi) [16:01:37] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission 💀), 10SRE, 10decommission-hardware, and 2 others: decommission gerrit2001.wikimedia.org (dcops) - https://phabricator.wikimedia.org/T315040 (10Papaul) [16:02:14] 10Gerrit, 10Release-Engineering-Team (The Decommission Mission 💀), 10SRE, 10decommission-hardware, and 2 others: decommission gerrit2001.wikimedia.org (dcops) - https://phabricator.wikimedia.org/T315040 (10Papaul) 05Open→03Resolved complete [16:05:56] (03PS1) 10Ahmon Dancy: Release 4.13.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/823183 [16:05:58] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.13.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/823183 (owner: 10Ahmon Dancy) [16:14:52] (03Merged) 10jenkins-bot: Release 4.13.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/823183 (owner: 10Ahmon Dancy) [16:34:24] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/821204/ - https://integration.wikimedia.org/ci/job/mwext-php72-phan-docker/197627/console [16:34:35] I've exceeded my blink budget on this one. [16:34:36] https://github.com/wikimedia/mediawiki-extensions-FlaggedRevs/blob/78c85ce264a82211bcb3140a48f8ad35c115d182/frontend/FlaggablePageView.php#L56 [16:34:40] Static call to undeclared method \FlaggablePageView::newFromTitle [16:34:49] It very obviously is declared. [16:36:52] or at least as declared as singleton() is, if... [16:37:08] .. if the extension is analyzed by Phan at all [16:37:11] unless [16:37:13] right [16:37:13] https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/VisualEditor/+/refs/heads/master/.phan/stubs/FlaggedRevs.php [16:39:03] stubs are awesome [17:20:55] (03PS1) 10Ahmon Dancy: Handle offline nodes properly [integration/config] - 10https://gerrit.wikimedia.org/r/823188 (https://phabricator.wikimedia.org/T315106) [17:22:40] (03CR) 10Ahmon Dancy: "I already deployed this manually in Jenkins to revive the job." [integration/config] - 10https://gerrit.wikimedia.org/r/823188 (https://phabricator.wikimedia.org/T315106) (owner: 10Ahmon Dancy) [17:23:40] (03CR) 10CI reject: [V: 04-1] Handle offline nodes properly [integration/config] - 10https://gerrit.wikimedia.org/r/823188 (https://phabricator.wikimedia.org/T315106) (owner: 10Ahmon Dancy) [17:25:35] (03PS2) 10Ahmon Dancy: Handle offline nodes properly [integration/config] - 10https://gerrit.wikimedia.org/r/823188 (https://phabricator.wikimedia.org/T315106) [17:35:39] Is it possible to see all gitlab commits, similar to a `status:merged` search in gerrit? I'm trying to see what changed around aug 9 at 09:00 UTC and i don't know what repository, so i'm just looking at all merges in that time frame [17:36:39] (i realize merge != deploy, but for the purposes of what i'm looking for they are likely to be siimlar) [17:38:39] ebernhardson: how about https://gitlab.wikimedia.org/groups/repos/-/merge_requests?scope=all&state=merged [17:38:55] (03PS1) 10BryanDavis: dockerfiles: update commit-message-validator [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) [17:39:21] mutante: that looks pretty good, thanks! [17:39:52] had to search around a bit, wasnt super obvious [17:40:04] (03CR) 10Jforrester: "Should the docker images be versioned with the software (so this would be 1.0.0)? We do this for some software-specific images like quibbl" [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) (owner: 10BryanDavis) [17:42:18] i do wonder a little that there are so few merges, i suppose i have no particular reason to expect more or less but for some reason i thought there would be more [17:45:46] there just aren't that many repos https://gitlab.wikimedia.org/repos?sort=name_asc [17:46:16] well..those are groups you can expand.. but yea [17:47:20] i suppose i happen to interact more with teams that use gitlab than in general, quite a few of the teams i interact with are doing things there, but there might not be many beyond them [17:48:27] (03CR) 10BryanDavis: dockerfiles: update commit-message-validator (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) (owner: 10BryanDavis) [18:03:12] (03CR) 10Jeena Huneidi: [C: 03+1] Handle offline nodes properly [integration/config] - 10https://gerrit.wikimedia.org/r/823188 (https://phabricator.wikimedia.org/T315106) (owner: 10Ahmon Dancy) [18:10:57] (03PS2) 10BryanDavis: dockerfiles: update commit-message-validator [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) [18:10:59] (03PS1) 10BryanDavis: dockerfiles: Add optional tag argument to debug-image [integration/config] - 10https://gerrit.wikimedia.org/r/823195 [18:15:51] (03CR) 10BryanDavis: dockerfiles: update commit-message-validator (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) (owner: 10BryanDavis) [18:20:11] (03PS3) 10BryanDavis: dockerfiles: update commit-message-validator [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) [18:20:13] (03PS2) 10BryanDavis: dockerfiles: Add optional tag argument to debug-image [integration/config] - 10https://gerrit.wikimedia.org/r/823195 [18:21:14] (03CR) 10BryanDavis: dockerfiles: update commit-message-validator (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/823193 (https://phabricator.wikimedia.org/T315159) (owner: 10BryanDavis) [18:29:00] (03PS1) 10Jeena Huneidi: scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 [18:32:51] (03CR) 10CI reject: [V: 04-1] scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 (owner: 10Jeena Huneidi) [18:34:36] (03PS2) 10Jeena Huneidi: scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 [18:38:39] (03CR) 10CI reject: [V: 04-1] scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 (owner: 10Jeena Huneidi) [18:53:56] (03PS3) 10Jeena Huneidi: scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 [19:03:46] 10Release-Engineering-Team, 10Foundational Technology Requests: Deploy to Wikimedia Production from GitLab - https://phabricator.wikimedia.org/T315253 (10thcipriani) [19:12:07] 10Scap, 10MediaWiki-ResourceLoader, 10MediaWiki-extensions-WikimediaMaintenance, 10Performance-Team, 10Technical-Debt: Remove old refreshMessageBlobs.php script from WikimediaMaintenance - https://phabricator.wikimedia.org/T314947 (10Krinkle) [19:12:12] 10Scap, 10MediaWiki-ResourceLoader, 10MediaWiki-extensions-WikimediaMaintenance, 10Performance-Team, 10Technical-Debt: Remove old refreshMessageBlobs.php script from WikimediaMaintenance - https://phabricator.wikimedia.org/T314947 (10Krinkle) [19:23:49] 10Scap, 10MediaWiki-ResourceLoader, 10MediaWiki-extensions-WikimediaMaintenance, 10Performance-Team, 10Technical-Debt: Remove old refreshMessageBlobs.php script from WikimediaMaintenance - https://phabricator.wikimedia.org/T314947 (10Krinkle) This is an easy task but touches numerous different repositori... [19:39:27] (03CR) 10Ahmon Dancy: scap backport: improve performance of validation (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 (owner: 10Jeena Huneidi) [19:40:13] 10Phabricator-Bot-Requests: Phabricator bot for train-blockers tool - https://phabricator.wikimedia.org/T315256 (10taavi) [19:55:28] 10Release-Engineering-Team, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fundraising Sprint NaN is a Number: Releng - please help us decommission our crm/civicrm git repo - https://phabricator.wikimedia.org/T314995 (10greg) Hi @thcipriani :) [20:19:41] 10Scap, 10Patch-For-Review: Allow scap to query Phabricator - https://phabricator.wikimedia.org/T315255 (10dancy) 05Open→03Resolved [21:53:36] 10Phabricator-Bot-Requests: Phabricator bot for train-blockers tool - https://phabricator.wikimedia.org/T315256 (10thcipriani) > Could I get a bot token instead? Sure! @train-blockers-bot exists now. Here's your token: P32392 Once you've verified this token works, please close this task. Otherwise followup w... [21:59:18] (03PS1) 10Thcipriani: CiviCRM: Decommission crm/civicrm [integration/config] - 10https://gerrit.wikimedia.org/r/823221 (https://phabricator.wikimedia.org/T314995) [22:00:39] 10Release-Engineering-Team, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fundraising Sprint NaN is a Number, 10Patch-For-Review: Releng - please help us decommission our crm/civicrm git repo - https://phabricator.wikimedia.org/T314995 (10thcipriani) I think that patch is all that's needed.... [22:02:22] (03CR) 10Thcipriani: [C: 04-1] "Hold for coordination with fr-tech." [integration/config] - 10https://gerrit.wikimedia.org/r/823221 (https://phabricator.wikimedia.org/T314995) (owner: 10Thcipriani) [22:14:42] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [22:24:24] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [22:39:01] hrm - zuul restart in order here? [22:49:42] (Queue (Jenkins jobs + Zuul functions) alert) firing: (2) Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [22:50:29] (03PS4) 10Jeena Huneidi: scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 [22:50:48] (03CR) 10Jeena Huneidi: scap backport: improve performance of validation (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 (owner: 10Jeena Huneidi) [22:51:37] (03PS5) 10Jeena Huneidi: scap backport: improve performance of validation [tools/scap] - 10https://gerrit.wikimedia.org/r/823197 [22:56:55] brennen: doesn't look like it. a lot of the time when this happens, it's actually zuul merger creating many possible future states. If you can catch it in time, you can avoid having too many jobs queued up at once. In this case, it looks like changes are clearing out on their own. That alarm seems a little too sensitive. [23:05:47] yeah, seems like it's ticking down. [23:08:29] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [200.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [23:09:42] (Queue (Jenkins jobs + Zuul functions) alert) resolved: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [23:11:42] (Queue (Jenkins jobs + Zuul functions) alert) firing: Queue (Jenkins jobs + Zuul functions) alert - https://alerts.wikimedia.org/?q=alertname%3DQueue+%28Jenkins+jobs+%2B+Zuul+functions%29+alert [23:26:37] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [400.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/d/000000322/zuul-gearman?orgId=1&viewPanel=10 [23:45:29] PROBLEM - PHD should be running on phab2002 is CRITICAL: PROCS CRITICAL: 0 processes with regex args php ./phd-daemon, UID = 498 (phd) https://wikitech.wikimedia.org/wiki/Phabricator [23:45:57] no, it shouldn't. I gave up on that :( [23:48:07] PROBLEM - PHD should be supervising processes on phab2002 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 498 (phd) https://wikitech.wikimedia.org/wiki/Phabricator [23:49:29] this is soo frustrating.. exactly what I kept saying would happen and now it's all over monitoring