[02:12:44] 10Continuous-Integration-Infrastructure, 10LibUp, 10ci-test-error: Git repos with direct pushes bypassing gerrit/CI could be a maintenance burden for libup patches - https://phabricator.wikimedia.org/T301201 (10Legoktm) My understanding was that we did not allow direct "open push" for master of extensions be... [07:57:48] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [07:58:16] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) p:05Triageβ†’03Medium [08:06:07] (03PS1) 10Majavah: release-scripts: Query installed scaps from PuppetDB [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 [08:09:29] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [09:35:45] Project beta-scap-sync-world build #38505: 04FAILURE in 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/38505/ [09:43:03] Project beta-code-update-eqiad build #379189: 04FAILURE in 2.9 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379189/ [09:53:01] Project beta-code-update-eqiad build #379190: 04STILL FAILING in 0.73 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379190/ [10:03:01] Project beta-code-update-eqiad build #379191: 04STILL FAILING in 0.8 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379191/ [10:04:32] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [10:09:54] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.0 - https://phabricator.wikimedia.org/T300804 (10JMeybohm) 05Openβ†’03Resolved [10:13:01] Project beta-code-update-eqiad build #379192: 04STILL FAILING in 0.95 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379192/ [10:23:01] Project beta-code-update-eqiad build #379193: 04STILL FAILING in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379193/ [10:30:08] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [10:30:33] 10Phabricator, 10DBA: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [10:31:12] 10Phabricator, 10DBA, 10User-notice: Switchover m3 master (db1107 -> db1183) - https://phabricator.wikimedia.org/T301219 (10Marostegui) [10:33:01] Project beta-code-update-eqiad build #379194: 04STILL FAILING in 0.75 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379194/ [10:43:01] Project beta-code-update-eqiad build #379195: 04STILL FAILING in 0.76 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379195/ [10:53:01] Project beta-code-update-eqiad build #379196: 04STILL FAILING in 0.68 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379196/ [11:03:01] Project beta-code-update-eqiad build #379197: 04STILL FAILING in 0.68 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379197/ [11:13:01] Project beta-code-update-eqiad build #379198: 04STILL FAILING in 0.65 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379198/ [11:23:01] Project beta-code-update-eqiad build #379199: 04STILL FAILING in 0.7 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379199/ [11:33:01] Project beta-code-update-eqiad build #379200: 04STILL FAILING in 0.75 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379200/ [11:43:02] Project beta-code-update-eqiad build #379201: 04STILL FAILING in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379201/ [11:53:01] Project beta-code-update-eqiad build #379202: 04STILL FAILING in 0.68 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379202/ [11:59:19] 10Continuous-Integration-Infrastructure, 10LibUp, 10ci-test-error: Git repos with direct pushes bypassing gerrit/CI could be a maintenance burden for libup patches - https://phabricator.wikimedia.org/T301201 (10MarcoAurelio) It looks like this is wanted by the maintainer (@RV1971)? * https://gerrit.wikimedi... [12:01:58] Project beta-mediawiki-config-update-eqiad build #126: 04FAILURE in 0.69 sec: https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update-eqiad/126/ [12:03:01] Project beta-code-update-eqiad build #379203: 04STILL FAILING in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379203/ [12:13:01] Project beta-code-update-eqiad build #379204: 04STILL FAILING in 0.68 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379204/ [12:23:01] Project beta-code-update-eqiad build #379205: 04STILL FAILING in 0.76 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379205/ [12:33:01] Project beta-code-update-eqiad build #379206: 04STILL FAILING in 0.7 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379206/ [12:39:50] 12:33:01 12:33:01 prep failed: Command 'git remote set-url origin https://gerrit.wikimedia.org/r/operations/mediawiki-config' failed with exit code 128; [12:39:54] Well, isn't that fun [12:43:01] Project beta-code-update-eqiad build #379207: 04STILL FAILING in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379207/ [12:53:01] Project beta-code-update-eqiad build #379208: 04STILL FAILING in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379208/ [13:03:01] Project beta-code-update-eqiad build #379209: 04STILL FAILING in 0.66 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379209/ [13:13:01] Project beta-code-update-eqiad build #379210: 04STILL FAILING in 0.88 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379210/ [13:23:01] Project beta-code-update-eqiad build #379211: 04STILL FAILING in 0.71 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379211/ [13:33:01] Project beta-code-update-eqiad build #379212: 04STILL FAILING in 0.7 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379212/ [13:33:35] this is deployment-deploy03: /dev/sdb 40G 38G 0 100% /srv [13:39:03] !log delete /srv/mediawiki-staging.save on deployment-deploy03 [13:39:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:43:09] Project beta-mediawiki-config-update-eqiad build #127: 04STILL FAILING in 4.9 sec: https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update-eqiad/127/ [13:46:36] Yippee, build fixed! [13:46:36] Project beta-code-update-eqiad build #379213: 09FIXED in 3 min 35 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/379213/ [13:51:33] Yippee, build fixed! [13:51:33] Project beta-scap-sync-world build #38506: 09FIXED in 4 min 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-sync-world/38506/ [15:51:41] taavi: Thanks for cleaning that up. I left that around a couple of weeks ago. [16:15:31] (03CR) 10Ahmon Dancy: "First, I love this change. It make the operation much more efficient." [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 (owner: 10Majavah) [16:23:37] !log hard reboot misbehaving deployment-echostore01 [16:23:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:32:28] (03CR) 10Majavah: release-scripts: Query installed scaps from PuppetDB (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 (owner: 10Majavah) [16:50:00] 10Release-Engineering-Team, 10Scap, 10serviceops: Deploy Scap version 4.3.0 - https://phabricator.wikimedia.org/T300804 (10dancy) Thanks @JMeybohm! [17:08:58] Yippee, build fixed! [17:08:58] Project beta-mediawiki-config-update-eqiad build #128: 09FIXED in 1 min 48 sec: https://integration.wikimedia.org/ci/job/beta-mediawiki-config-update-eqiad/128/ [17:09:00] 10GitLab (CI & Job Runners), 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10Security-Team, 10Patch-For-Review, and 2 others: Limit GitLab shared runners to images from Wikimedia Docker registry - https://phabricator.wikimedia.org/T291978 (10thcipriani) [17:09:14] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10Scap, 10Patch-For-Review: Improve efficiency of scap l10n operations - https://phabricator.wikimedia.org/T295304 (10thcipriani) [17:12:55] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.20 deployment blockers - https://phabricator.wikimedia.org/T293961 (10jeena) [17:13:38] 10GitLab (Project Migration), 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10User-brennen: Migrate mediawiki/tools/release/ to GitLab - https://phabricator.wikimedia.org/T290260 (10thcipriani) [17:17:24] 10GitLab (Infrastructure), 10SRE, 10serviceops, 10Patch-For-Review: gitlab: enable IPv6 for https - https://phabricator.wikimedia.org/T300816 (10Jelto) 05Openβ†’03In progress p:05Triageβ†’03Medium a:03Jelto [17:19:20] 10GitLab (Project Migration), 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10User-brennen: Migrate mediawiki/tools/release/ to GitLab - https://phabricator.wikimedia.org/T290260 (10thcipriani) [17:22:37] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10Scap: Scap backport change_url: Implement β€œwhat are the commit hashes from the two repos that correspond to my merged commit(s)”. - https://phabricator.wikimedia.org/T295495 (10thcipriani) [17:23:09] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: legacy deployment - https://phabricator.wikimedia.org/T294466 (10thcipriani) [17:23:21] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10MW-on-K8s, 10Release Pipeline, 10User-brennen: Scap backport change_url: copy files for legacy deployment - https://phabricator.wikimedia.org/T294457 (10thcipriani) [17:29:41] 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10MediaWiki Train Development Environment: Train-dev: Update to helm 3 - https://phabricator.wikimedia.org/T301266 (10jeena) [17:34:51] !log remove scap from deployment-kafka-main/jumbo [17:34:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:35:23] (03CR) 10Majavah: release-scripts: Query installed scaps from PuppetDB (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 (owner: 10Majavah) [17:35:30] dancy: ^^ [17:35:45] Taking a look [17:46:35] (03CR) 10Ahmon Dancy: [C: 03+2] release-scripts: Query installed scaps from PuppetDB [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 (owner: 10Majavah) [17:47:17] (03Merged) 10jenkins-bot: release-scripts: Query installed scaps from PuppetDB [tools/scap] - 10https://gerrit.wikimedia.org/r/760882 (owner: 10Majavah) [17:49:34] Thanks taavi. That's a great mod. [18:00:41] 10GitLab (Project Migration), 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10User-brennen: Early adoption signup for WMF GitLab - https://phabricator.wikimedia.org/T282842 (10brennen) Will create appropriate `people/` and `repos/` groups for everyone on this list who doesn't currently have access within thi... [18:36:50] 10GitLab (Project Migration), 10Release-Engineering-Team (Done by Feb 23πŸ”₯): Create new GitLab project group: API Platform - https://phabricator.wikimedia.org/T301164 (10brennen) [18:39:15] 10Phabricator, 10Release-Engineering-Team (Done by Feb 23πŸ”₯), 10User-brennen: Phabricator admin knowledge transfer - https://phabricator.wikimedia.org/T300693 (10brennen) [19:01:24] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.38.0-wmf.22 deployment blockers - https://phabricator.wikimedia.org/T300198 (10thcipriani) [19:13:03] 10Release-Engineering-Team (Seen), 10local-charts, 10Developer Productivity, 10MW-1.35-notes (1.35.0-wmf.5; 2019-11-05), 10User-brennen: mediawiki/core/.pipeline/dev_prereq.sh: fails if VisualEditor lib/ve submodule is at a commit not present in shallow clone - https://phabricator.wikimedia.org/T237310 (1... [19:37:10] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Tgr) Whatever thumbnailing problem beta had s... [19:48:42] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Tgr) >>! In T300822#7673709, @Tgr wrote: > Th... [20:37:40] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Tgr) >>! In T300822#7694753, @Tgr wrote: > Th... [20:52:03] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Tgr) ...or rather, can't reproduce at all any... [20:52:35] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Tgr) a:03Tgr [21:00:16] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Etonkovidova) >>! In T300822#7695060, @Tgr wr... [22:31:17] 10Continuous-Integration-Infrastructure, 10LibUp, 10ci-test-error: Git repos with direct pushes bypassing gerrit/CI could be a maintenance burden for libup patches - https://phabricator.wikimedia.org/T301201 (10RV1971) I did not intentionally configure anything different from the way it is supposed to work.... [22:31:47] (03PS1) 10Ahmon Dancy: Fix sync-world: Change handling of wikiversions.php [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 [22:32:48] (03CR) 10Ahmon Dancy: "Jeena, this addresses the problem you ran into today." [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 (owner: 10Ahmon Dancy) [22:34:04] (03PS1) 10MarcoAurelio: Review access change [extensions/DynamicPageListEngine] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760983 [22:34:37] (03PS2) 10MarcoAurelio: Remove direct push access from the repository [extensions/DynamicPageListEngine] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760983 (https://phabricator.wikimedia.org/T301201) [22:35:34] (03PS1) 10MarcoAurelio: Review access change [extensions/DataTable2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760984 [22:36:11] (03PS2) 10MarcoAurelio: Modify repository access settings [extensions/DataTable2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760984 (https://phabricator.wikimedia.org/T301201) [22:40:12] (03CR) 10Zabe: [V: 03+2 C: 03+2] Remove direct push access from the repository [extensions/DynamicPageListEngine] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760983 (https://phabricator.wikimedia.org/T301201) (owner: 10MarcoAurelio) [22:41:58] (03CR) 10Zabe: [V: 03+2 C: 03+2] Modify repository access settings [extensions/DataTable2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/760984 (https://phabricator.wikimedia.org/T301201) (owner: 10MarcoAurelio) [22:42:06] thanks zabe :) [22:42:36] yw :) [22:48:09] (03CR) 10Jeena Huneidi: "I'm a bit confused about the commit message explanation since I ended up using sync-world to update the wikiversions." [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 (owner: 10Ahmon Dancy) [22:49:25] (03PS1) 10Ahmon Dancy: Make www container handle www.mediawiki.org too [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761065 [22:49:27] (03PS1) 10Ahmon Dancy: train-dev: misc improvements [tools/train-dev] - 10https://gerrit.wikimedia.org/r/761066 [22:49:49] (03CR) 10Ahmon Dancy: Fix sync-world: Change handling of wikiversions.php (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 (owner: 10Ahmon Dancy) [22:51:15] (03PS2) 10Ahmon Dancy: Fix sync-wikiversions [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 [22:59:54] (03CR) 10Krinkle: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [23:02:07] (03CR) 10Krinkle: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [23:05:48] 10Beta-Cluster-Infrastructure, 10Growth-Structured-Tasks, 10Image-Suggestions, 10Growth-Team (Current Sprint): [betalabs] "Suggestions are no longer available on this article" is show for every image recommendation - https://phabricator.wikimedia.org/T300822 (10Etonkovidova) 05Openβ†’03Resolved [23:06:16] (03CR) 10Ahmon Dancy: [C: 04-1] "holding for mods" [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [23:31:05] (03PS3) 10Ahmon Dancy: Fix "sync-world: Change handling of wikiversions.php" [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 [23:32:15] (03PS4) 10Ahmon Dancy: Fix "sync-world: Change handling of wikiversions.php" [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 [23:35:05] (03PS5) 10Ahmon Dancy: Make sync-world, etc work without /srv/mediawiki on deploy server [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) [23:36:00] (03CR) 10Ahmon Dancy: Make sync-world, etc work without /srv/mediawiki on deploy server (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/757070 (https://phabricator.wikimedia.org/T253822) (owner: 10Ahmon Dancy) [23:56:51] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM" [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 (owner: 10Ahmon Dancy) [23:57:58] (03Merged) 10jenkins-bot: Fix "sync-world: Change handling of wikiversions.php" [tools/scap] - 10https://gerrit.wikimedia.org/r/761061 (owner: 10Ahmon Dancy)