[00:18:59] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-jeena (Sprint 7/28-8/10): 1.37.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T281159 (10Krinkle) [00:33:36] as far as I can tell, I'm not listed on the attention set for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ConfirmEdit/+/680756 but it still shows up in my dashboard as "my turn" [01:36:26] legoktm: I noticed that for one or two changes as well. In my case it was because it used to be assigned [01:36:27] https://gerrit.wikimedia.org/r/q/assignee:legoktm+is:open [01:36:34] but doesn't look like that's the case for this one [05:04:18] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T281158 (10DannyS712) [05:14:46] 10Gerrit: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10Legoktm) [05:18:47] 10Gerrit: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10santhosh) I also faced the issue. The dropdown did not go away. I refreshed the page and avoided clicking on reviewers field to enter the comment. [05:47:16] 10Gerrit: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10Paladox) @hashar back ported the fix here https://gerrit-review.googlesource.com/c/gerrit/+/313689 [05:48:02] 10Gerrit, 10Upstream: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10Paladox) [06:05:18] paladox: <3 [06:05:47] oh, hashar was the one who backported it, <3 to him as well [08:22:46] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) [08:23:11] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) [08:23:39] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) p:05Triage→03Medium [08:24:08] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) @mmodell if you can confirm that the above method to set phabricator to read-only is correct, I can take care of this myself. Thanks! [08:25:29] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10mmodell) @Marostegui indeed that's the correct command for making phabricator `read-only`. [08:25:47] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) Excellent - thanks! [08:27:17] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) I am going to aim for Thursday 12th at 06:00 AM UTC for this. [08:27:27] 10Phabricator, 10DBA: Failover m3 (phabricator) master (db1132) to a different host to upgrade its kernel - https://phabricator.wikimedia.org/T288197 (10Marostegui) [09:02:03] 10Release-Engineering-Team (Radar), 10Quality-and-Test-Engineering-Team (QTE), 10serviceops-radar, 10CommRel-Specialists-Support (Jul-Sep-2021): Expand the list of group 1 wikis to contain at least one (preferably 2) smaller "top ten size" wikis - https://phabricator.wikimedia.org/T286664 (10Elitre) [09:16:12] 10Gerrit, 10Upstream: Can't dismiss reviewer autosuggestion dropdown in new Gerrit - https://phabricator.wikimedia.org/T288193 (10hashar) a:03hashar The [[ https://gerrit-review.googlesource.com/c/gerrit/+/313689 | upstream patch is merged ]], gotta wait for the next patch release. [09:27:42] 10Gerrit, 10Release-Engineering-Team (Radar), 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10hashar) [09:28:07] 10Gerrit, 10Release-Engineering-Team (Radar), 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10hashar) p:05Triage→03Medium [13:37:29] 10Release-Engineering-Team (Radar), 10SRE, 10serviceops: Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Dzahn) 05Open→03Stalled this is only open due to a single remaining server, the mwmaint servers in codfw. this will be upgraded after we switch D... [13:37:41] 10Continuous-Integration-Infrastructure, 10Patch-For-Review: Migrate all CI jobs from stretch to buster or later and drop stretch testing support - https://phabricator.wikimedia.org/T278203 (10Dzahn) [13:37:49] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Patch-For-Review: Drop MediaWiki testing in stretch and instead test only in buster - https://phabricator.wikimedia.org/T252432 (10Dzahn) [14:03:45] PROBLEM - Work requests waiting in Zuul Gearman server on contint2001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [150.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [14:10:41] RECOVERY - Work requests waiting in Zuul Gearman server on contint2001 is OK: OK: Less than 100.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1 [14:38:22] whenever i +2 a change, gerrit says that "Attention is currently required from" the change owner, as if i had left a -1 review. can we configure it so that it requires attention from jenkins-bot instead? [14:38:52] (and should i file a task about this instead of bothering you here?) [14:39:03] MatmaRex: jenkins-bot is .. a bot [14:39:08] so it is not going to do anything [14:39:37] well, it is literally going to review and then perhaps merge the change [14:39:39] I am not sure how configurable the feature is though [14:39:45] (i know that requiring attention won't do anything) [14:39:50] (but it would look nice) [14:40:22] then jenkins-bot does not have any knowledge about attention being raised to it [14:42:17] I think in upstream workflow, someone CR+2 and raise attention to the change owner who can then submit it [14:42:36] when in our, that raises attention to CI that the change should be acted on [14:42:42] and the owner probably don't need to be bother [14:42:59] but I guess the feat raises attention back to whoever acted on the task [14:43:19] so your action would spam all the others indeed [14:57:45] (03PS1) 10Elukey: helm-linter: force docker image rebuild [integration/config] - 10https://gerrit.wikimedia.org/r/710276 [15:00:20] (03CR) 10Hashar: [C: 03+2] helm-linter: force docker image rebuild [integration/config] - 10https://gerrit.wikimedia.org/r/710276 (owner: 10Elukey) [15:00:41] (03PS1) 10Elukey: operations-misc: update helm-linter's image to 0.2.15 [integration/config] - 10https://gerrit.wikimedia.org/r/710277 [15:02:00] (03Merged) 10jenkins-bot: helm-linter: force docker image rebuild [integration/config] - 10https://gerrit.wikimedia.org/r/710276 (owner: 10Elukey) [15:02:25] !log Building Docker image helm-linter for https://gerrit.wikimedia.org/r/c/integration/config/+/710276 [15:02:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:05:01] (03PS1) 10Hashar: jjb: update helm-linter to 3.6.3 [integration/config] - 10https://gerrit.wikimedia.org/r/710282 [15:05:20] (03CR) 10Hashar: "the job is updated by https://gerrit.wikimedia.org/r/c/integration/config/+/710282/" [integration/config] - 10https://gerrit.wikimedia.org/r/710276 (owner: 10Elukey) [15:07:36] hashar: https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/106468/consoleFull failed with messages like "no space left on device". Did our CI run out of disk space? [15:08:00] urbanecm: I guess the instance requires some cleanup. Might be docker images/container pilled up on it [15:08:24] * hashar checks [15:08:27] Thanks! [15:08:45] yeah that is pipeline lib filling it :-\ [15:09:07] !log integration-agent-docker-1020: docker container prune && docker image prune [15:09:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:09:59] so, now i try again with "recheck", right hashar ? [15:10:15] urbanecm: right! [15:10:21] !log integration: sudo cumin --force 'name:docker' 'docker container prune -f && docker image prune -f' [15:10:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:10:28] and that command cleans all the agents [15:10:29] thanks for fixing it so quickly hashar :) [15:10:39] there is a task about pipeline lib not reclaiming images/container [15:19:55] (03CR) 10Hashar: [C: 03+2] "Successfully published image docker-registry.discovery.wmnet/releng/helm-linter:0.2.15" [integration/config] - 10https://gerrit.wikimedia.org/r/710282 (owner: 10Hashar) [15:21:52] urbanecm: all agents have been purged successfuly [15:22:23] (03CR) 10Klausman: [C: 03+1] jjb: update helm-linter to 3.6.3 [integration/config] - 10https://gerrit.wikimedia.org/r/710282 (owner: 10Hashar) [15:22:25] (03Merged) 10jenkins-bot: jjb: update helm-linter to 3.6.3 [integration/config] - 10https://gerrit.wikimedia.org/r/710282 (owner: 10Hashar) [15:28:17] thanks hashar [15:29:17] (03PS1) 10Ahmon Dancy: Use numeric USER in multiversion-base Dockerfile [tools/release] - 10https://gerrit.wikimedia.org/r/710289 [15:29:50] (03CR) 10Ahmon Dancy: [C: 03+2] Use numeric USER in multiversion-base Dockerfile [tools/release] - 10https://gerrit.wikimedia.org/r/710289 (owner: 10Ahmon Dancy) [15:31:51] (03Merged) 10jenkins-bot: Use numeric USER in multiversion-base Dockerfile [tools/release] - 10https://gerrit.wikimedia.org/r/710289 (owner: 10Ahmon Dancy) [16:15:19] (03PS1) 10Ahmon Dancy: Update purge-restricted-images Makefile target [tools/release] - 10https://gerrit.wikimedia.org/r/710299 [16:15:42] (03CR) 10Ahmon Dancy: [C: 03+2] Update purge-restricted-images Makefile target [tools/release] - 10https://gerrit.wikimedia.org/r/710299 (owner: 10Ahmon Dancy) [16:16:46] (03Merged) 10jenkins-bot: Update purge-restricted-images Makefile target [tools/release] - 10https://gerrit.wikimedia.org/r/710299 (owner: 10Ahmon Dancy) [16:34:42] 10Release-Engineering-Team (Doing), 10Infrastructure-Foundations, 10CAS-SSO, 10GitLab (Initialization), and 2 others: Open gitlab.wikimedia.org to all users with Wikimedia developer accounts - https://phabricator.wikimedia.org/T288162 (10brennen) cc: @Muehlenhoff as I think John's AFK for a bit. [16:36:49] 10Release-Engineering-Team (Doing), 10Infrastructure-Foundations, 10CAS-SSO, 10GitLab (Initialization), and 2 others: Open gitlab.wikimedia.org to all users with Wikimedia developer accounts - https://phabricator.wikimedia.org/T288162 (10brennen) [17:26:26] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10NRodriguez) Hi there, I would love to be able to create project labels for Comm Tech wishes. I am a product manager for CommTech [17:45:09] 10Project-Admins: Requests for addition to the #acl*Project-Admins group (in comments) - https://phabricator.wikimedia.org/T706 (10Ladsgroup) Done: >//Usual disclaimer: Please follow [guidelines](https://www.mediawiki.org/wiki/Phabricator/Creating_and_renaming_projects#Creating_new_projects) (naming, an understa... [20:29:02] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T281158 (10Raymond) [20:42:03] (03Abandoned) 10Hashar: operations-misc: update helm-linter's image to 0.2.15 [integration/config] - 10https://gerrit.wikimedia.org/r/710277 (owner: 10Elukey) [21:09:24] so here's a question for the group: should the "operations" namespace carry over from gerrit to gitlab? [21:09:42] i'm inclined to think yes, but if we wanted to rethink anything about that, now is probably the time [21:10:03] seems like a question for #wiimedia-sre [21:10:05] 10Phabricator: Add a Herald rule for User-MediaJS - https://phabricator.wikimedia.org/T286077 (10MediaJS) 05Declined→03Open When any of these conditions are met: Project tags include any of #Material Author is any of: @RhinosF1, @Universal_Omega, @R4356th Take these actions the first time this rule matches:... [21:12:54] bd808: yeah, good call. [21:13:40] hey twentyafterfour I have a fix for that Monobook issue [21:13:42] https://gerrit.wikimedia.org/r/710359 [21:13:56] Just trying to find a reviewer. [21:14:12] Jdlrobson: awesome thanks [21:16:43] twentyafterfour: it shouldn't block the train though (and assume it isn't) [21:17:02] but definitely worth getting the fix out today if we can [21:18:44] anyway, glad that use of risky template worked out [21:22:12] Jdlrobson: C+2'ed. [21:25:54] thanks James_F [21:26:14] twentyafterfour: are you able to help me backport the patch now or do I need to use the 4pm window? [21:26:36] I can deploy it now too, if needed. [21:26:48] Ah that would be great James_F 'cause I want to go to the staff meeting [21:27:14] Jdlrobson: James_F: I can help but I've got a 1:1 in 30 minutes... [21:28:00] I'll do it. [21:49:48] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T281158 (10Jdforrester-WMF) [21:50:34] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments, 10User-brennen: 1.37.0-wmf.12 deployment blockers - https://phabricator.wikimedia.org/T281153 (10Jdforrester-WMF) [21:58:32] 10Gerrit: Syntaxhighlighting in Gerrit is broken on php here doc syntax - https://phabricator.wikimedia.org/T288304 (10Umherirrender) [22:01:52] Can I beg for some upvotes/comments on T281315 ? ;-) [22:01:52] T281315: Maybe pass `--qq` to apt-get when in --info rather than --debug mode - https://phabricator.wikimedia.org/T281315 [22:02:07] (03CR) 10Jeena Huneidi: [C: 03+2] "LGTM!" [integration/config] - 10https://gerrit.wikimedia.org/r/710108 (https://phabricator.wikimedia.org/T287786) (owner: 10Accraze) [22:02:40] And if anyone wants to be nerd-sniped, T281312 [22:02:40] T281312: Provide some indicator of progress - https://phabricator.wikimedia.org/T281312 [22:03:26] (03Merged) 10jenkins-bot: inference-services: added articlequality pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/710108 (https://phabricator.wikimedia.org/T287786) (owner: 10Accraze) [22:04:34] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/710108 [22:04:36] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:13:37] 10Release-Engineering-Team (Doing), 10Release, 10Train Deployments: 1.37.0-wmf.17 deployment blockers - https://phabricator.wikimedia.org/T281158 (10Jdforrester-WMF) [22:14:40] Okie-dokie. Two UBNs back-ported. Now for some coffee. [22:26:17] !log gitlab: setting CI access on all repos to "Project members only", per https://www.mediawiki.org/wiki/GitLab/Policy#Permissions - may need revisited depending on effects [22:26:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:33:47] 10Gerrit: Syntaxhighlighting in Gerrit is broken on php here doc syntax - https://phabricator.wikimedia.org/T288304 (10Paladox) Gerrit 3.3 uses highlight.js 10.7. Looks like the newer HEREdoc support was added here https://github.com/highlightjs/highlight.js/commit/2a65144dfe92bc2eca0af913b29be56a97e66eae#diff-b... [22:40:11] (03CR) 10Addshore: WIP: add buster-apache2, log errors to stdio (031 comment) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/709831 (https://phabricator.wikimedia.org/T283416) (owner: 10Brennen Bearnes) [22:41:14] 10Gerrit, 10Release-Engineering-Team (Radar), 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10Paladox) [22:42:03] (03CR) 10Brennen Bearnes: WIP: add buster-apache2, log errors to stdio (031 comment) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/709831 (https://phabricator.wikimedia.org/T283416) (owner: 10Brennen Bearnes) [23:26:29] 10Release-Engineering-Team (Seen), 10GitLab, 10User-brennen: Figure out submodule updating in GitLab - https://phabricator.wikimedia.org/T268283 (10Legoktm) The mediawiki/extensions and mediawiki/skins repositories with auto-updating submodules is great in absence of a real monorepo. And submodules in mediaw... [23:27:37] 10Gerrit: Gerrit notification email contains "null" - https://phabricator.wikimedia.org/T288312 (10Umherirrender) [23:28:55] 10Gerrit, 10Release-Engineering-Team (Radar), 10Upstream: Gerrit names misalignment in 3.3.5 - https://phabricator.wikimedia.org/T288035 (10Paladox) Made this fix: https://gerrit-review.googlesource.com/c/gerrit/+/313982/4 [23:38:04] paladox: well done! :-] [23:38:20] thanks :) [23:38:58] paladox: this morning I managed to test a js fix I wrote,from start to review send: 1 hour! [23:39:07] nice! [23:39:16] I just followed their doc to have my local gerrit to load js directly from disk [23:39:20] upstream are in the process of moving to lit elements [23:39:21] went hacking and verifying. Solved! [23:39:27] (the successor to polymer) [23:39:45] I guess Polymer hasn't took off? [23:40:06] it has, i mean lit element is a lot similar to polymer [23:40:15] https://lit.dev/ [23:41:03] from my understanding that looks a bit like the OOUI thing we have (had?) [23:41:59] https://github.com/GerritCodeReview/gerrit/commit/e893feed71f1703442d2552846ffa62953066691 [23:42:01] looks like ^