[01:50:17] 10Beta-Cluster-Infrastructure, 10SRE, 10Traffic, 10Beta-Cluster-reproducible: Beta cluster down: Error: 502, Next Hop Connection Failed (Feb 2022) - https://phabricator.wikimedia.org/T302699 (10Krinkle) 05Openβ†’03Resolved Appears resolved. Unless there is a specific common cause recurring, I assume ther... [01:50:24] 10Beta-Cluster-Infrastructure, 10Infrastructure-Foundations, 10Beta-Cluster-reproducible, 10Puppet: Beta cluster MediaWiki code not updating - https://phabricator.wikimedia.org/T300591 (10Krinkle) 05Openβ†’03Resolved [01:52:04] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Seen), 10Patch-For-Review: Beta puppetmaster cherry-pick process - https://phabricator.wikimedia.org/T135427 (10Krinkle) I've written a runbook for our team which I understand is being circulated by various people to reference how to do puppet cherry... [01:52:43] 10Beta-Cluster-Infrastructure: Move upload.beta.wmflabs.org to upload.wikimedia.beta.wmflabs.org - https://phabricator.wikimedia.org/T281650 (10Krinkle) 05Openβ†’03Resolved a:03Krinkle [01:52:49] 10Beta-Cluster-Infrastructure, 10SRE, 10Technical-Debt, 10Tracking-Neverending: Minimize infrastructure differences between Beta Cluster and production - https://phabricator.wikimedia.org/T87220 (10Krinkle) [02:00:31] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Seen), 10Scap, 10SRE, 10serviceops: Scap can't clear opcache on mw servers in Beta Cluster - https://phabricator.wikimedia.org/T237033 (10Krinkle) @thcipriani @dancy I believe the equivalent of the `beta-scap-eqiad` job from back then (which is n... [08:06:33] there is merge blocker: https://phabricator.wikimedia.org/T309764 [08:13:34] 10Release-Engineering-Team, 10Scap, 10SRE, 10serviceops: Deploy Scap version 4.8.1 - https://phabricator.wikimedia.org/T309116 (10JMeybohm) a:03JMeybohm [08:36:12] 10Release-Engineering-Team, 10Scap, 10SRE, 10serviceops: Deploy Scap version 4.8.1 - https://phabricator.wikimedia.org/T309116 (10JMeybohm) ` mwdebug1002:~$ scap pull Traceback (most recent call last): File "/usr/bin/scap", line 32, in from scap import cli File "/usr/lib/python3/dist-packa... [08:37:00] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T308067 (10Tacsipacsi) I’m not sure if {T309770} is a train blocker, but looking at the code, it may cause TimedText namespace appearing... [08:39:48] 10Release-Engineering-Team, 10Scap, 10SRE, 10serviceops: Deploy Scap version 4.8.1 - https://phabricator.wikimedia.org/T309116 (10JMeybohm) 05Openβ†’03Stalled [08:43:51] (03CR) 10Michael Große: [C: 03+1] "Looks good to me. I'm probably supposed to know this, but has WikibaseLexeme already been "deployed" to doc.wikimedia.org, or is every ext" [integration/docroot] - 10https://gerrit.wikimedia.org/r/802138 (owner: 10Lucas Werkmeister (WMDE)) [08:45:17] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build filure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 (10kostajh) p:05Triageβ†’03Unbreak! >>! In T309764#7975738, @Urbanecm_WMF wrote: > Looks to affect... [09:02:09] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build filure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 (10hashar) The Zip file for the composer dependency `wmde/hamcrest-html-matchers` is corrupt: ` is a... [09:09:58] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build filure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 (10hashar) The cache system is more or less described https://www.mediawiki.org/wiki/Continuous_integ... [09:13:18] hashar: are we getting phpdocumentor/type-resolver from cache, or downloading it? It looks like it is being downloaded [09:13:46] e.g. in https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php72-docker/69272/console, " - Downloading phpdocumentor/type-resolver (1.6.1)" [09:14:25] kostajh: hi, I went to first validate whether the cache is corrupt which occurs from time to time [09:14:28] when I try to download locally, it works fine, so perhaps it is some network issue from our CI boxes? [09:15:19] possibly :-\ [09:15:38] there are multiple dependencies suffering the same issue [09:15:57] can you try `composer require phpdocumentor/type-resolver --no-cache` some tmp directory on a ci box to see? [09:16:35] the build I looked at did not download the zip file [09:16:42] so it got it frm the cache but the cached file looks valid [09:17:16] (I looked at the two build logs listed on T309764 [09:17:17] T309764: Gate pipeline build filure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 [09:17:38] maybe the disks have some issues [09:18:06] (back in a bit) [09:37:56] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 (10Aklapper) [09:44:39] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver failed - https://phabricator.wikimedia.org/T309764 (10hashar) Kosta pointed out the dependencies are downloaded a build like https://integration.wikime... [09:45:38] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver or wmde/hamcrest-html-matchers fail - https://phabricator.wikimedia.org/T309764 (10hashar) [09:53:13] hmm. they did have some incident last week with GitHub packages [09:53:37] and I found exactly one tweet from today asking if codeload.github.com is down, but otherwise haven't seen anything [09:54:05] so hashar if you curl https://codeload.github.com/phpDocumentor/TypeResolver/legacy.zip/77a32518733312af16a44300404e945338981de3 in a CI machine, the file will unzip correctly? [09:56:21] kostajh: yeah I tried your command above using a quibble docker image [09:56:24] and it worked fine [09:56:29] there are build working just fine now [09:57:00] hm [09:57:00] https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/buildTimeTrend is triggered by bunch of repositories so it often runs [09:57:17] and there is a trend of build times which shows a bunch of failure [09:57:20] but it apparently no more happens [09:57:39] https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/buildTimeGraph/png [09:58:16] hmm, ok [09:59:37] 10Release-Engineering-Team, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver or wmde/hamcrest-html-matchers fail - https://phabricator.wikimedia.org/T309764 (10hashar) If I look at the build trend for the job https://integration... [09:59:44] I am really tempted to put the blame on that codeload github system [10:01:29] https://status.mondoo.com/incidents/hx4ckvgm2qdp?u=8sxb6bxvvj6c Github's Asset Servers seem to be experiencing intermittent problems [10:17:17] (03PS5) 10Jaime Nuche: install-world: add new operation [tools/scap] - 10https://gerrit.wikimedia.org/r/802131 (https://phabricator.wikimedia.org/T307081) [10:17:25] (03CR) 10Jaime Nuche: install-world: add new operation (0314 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/802131 (https://phabricator.wikimedia.org/T307081) (owner: 10Jaime Nuche) [10:41:45] 10Release-Engineering-Team, 10Upstream, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver or wmde/hamcrest-html-matchers fail - https://phabricator.wikimedia.org/T309764 (10hashar) 05Openβ†’03Resolved a:03hashar I believe i... [10:44:09] (03CR) 10Dan Shick: [C: 03+1] "Looks good" [integration/docroot] - 10https://gerrit.wikimedia.org/r/802138 (owner: 10Lucas Werkmeister (WMDE)) [11:07:27] 10Release-Engineering-Team, 10Upstream, 10ci-test-error (WMF-deployed Build Failure): Gate pipeline build failure on mediawiki/core: Install of phpdocumentor/type-resolver or wmde/hamcrest-html-matchers fail - https://phabricator.wikimedia.org/T309764 (10MoritzMuehlenhoff) >>! In T309764#7976064, @hashar wro... [11:19:41] !log Restarting Jenkins on releases1002 [11:19:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:26:32] !log Restarting Jenkins on contint2001 [11:26:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:17:38] hashar: if you wanna walk through the doc switchover (or just do it), I can be a sound board :) [12:18:02] I've just read up on https://phabricator.wikimedia.org/T247653#7974301 [13:30:55] Krinkle: I am guessing we should just do it [13:31:12] looks like mutante made it easy to switch over between hosts [13:31:47] (03CR) 10Hashar: [C: 03+2] "Thanks, will deploy with scap once merged :)" [integration/docroot] - 10https://gerrit.wikimedia.org/r/793623 (owner: 10Libraryupgrader) [13:44:15] (03CR) 10Hashar: Update Wikibase section (032 comments) [integration/docroot] - 10https://gerrit.wikimedia.org/r/802138 (owner: 10Lucas Werkmeister (WMDE)) [13:50:37] I haven't been able to match the ssh fingerprints from gitlab.wikimedia.org to anything found at https://config-master.wikimedia.org/ssh-fingerprints.txt , is anyone else having that problem? [13:50:46] here's what I'm getting: https://phabricator.wikimedia.org/P29344 [13:51:05] hashar: ack, so once we have an SRE onboard we can do it in quick succession. what order should we do the ops in? are all the patches lined up? [13:52:10] inflatador: this are the fingerprints for the servers themselves rather than the service listening on gitlab.wikimedia.org [13:52:27] inflatador: I have no idea where we store our services ssh fingerprints :/ [13:52:52] interesting [13:53:04] Krinkle: pretty much the order: change dns, merge the puppet change, run puppet on all hosts [13:54:15] there might be some unbalance until the dns entry has expired on all hosts [13:54:48] I'll YOLO it for now, guess this is a known issue for stuff like git/gerrit [13:55:26] (or more accurately, with the way we collect fingerprints) [13:55:49] inflatador: for Gerrit it is at https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/gerrit.wikimedia.org:29418 [13:56:16] and from the parent page there is gitlab listed https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/gitlab.wikimedia.org [13:56:17] inflatador: https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/gitlab.wikimedia.org [13:56:44] Thanks hashar and taavi , checking now [13:58:49] all good, thanks again [15:17:18] (03PS1) 10Ahmon Dancy: build-deb-in-docker: Rename a variable [tools/scap] - 10https://gerrit.wikimedia.org/r/802545 [15:17:20] (03PS1) 10Ahmon Dancy: Added python3-packaging to deb deps list [tools/scap] - 10https://gerrit.wikimedia.org/r/802546 [15:20:35] (03PS2) 10Ahmon Dancy: Added python3-packaging to deb deps list [tools/scap] - 10https://gerrit.wikimedia.org/r/802546 (https://phabricator.wikimedia.org/T309116) [15:21:56] (03CR) 10Ahmon Dancy: [C: 03+2] build-deb-in-docker: Rename a variable [tools/scap] - 10https://gerrit.wikimedia.org/r/802545 (owner: 10Ahmon Dancy) [15:26:24] (03Merged) 10jenkins-bot: build-deb-in-docker: Rename a variable [tools/scap] - 10https://gerrit.wikimedia.org/r/802545 (owner: 10Ahmon Dancy) [15:26:45] (03CR) 10Ahmon Dancy: [C: 03+2] Added python3-packaging to deb deps list [tools/scap] - 10https://gerrit.wikimedia.org/r/802546 (https://phabricator.wikimedia.org/T309116) (owner: 10Ahmon Dancy) [15:31:01] (03Merged) 10jenkins-bot: Added python3-packaging to deb deps list [tools/scap] - 10https://gerrit.wikimedia.org/r/802546 (https://phabricator.wikimedia.org/T309116) (owner: 10Ahmon Dancy) [15:33:50] !log Upgrading scap to 4.8.1-1+0~20220602153109.295~1.gbp318d9c in beta cluster [15:33:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:37:03] (03PS1) 10Ahmon Dancy: Release 4.8.2-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/802550 [15:37:05] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.8.2-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/802550 (owner: 10Ahmon Dancy) [15:37:31] 10Continuous-Integration-Config, 10Release-Engineering-Team (Next), 10MediaWiki-Core-Tests, 10Code-Health, and 8 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Jdlrobson) In T304114 I was made aware that we're running Minerva Selenium tests on... [15:40:47] (03Merged) 10jenkins-bot: Release 4.8.2-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/802550 (owner: 10Ahmon Dancy) [15:42:03] 10Release-Engineering-Team, 10Scap, 10SRE, 10serviceops, 10Patch-For-Review: Deploy Scap version 4.8.2 - https://phabricator.wikimedia.org/T309116 (10dancy) 05Stalledβ†’03Open [15:42:30] 10Release-Engineering-Team, 10Scap, 10SRE, 10serviceops, 10Patch-For-Review: Deploy Scap version 4.8.2 - https://phabricator.wikimedia.org/T309116 (10dancy) Fixed at tag 4.8.2. >>! In T309116#7975904, @JMeybohm wrote: > Probably missing dependencies: > ` > mwdebug1002:~$ scap pull > Traceback (most rece... [15:57:37] (03CR) 10Lucas Werkmeister (WMDE): Update Wikibase section (032 comments) [integration/docroot] - 10https://gerrit.wikimedia.org/r/802138 (owner: 10Lucas Werkmeister (WMDE)) [16:38:42] (03PS1) 10Ahmon Dancy: Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 [16:38:44] (03PS1) 10Ahmon Dancy: Makefile tweaks to support parallel builds [tools/release] - 10https://gerrit.wikimedia.org/r/802563 [16:39:34] (03CR) 10CI reject: [V: 04-1] Makefile tweaks to support parallel builds [tools/release] - 10https://gerrit.wikimedia.org/r/802563 (owner: 10Ahmon Dancy) [16:39:42] (03CR) 10CI reject: [V: 04-1] Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 (owner: 10Ahmon Dancy) [16:49:18] 10GitLab (Infrastructure), 10serviceops, 10Patch-For-Review: bring new gitlab hardware servers into production - https://phabricator.wikimedia.org/T307142 (10Jelto) Migration of production GitLab from `gitlab1001` to `gitlab1004` was successful. Downtime was around 65 minutes. Backups were configured for th... [16:56:30] (03PS2) 10Ahmon Dancy: Makefile tweaks to support parallel builds [tools/release] - 10https://gerrit.wikimedia.org/r/802563 [16:56:36] (03PS2) 10Ahmon Dancy: Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 [16:57:29] (03CR) 10CI reject: [V: 04-1] Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 (owner: 10Ahmon Dancy) [16:58:04] (03CR) 10Ahmon Dancy: [C: 03+2] Makefile tweaks to support parallel builds [tools/release] - 10https://gerrit.wikimedia.org/r/802563 (owner: 10Ahmon Dancy) [16:58:27] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for bking - https://phabricator.wikimedia.org/T308147 (10thcipriani) 05Openβ†’03Resolved a:03thcipriani ## {icon check-circle-o} Training completed! Hey @bking thanks for attending deployment training! (And for deploying... [16:59:08] (03Merged) 10jenkins-bot: Makefile tweaks to support parallel builds [tools/release] - 10https://gerrit.wikimedia.org/r/802563 (owner: 10Ahmon Dancy) [17:00:25] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for TheresNoTime - https://phabricator.wikimedia.org/T305191 (10thcipriani) hiya @TheresNoTime β€” do you have an invite to the training you were hoping to attend (either UTC early or UTC late)? [17:02:32] (03PS3) 10Ahmon Dancy: Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 [17:06:30] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for TheresNoTime - https://phabricator.wikimedia.org/T305191 (10TheresNoTime) >>! In T305191#7976993, @thcipriani wrote: > hiya @TheresNoTime β€” do you have an invite to the training you were hoping to attend (either UTC early... [17:09:19] (03CR) 10Ahmon Dancy: [C: 03+2] Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 (owner: 10Ahmon Dancy) [17:09:45] thcipriani: ( thank you for the prod ^ ) [17:10:44] (03Merged) 10jenkins-bot: Factor out incremental image build implemention [tools/release] - 10https://gerrit.wikimedia.org/r/802562 (owner: 10Ahmon Dancy) [17:13:42] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for TheresNoTime - https://phabricator.wikimedia.org/T305191 (10ArielGlenn) While we're at it, I've been the only one around the last couple of times and um, the point of having trainings was so no one is the lone deployer. Ca... [17:40:12] 10Release-Engineering-Team (Deployment Training Requests): Deployment training request for TheresNoTime - https://phabricator.wikimedia.org/T305191 (10thcipriani) >>! In T305191#7977040, @TheresNoTime wrote: >>>! In T305191#7976993, @thcipriani wrote: >> hiya @TheresNoTime β€” do you have an invite to the training... [17:48:52] thcipriani: would you mind inviting my wikimedia email instead? Sorry to be a pain! [17:50:18] 10Release-Engineering-Team, 10NFDI: Do mediawiki docker images from wmf branch exist - https://phabricator.wikimedia.org/T309458 (10dancy) Hi @Physikerwelt. We do not currently build images for individual wmf branches. At this time we build and publish images that contain directories for all live branches (a... [17:50:32] TheresNoTime: So demanding! [17:50:40] :((( [17:50:43] :P [17:50:44] hehe. [17:50:49] TheresNoTime: sure! no problem :) [17:50:55] Tyler is very accomodating [17:51:08] only sometimes. [17:51:36] TheresNoTime: Thanks to you and Majavah for working on beta cluster! [17:52:16] I'm definitely more on the `s/working on/breaking` side there I think :D [17:52:41] that's what work looks like some days [17:52:47] Nod.. most days [17:53:00] TheresNoTime: you should have an updated invite now [17:53:22] Got it, thank you :) [17:54:31] sure thing, thanks for taking an interest in deployment! [18:06:32] (03CR) 10Ahmon Dancy: [C: 03+2] install-world: add new operation [tools/scap] - 10https://gerrit.wikimedia.org/r/802131 (https://phabricator.wikimedia.org/T307081) (owner: 10Jaime Nuche) [18:10:38] (03Merged) 10jenkins-bot: install-world: add new operation [tools/scap] - 10https://gerrit.wikimedia.org/r/802131 (https://phabricator.wikimedia.org/T307081) (owner: 10Jaime Nuche) [18:11:46] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Seen), 10Scap, 10SRE, 10serviceops: Scap can't clear opcache on mw servers in Beta Cluster - https://phabricator.wikimedia.org/T237033 (10dancy) >>! In T237033#7975492, @Krinkle wrote: > @thcipriani @dancy I believe the equivalent of the `beta-sc... [19:12:43] 10Release-Engineering-Team (Priority Backlog πŸ“₯), 10Patch-For-Review, 10Release, 10Train Deployments: 1.39.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T308067 (10thcipriani) >>! In T308067#7975902, @Tacsipacsi wrote: > I’m not sure if {T309770} is a train blocker, but looking at the cod... [19:59:55] mutante: about the doc.wm.o switch do you think Timo and I can do that with jelto ? Would work better for me timezone related :] [20:27:28] James_F: why override the -1 on https://gerrit.wikimedia.org/r/c/mediawiki/core/+/795246 [20:27:40] I never updated that task on it being our stupid extensions [20:30:00] RhinosF1: You replied explaining why the patch was needed. The C-1 should have been dismissed at that point, right? [20:30:50] RhinosF1: Or is that not the question you're asking? [20:30:58] James_F: I replied saying because I think it's a bug but then we eventually realised with taavi that ManageWiki turns [] into [""] in some cases [20:31:16] So I guess it's a question of should mediawiki guard against nonsense [20:31:36] Oh. This isn't on gerrit or on the task. I'm not totally psychic, you know. :-) [20:32:05] James_F: it's entirely my fault for not updating the task or patch [20:32:17] I think having MW work around accidents is fine, but ideally we should have a test to avoid things regressing when adding such checks. But let's not hold up the REL1_38 release yet further. [20:32:27] Yeah [20:32:35] We're already 2 days late. [20:32:46] Although I wouldn't mind the release being help up [20:32:52] We haven't even started testing yet [20:33:02] I haven't ran most of the schema changes [20:33:19] Stupid exams [20:34:10] :-) [21:13:57] https://gerrit.wikimedia.org/r/c/mediawiki/core/+/802640 is a bit of a hack but it should at least make Timo happy for now. [21:14:59] Does anyone have time to quickly deploy https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/802637/ before the week ends? [21:15:20] zabe: That's not the kind of thing that should ever be deployed quickly. Leave it to the DBAs. [21:15:46] Oh, rolling back everything to COMPAT_OLD? Hmm, maybe. [21:16:00] (But this should be in -operations.) [21:16:00] Yeah, it's basically a revert because the writing makes problems [21:16:22] (I also asked in -operations, but it usually goes under in the wikibugs messages) [21:45:59] (03PS1) 10Ahmon Dancy: make-container-image: push -q to all docker-pusher calls [tools/release] - 10https://gerrit.wikimedia.org/r/802648 [21:47:47] 10MediaWiki-Releasing, 10MW-1.38-release: Write and send release announcement for 1.38.0 - https://phabricator.wikimedia.org/T302914 (10Reedy) [21:48:08] 10MediaWiki-Releasing, 10MW-1.38-release: Write and send release announcement for 1.38.0 - https://phabricator.wikimedia.org/T302914 (10Reedy) 05Openβ†’03In progress p:05Triageβ†’03Medium [21:48:12] 10MediaWiki-Releasing, 10MW-1.38-release, 10Patch-For-Review: Release MW 1.38.0 - https://phabricator.wikimedia.org/T302893 (10Reedy) [21:51:31] (03CR) 10Ahmon Dancy: [C: 04-1] "Holding because releases1002.eqiad.wmnet is running Docker version 18.09.1 which does not yet have the -q flag for docker push." [tools/release] - 10https://gerrit.wikimedia.org/r/802648 (owner: 10Ahmon Dancy) [21:57:58] (03PS1) 10Ahmon Dancy: Add mediawiki version to the eval.php debug message [tools/scap] - 10https://gerrit.wikimedia.org/r/802651 [21:58:34] (03CR) 10Ahmon Dancy: [C: 03+2] Add mediawiki version to the eval.php debug message [tools/scap] - 10https://gerrit.wikimedia.org/r/802651 (owner: 10Ahmon Dancy) [22:02:25] (03Merged) 10jenkins-bot: Add mediawiki version to the eval.php debug message [tools/scap] - 10https://gerrit.wikimedia.org/r/802651 (owner: 10Ahmon Dancy) [22:41:45] 10MediaWiki-Releasing, 10MW-1.38-release: Write and send release announcement for 1.38.0 - https://phabricator.wikimedia.org/T302914 (10Reedy) [22:58:53] 10MediaWiki-Releasing, 10MW-1.38-release: Release MW 1.38.0 - https://phabricator.wikimedia.org/T302893 (10Reedy) [22:58:57] 10MediaWiki-Releasing, 10MW-1.38-release: Write and send release announcement for 1.38.0 - https://phabricator.wikimedia.org/T302914 (10Reedy) 05In progressβ†’03Resolved a:03Reedy [23:03:53] 10MediaWiki-Releasing, 10MW-1.38-notes, 10MW-1.38-release: Release MW 1.38.0 - https://phabricator.wikimedia.org/T302893 (10Reedy) 05In progressβ†’03Resolved a:03Reedy [23:34:11] 10Phabricator (Upstream), 10Upstream: Option to measure WIP limits based on card count instead of points, to more closely adhere to Kanban standards. - https://phabricator.wikimedia.org/T120042 (10MBinder_WMF) Adding @LGoto as an interested party. @Aklapper What is the procedure for upstreaming these days, wi...