[00:06:45] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) The reason p75=p99 in the sample at 04:26 is... [01:17:05] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) >>! In T317187#8217327, @kostajh wrote: > An... [01:29:34] 10Phabricator: Herald rule for Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10Reedy) [01:38:42] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) It could be our general Kubernetes problem,... [02:04:53] 10GitLab (Project Migration), 10Striker, 10Tools, 10Developer-Advocacy (Jul-Sep 2022), and 3 others: Replace Diffusion integration with Gitlab integration in Striker (toolsadmin) - https://phabricator.wikimedia.org/T296893 (10bd808) [02:04:59] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹️), 10Epic, and 2 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10bd808) [02:16:48] 10GitLab (Project Migration), 10Phabricator, 10Release-Engineering-Team (Bonus Level 🕹️), 10Epic, and 2 others: Migrate active repositories in Phabricator Differential to GitLab - https://phabricator.wikimedia.org/T191182 (10bd808) 05Stalled→03Open Removing the stalled status. #striker no longer allows... [02:22:56] 10GitLab, 10Striker, 10User-bd808: Inactive Diffusion repos for missingpages and wiki2email were migrated to GitLab - https://phabricator.wikimedia.org/T317260 (10bd808) Diffusion repos that were mirrored: * https://phabricator.wikimedia.org/source/tool-wiki2email/ * https://phabricator.wikimedia.org/source/... [02:23:41] 10GitLab, 10Striker, 10User-bd808: Inactive Diffusion repos for missingpages and wiki2email were migrated to GitLab - https://phabricator.wikimedia.org/T317260 (10bd808) [02:57:28] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) 503 responses as a percentage of all respons... [03:56:01] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) ores and image-suggestion almost always appe... [04:29:35] (03CR) 10Jeena Huneidi: "This change is ready for review." [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [04:33:09] (03CR) 10CI reject: [V: 04-1] Scap backport --revert: improve commit message [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) (owner: 10Jeena Huneidi) [04:34:26] (03PS4) 10Jeena Huneidi: Scap backport --revert: improve commit message [tools/scap] - 10https://gerrit.wikimedia.org/r/829249 (https://phabricator.wikimedia.org/T287046) [05:08:30] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) The appserver envoy metrics show many 503s,... [05:48:50] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10Joe) >>! In T317187#8219775, @tstarling wrote: > The ap... [06:28:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Developer Productivity, 10Regression: Unable to view mw-debug log in Jenkins - https://phabricator.wikimedia.org/T285335 (10hashar) We previously (T249268) added a hack to stream compressed fresnel.json.gz files which was https:/... [07:01:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Seen), 10Developer Productivity, 10Regression: Unable to view mw-debug log in Jenkins - https://phabricator.wikimedia.org/T285335 (10hashar) The relevant code in Jenkins is: ` lang=java,name=core/src/main/java/hudson/model/DirectoryBrowse... [08:03:58] 10Release-Engineering-Team (Bonus Level 🕹️), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10jnuche) @Joe we were thinking we can flatten the configs one level, since we are already parsing the ent... [08:17:51] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) [08:18:33] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) [08:22:16] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10hashar) [08:23:40] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) [08:26:03] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) Hello, I added some more information because it seems that this problem comes from ph... [08:27:03] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10hashar) We are in process of upgrading our ElasticSearch cluster from 6.2.0 to 7.1.5, from the git... [08:28:02] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) >>! In T317187#8219640, @tstarling wrote: >>>!... [08:30:58] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [08:31:54] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search): The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) Hello, Good deduction, the error came from elastica which does not update its depende... [08:44:59] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10JMeybohm) >>! In T317187#8219642, @tstarling wrote: > I... [08:48:16] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) >>! In T317187#8217380, @dcausse wrote: > @kos... [08:49:02] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10kostajh) [08:50:43] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search), 10PHP 8.1 support: The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10kostajh) 05Open→03Resolved a:03kostajh >>! In T317284#8220122, @Fallout... [08:54:33] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search), 10PHP 8.1 support: The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10Falloutgen) Hello, Everything is good on our side, I can even say that on a... [09:00:40] 10Phabricator (Search), 10CirrusSearch, 10Discovery-Search, 10MediaWiki-Interface (autocomplete search), 10PHP 8.1 support: The search no longer works with the WMF28 (CirrusSearch) - https://phabricator.wikimedia.org/T317284 (10hashar) @Falloutgen for Wikimedia deployment we copy the composer dependencie... [09:10:30] (03CR) 10Hashar: [C: 04-2] "I talked with David about the code I used to extract the list of events from the private field com.google.gerrit.server.events.Event.types" [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/830654 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [11:15:27] (03CR) 10Hashar: [C: 04-2] "On hold, pending a change proposed upstream to add getters https://gerrit-review.googlesource.com/c/gerrit/+/345017" [software/gerrit/plugins/events-wikimedia] - 10https://gerrit.wikimedia.org/r/830654 (https://phabricator.wikimedia.org/T304947) (owner: 10Hashar) [11:52:38] (03PS3) 10Jaime Nuche: K8s: add parsing of deployments configuration file [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) [11:53:08] (03CR) 10Jaime Nuche: K8s: add parsing of deployments configuration file (038 comments) [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [13:02:14] 10Phabricator: Herald rule for Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10Aklapper) I'm not sure why this task was created as T304572 is exactly the same [13:02:20] 10Phabricator: Herald rule for Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10Aklapper) [13:02:24] 10Phabricator, 10Data Engineering Planning, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10Aklapper) [13:02:37] 10Phabricator: Herald rule for Product Analytics and Data Engineering tags to Wmfdata-Python tasks Open, Needs TriagePublic - https://phabricator.wikimedia.org/T317266 (10Aklapper) [13:02:41] 10Phabricator, 10Data Engineering Planning, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10Aklapper) [13:04:38] (03Abandoned) 10Jforrester: Zuul: [mediawiki/extensions/ImageSuggestions] Add to extension gate [integration/config] - 10https://gerrit.wikimedia.org/r/803291 (owner: 10Jforrester) [13:26:36] (03PS1) 10Jaime Nuche: k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) [14:27:25] (03CR) 10Jaime Nuche: "Verified in traindev" [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830693 (owner: 10Ahmon Dancy) [14:27:27] (03CR) 10Jaime Nuche: [C: 03+2] Make test-scap-backport bootstrap itself [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830693 (owner: 10Ahmon Dancy) [14:27:53] (03Merged) 10jenkins-bot: Make test-scap-backport bootstrap itself [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830693 (owner: 10Ahmon Dancy) [14:56:30] (03CR) 10Ahmon Dancy: "I will retest in train-dev" [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [14:59:53] (03PS1) 10Jaime Nuche: k8s: update format of mediawiki k8s deployments file [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830884 [15:06:45] (03CR) 10Jaime Nuche: K8s: add parsing of deployments configuration file (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [15:09:46] (03CR) 10Ahmon Dancy: K8s: add parsing of deployments configuration file (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [15:13:27] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.39-release: Expand the set of bundled extensions and skins in MediaWiki 1.39 - https://phabricator.wikimedia.org/T305072 (10Jdforrester-WMF) 05Open→03Resolved a:03Jdforrester-WMF To the extent that it's now... [15:34:31] 10GitLab (Project Migration), 10Quarry: Move quarry to gitlab or github - https://phabricator.wikimedia.org/T308978 (10rook) As mentioned above github appears to serve the project requirements where gitlab is not quite ready. I don't believe I mentioned it here but one of the main problems is the lack of being... [15:39:52] 10GitLab, 10Striker, 10User-bd808: Inactive Diffusion repos for missingpages and wiki2email were migrated to GitLab - https://phabricator.wikimedia.org/T317260 (10bd808) 05Open→03Resolved [15:47:21] !log Upgrading scap to latest code revision in beta cluster [15:47:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:49:35] (03PS1) 10Ahmon Dancy: Release 4.17.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/830899 [15:49:44] (03CR) 10Ahmon Dancy: [C: 03+2] Release 4.17.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/830899 (owner: 10Ahmon Dancy) [15:55:50] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Jdforrester-WMF) [15:55:54] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.39-release: Expand the set of bundled extensions and skins in MediaWiki 1.39 - https://phabricator.wikimedia.org/T305072 (10Jdforrester-WMF) [15:56:19] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Jdforrester-WMF) [15:56:21] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Tracking-Neverending: Expand the set of bundled extensions and skins to achieve a default MediaWiki experience that's comparable to Wikimedia sites - https://phabricator.wikimedia.org/T178349 (10Jdforrester-WMF) [15:57:46] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.40-release: Expand the set of bundled extensions and skins in MediaWiki 1.40 - https://phabricator.wikimedia.org/T317146 (10Jdforrester-WMF) [15:57:49] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Tracking-Neverending: Expand the set of bundled extensions and skins to achieve a default MediaWiki experience that's comparable to Wikimedia sites - https://phabricator.wikimedia.org/T178349 (10Jdforrester-WMF) [15:58:07] 10MediaWiki-Releasing, 10MediaWiki-Installer, 10MediaWiki-Stakeholders-Group, 10Epic, 10MW-1.39-release: Expand the set of bundled extensions and skins in MediaWiki 1.39 - https://phabricator.wikimedia.org/T305072 (10Jdforrester-WMF) [15:58:09] 10MediaWiki-Releasing, 10Reference Previews, 10MW-1.39-release: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797 (10Jdforrester-WMF) [15:58:19] 10MediaWiki-Releasing, 10MediaWiki-extensions-LoginNotify, 10MW-1.39-release: Bundle LoginNotify extension with MediaWiki - https://phabricator.wikimedia.org/T191746 (10Jdforrester-WMF) [15:58:29] 10MediaWiki-Releasing, 10AntiSpoof, 10MW-1.39-release: Bundle AntiSpoof extension with MediaWiki - https://phabricator.wikimedia.org/T191736 (10Jdforrester-WMF) [15:58:37] 10MediaWiki-Releasing, 10Growth-Team, 10Growth-Team-Filtering, 10Thanks, 10MW-1.39-release: Bundle Thanks extension with MediaWiki - https://phabricator.wikimedia.org/T191739 (10Jdforrester-WMF) [15:58:43] 10MediaWiki-Releasing, 10MW-1.39-release, 10MobileFrontend (Tracking): Bundle MobileFrontend extension with MediaWiki - https://phabricator.wikimedia.org/T191734 (10Jdforrester-WMF) [15:58:51] 10MediaWiki-Releasing, 10Growth-Team, 10Growth-Team-Filtering, 10Notifications, and 2 others: Bundle Echo extension with MediaWiki - https://phabricator.wikimedia.org/T191738 (10Jdforrester-WMF) [16:00:42] (03Merged) 10jenkins-bot: Release 4.17.0-1 [tools/scap] - 10https://gerrit.wikimedia.org/r/830899 (owner: 10Ahmon Dancy) [16:09:25] 10MediaWiki-Releasing, 10Reference Previews, 10MW-1.39-release: Bundle Popups extension with MediaWiki - https://phabricator.wikimedia.org/T231797 (10Krinkle) >>! In T231797#5818511, @Legoktm wrote: > Popups is a pain for redistributors because it distributes minified, compiled JavaScript without the accompa... [16:12:20] 10Release-Engineering-Team, 10Scap: Scap backport should include the deployers username on the Gerrit approval comment - https://phabricator.wikimedia.org/T317163 (10dancy) 05Open→03Resolved a:03dancy scap 4.17.0 (which contains this modification) has been deployed. Thanks for the suggestion @taavi ! [16:19:01] (03CR) 10Ahmon Dancy: [C: 03+2] k8s: update format of mediawiki k8s deployments file [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830884 (owner: 10Jaime Nuche) [16:19:29] (03Merged) 10jenkins-bot: k8s: update format of mediawiki k8s deployments file [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830884 (owner: 10Jaime Nuche) [16:19:50] (03CR) 10Ahmon Dancy: [C: 03+1] k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [16:26:01] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab, 10Security: Findings in Security Readiness Reviews of Trusted GitLab Runners - https://phabricator.wikimedia.org/T317341 (10Jelto) [16:29:17] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab, 10Security: Findings in Security Readiness Reviews of Trusted GitLab Runners - https://phabricator.wikimedia.org/T317341 (10Jelto) [16:38:16] 10GitLab (Project Migration), 10Striker, 10Tools, 10Developer-Advocacy (Jul-Sep 2022), and 3 others: Replace Diffusion integration with Gitlab integration in Striker (toolsadmin) - https://phabricator.wikimedia.org/T296893 (10bd808) [17:34:54] 10Phabricator, 10Data-Engineering, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10Aklapper) 05Open→03Resolved a:03Aklapper Created H407: When all of these conditions are met:... [17:35:47] 10Phabricator, 10Data-Engineering, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10Aklapper) Note that this does not apply retroactively. Should I mass-add to the open tasks at https... [18:08:16] (03CR) 10Jaime Nuche: [C: 03+2] "Ahmon gave his ok to go ahead and merge." [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:11:57] 10Release-Engineering-Team (Bonus Level 🕹️), 10Scap: Scap backport should include the deployers username on the Gerrit approval comment - https://phabricator.wikimedia.org/T317163 (10dancy) [18:12:29] (03Merged) 10jenkins-bot: K8s: add parsing of deployments configuration file [tools/scap] - 10https://gerrit.wikimedia.org/r/830675 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:12:38] (03CR) 10Jaime Nuche: [C: 03+2] k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:12:47] (03CR) 10CI reject: [V: 04-1] k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:13:28] (03PS2) 10Jaime Nuche: k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) [18:19:13] (03CR) 10Jaime Nuche: k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:19:19] (03CR) 10Jaime Nuche: [C: 03+2] k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:23:05] (03Merged) 10jenkins-bot: k8s: reset local logs between runs [tools/scap] - 10https://gerrit.wikimedia.org/r/830864 (https://phabricator.wikimedia.org/T299648) (owner: 10Jaime Nuche) [18:27:30] (03PS1) 10Ahmon Dancy: train-dev: Fix check for --help in bash 3 [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830910 [18:37:10] 10Release-Engineering-Team (Bonus Level 🕹️), 10MW-on-K8s, 10serviceops, 10Patch-For-Review: Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10dancy) [18:39:34] 10GitLab (CI & Job Runners), 10serviceops, 10serviceops-collab, 10Security: Findings in Security Readiness Reviews of Trusted GitLab Runners - https://phabricator.wikimedia.org/T317341 (10Mstyles) [18:42:54] (03PS1) 10Dduvall: dockerfiles: Provide basic buildctl image [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) [18:45:27] (03PS2) 10Dduvall: dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) [18:54:53] (03CR) 10Ahmon Dancy: [C: 03+2] train-dev: Fix check for --help in bash 3 [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830910 (owner: 10Ahmon Dancy) [18:55:19] (03Merged) 10jenkins-bot: train-dev: Fix check for --help in bash 3 [tools/train-dev] - 10https://gerrit.wikimedia.org/r/830910 (owner: 10Ahmon Dancy) [18:56:42] (03PS2) 10Jforrester: Add DiscussionTools to the default MediaWiki bundle [tools/release] - 10https://gerrit.wikimedia.org/r/720490 (https://phabricator.wikimedia.org/T290464) (owner: 10Gergő Tisza) [18:59:18] (03CR) 10Jforrester: [C: 03+2] Add Linter to the MediaWiki bundle [tools/release] - 10https://gerrit.wikimedia.org/r/830288 (https://phabricator.wikimedia.org/T290465) (owner: 10Reedy) [18:59:29] (03CR) 10Jforrester: [C: 03+2] Add DiscussionTools to the default MediaWiki bundle [tools/release] - 10https://gerrit.wikimedia.org/r/720490 (https://phabricator.wikimedia.org/T290464) (owner: 10Gergő Tisza) [19:00:23] (03Merged) 10jenkins-bot: Add Linter to the MediaWiki bundle [tools/release] - 10https://gerrit.wikimedia.org/r/830288 (https://phabricator.wikimedia.org/T290465) (owner: 10Reedy) [19:00:28] (03Merged) 10jenkins-bot: Add DiscussionTools to the default MediaWiki bundle [tools/release] - 10https://gerrit.wikimedia.org/r/720490 (https://phabricator.wikimedia.org/T290464) (owner: 10Gergő Tisza) [19:06:18] (03CR) 10Ahmon Dancy: [C: 03+1] "Looks reasonable. One minor suggestion." [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [19:54:01] 10Phabricator, 10Data-Engineering, 10Product-Analytics, 10wmfdata-python: Herald rule to add Product Analytics and Data Engineering tags to Wmfdata-Python tasks - https://phabricator.wikimedia.org/T304572 (10nshahquinn-wmf) Thank you @Aklapper! >>! In T304572#8222816, @Aklapper wrote: > Note that this doe... [19:54:40] (03PS3) 10Dduvall: dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) [19:54:51] (03CR) 10Dduvall: "Thanks, Ahmon!" [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [19:56:27] (03PS4) 10Dduvall: dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) [19:57:11] (03CR) 10Dduvall: "Made one additional tweak, setting the WORKDIR to $BUILDKIT_HOME. Ready again." [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [19:59:37] (03CR) 10CI reject: [V: 04-1] dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [20:02:54] (03CR) 10Dduvall: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [20:18:17] heads up to folks that watch for problems in deployment-prep: we just turned on a simple ip based rate limiter capping at 100/s for *.wmcloud.org. I just saw that deployment-eventstreams-2.deployment-prep.eqiad1.wikimedia.cloud hit the limit while polling schema-beta.wmflabs.org. [20:18:58] (please consider moving the deployment-prep stuff to use the deployment-prep cache hosts instead of the shared infra) [20:19:04] hm? [20:19:38] you are saying that there are > 100/s requests going to schema-beta.wmflabs.org [20:20:30] we don't even have that in prod, and the connections are timed out there every 15 minutes! [20:20:34] ottomata: there at least were during a given second [20:20:36] (03CR) 10Dduvall: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [20:20:43] hm! weird. [20:21:18] bd808: if that is happenign it should def be throtttled, so good stuff :) [20:22:14] ottomata: https://phabricator.wikimedia.org/P34317 some numbers there [20:23:10] (03PS1) 10Dduvall: Lock version of editorconfig-checker to 2.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/830940 [20:24:05] aye, there is no good reason for anyone person to make many requests to that service. its a streamed response. so throttle away plz! :) [20:24:47] deployment-eventstreams-2 is the thing being throttled [20:25:18] (the call is coming from inside the house!) [20:25:43] deployment-eventstreams-2 also seems not to set a UA for its requests which seems less than ideal [20:27:26] (03PS2) 10Dduvall: Constrain version of editorconfig-checker to 2.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/830940 [20:27:52] (03CR) 10Dduvall: [C: 03+2] Constrain version of editorconfig-checker to 2.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/830940 (owner: 10Dduvall) [20:29:54] (03Merged) 10jenkins-bot: Constrain version of editorconfig-checker to 2.4.0 [integration/config] - 10https://gerrit.wikimedia.org/r/830940 (owner: 10Dduvall) [20:37:09] (03PS5) 10Dduvall: dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) [20:45:03] (03CR) 10Ahmon Dancy: [C: 03+2] dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [20:47:31] (03Merged) 10jenkins-bot: dockerfiles: Provide basic buildctl image for use with GitLab pipelines [integration/config] - 10https://gerrit.wikimedia.org/r/830919 (https://phabricator.wikimedia.org/T308501) (owner: 10Dduvall) [20:50:41] !log running `./fab deploy_docker` to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/830919 [20:50:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:01:16] (03PS1) 10Dduvall: dockerfile: Fix use of pipefail in buildctl Dockerfile [integration/config] - 10https://gerrit.wikimedia.org/r/830943 [21:05:24] (03CR) 10Dduvall: [C: 03+2] dockerfile: Fix use of pipefail in buildctl Dockerfile [integration/config] - 10https://gerrit.wikimedia.org/r/830943 (owner: 10Dduvall) [21:08:04] (03Merged) 10jenkins-bot: dockerfile: Fix use of pipefail in buildctl Dockerfile [integration/config] - 10https://gerrit.wikimedia.org/r/830943 (owner: 10Dduvall) [23:53:41] 10Release-Engineering-Team, 10GrowthExperiments, 10Performance-Team, 10Growth-Team (Current Sprint), 10Performance Issue: GrowthExperiments Special:Homepage: investigate performance regression in wmf.28 - https://phabricator.wikimedia.org/T317187 (10tstarling) >>! In T317187#8220183, @JMeybohm wrote: > A...