[00:08:48] 10Release-Engineering-Team, 10Dumps-Generation, 10MW-on-K8s, 10serviceops: Migrate current-generation dumps to run from our containerized images - https://phabricator.wikimedia.org/T352650 (10VirginiaPoundstone) Thanks for the flag @Milimetric. @Joe thanks for thinking this through. I have three follow-u... [00:11:08] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [00:24:38] !log Created deployment-mwlog02 on Bullseye in Beta Cluster for T345566 [00:24:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:24:42] T345566: error.log is not rotated in beta - https://phabricator.wikimedia.org/T345566 [00:36:55] James_F: so the new plan is that instead of rotating the logs we'll rotate the log hosts? [00:37:32] tgr: It's an out-of-date instance and apparently the logrotate on debian buster doesn't support the flags that prod puppet uses? [00:43:07] 10Phabricator (phabricator-next), 10Release-Engineering-Team (Priority Backlog 📥), 10Wikimedia-Phabricator-Extensions, 10Voice & Tone: MediaWiki OAuth provider: Update instructions to match MediaWiki strings - https://phabricator.wikimedia.org/T343579 (10Tgr) >>! In T343579#9071262, @Aklapper wrote: > Also... [00:47:04] James_F: yeah, although logrotate should in theory just ignore unknown flags so I'm not sure what's going on there. Anyway it was just a poor attempt at a joke, not complaining about it. [01:16:50] 10Fresh, 10ARM support, 10Upstream, 10User-zeljkofilipin: ECONNREFUSED error when running Selenium tests on M1 Mac - https://phabricator.wikimedia.org/T308889 (10Peter) With the help from @hashar I finally understand what's going on. Somehow `chromium --headless` do not work anymore without a display in ou... [01:23:57] 10Phabricator: move_project script does not check if task is in both future parent and future subproject and does not remove future parent project - https://phabricator.wikimedia.org/T224421 (10Aklapper) 05Open→03Declined [02:02:53] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [02:08:28] tgr: Aha. :-) [02:08:43] !log Switching deployment-mwlog01 off in Beta Cluster for T345566 [02:08:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:08:47] T345566: error.log is not rotated in beta - https://phabricator.wikimedia.org/T345566 [02:10:10] (03PS1) 10BCornwall: Archive libvmod-{header,tbf,uuid,vslp} [integration/config] - 10https://gerrit.wikimedia.org/r/980975 (https://phabricator.wikimedia.org/T347623) [02:18:53] 10Beta-Cluster-Infrastructure: error.log is not rotated in beta - https://phabricator.wikimedia.org/T345566 (10Jdforrester-WMF) OK, update: mwlog02 now exists, has the mount, all the config has switched over, and from the network trace it's receiving the files, but it's not updating the files. [05:09:07] 10Phabricator: Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b - https://phabricator.wikimedia.org/T352943 (10Aklapper) p:05Triage→03Low [05:13:20] 10Phabricator, 10Patch-For-Review: Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b - https://phabricator.wikimedia.org/T352943 (10CodeReviewBot) aklapper opened https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/29 Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b [07:55:12] 10Release-Engineering-Team, 10Dumps-Generation, 10MW-on-K8s, 10serviceops: Migrate current-generation dumps to run from our containerized images - https://phabricator.wikimedia.org/T352650 (10Joe) >>! In T352650#9388845, @VirginiaPoundstone wrote: > @Joe thanks for thinking this through. I have three follo... [08:52:53] 10Project-Admins, 10Machine-Learning-Team: Create three Phab Projects for Machine Learning: Lift Wing, Pilot Flag, Test Grounds - https://phabricator.wikimedia.org/T264774 (10isarantopoulos) p:05Triage→03Medium [08:53:10] 10Beta-Cluster-Infrastructure, 10Machine-Learning-Team, 10ORES, 10PageTriage: Special:NewPagesFeed broken on beta cluster testwiki - https://phabricator.wikimedia.org/T349635 (10isarantopoulos) p:05Triage→03Medium [09:20:42] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, 10serviceops: Move MediaWiki jobs to mw-on-k8s - https://phabricator.wikimedia.org/T349796 (10JMeybohm) [11:20:22] (03CR) 10Dreamy Jazz: Zuul: Add TorBlock to CheckUser extension dependency list (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/979998 (https://phabricator.wikimedia.org/T352681) (owner: 10Dreamy Jazz) [11:23:47] 10GitLab (Administration, Settings & Policy), 10collaboration-services, 10Patch-For-Review: Wikimedia GitLab should use a Wikimedia logo - https://phabricator.wikimedia.org/T349167 (10CodeReviewBot) jelto merged https://gitlab.wikimedia.org/repos/releng/gitlab-settings/-/merge_requests/55 manage favicon wit... [11:25:57] !log run gitlab-settings and update GitLab favicon - T349167 [11:26:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:26:00] T349167: Wikimedia GitLab should use a Wikimedia logo - https://phabricator.wikimedia.org/T349167 [11:26:37] 10GitLab (Administration, Settings & Policy), 10collaboration-services, 10Patch-For-Review: Wikimedia GitLab should use a Wikimedia logo - https://phabricator.wikimedia.org/T349167 (10Jelto) [11:29:30] 10GitLab (Administration, Settings & Policy), 10Release-Engineering-Team (Quid Pro Crow 🦃), 10gitlab-settings, 10collaboration-services, 10User-brennen: Define "Terms of Service and Privacy Policy" text for GitLab - https://phabricator.wikimedia.org/T285354 (10Jelto) [11:30:11] 10GitLab (Administration, Settings & Policy), 10collaboration-services, 10Patch-For-Review: Wikimedia GitLab should use a Wikimedia logo - https://phabricator.wikimedia.org/T349167 (10Jelto) 05Open→03Resolved I changed the favicon to the old logo on all instances. So all logos are customized and differen... [13:08:58] 10Release-Engineering-Team (Radar), 10Tech-Docs-Team, 10Documentation: Create repository with configuration files for common CI use cases - https://phabricator.wikimedia.org/T352273 (10KBach) A sandbox repository for this task is available on GitLab: https://gitlab.wikimedia.org/repos/technical-documentation... [13:58:06] (03PS1) 10Hashar: dockerfiles: node*-test: Don't cd /src [integration/config] - 10https://gerrit.wikimedia.org/r/981334 [14:01:40] (03CR) 10Hashar: "If one wants to debug Fresh (which uses those images), they would want to run the entrypoints such as `/run.sh` or `/run-with-xvfb.sh` wh" [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [14:50:58] 10Fresh, 10ARM support, 10Upstream, 10User-zeljkofilipin: ECONNREFUSED error when running Selenium tests on M1 Mac - https://phabricator.wikimedia.org/T308889 (10Peter) I did try a couple of other things: I rollbacked the docker container to the version that uses Chrome 103 and then headless works. Trying... [15:23:13] 10Release-Engineering-Team (Radar), 10Tech-Docs-Team, 10Documentation: Create repository with configuration files for common CI use cases - https://phabricator.wikimedia.org/T352273 (10KBach) Submitted a merge request for the example in Rust. [15:31:46] 10Beta-Cluster-Infrastructure: error.log is not rotated in beta - https://phabricator.wikimedia.org/T345566 (10Jdforrester-WMF) Notes from last night: * Despite having the `role::logging::mediawiki::udp2log` role, mwlog02 didn't have Nagios installed; as a short-term fix, we applied a (non-puppetised) install of... [15:39:43] 10Continuous-Integration-Infrastructure, 10Code-Health, 10Developer Productivity, 10MediaWiki CodeJam Dec 2023: Regularly generate a dependency graph for MediaWiki code - https://phabricator.wikimedia.org/T351336 (10daniel) [15:43:23] 10Release-Engineering-Team (Radar), 10Anti-Harassment, 10Security-Team, 10iPoid-Service: Use Gitlab Security Pipeline for ipoid - https://phabricator.wikimedia.org/T338238 (10mmartorana) [15:44:21] 10Gitlab-Application-Security-Pipeline, 10Security-Team, 10Documentation, 10SecTeam-Processed, 10Security: Update on-wiki documentation for the AppSec Pipeline for phase 2 work - https://phabricator.wikimedia.org/T346802 (10mmartorana) [15:45:31] 10Gitlab-Application-Security-Pipeline, 10Security Team AppSec, 10Security-Team, 10Patch-For-Review, and 2 others: Address issues within certain Gitlab CI security templates - https://phabricator.wikimedia.org/T338034 (10mmartorana) [15:45:46] 10Gitlab-Application-Security-Pipeline, 10SecTeam-Processed, 10Security: Provide use case about specific workflow in documentation - https://phabricator.wikimedia.org/T348548 (10mmartorana) [16:12:46] 10Project-Admins: Create project tag for "Machine-Learning-Backlog" - https://phabricator.wikimedia.org/T352997 (10calbon) [16:30:48] 10Continuous-Integration-Config, 10MobileFrontend, 10Developer Productivity, 10Web-Team-Backlog (Needs Prioritization (Tech)): Export MobileFrontend dist files as CI artifacts - https://phabricator.wikimedia.org/T352240 (10Jdlrobson) I think it might be worth re-evaluating the use of Webpack in MobileFront... [16:30:54] 10Continuous-Integration-Config, 10MobileFrontend, 10Developer Productivity, 10Web-Team-Backlog (Needs Prioritization (Tech)): Export MobileFrontend dist files as CI artifacts - https://phabricator.wikimedia.org/T352240 (10Jdlrobson) p:05Triage→03Medium [16:34:12] 10Continuous-Integration-Infrastructure, 10Quibble: api.php?format=json (FormatJson.php) Allowed memory size fatal error in quibble-vendor-mysql-php74-selenium-docker - https://phabricator.wikimedia.org/T353001 (10sbassett) [16:34:50] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Quibble: api.php?format=json (FormatJson.php) Allowed memory size fatal error in quibble-vendor-mysql-php74-selenium-docker - https://phabricator.wikimedia.org/T353001 (10sbassett) [16:37:33] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-extensions-StopForumSpam, 10Quibble: api.php?format=json (FormatJson.php) Allowed memory size fatal error in quibble-vendor-mysql-php74-selenium-docker - https://phabricator.wikimedia.org/T353001 (10sbassett) [16:40:16] (03CR) 10Ahmon Dancy: [C: 03+1] "lgtm!" [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [17:14:19] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [17:17:51] (03PS2) 10BCornwall: Archive libvmod-{header,re2,tbf,uuid,vslp} [integration/config] - 10https://gerrit.wikimedia.org/r/980975 (https://phabricator.wikimedia.org/T347623) [17:40:56] (03CR) 10Hashar: [C: 03+2] dockerfiles: node*-test: Don't cd /src [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [17:42:30] (03CR) 10CI reject: [V: 04-1] dockerfiles: node*-test: Don't cd /src [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [17:42:44] * hashar starts a protest and burns tires [18:07:59] 10Project-Admins: Create project tag for "Machine-Learning-Backlog" - https://phabricator.wikimedia.org/T352997 (10Aklapper) Hi, please provide sufficient context and links. For example, how would this project tag relate to #machine-learning-team? Thanks. [18:13:15] (03CR) 10Hashar: [C: 03+2] dockerfiles: node*-test: Don't cd /src [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [18:14:28] (03Merged) 10jenkins-bot: dockerfiles: node*-test: Don't cd /src [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [18:16:54] (03CR) 10Hashar: [C: 03+2] "For Fresh specifically, that requires a new version of Fresh to be cut since it hardcodes the image tags being used." [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [18:26:10] 10Beta-Cluster-Infrastructure: Change the favicon on https://en.wikipedia.beta.wmflabs.org/ - https://phabricator.wikimedia.org/T352951 (10Aklapper) [18:29:10] (03CR) 10Hashar: [C: 03+2] "Successfully published image docker-registry.wikimedia.org/releng/node20-test-browser-php80-composer:0.1.0-s2" [integration/config] - 10https://gerrit.wikimedia.org/r/981334 (owner: 10Hashar) [18:29:36] 10Project-Admins: Create project tag for "Machine-Learning-Backlog" - https://phabricator.wikimedia.org/T352997 (10calbon) Hey @Aklapper, we have our work board [[ https://phabricator.wikimedia.org/project/view/1901/ | here ]] but it is getting unmanageable large. We wanted to move backlog tickets to their own d... [19:07:07] (03PS1) 10BCornwall: Archive operations/debs/coredns [integration/config] - 10https://gerrit.wikimedia.org/r/981384 (https://phabricator.wikimedia.org/T347623) [19:35:08] 10Phabricator, 10Patch-For-Review: Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b - https://phabricator.wikimedia.org/T352943 (10CodeReviewBot) aklapper merged https://gitlab.wikimedia.org/repos/phabricator/phabricator/-/merge_requests/29 Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b [19:36:02] 10Phabricator (phabricator-next): Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b - https://phabricator.wikimedia.org/T352943 (10Aklapper) https://phabricator.wikimedia.org/rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b touched two source files and got merged on Jul 10 2016. The one file https://phabricator... [19:36:41] 10Phabricator (phabricator-next): Revert rPHAB9f143f8321b9160e5f11810f2e82fd980840d36b - https://phabricator.wikimedia.org/T352943 (10Aklapper) a:03Aklapper [19:43:06] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [21:07:50] 10Deployments, 10Release-Engineering-Team: Update [[wikitech:Backport_windows/Deployers]] for scap backport - https://phabricator.wikimedia.org/T353024 (10taavi) [21:27:03] (03PS1) 10BCornwall: Archive operations/software/varnish/vhtcpd [integration/config] - 10https://gerrit.wikimedia.org/r/981394 [21:28:59] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [22:36:01] 10Diffusion, 10Release-Engineering-Team, 10collaboration-services: Make https://git.wikimedia.org not redirect to Phabricator Diffusion - https://phabricator.wikimedia.org/T323073 (10Dzahn) made a repo to hold the landing / disambiguation page: https://gitlab.wikimedia.org/repos/sre/miscweb/git-landing-page [22:41:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-extensions-StopForumSpam, 10Quibble: api.php?format=json (FormatJson.php) Allowed memory size fatal error in quibble-vendor-mysql-php74-selenium-docker - https://phabricator.wikimedia.org/T353001 (10Umherirrender) The correspo... [23:03:42] 10GitLab (Project Migration), 10Traffic, 10Patch-For-Review: Migrate Traffic repositories from Gerrit to Gitlab - https://phabricator.wikimedia.org/T347623 (10BCornwall) [23:05:09] 10Continuous-Integration-Config, 10Codex, 10Design-Systems-Team, 10Patch-For-Review: Migrate Codex CI from Node 16 to Node 18 - https://phabricator.wikimedia.org/T352768 (10egardner) >>! In T352768#9382994, @Jdforrester-WMF wrote: > @egardner I've teed-up the patches to switch Codex, but I've not deployed... [23:39:29] 10Phabricator: Create Wikidata WikiProject Ontology - https://phabricator.wikimedia.org/T353033 (10Lectrician1) [23:39:53] 10Phabricator: Create Wikidata WikiProject Ontology Phabricator project - https://phabricator.wikimedia.org/T353033 (10Lectrician1) [23:52:56] 10Project-Admins: Create Wikidata WikiProject Ontology Phabricator project - https://phabricator.wikimedia.org/T353033 (10JJMC89) [23:58:31] 10Project-Admins: Create project tag for "Machine-Learning-Backlog" - https://phabricator.wikimedia.org/T352997 (10Aklapper) I'm still curious how and why things got "unmanageable". Large backlogs are not necessarily a problem per-se but sometimes imply missing approaches to divide and conquer (or some automatio...