[05:27:55] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Joe) p:05Low→03High The issue here is that all of the stretch images fail to report because they've not been rebuilt on top of the latest libssl updates. This i... [07:59:00] 10Release-Engineering-Team (Done by Wed 06 Oct), 10Security-Team, 10ContentSecurityPolicy, 10GitLab (Administration, Settings & Policy), and 4 others: Define a Content Security Policy for GitLab - https://phabricator.wikimedia.org/T285363 (10Jelto) I deployed https://gerrit.wikimedia.org/r/725900 to `gitla... [08:56:55] 10Project-Admins: Create dedicated project tag for User-dcaro - https://phabricator.wikimedia.org/T292450 (10Aklapper) 05Open→03Resolved a:03Aklapper Requested public project #User-dcaro has been created: https://phabricator.wikimedia.org/project/view/5584/ Interested people are welcome to join the projec... [09:04:14] 10Release-Engineering-Team (Next), 10Release Pipeline, 10GitLab (CI & Job Runners), 10User-dcaro, 10cloud-services-team (Kanban): Proof of concept for Buildpacks as WMF image build tool in GitLab - https://phabricator.wikimedia.org/T291017 (10dcaro) [09:04:25] 10Release-Engineering-Team (Next), 10Release Pipeline, 10GitLab (CI & Job Runners), 10User-dcaro: Proof of concept for Buildpacks as WMF image build tool in GitLab - https://phabricator.wikimedia.org/T291017 (10dcaro) [09:10:53] 10Release-Engineering-Team (Next), 10Release Pipeline, 10GitLab (CI & Job Runners): Proof of concept for Buildpacks as WMF image build tool in GitLab - https://phabricator.wikimedia.org/T291017 (10dcaro) [09:24:20] 10Phabricator (Upstream), 10Developer-Advocacy, 10Upstream: Visually indicate when a Phabricator user is new (Welcome culture) - https://phabricator.wikimedia.org/T64324 (10Aklapper) //(This ticket came back to my mind when reading WMF's internal conversations on communication ("ERC").)// IIUC, GitLab only... [09:48:05] 10Phabricator: Allow partially locking a task - https://phabricator.wikimedia.org/T292505 (10Aklapper) [10:05:09] 10Phabricator: Unclear what the point of tokens in Phabricator is - https://phabricator.wikimedia.org/T899 (10Aklapper) As tokens are per task and not per a specific comment, I'm unsure whether a token added at a certain point in time refers to a recently added previous comment, or really to the task itself. Gi... [11:00:07] (03PS1) 10Eugene233: Propose Zuul CI config for Wikimedia Commons reconcialiation tool [integration/config] - 10https://gerrit.wikimedia.org/r/726576 (https://phabricator.wikimedia.org/T292510) [11:03:07] (03PS2) 10Ladsgroup: Propose Zuul CI config for Wikimedia Commons reconciliation tool [integration/config] - 10https://gerrit.wikimedia.org/r/726576 (https://phabricator.wikimedia.org/T292510) (owner: 10Eugene233) [11:03:59] (03PS3) 10Ladsgroup: zuul: CI config for Wikimedia Commons reconciliation tool [integration/config] - 10https://gerrit.wikimedia.org/r/726576 (https://phabricator.wikimedia.org/T292510) (owner: 10Eugene233) [11:07:25] (03CR) 10Ladsgroup: [C: 03+2] zuul: CI config for Wikimedia Commons reconciliation tool [integration/config] - 10https://gerrit.wikimedia.org/r/726576 (https://phabricator.wikimedia.org/T292510) (owner: 10Eugene233) [11:09:08] (03Merged) 10jenkins-bot: zuul: CI config for Wikimedia Commons reconciliation tool [integration/config] - 10https://gerrit.wikimedia.org/r/726576 (https://phabricator.wikimedia.org/T292510) (owner: 10Eugene233) [11:11:24] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/c/integration/config/+/726576 [11:11:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:17:25] 10Beta-Cluster-Infrastructure: Figure out why deployment-cache-text06 keeps crashing - https://phabricator.wikimedia.org/T286502 (10ema) The instance has 4G of memory, of which up to 1G is used by the varnish cache and 2G by the ATS backend ram cache (setting `proxy.config.cache.ram_cache.size`). Indeed the syst... [12:27:28] 10Beta-Cluster-Infrastructure, 10Traffic: Figure out why deployment-cache-text06 keeps crashing - https://phabricator.wikimedia.org/T286502 (10ema) [13:42:51] https://tech.trivago.com/2021/10/05/postmortem-removing-all-users-from-github.com/trivago/ [13:45:17] 10Phabricator, 10WMNO-General: Disable Phabricator account WMNOaasapaaske - https://phabricator.wikimedia.org/T292529 (10jhsoby-WMNO) [13:49:09] 10Phabricator, 10WMNO-General: Disable Phabricator account WMNOaasapaaske - https://phabricator.wikimedia.org/T292529 (10Reedy) 05Open→03Resolved a:03Reedy Done [14:14:15] 10Beta-Cluster-Infrastructure: Upload cache not invalidated after purge - https://phabricator.wikimedia.org/T291643 (10dom_walden) >>! In T291643#7380223, @dom_walden wrote: >>>! In T291643#7374494, @gerritbot wrote: >> Change 723216 **merged** by jenkins-bot: >> %%%[operations/mediawiki-config@master] [beta] Up... [15:31:07] hi all! whom and where should I pester about being unable to activate 2FA on my WMF Gitlab account? [15:32:37] https://phabricator.wikimedia.org/T292431 [16:07:35] AntiComposite: ah fantastic thanks! yeah I'll comment there :) [16:26:05] 10Release-Engineering-Team, 10GitLab (Auth & Access): Gitlab 2fa password validation seems bugged - https://phabricator.wikimedia.org/T292431 (10AndyRussG) Hi! I have the same, or at least a similar, issue. I've previously been logging in without issue using my WMF developer account. I didn't have 2FA enabled... [16:35:41] I'd like to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/724430 tomorrow on gitlab2001 (replica). For that I will disable puppet on gitlab1001 (production gitlab). Let me know if this interferes with some deployments planned for production gitlab [16:36:32] 10Release-Engineering-Team (Radar), 10Infrastructure-Foundations, 10GitLab (Infrastructure), 10Patch-For-Review, and 3 others: Puppetise gitlab-ansible playbook - https://phabricator.wikimedia.org/T283076 (10Jelto) I'd like to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/724430 tomorrow on `gi... [16:45:00] 10Beta-Cluster-Infrastructure: Upload cache not invalidated after purge - https://phabricator.wikimedia.org/T291643 (10Reedy) 05Open→03Resolved a:03Reedy Gonna mark it as closed then. Feel free to reopen if you experience it again [17:06:58] jelto: works for me. [17:12:14] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10brennen) [17:55:21] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10thcipriani) ` Oct 04 18:44:08 deneb docker-report-releng[23989]: docker-registry.wikimedia.org/releng/typos:0.0.3-s4 [OK] Oct 04 18:44:08 deneb do... [18:00:31] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10thcipriani) >>! In T292485#7403167, @thcipriani wrote: > * `quibble-stretch*`: I think we can delete all the quibble-stretch images, poking @hashar to check me on... [18:01:21] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Legoktm) >>! In T292485#7403167, @thcipriani wrote: > * `audit-resources`: python3 //should be safe to rebuild//, poking @Legoktm in case he can foresee issues Unf... [18:27:50] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Legoktm) >>! In T292485#7403173, @thcipriani wrote: > No stupid questions: do I have the power to delete images and are there docs for that? https://wikitech.wikim... [18:30:22] 10Release-Engineering-Team (Seen), 10MW-on-K8s, 10SRE, 10Traffic, and 2 others: Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Krinkle) [18:31:03] 10Project-Admins: Create project tag for Codex - https://phabricator.wikimedia.org/T292565 (10Catrope) [18:32:42] 10Project-Admins: Create project tag for Codex - https://phabricator.wikimedia.org/T292565 (10Catrope) Associated with the `codex` repository in Gerrit, whose creation I have also [[https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests|requested]] [19:13:38] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10jeena) [19:46:53] 10Release-Engineering-Team (Doing), 10mwcli, 10GitLab (CI & Job Runners), 10User-brennen: Increase maximum artifacts size for gitlab CI - https://phabricator.wikimedia.org/T292372 (10Legoktm) >>! In T292372#7399089, @Addshore wrote: >>>! In T292372#7399078, @brennen wrote: >> In the short term, I'll put to... [19:56:17] 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10hashar) Last time that was for Jessie images and we ended up deleting them (see T251918#7138799 and following comments). We still rely on Stretch and I had all ima... [19:56:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10hashar) [20:08:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10brennen) We should also likely delete the no-longer-updated `dev/stretch-*` images, or at least most of them. I can file... [20:16:20] (03PS1) 10Kosta Harlan: Add support for phpbench [integration/quibble] - 10https://gerrit.wikimedia.org/r/726681 (https://phabricator.wikimedia.org/T291549) [20:20:20] (03CR) 10jerkins-bot: [V: 04-1] Add support for phpbench [integration/quibble] - 10https://gerrit.wikimedia.org/r/726681 (https://phabricator.wikimedia.org/T291549) (owner: 10Kosta Harlan) [20:50:40] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10DannyS712) @brennen would it be possible for the commits that revert/rollback the train to also get tagged with this task... [20:57:04] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10brennen) @DannyS712 That's a reasonable enough ask, though this is all still manual and tends to vary some between deploy... [20:59:14] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10brennen) [21:47:17] !log updating dev-images docker-pkg files on primary contint for https://gitlab.wikimedia.org/releng/dev-images/-/merge_requests/7 [21:47:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:00:38] 10Release-Engineering-Team (Doing), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.38.0-wmf.3 deployment blockers - https://phabricator.wikimedia.org/T281167 (10brennen) [22:04:43] (03PS3) 10Ahmon Dancy: build-mv-image: Handle mid-week L10N rebuilds gracefully [tools/release] - 10https://gerrit.wikimedia.org/r/725959 [22:24:40] (03CR) 10Dduvall: [C: 03+2] build-mv-image: Handle mid-week L10N rebuilds gracefully [tools/release] - 10https://gerrit.wikimedia.org/r/725959 (owner: 10Ahmon Dancy) [22:26:13] (03Merged) 10jenkins-bot: build-mv-image: Handle mid-week L10N rebuilds gracefully [tools/release] - 10https://gerrit.wikimedia.org/r/725959 (owner: 10Ahmon Dancy) [23:06:58] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Legoktm) btw, when I ctrl+f for jessie on https://docker-registry.wikimedia.org/ I see: ` releng/quibble-jessie-php55 rel... [23:07:29] legoktm: The 'wikimedia-audit-resources' docker image is only used for the 'audit-resources' job which isn't used anywhere; do you want me to just delete the lot for T292485? [23:07:29] T292485: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 [23:08:02] the job runs on a timer https://integration.wikimedia.org/ci/job/audit-resources/ but it's totally broken, so I think just deleting it all for now makes sense [23:08:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Jdforrester-WMF) >>! In T292485#7404119, @Legoktm wrote: > btw, when I ctrl+f for jessie on https://docker-registry.wikim... [23:08:19] > ERROR: ng.wikipedia.org: JSON errorGecko error: it seems /usr/bin/firefox is not compatible with SlimerJS. [23:08:19] * legoktm cries in slimerjs [23:08:21] Ack. [23:08:36] so yes please :) otherwise I'll get to it...soon. [23:09:30] (03PS1) 10Jforrester: jjb: [audit-resources] Drop job, doesn't work any more [integration/config] - 10https://gerrit.wikimedia.org/r/726705 [23:09:32] (03PS1) 10Jforrester: dockerfiles: [wikimedia-audit-resources] Drop image, not used any more [integration/config] - 10https://gerrit.wikimedia.org/r/726706 [23:10:11] legoktm: Should we also drop https://gerrit.wikimedia.org/r/plugins/gitiles/integration/audit-resources/+/refs/heads/master ? [23:12:55] (03CR) 10Jforrester: [C: 03+2] "Dropped." [integration/config] - 10https://gerrit.wikimedia.org/r/726705 (owner: 10Jforrester) [23:13:24] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [wikimedia-audit-resources] Drop image, not used any more [integration/config] - 10https://gerrit.wikimedia.org/r/726706 (owner: 10Jforrester) [23:13:35] I guess so :( [23:13:38] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Jdforrester-WMF) >>! In T292485#7403175, @Legoktm wrote: >>>! In T292485#7403167, @thcipriani wrote: >> * `audit-resource... [23:13:43] legoktm: Ack. [23:15:04] (03Merged) 10jenkins-bot: jjb: [audit-resources] Drop job, doesn't work any more [integration/config] - 10https://gerrit.wikimedia.org/r/726705 (owner: 10Jforrester) [23:15:08] (03Merged) 10jenkins-bot: dockerfiles: [wikimedia-audit-resources] Drop image, not used any more [integration/config] - 10https://gerrit.wikimedia.org/r/726706 (owner: 10Jforrester) [23:15:14] (03PS1) 10Jforrester: Archive this repo, not used or working [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/726708 [23:15:46] ^^ [23:16:18] (03CR) 10Legoktm: [V: 03+2 C: 03+2] "Regretfully." [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/726708 (owner: 10Jforrester) [23:16:24] (03CR) 10jerkins-bot: [V: 04-1] Archive this repo, not used or working [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/726708 (owner: 10Jforrester) [23:17:17] meh, don't have submit permissions [23:17:20] (03CR) 10jerkins-bot: [V: 04-1] Archive this repo, not used or working [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/726708 (owner: 10Jforrester) [23:17:49] Ha. [23:19:21] Force-submitted. [23:19:53] (03PS1) 10Jforrester: Zuul: [integration/audit-resources] Mark repo as archived [integration/config] - 10https://gerrit.wikimedia.org/r/726712 [23:20:10] (03CR) 10Jforrester: [C: 03+2] Zuul: [integration/audit-resources] Mark repo as archived [integration/config] - 10https://gerrit.wikimedia.org/r/726712 (owner: 10Jforrester) [23:22:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Legoktm) >>! In T292485#7403533, @hashar wrote: > Thus please delete: > * `releng/quibble-stretch*` images. They are no m... [23:22:56] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: docker-reporter-releng-images => docker registry: status=3/NOTIMPLEMENTED - https://phabricator.wikimedia.org/T292485 (10Legoktm) Deleted wikimedia-audit-resources too. [23:23:44] (03PS1) 10Dduvall: Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 [23:25:25] (03CR) 10Jforrester: [C: 03+2] "Let's go for next branch." [tools/release] - 10https://gerrit.wikimedia.org/r/725980 (https://phabricator.wikimedia.org/T235752) (owner: 10Jforrester) [23:25:38] (03CR) 10jerkins-bot: [V: 04-1] Replace simple git server with Gerrit [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [23:25:51] I'm re-running the docker report now [23:26:50] Excellent. [23:27:27] (03CR) 10Dduvall: "So many linter errors... I will address those tomorrow." [tools/train-dev] - 10https://gerrit.wikimedia.org/r/726714 (owner: 10Dduvall) [23:28:07] (03Merged) 10jenkins-bot: Zuul: [integration/audit-resources] Mark repo as archived [integration/config] - 10https://gerrit.wikimedia.org/r/726712 (owner: 10Jforrester) [23:28:32] (03Merged) 10jenkins-bot: Stop branching GettingStarted for Wikimedia production [tools/release] - 10https://gerrit.wikimedia.org/r/725980 (https://phabricator.wikimedia.org/T235752) (owner: 10Jforrester)