[00:09:25] 10Continuous-Integration-Infrastructure (Zuul upgrade), 06collaboration-services: puppetize setup of new zuul VMs - https://phabricator.wikimedia.org/T395938#11756494 (10Dzahn) So if i DROP that table and let zuul-web restart, it creates it again and again with `varbinary` type. Then I tried adding `?charset=... [00:18:10] !log Unblock 95.90.238.0/23 (T421447) [00:18:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:36:51] 10Gerrit, 10Wikimedia-GitHub: Core Github mirror is out of sync - https://phabricator.wikimedia.org/T421452 (10SomeRandomDeveloper) 03NEW [00:40:27] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#11756593 (10bd808) [06:13:59] 10Gerrit, 06collaboration-services: Harmonize DNS on all gerrit instances - https://phabricator.wikimedia.org/T417279#11756882 (10ABran-WMF) 05Open→03Resolved a:03ABran-WMF We moved all gerrit instances behind CDN, they all have the same DNS pattern [06:15:48] 10Gerrit, 06Release-Engineering-Team, 14Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 06collaboration-services, and 2 others: Jenkins job failing intermittently due to Gerrit HTTP 502 errors when interacting with repos - https://phabricator.wikimedia.org/T246763#11756887 (10ABran-WMF) >>! In... [06:30:32] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, 13Patch-For-Review: gerrit: Add Envoy in Gerrit's stack - https://phabricator.wikimedia.org/T420909#11756910 (10ABran-WMF) >>! In T420909#11756114, @hashar wrote: > ` > lang=yaml > profile::tlsproxy::envoy::upstream_tls: true > pr... [07:27:07] (03CR) 10Novem Linguae: [C:03+1] "I suspect this will get rid of the Phan CI error in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/SecurePoll/+/1256892" [integration/config] - 10https://gerrit.wikimedia.org/r/1259014 (owner: 10Bhsd) [07:35:10] 10Beta-Cluster-Infrastructure, 06Content-Transform-Team (Work In Progress), 07Essential-Work: Maps don't display on the beta cluster; tile server offline on deployment-maps-master02.deployment-prep.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T420299#11756968 (10Jgiannelos) I can cleanup th... [07:40:42] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic: gerrit: Add Envoy in Gerrit's stack - https://phabricator.wikimedia.org/T420909#11756973 (10hashar) @ABran-WMF I think I have mixed up `downstream` and `upstream. [[ https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/in... [08:44:44] 06Release-Engineering-Team (Doing 😎), 10Catalyst (Luka Ijo Pimeja Jan), 07Essential-Work: Patchdemo timing out while streaming env creation logs - https://phabricator.wikimedia.org/T421394#11757090 (10jnuche) 05Open→03Resolved a:05jnuche→03brennen https://gitlab.wikimedia.org/repos/test-platform/... [09:11:02] 10Phabricator-Bot-Requests, 06Product Safety and Integrity, 06Security-Team: Create a Mailing list account to replace Slack_connector_for_TSP_team_triage - https://phabricator.wikimedia.org/T421406#11757179 (10A_smart_kitten) >>! In T421406#11756362, @Aklapper wrote: >> Please also disable all email notifica... [09:12:44] 10Phabricator-Bot-Requests, 06Product Safety and Integrity, 06Security-Team: Create a Mailing list account to replace Slack_connector_for_TSP_team_triage - https://phabricator.wikimedia.org/T421406#11757200 (10Aklapper) Uh, thanks. Yes, my previous comment here is wrong (and I wonder what I checked yesterday). [09:34:50] Project beta-code-update-eqiad build #593449: 04FAILURE in 20 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/593449/ [09:42:50] 10Gerrit, 06Release-Engineering-Team, 06collaboration-services, 06Traffic, 13Patch-For-Review: gerrit: Add Envoy in Gerrit's stack - https://phabricator.wikimedia.org/T420909#11757279 (10ABran-WMF) we tweaked several knobs on httpd and Envoy and still have the same underlying issue, I think aligning Jet... [09:45:23] Yippee, build fixed! [09:45:23] Project beta-code-update-eqiad build #593450: 09FIXED in 2 min 22 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/593450/ [10:24:52] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#11757613 (10LucasWerkmeister) >>! In T420833#11752055, @bd808 wrote: > We need the https://wikitech.wikimedia.org/wiki/Requestctl stack in Beta. I gues... [11:28:18] !log deployment-prep: removed block for `143.176.0.0/15` and blocked subblock `143.176.0.0/16` instead. This unblocks `143.177.0.0/16` # T421420 [11:28:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:53:03] 10Continuous-Integration-Infrastructure, 10Quibble: Parallel npm install times out in Quibble - https://phabricator.wikimedia.org/T303270#11757899 (10hashar) From P22106 , threads are incrementing the same variable (`n`) and that is not thread safe. There is a similar one with the `logging` handlers not being... [14:49:18] 10Gerrit, 10Wikimedia-GitHub: Core Github mirror is out of sync - https://phabricator.wikimedia.org/T421452#11758576 (10SomeRandomDeveloper) 05Open→03Resolved Seems to be resolved now. [14:55:49] (03PS1) 10Phedenskog: Skip composer dev-requires for browser test only jobs [integration/quibble] - 10https://gerrit.wikimedia.org/r/1262148 (https://phabricator.wikimedia.org/T421500) [14:59:31] 10Beta-Cluster-Infrastructure: Beta cluster is being crawled to death by bot traffic coming from residential proxies - https://phabricator.wikimedia.org/T420833#11758619 (10bd808) >>! In T420833#11757613, @LucasWerkmeister wrote: >>>! In T420833#11752055, @bd808 wrote: >> We need the https://wikitech.wikimedia.o... [15:36:02] Project mediawiki-core-phpmetrics build #701: 04STILL FAILING in 2 min 1 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-phpmetrics/701/ [15:39:57] 10Beta-Cluster-Infrastructure, 06Content-Transform-Team (Work In Progress), 07Essential-Work: Maps don't display on the beta cluster; tile server offline on deployment-maps-master02.deployment-prep.eqiad1.wikimedia.cloud - https://phabricator.wikimedia.org/T420299#11758732 (10bd808) >>! In T420299#117569... [16:19:21] (03open) 10dancy: jenkins.yaml: Allow read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:19:29] (03update) 10dancy: jenkins.yaml: Allow read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:19:42] (03update) 10dancy: jenkins.yaml: Grant read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:19:51] (03update) 10dancy: jenkins.yaml: Grant read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:20:29] (03update) 10dancy: jenkins.yaml: Grant read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:39:10] (03approved) 10thcipriani: jenkins.yaml: Grant read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 (owner: 10dancy) [16:39:24] (03merge) 10dancy: jenkins.yaml: Grant read access to Arnaudb [repos/releng/jenkins-deploy] - 10https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/112 [16:55:45] 10Gerrit, 06Release-Engineering-Team, 14Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 06collaboration-services, and 2 others: Jenkins job failing intermittently due to Gerrit HTTP 502 errors when interacting with repos - https://phabricator.wikimedia.org/T246763#11759008 (10dancy) >>! In T24... [20:13:30] 10Phabricator, 10Bitu, 06collaboration-services, 06Infrastructure-Foundations, 10VPS-project-Phabricator: Allow Bitu to link Phabricator account - https://phabricator.wikimedia.org/T406495#11759750 (10A_smart_kitten) To confirm, has this been completed now (or e.g. is there more still to do)? I was able... [20:58:29] (03CR) 10Jforrester: [C:03+2] Docker: [quibble-bullseye] Drop Python 2 from images [integration/config] - 10https://gerrit.wikimedia.org/r/1259122 (owner: 10Jforrester) [21:00:12] (03Merged) 10jenkins-bot: Docker: [quibble-bullseye] Drop Python 2 from images [integration/config] - 10https://gerrit.wikimedia.org/r/1259122 (owner: 10Jforrester) [21:00:46] !log Docker: [quibble-bullseye] Drop Python 2 from images [21:00:47] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:03:15] (03CR) 10Majavah: "i might be missing something but that seems like an outdated comment and the image isn't actually installing gcc?" [integration/config] - 10https://gerrit.wikimedia.org/r/1259122 (owner: 10Jforrester) [21:04:17] (03CR) 10Jforrester: [C:03+2] "build-essential pulls in dpkg-dev, gcc, g++, libc6-dev, and make, I believe." [integration/config] - 10https://gerrit.wikimedia.org/r/1259122 (owner: 10Jforrester) [21:12:36] (03PS1) 10Jforrester: jjb: Switch jobs to Quibble images without Python 2 [integration/config] - 10https://gerrit.wikimedia.org/r/1262291 [21:27:34] (03CR) 10Jforrester: [C:03+2] jjb: Switch jobs to Quibble images without Python 2 [integration/config] - 10https://gerrit.wikimedia.org/r/1262291 (owner: 10Jforrester) [21:29:23] (03Merged) 10jenkins-bot: jjb: Switch jobs to Quibble images without Python 2 [integration/config] - 10https://gerrit.wikimedia.org/r/1262291 (owner: 10Jforrester) [22:34:28] 10Phabricator: Tasks can be set to the 'Lowest' priority using a priority-grouped workboard, even though that priority is set as disabled within Phab config - https://phabricator.wikimedia.org/T421554 (10A_smart_kitten) 03NEW