[07:43:50] 10serviceops, 10SRE-swift-storage: Corrupted helm charts in Swift that are not properly showing up in chartmuseum - https://phabricator.wikimedia.org/T283147 (10JMeybohm) [07:48:05] 10serviceops, 10SRE-swift-storage: Corrupted helm charts in Swift that are not properly showing up in chartmuseum - https://phabricator.wikimedia.org/T283147 (10JMeybohm) > On a hunch, I went and looked into swift directly for those charts. After doing the curl dance[1] to get a token from swift the following... [10:06:21] 10serviceops, 10SRE-swift-storage: Corrupted helm charts in Swift that are not properly showing up in chartmuseum - https://phabricator.wikimedia.org/T283147 (10JMeybohm) >>! In T283147#7097842, @akosiaris wrote: > This is not good, it could baldy affect deploys. Setting to `High`. > > I am wondering whether... [11:04:19] 10serviceops, 10Maps, 10Packaging, 10Product-Infrastructure-Team-Backlog, 10SRE: Packaging PostGIS 3.1 for the new Maps stack - https://phabricator.wikimedia.org/T277064 (10hnowlan) postgis 3.1.1 running on maps1009 successfully. Just to note for posterity and reference, we hit some issues upgrading, w... [11:05:04] 10serviceops, 10Maps, 10Packaging, 10Product-Infrastructure-Team-Backlog, 10SRE: Packaging PostGIS 3.1 for the new Maps stack - https://phabricator.wikimedia.org/T277064 (10hnowlan) 05Open→03Resolved [11:37:58] 10serviceops, 10SRE-swift-storage: Corrupted helm charts in Swift that are not properly showing up in chartmuseum - https://phabricator.wikimedia.org/T283147 (10JMeybohm) a:03JMeybohm Both nodes run packaging timers again and the cache-interval is set to 60s. I'll keep an eye. [14:28:06] wkandek: are we going to skip the meeting today? [14:28:13] most of us are at the training [14:39:49] <_joe_> I'm ok with leaving the training and go to the meeting tbh [14:40:01] <_joe_> but I guess not everyone will be, and it's ok [14:55:14] 10serviceops, 10SRE: ifup@eno1.service failed on some buster hosts - https://phabricator.wikimedia.org/T270220 (10jbond) 05Open→03Resolved [14:58:23] If you are in the training, then stay - probably cut the meeting short.... [15:11:39] win 2 [16:31:40] 10serviceops, 10Security-Team: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) [16:32:04] 10serviceops, 10Peek, 10Security-Team: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) [16:44:18] 10serviceops, 10Peek, 10Security-Team: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) p:05Triage→03Medium [16:46:20] 10serviceops, 10Peek, 10Security-Team: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10MoritzMuehlenhoff) It depends the plans for peek I'd say. https://gerrit.wikimedia.org/r/697628 should be enough to silence the crons. We can keep the rest of the Puppet code in the tree i... [16:58:57] 10serviceops, 10Peek, 10Security-Team: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) 05Open→03Stalled a:03sbassett @MoritzMuehlenhoff - ok, sounds good. +1 to the patch above, not sure about that Tox CI error though, seems unrelated to the patch. Would yo... [16:59:49] 10serviceops, 10Peek, 10Security-Team, 10user-sbassett: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) p:05Medium→03Low [17:03:56] 10serviceops, 10Peek, 10Security-Team, 10user-sbassett: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10MoritzMuehlenhoff) >>! In T284090#7126792, @sbassett wrote: > @MoritzMuehlenhoff - ok, sounds good. +1 to the patch above, not sure about that Tox CI error though, seem... [17:05:56] moritzm: CI error seems to saying you're missing a T before the bug number but I don't want to step on toes [17:46:39] 10serviceops, 10Peek, 10Security-Team, 10Patch-For-Review, 10user-sbassett: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10MoritzMuehlenhoff) >>! In T284090#7126752, @MoritzMuehlenhoff wrote: > It depends the plans for peek I'd say. https://gerrit.wikimedia.org/r/697628... [18:02:05] 10serviceops, 10Peek, 10Security-Team, 10Patch-For-Review, 10user-sbassett: Disable peek for the Security Team - https://phabricator.wikimedia.org/T284090 (10sbassett) Thanks, @MoritzMuehlenhoff. [18:29:08] 10serviceops, 10TimedMediaHandler-TimedText, 10Patch-For-Review: Create cron job to update query page Special:OrphanedTimedText on wikis with $wgEnableLocalTimedText = true - https://phabricator.wikimedia.org/T283975 (10Dzahn) note: if a 'periodic job' is needed these are not literally cron jobs anymore, the... [22:28:30] 10serviceops, 10Release Pipeline: Production buster-nodejs10-devel image has npm 5.x, which is not actually compatible with node 10.x - https://phabricator.wikimedia.org/T284112 (10Jdforrester-WMF) [23:11:29] 10serviceops, 10Parsoid (Tracking), 10User-jijiki: Remove parsoidJS leftovers from production - https://phabricator.wikimedia.org/T279059 (10Izno) [23:11:33] 10serviceops, 10SRE, 10Parsoid (Tracking), 10Patch-For-Review: Upgrade Parsoid servers to buster - https://phabricator.wikimedia.org/T268524 (10Izno) [23:11:37] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno) [23:12:16] 10serviceops, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services): Replace production deployment servers and update them to Buster - https://phabricator.wikimedia.org/T265963 (10Izno) [23:12:21] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno) [23:13:16] 10serviceops, 10Patch-For-Review, 10Release-Engineering-Team (Deployment services): Replace production deployment servers and update them to Buster - https://phabricator.wikimedia.org/T265963 (10Izno) [23:13:21] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno) [23:15:32] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno) [23:16:17] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno) [23:17:43] 10serviceops, 10SRE, 10Patch-For-Review, 10Release-Engineering-Team (Radar): Upgrade MediaWiki clusters to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10Izno)