[02:51:51] 10serviceops, 10Performance-Team, 10SRE, 10SRE-swift-storage, and 2 others: Progressive Multi-DC roll out - https://phabricator.wikimedia.org/T279664 (10tstarling) [03:01:48] 10serviceops, 10Performance-Team, 10SRE, 10SRE-swift-storage, and 2 others: Progressive Multi-DC roll out - https://phabricator.wikimedia.org/T279664 (10tstarling) Stage 3 (traffic percentage) is useful for capacity modelling, but it's not expected to be optimal for data store consistency, since the stabil... [04:31:46] 10serviceops, 10Performance-Team, 10SRE, 10SRE-swift-storage, and 3 others: Progressive Multi-DC roll out - https://phabricator.wikimedia.org/T279664 (10tstarling) Planning for stage 3/4 capacity monitoring. > Observe cross-DC database connection rate, analyse sources In the DBPerformance logs, we see a... [05:44:54] 10serviceops, 10SRE, 10Wikimedia-Etherpad: Upgrade etherpad.wikimedia.org to (more) recent Etherpad version with more rich end-user features - https://phabricator.wikimedia.org/T316421 (10Spinster) FWIW, I filed this ticket because I was using the public https://board.net/ Etherpad instance, and saw all thos... [08:25:50] 10serviceops, 10SRE, 10Traffic: Set per-request timeout on ATS-BE - https://phabricator.wikimedia.org/T315533 (10Vgutierrez) 05Open→03Resolved [09:24:38] 10serviceops, 10Patch-For-Review: Cleanup profile::docker::engine::version - https://phabricator.wikimedia.org/T316341 (10MoritzMuehlenhoff) JFTR, there is also some code in class docker which can be removed now: With all hosts on Buster or later, the debian::codename::lt (where lt stands for "less than") cond... [09:35:12] 10serviceops, 10Patch-For-Review: Cleanup profile::docker::engine::version - https://phabricator.wikimedia.org/T316341 (10Clement_Goubert) Thanks for pointing it out, it's the next cleanup step as I wanted to do it gradually. Removing that codepath allows us to remove the hiera setting `profile::docker::engine... [09:40:41] 10serviceops, 10Phabricator, 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹ī¸): Move Phabricator to new hardware generation - https://phabricator.wikimedia.org/T280597 (10hashar) [09:40:46] 10serviceops, 10Phabricator, 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹ī¸): move phabricator to new hardware generation - https://phabricator.wikimedia.org/T280597 (10hashar) [09:46:08] 10serviceops, 10Patch-For-Review: Cleanup profile::docker::engine::version - https://phabricator.wikimedia.org/T316341 (10MoritzMuehlenhoff) >>! In T316341#8197781, @Clement_Goubert wrote: > I wonder if we shouldn't set the `package_name` default for the `docker` class to `docker.io` in the class definition... [09:46:52] FYI, I'm temporarily switching kubestagetcd2001 to DRBD to move it to a different Ganeti node for reimage [09:53:13] 10serviceops: Cleanup profile::docker::engine::packagename - https://phabricator.wikimedia.org/T316639 (10Clement_Goubert) [09:53:47] 10serviceops, 10Patch-For-Review: Cleanup profile::docker::engine::version - https://phabricator.wikimedia.org/T316341 (10Clement_Goubert) 05Open→03Resolved [09:54:10] 10serviceops, 10Patch-For-Review: Cleanup profile::docker::engine::version - https://phabricator.wikimedia.org/T316341 (10Clement_Goubert) Cleanup continued in https://phabricator.wikimedia.org/T316639 [10:21:10] kubestagetcd2001 is back to plain disk storage [13:14:51] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install new eqiad memcached hosts - https://phabricator.wikimedia.org/T313963 (10Jclark-ctr) [13:14:56] 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install new eqiad memcached hosts - https://phabricator.wikimedia.org/T313963 (10Jclark-ctr) a:05Jclark-ctr→03Cmjohnson mc-wf1001 B8 U25 Port 27 Cableid 3286 mc-wf1002 D8 U26 Port 30 Cableid 2013339101803 [13:51:13] 10serviceops: Cleanup profile::docker::engine::packagename - https://phabricator.wikimedia.org/T316639 (10Clement_Goubert) p:05Triage→03Low [15:18:26] <_joe_> claime: jelto is reporting puppet failures on all the gitlab runners [15:18:33] <_joe_> related to docker parameters [15:20:40] claime: profile::gitlab::runner still passes the version parameter (which is now removed) to the docker class, simply removing it from the profile will fix it [15:23:05] I'm here sorry for the lag [15:23:14] Fixing [15:26:41] I'm finding a docker version setting in profile::ci::docker [15:27:05] I'll do a first change to remove from gitlab runner, and a second for profile::ci::docker [15:30:35] I'm sorry jelto [15:33:58] Patch pushed https://gerrit.wikimedia.org/r/c/operations/puppet/+/828041 [15:39:49] <_joe_> shipit! [15:40:20] shipingit [15:43:10] jelto, puppet runs ok on gitlab runners [15:43:24] ci::docker hosts will fail too, prepping patch [15:52:10] Hmm actually contint are not failing due to docker setup [15:52:22] They're failing due to this https://gerrit.wikimedia.org/r/c/operations/puppet/+/825755 [15:52:43] Well, that patch should fix why they're failing, actually [15:52:52] claime: sorry I was in a meeting. Thanks for fixing the runners :) I can confirm puppet is happy again now [15:53:18] jelto: no worries, it was my fault they were broken in the first place, it's only just I fix them [17:20:46] 10serviceops, 10Prod-Kubernetes, 10Wikidata, 10Wikidata-Query-Service, and 2 others: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes - https://phabricator.wikimedia.org/T293063 (10JMeybohm) [17:24:39] 10serviceops, 10Prod-Kubernetes, 10Wikidata, 10Wikidata-Query-Service, and 2 others: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes - https://phabricator.wikimedia.org/T293063 (10JMeybohm) [21:07:41] 10serviceops, 10MediaWiki-Releasing, 10PHP 7.2 support, 10PHP 7.3 support, 10Patch-For-Review: Drop PHP 7.2 & 7.3 support from MediaWiki master branch, once Wikimedia production is on 7.4 - https://phabricator.wikimedia.org/T261872 (10Jdforrester-WMF) [21:08:20] 10serviceops, 10MediaWiki-Releasing, 10PHP 7.2 support, 10PHP 7.3 support, 10Patch-For-Review: Drop PHP 7.2 & 7.3 support from MediaWiki master branch, once Wikimedia production is on 7.4 - https://phabricator.wikimedia.org/T261872 (10Jdforrester-WMF)