[00:02:14] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10083349 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bullseye... [00:08:10] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10083351 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bull... [00:08:14] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10083352 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bullseye... [00:12:10] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10083364 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bull... [00:12:20] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10083365 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bullseye... [07:17:42] 06serviceops, 06DBA, 06SRE, 10MediaWiki-Platform-Team (Radar), 10Sustainability (Incident Followup): In the aftermath of T370304: Brainstorming of short- and medium-term observability / quality-of-life production changes - https://phabricator.wikimedia.org/T372943#10083764 (10Marostegui) >>! In T372943#1... [08:58:09] 06serviceops, 10MW-on-K8s, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Create ValidatingAdmissionPolicies to replace mediawiki PSP - https://phabricator.wikimedia.org/T368251#10083957 (10JMeybohm) 05Open→03Resolved [09:37:00] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084070 (10JMeybohm) [09:42:01] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084082 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=6087c1fc-d7cf-44c1-8f1a-328699314b21) set by jayme@cumin1002 for 1 day, 0:00:00 on 1 host(s)... [09:47:38] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084101 (10JMeybohm) [10:12:13] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084164 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=962b8f3e-5b6f-4ce2-b907-37b801206cda) set by jayme@cumin1002 for 1 day, 0:00:00 on 1 host(s)... [10:21:06] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084178 (10JMeybohm) [11:19:20] 06serviceops, 06Infrastructure-Foundations, 10netops, 06SRE, 13Patch-For-Review: Re-IP wikikube servers in codfw row A/B moving to per-rack subnets - https://phabricator.wikimedia.org/T372878#10084337 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cgoubert@cumin1002 for host w... [11:22:31] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084340 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bull... [11:23:10] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084341 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bullseye... [11:25:03] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084342 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bull... [11:41:17] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10084372 (10JMeybohm) [12:08:51] 06serviceops, 06Infrastructure-Foundations, 10netops, 06SRE, 13Patch-For-Review: Re-IP wikikube servers in codfw row A/B moving to per-rack subnets - https://phabricator.wikimedia.org/T372878#10084514 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cgoubert@cumin1002 for host wikik... [12:22:21] 06serviceops, 06Infrastructure-Foundations, 10netops, 06SRE, 13Patch-For-Review: Re-IP wikikube servers in codfw row A/B moving to per-rack subnets - https://phabricator.wikimedia.org/T372878#10084589 (10Clement_Goubert) [12:26:01] 06serviceops, 06Infrastructure-Foundations, 10netops, 06SRE, 13Patch-For-Review: Re-IP wikikube servers in codfw row A/B moving to per-rack subnets - https://phabricator.wikimedia.org/T372878#10084622 (10Clement_Goubert) [12:26:11] 06serviceops, 06Infrastructure-Foundations, 10netops, 06SRE, 13Patch-For-Review: Re-IP wikikube servers in codfw row A/B moving to per-rack subnets - https://phabricator.wikimedia.org/T372878#10084623 (10Clement_Goubert) [13:16:06] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084841 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1298.eqiad.wmnet with OS bullseye... [13:16:27] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084843 (10Jclark-ctr) [13:17:14] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084849 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bull... [13:17:22] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084851 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bullseye... [13:17:52] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084852 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bull... [13:54:22] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10084970 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jclark@cumin1002 for host wikikube-worker1296.eqiad.wmnet with OS bullseye... [14:24:29] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10085124 (10Jclark-ctr) [14:24:51] 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 - https://phabricator.wikimedia.org/T369743#10085125 (10Jclark-ctr) 05Open→03Resolved [15:22:57] 06serviceops, 13Patch-For-Review: kafka-main200[6789] and kafka-main2010 implementation tracking - https://phabricator.wikimedia.org/T363210#10085346 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=61035eab-a385-4c69-8001-e2c6caf52946) set by jayme@cumin1002 for 2 days, 0:00:00 on 1 host(s)... [15:28:06] 06serviceops, 06Data-Platform-SRE, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate charts to Calico Network Policies - https://phabricator.wikimedia.org/T359423#10085368 (10JMeybohm) [15:37:23] 06serviceops, 06Data-Platform-SRE, 10Prod-Kubernetes, 07Kubernetes, 13Patch-For-Review: Migrate charts to Calico Network Policies - https://phabricator.wikimedia.org/T359423#10085400 (10brouberol) [19:31:53] 06serviceops, 13Patch-For-Review: Prepare WMF PHP 8.1 packages for Bullseye - https://phabricator.wikimedia.org/T372507#10086185 (10Scott_French) Alright, I think I see a way out of this: I'd overlooked that the `debian/rules` files for these packages set an explicit `INSTALL_ROOT` for `make install` in `overr... [20:33:49] 06serviceops, 13Patch-For-Review: Prepare WMF PHP 8.1 packages for Bullseye - https://phabricator.wikimedia.org/T372507#10086354 (10Scott_French) Shared objects are now present in all three packages, as well as a previously missing .ini file from wikidiff2. Verified that a local build of `docker-registry.wikim...