[10:02:28] <wikibugs>	 06serviceops, 10ConfirmEdit (CAPTCHA extension), 05WE4.2 Bot detection (WE4.2 hCaptcha editing trial): Increase in memcached get misses from mc1047 and mc1042 - https://phabricator.wikimedia.org/T412947#11511454 (10jijiki) >>! In T412947#11480759, @kostajh wrote: >> Since 2 Dec 2025, I have observed an odd i...
[10:46:05] <wikibugs>	 06serviceops, 06SRE Observability, 06MediaWiki-Platform-Team (Kanban Board): Improve MediaWiki periodic job alerts - https://phabricator.wikimedia.org/T412799#11511638 (10Clement_Goubert) 05In progress→03Resolved
[11:09:08] <wikibugs>	 06serviceops, 06MediaWiki-Platform-Team (Kanban Board), 13Patch-For-Review: Create namespace and support files for redioscope - https://phabricator.wikimedia.org/T413999#11511760 (10Clement_Goubert)
[11:09:21] <wikibugs>	 06serviceops, 06MediaWiki-Platform-Team (Kanban Board), 13Patch-For-Review: Create namespace and support files for redioscope - https://phabricator.wikimedia.org/T413999#11511761 (10Clement_Goubert) 05Open→03In progress
[11:37:16] <wikibugs>	 06serviceops, 10Ceph, 06Data-Persistence, 10SRE-swift-storage: Onboard the Docker Registry to apus - https://phabricator.wikimedia.org/T394476#11511867 (10elukey) Adding an IRC conversation between me and Matthew about DC replication:  ` <elukey> Emperor: o/ morning :) I have been testing the apus S3 api v...
[12:03:29] <wikibugs>	 06serviceops, 06MediaWiki-Platform-Team (Kanban Board), 13Patch-For-Review: Create namespace and support files for redioscope - https://phabricator.wikimedia.org/T413999#11511959 (10Clement_Goubert) 05In progress→03Resolved
[12:34:57] <wikibugs>	 06serviceops, 07Epic, 07OKR-Work: Use external_services for redis network policies - https://phabricator.wikimedia.org/T414333 (10Clement_Goubert) 03NEW
[12:35:05] <wikibugs>	 06serviceops, 07Epic, 07OKR-Work: Use external_services for redis network policies - https://phabricator.wikimedia.org/T414333#11512040 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Medium
[12:56:28] <wikibugs>	 06serviceops, 10Scap, 06SRE, 07Datacenter-Switchover: Add scap lock/unlock steps to sre.switchdc.mediawiki cookbook - https://phabricator.wikimedia.org/T330996#11512111 (10Clement_Goubert) a:03Blake
[14:41:16] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker1335-59 - https://phabricator.wikimedia.org/T408752#11512481 (10Jclark-ctr)
[14:42:48] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker1335-59 - https://phabricator.wikimedia.org/T408752#11512484 (10Jclark-ctr) All servers racked and netbox configured holding for update on 4ft power cables
[15:06:52] <wikibugs>	 06serviceops, 06Traffic, 07Epic, 05FY2025-26 KR 5.1, 07OKR-Work: Log rate limits from rest-gateway in webrequests - https://phabricator.wikimedia.org/T414349 (10Clement_Goubert) 03NEW
[15:07:22] <wikibugs>	 06serviceops, 06Traffic, 07Epic, 05FY2025-26 KR 5.1, 07OKR-Work: Log rate limits from rest-gateway in webrequests - https://phabricator.wikimedia.org/T414349#11512639 (10Clement_Goubert) p:05Triage→03Medium
[17:44:27] <wikibugs>	 06serviceops: Proof of Concept: SquareOne Dashboards - https://phabricator.wikimedia.org/T411202#11513446 (10jijiki)
[18:53:42] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513719 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vriley@cumin1003 for host wikikube-worker1366.eqiad.wmnet with OS trixie
[18:56:23] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513724 (10VRiley-WMF)
[19:07:02] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513775 (10VRiley-WMF)
[19:10:12] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513789 (10VRiley-WMF)
[19:14:38] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513806 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vriley@cumin1003 for host wikikube-worker1367.eqiad.wmnet with OS trixie
[19:26:22] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513866 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vriley@cumin1003 for host wikikube-worker1366.eqiad.wmnet with OS trixie completed: - wikikub...
[19:26:26] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513867 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vriley@cumin1003 for host wikikube-worker1366.eqiad.wmnet with OS trixie executed with errors...
[19:35:02] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-codfw, 06SRE: Q2:rack/setup/install wikikube-worker2332-56 - https://phabricator.wikimedia.org/T408757#11513905 (10Jhancock.wm) @Clement_Goubert the servers landed last week. Gonna start unpacking them tomorrow or wednesday. Are they able to be racked int he E/F?
[19:36:11] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513913 (10VRiley-WMF)
[19:49:50] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513959 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vriley@cumin1003 for host wikikube-worker1367.eqiad.wmnet with OS trixie completed: - wikikub...
[19:50:23] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11513980 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vriley@cumin1003 for host wikikube-worker1368.eqiad.wmnet with OS trixie
[20:08:30] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11514017 (10VRiley-WMF)
[20:10:48] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11514029 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by vriley@cumin1003 for host wikikube-worker1369.eqiad.wmnet with OS trixie
[20:29:28] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11514105 (10VRiley-WMF)
[20:31:10] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11514108 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vriley@cumin1003 for host wikikube-worker1368.eqiad.wmnet with OS trixie completed: - wikikub...
[20:45:19] <wikibugs>	 06serviceops, 06DC-Ops, 10ops-eqiad, 06SRE: Q2:rack/setup/install wikikube-worker refresh - https://phabricator.wikimedia.org/T408760#11514191 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by vriley@cumin1003 for host wikikube-worker1369.eqiad.wmnet with OS trixie completed: - wikikub...