[08:56:25] 06serviceops, 10Similarusers, 13Patch-For-Review: Remove similar-users service from k8s - https://phabricator.wikimedia.org/T345274#9834147 (10kostajh) a:03hnowlan [10:31:18] 06serviceops, 10MoveComms-Support, 10MW-on-K8s, 06SRE, and 2 others: Move 100% of external traffic to Kubernetes (excluding Votewiki) - https://phabricator.wikimedia.org/T362323#9834404 (10Clement_Goubert) As far as mediawiki calling itself goes (I see it was removed from the task description, but it is te... [11:05:18] 06serviceops, 10MW-on-K8s, 06Quality-and-Test-Engineering-Team, 06SRE, 13Patch-For-Review: Move testwiki over to mw-on-k8s - https://phabricator.wikimedia.org/T355534#9834501 (10Clement_Goubert) 05Open→03In progress [11:12:22] 06serviceops, 10Phabricator, 13Patch-For-Review, 07Technical-Debt: Investigate / remove custom downstream changes to PhabricatorClientRateLimit.php - https://phabricator.wikimedia.org/T364839#9834525 (10Aklapper) [11:14:25] 06serviceops, 10MW-on-K8s, 10Observability-Metrics, 13Patch-For-Review, 10SRE Observability (FY2023/2024-Q4): Create a per-release deployment of statsd-exporter for mw-on-k8s - https://phabricator.wikimedia.org/T365265#9834539 (10Clement_Goubert) >>! In T365265#9815590, @JMeybohm wrote: > I might be miss... [11:44:54] 06serviceops, 10MW-on-K8s, 06Quality-and-Test-Engineering-Team, 06SRE, 13Patch-For-Review: Move testwiki over to mw-on-k8s - https://phabricator.wikimedia.org/T355534#9834567 (10Clement_Goubert) 05In progress→03Resolved `testwiki` and `testcommonswiki` are now moved over to #mw-on-k8s [11:55:36] 06serviceops, 06Infrastructure-Foundations, 06Release-Engineering-Team: Deprecate buster-backports - https://phabricator.wikimedia.org/T362518#9834634 (10Clement_Goubert) Just to be completely sure before deleting >>! In T362518#9805179, @dancy wrote: >>>! In T362518#9803705, @Clement_Goubert wrote: >> Stil... [12:55:33] o/ not sure if it is supposed to or not, but pybal is complaining that parse1002 is listed as down for thumbor [12:55:39] since days ago [12:56:16] and there are also some alerts about thumbor containers being OOM killed :( [13:05:18] thanks for the heads-up [13:05:19] OOMs are unfortunately expected [13:06:14] looks like parse1002 has long been unwell https://phabricator.wikimedia.org/T363086#9776549 [13:17:40] depooled for now anyway [14:00:34] 06serviceops, 06Content-Transform-Team, 07Essential-Work, 13Patch-For-Review, 07Wikimedia-Incident: Maps Unavailability due to thanos-swift cfssl rollout (14 Aug 2023) - https://phabricator.wikimedia.org/T344324#9835018 (10elukey) Tegola on staging and production is now connecting to Thanos via the enov... [15:28:54] 06serviceops, 13Patch-For-Review: Upgrade memcache and memcached gutter pools to Bookworm - https://phabricator.wikimedia.org/T352891#9835382 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jiji@cumin2002 for host mc2049.codfw.wmnet with OS bookworm [15:57:12] 06serviceops, 06SRE, 10Data Products (Data Products Sprint 14), 13Patch-For-Review, 07Service-deployment-requests: Commons Impact Metrics AQS 2.0 Deployment to Staging and Production - https://phabricator.wikimedia.org/T361835#9835532 (10WDoranWMF) [15:58:18] 06serviceops, 13Patch-For-Review: Upgrade memcache and memcached gutter pools to Bookworm - https://phabricator.wikimedia.org/T352891#9835572 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jiji@cumin2002 for host mc2049.codfw.wmnet with OS bookworm executed with errors: - mc2049 (**FAIL... [15:59:19] 06serviceops, 13Patch-For-Review: Upgrade memcache and memcached gutter pools to Bookworm - https://phabricator.wikimedia.org/T352891#9835593 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by jiji@cumin2002 for host mc2049.codfw.wmnet with OS bookworm [16:36:16] 06serviceops, 13Patch-For-Review: Upgrade memcache and memcached gutter pools to Bookworm - https://phabricator.wikimedia.org/T352891#9835695 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by jiji@cumin2002 for host mc2049.codfw.wmnet with OS bookworm executed with errors: - mc2049 (**FAIL... [19:52:19] 06serviceops, 06Growth-Team, 10Growth-Team-Filtering, 10StructuredDiscussions, and 2 others: [{exception_id}] {exception_url} Flow\Exception\FlowException from line 397 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Flow/includes/Block/TopicListBlock.php: T... - https://phabricator.wikimedia.org/T211798#9836119