[08:38:53] hnowlan: Hey, I gathered another round of URLs to purge from the weekend so we can minimize the long tail of errors. Error rate is low so its better to focus on the switchover fixes instead of the restbase errors. [15:43:05] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: prometheus-apache-exporter in buster does not support -log.format json - https://phabricator.wikimedia.org/T283861 (10Clement_Goubert) 05Open→03In progress p:05Triage→03Low a:03Clement_Goubert [15:52:07] 10serviceops, 10Infrastructure-Foundations, 10Puppet-Core, 10SRE, and 5 others: Migrate roles to puppet7 - https://phabricator.wikimedia.org/T349619 (10MoritzMuehlenhoff) [17:23:58] 10serviceops: WikiKube runc upgrades - https://phabricator.wikimedia.org/T356661 (10akosiaris) [18:22:14] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1386.eqiad.wmnet with OS bullseye [18:22:52] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1388.eqiad.wmnet with OS bullseye [18:23:18] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1390.eqiad.wmnet with OS bullseye [18:24:43] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1392.eqiad.wmnet with OS bullseye [18:44:30] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1394.eqiad.wmnet with OS bullseye [18:45:25] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1396.eqiad.wmnet with OS bullseye [18:46:03] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1408.eqiad.wmnet with OS bullseye [19:42:08] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1386.eqiad.wmnet with OS bullseye executed with errors: - mw1... [19:43:28] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1388.eqiad.wmnet with OS bullseye executed with errors: - mw1... [19:43:38] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1390.eqiad.wmnet with OS bullseye executed with errors: - mw1... [19:44:34] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1392.eqiad.wmnet with OS bullseye executed with errors: - mw1... [19:47:30] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw1386.eqiad.wmnet with OS bullseye [19:58:26] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by kamila@cumin1002 for host mw2317.codfw.wmnet with OS bullseye [20:04:34] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1394.eqiad.wmnet with OS bullseye executed with errors: - mw1... [20:05:59] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1408.eqiad.wmnet with OS bullseye executed with errors: - mw1... [20:06:31] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw1396.eqiad.wmnet with OS bullseye executed with errors: - mw1... [21:16:38] 10serviceops, 10MW-on-K8s, 10Patch-For-Review: Move servers from the appserver/api cluster to kubernetes - https://phabricator.wikimedia.org/T351074 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by kamila@cumin1002 for host mw2317.codfw.wmnet with OS bullseye executed with errors: - mw2... [21:17:16] 10serviceops, 10Content-Transform-Team-WIP, 10Maintenance-Worktype, 10Wikimedia-production-error: TypeError: Argument 4 passed to Wikimedia\Parsoid\Utils\Title::__construct() must be of the type string, null given, called in /srv/mediawiki/php-1.42.0-wmf.15/vendor/... - https://phabricator.wikimedia.org/T356024 [21:45:18] 10serviceops, 10Infrastructure-Foundations: Debian installer waits for input for network config during host reimage - https://phabricator.wikimedia.org/T356709 (10kamila) [23:24:41] 10serviceops, 10Maps, 10Patch-For-Review: Repool maps primaries in Kartotherian - https://phabricator.wikimedia.org/T355892 (10Jgiannelos) This change looks like is causing some issue in production traffic. From apps team: ` We seem to be getting intermittent 404s for certain urls, e.g. https://maps.wikimedi...