[06:34:54] <wikibugs>	 10serviceops, 10serviceops-collab, 10vrts: vrts - spamassassin icinga alerts - https://phabricator.wikimedia.org/T316903 (10SLyngshede-WMF) It seems weird if the script didn't also sometimes fail under cron. Perhaps we just didn't notice.  But yes, maybe we just need to modify the script a bit and have et se...
[07:33:38] <wikibugs>	 10serviceops, 10serviceops-collab, 10vrts, 10Patch-For-Review: vrts - spamassassin icinga alerts - https://phabricator.wikimedia.org/T316903 (10MoritzMuehlenhoff) >>! In T316903#8207698, @SLyngshede-WMF wrote: > It seems weird if the script didn't also sometimes fail under cron. Perhaps we just didn't noti...
[08:47:38] <wikibugs>	 10serviceops: Put mw14[57-98] in production - https://phabricator.wikimedia.org/T313327 (10ayounsi)
[08:47:55] <wikibugs>	 10serviceops, 10DC-Ops, 10SRE, 10ops-eqiad: Q4: (Need By: TBD) rack/setup/install mw14[57-98] - https://phabricator.wikimedia.org/T306121 (10ayounsi) 05Resolved→03Open @Cmjohnson @Dzahn   The following hosts are alerting in https://netbox.wikimedia.org/extras/reports/puppetdb.PhysicalHosts/ ` mw1459 (W...
[12:10:30] <wikibugs>	 10serviceops, 10DBA, 10Patch-For-Review, 10Performance-Team (Radar): Update wgLBFactoryConf for x2 to register only the local primary - https://phabricator.wikimedia.org/T316482 (10CDanis) 05Open→03Resolved
[13:30:36] <wikibugs>	 10serviceops, 10Data-Persistence-Backup, 10serviceops-collab, 10GitLab (Infrastructure), 10User-brennen: Setup partial backups for GitLab - https://phabricator.wikimedia.org/T316935 (10Jelto)
[13:35:44] <wikibugs>	 10serviceops, 10Data-Persistence-Backup, 10serviceops-collab, 10GitLab (Infrastructure), 10User-brennen: Setup partial backups for GitLab - https://phabricator.wikimedia.org/T316935 (10Jelto) p:05Triage→03Medium
[15:57:45] <wikibugs>	 10serviceops, 10Dumps-Generation, 10Patch-For-Review, 10Performance-Team (Radar): Migrate WMF production from PHP 7.2 to PHP 7.4 - https://phabricator.wikimedia.org/T271736 (10ArielGlenn) Given other delays, I am now planning to move the snapshot hosts to php7.4 when the current dump run completes, about t...
[16:38:31] <wikibugs>	 10serviceops, 10Prod-Kubernetes, 10Kubernetes: High API server request latencies (LIST) - https://phabricator.wikimedia.org/T303184 (10JMeybohm) Started looking into this again because we saw elevated LIST latencies again starting from 2022-08-31 ~22:20Z in codfw and 2022-09-01 02:55Z in eqiad.  There are mu...
[19:01:43] <wikibugs>	 10serviceops, 10MW-on-K8s, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹️): Make scap deploy to kubernetes together with the legacy systems - https://phabricator.wikimedia.org/T299648 (10dancy) I ran this successfully today: `~dancy/devel-scap sync-world -Dbuild_mw_container_image:True -Dde...