[09:45:30] <wikibugs>	 10serviceops, 10Observability-Logging, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Kubernetes logs (container stderr,strout) do not show up in Elasticsearch/Kibana - https://phabricator.wikimedia.org/T289766 (10JMeybohm) p:05High→03Low I've added a prometheus alert that fires if mmkubernetes...
[09:45:36] <wikibugs>	 10serviceops, 10Observability-Logging, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Kubernetes logs (container stderr,strout) do not show up in Elasticsearch/Kibana - https://phabricator.wikimedia.org/T289766 (10JMeybohm) a:05JMeybohm→03None
[11:23:13] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE-swift-storage, 10Shellbox: Support large files in Shellbox - https://phabricator.wikimedia.org/T292322 (10Joe) For the record, I'm taking care of this release, and given I am annoyed at how we manage image versions for shellbox, I'm also slightly modifying the procedure. I'...
[12:28:49] <wikibugs>	 10serviceops, 10SRE, 10Kubernetes, 10Patch-For-Review: Migrate to helm v3 - https://phabricator.wikimedia.org/T251305 (10Jelto)
[14:17:27] <wikibugs>	 10serviceops, 10Maps, 10Product-Infrastructure-Team-Backlog, 10Patch-For-Review, 10User-jijiki: Disable unused services on maps nodes - https://phabricator.wikimedia.org/T298246 (10Jgiannelos)
[14:17:38] <wikibugs>	 10serviceops, 10Maps, 10Product-Infrastructure-Team-Backlog, 10Patch-For-Review, 10User-jijiki: Disable unused services on maps nodes - https://phabricator.wikimedia.org/T298246 (10Jgiannelos)
[14:34:16] <wikibugs>	 10serviceops, 10Maps, 10Product-Infrastructure-Team-Backlog, 10Patch-For-Review, 10User-jijiki: Disable unused services on maps nodes - https://phabricator.wikimedia.org/T298246 (10Jgiannelos)
[17:14:59] <wikibugs>	 10serviceops, 10Maps, 10Product-Infrastructure-Team-Backlog, 10Patch-For-Review, 10User-jijiki: Disable unused services on maps nodes - https://phabricator.wikimedia.org/T298246 (10Jgiannelos)
[18:01:29] <wikibugs>	 10serviceops, 10Release-Engineering-Team, 10Scap: Deploy Scap version 4.1.1 - https://phabricator.wikimedia.org/T298986 (10dancy)
[18:12:07] <_joe_>	 dancy: I'll take a look tomorrow morning
[18:25:15] <wikibugs>	 10serviceops, 10Release-Engineering-Team, 10Scap: Deploy Scap version 4.1.1 - https://phabricator.wikimedia.org/T298986 (10Joe) a:03Joe
[18:49:08] <wikibugs>	 10serviceops, 10SRE, 10Patch-For-Review: Remove mediawiki::packages::fonts from non thumbor servers - https://phabricator.wikimedia.org/T294378 (10Dzahn) 05Resolved→03Open reopening  While we have purged all the font packages that were specifically in the config, these had also pulled in more font packag...
[18:50:13] <wikibugs>	 10serviceops, 10SRE, 10Patch-For-Review: Remove mediawiki::packages::fonts from non thumbor servers - https://phabricator.wikimedia.org/T294378 (10Dzahn) example:  a font that was in our list, fonts-alee is properly gone: https://debmonitor.wikimedia.org/packages/fonts-alee (except on thumbor, expected)  a f...
[20:59:31] <wikibugs>	 10serviceops, 10SRE, 10Patch-For-Review: Remove mediawiki::packages::fonts from non thumbor servers - https://phabricator.wikimedia.org/T294378 (10Dzahn) Doing the `apt-get autoremove` and accepting what it suggests also removes python packages in addition to font packages.  When running puppet afterwards th...
[23:38:00] <wikibugs>	 10serviceops, 10SRE, 10Wikimedia-production-error: PHP7 corruption reports in 2020-2022 (Call on wrong object, etc.) - https://phabricator.wikimedia.org/T245183 (10Krinkle)
[23:38:23] <wikibugs>	 10serviceops, 10SRE, 10Wikimedia-production-error: PHP7 corruption reports in 2020-2022 (Call on wrong object, etc.) - https://phabricator.wikimedia.org/T245183 (10Krinkle)
[23:52:56] <wikibugs>	 10serviceops, 10SRE, 10User-Ladsgroup, 10Wikimedia-production-error: wtp* hosts: Out of memory (allocated 39845888) (tried to allocate 131072 bytes) in OutputHandler.php - https://phabricator.wikimedia.org/T297517 (10Krinkle) 05Open→03Resolved a:03Ladsgroup The immediate issue appears resolved, as ev...
[23:59:55] <wikibugs>	 10serviceops, 10SRE, 10Wikimedia-production-error: PHP7 corruption reports in 2020-2022 (Call on wrong object, etc.) - https://phabricator.wikimedia.org/T245183 (10Krinkle) From {T297316}  ` from /srv/mediawiki/php-1.38.0-wmf.9/includes/libs/objectcache/MemcachedPeclBagOStuff.php(341) #0 /srv/mediawiki/php-1...