[06:16:49] <elukey>	 goog morning folks
[06:17:00] <elukey>	 I am going to finish the kafka main-eqiad rebalance
[06:22:05] <_joe_>	 elukey: ack
[07:46:53] <elukey>	 I am trying to make some sense of the current Kafka main mirror maker graphs
[07:47:15] <elukey>	 for example, main-eqiad, past 70d
[07:47:16] <elukey>	 https://grafana.wikimedia.org/d/000000521/kafka-mirrormaker?orgId=1&var-datasource=eqiad%20prometheus%2Fops&var-lag_datasource=codfw%20prometheus%2Fops&var-mirror_name=main-codfw_to_main-eqiad&refresh=5m&from=now-70d&to=now
[07:47:32] <elukey>	 same thing for main-codfw
[07:47:33] <elukey>	 https://grafana.wikimedia.org/d/000000521/kafka-mirrormaker?orgId=1&refresh=5m&from=now-70d&to=now&var-datasource=codfw%20prometheus%2Fops&var-lag_datasource=codfw%20prometheus%2Fops&var-mirror_name=main-eqiad_to_main-codfw
[07:47:57] <elukey>	 there is a pattern change when we switched over, but I'd have expected something different after the switchback
[07:48:18] <elukey>	 topics like codfw.resource-purge seem not used
[07:48:20] <elukey>	 https://grafana.wikimedia.org/d/000000234/kafka-by-topic?orgId=1&refresh=5m&from=now-70d&to=now&var-datasource=codfw%20prometheus%2Fops&var-kafka_cluster=main-codfw&var-kafka_broker=All&var-topic=codfw.resource-purge
[07:48:31] <elukey>	 did we change something on the change-prop/mw side?
[07:48:44] <elukey>	 (or it is probably me not getting it on monday)
[07:52:37] <elukey>	 of course self-answered, that was the switch-back
[07:53:27] <elukey>	 I'll check why we have those warnings for mirror maker in icinga then
[07:54:43] * elukey gets another coffee
[08:27:50] <elukey>	 created https://gerrit.wikimedia.org/r/c/operations/puppet/+/731332 for the mirror maker alerts
[10:42:50] <elukey>	 going afk for lunch, the rebalance is stopped for the moment, but very few topics left :)
[13:37:12] <wikibugs>	 10serviceops: Allow sending traffic to php 7.2 or 7.4 selectively in the apache configuration for MediaWiki - https://phabricator.wikimedia.org/T293451 (10jijiki) Which brings again our usual issue, to cache slot or not to cache slot? If we don't want to slot the caches, we could consider following what we are p...
[13:38:42] <wikibugs>	 10serviceops: Migrate WMF Production from PHP 7.2 to PHP 7.4 - https://phabricator.wikimedia.org/T271736 (10jijiki)
[14:07:33] <elukey>	 moved for kafka main-eqiad completed, it is not a perfect balance (like for codfw) between brokers but it is good enough for the moment in my opinion
[14:08:36] <wikibugs>	 10serviceops, 10Performance-Team: Investigate performance degradation at high concurrencies in php-fpm - https://phabricator.wikimedia.org/T293630 (10jijiki)
[14:09:05] <wikibugs>	 10serviceops, 10Performance-Team: Investigate performance degradation at high concurrencies in php-fpm - https://phabricator.wikimedia.org/T293630 (10jijiki)
[14:09:11] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Patch-For-Review, 10Performance-Team (Radar): Benchmark performance of MediaWiki on k8s - https://phabricator.wikimedia.org/T280497 (10jijiki)
[15:36:27] <wikibugs>	 10serviceops, 10Prod-Kubernetes, 10Wikidata, 10Wikidata-Query-Service, and 2 others: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes - https://phabricator.wikimedia.org/T293063 (10MPhamWMF)
[16:23:07] <wikibugs>	 10serviceops: Upgrade mc* and mc-gp* hosts to Debian Bullseye - https://phabricator.wikimedia.org/T293216 (10jijiki)
[16:23:59] <wikibugs>	 10serviceops: Upgrade mc* and mc-gp* hosts to Debian Bullseye - https://phabricator.wikimedia.org/T293216 (10jijiki)
[18:17:26] <wikibugs>	 10serviceops, 10Performance-Team: Investigate performance degradation at high concurrencies in php-fpm - https://phabricator.wikimedia.org/T293630 (10dpifke) a:03aaron
[18:44:43] <wikibugs>	 10serviceops, 10Performance-Team, 10SRE, 10MW-1.36-notes, and 3 others: Enable "/*/mw-with-onhost-tier/" route for MediaWiki where safe - https://phabricator.wikimedia.org/T264604 (10aaron)
[21:12:50] <wikibugs>	 10serviceops, 10MediaWiki-extensions-TranslationNotifications, 10SRE: Fatal error: Uncaught Error: Class 'MediaWiki\MediaWikiServices' not found - mediawiki_job_translationnotifications - https://phabricator.wikimedia.org/T293702 (10Dzahn)
[21:14:10] <wikibugs>	 10serviceops, 10MediaWiki-extensions-TranslationNotifications, 10SRE: Fatal error: Uncaught Error: Class 'MediaWiki\MediaWikiServices' not found - mediawiki_job_translationnotifications - https://phabricator.wikimedia.org/T293702 (10Dzahn) 21:09 < Reedy> https://github.com/wikimedia/mediawiki-extensions-Tran...