[06:24:41] just finished m5 switchover [06:56:15] Can someone double check that I picked the right proxy? (compare dbproxy1023.yaml with dbproxy1025) https://gerrit.wikimedia.org/r/c/operations/dns/+/980087/ [08:24:29] looking now [08:24:40] thank you <3 [08:26:28] LGTM, +1 [08:26:38] Thank you, I am going to proceed and failover m2 then! [11:55:55] I've migrated this repo to gitlab: https://gitlab.wikimedia.org/repos/sre/mediabackups [12:25:58] arnaudb: marostegui are you doing anything on s5? https://wikitech.wikimedia.org/wiki/Map_of_database_maintenance says not but double checking [12:26:09] not me [12:26:15] about to make a couple of large wikis there to read new for pagelinks [12:27:14] fun fact: pagelinks is roughly 30% of s5 🤦 [12:47:54] not me either! [14:06:42] afk for a bit but after switching large wikis of s5 to read new there is an uptick on slow queries there, it's so far seems to be the query planner being stupid (tbf, the queries are really complex) I'm optimizing dbstoree of s5 to see if that has any impact. If anything goes sideways when I'm afk (very unlikely), just revert my mw-config patch [14:07:56] Amir1: can you link it before you go? [14:09:11] I think it is this: https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/980370/1/wmf-config/InitialiseSettings.php [14:10:02] yup that's it [14:11:54] funnily enough, if the pagelinks gets cleaned up there, it'll drop 100GB-ish, making the whole section around 459GB. If we squeeze it a bit harder we can fit the whole thing into memory [16:53:46] as usual, it normally gets a little worse until it gets better :-D [16:56:58] <_joe_> hi, before upgrading any datastore that uses openssl to bookworm, please consider https://phabricator.wikimedia.org/T352744 [16:57:20] <_joe_> actually, you should probably add there the systems I forgot that use openssl directly either as servers or clients [16:59:25] <_joe_> like cassandra for instance, and I think the rewrite handler from swift to thumbor also uses openssl as it doesn't use envoy in the middle [17:01:21] it's using python's http(s) library, IIRC [17:01:36] * Emperor adds another item to the "why I hate the rewrite middleware" list