[06:42:47] <marostegui>	 federico3: hold the reboots for now, I've been talking to moritzm and we'll have a new task soon
[06:42:50] <marostegui>	 Amir1: ^
[08:01:47] <federico3>	 I'm having an outage on my ISP
[08:08:07] <Emperor>	 /o\
[08:25:04] <federico3>	 marostegui: ack, holding back 
[09:43:22] <Amir1>	 thanks. I work on some reboot automation then
[10:02:27] <jynus>	 remind me, is 10.11 the next target version for mariadb?
[10:03:31] <jynus>	 or is it 11, cannot remember
[10:04:06] <jynus>	 I think it is 10.11 T391921
[10:04:07] <stashbot>	 T391921: Migrate read only external store to MariaDB 10.11 - https://phabricator.wikimedia.org/T391921
[10:04:40] <jynus>	 yeah, at least it is what it says on phab at the moment: T365805
[10:04:41] <stashbot>	 T365805: Test MariaDB 10.11 - https://phabricator.wikimedia.org/T365805
[10:21:05] <marostegui>	 jynus: 10.11 yes
[10:29:21] <jynus>	 thanks
[11:22:55] <marostegui>	 Amir1: Any objection to go with 10.11 for x3?
[11:26:52] <Amir1>	 Sure 
[11:27:00] <marostegui>	 <3
[11:27:14] <marostegui>	 I will leave them serving in s8 of course now
[11:27:17] <taavi>	 would that mean we need to upgrade some of the clouddb replicas to 10.11 before setting up replication for that?
[11:27:53] <Amir1>	 taavi: I think they will be moved from s8 replicas that will be still on 10.6
[11:28:13] <marostegui>	 But x3 needs an specific instance in wikireplicas?
[11:28:25] <Amir1>	 (We need to move a couple of replicas from s8 to x3 after switch)
[11:28:28] <marostegui>	 If that is the case, then we do need to leave them on 10.6
[11:28:40] <marostegui>	 At least the main sanitarium master needs to be on 10.65
[11:28:41] <marostegui>	 10.6
[11:28:52] <Amir1>	 Yeah. That's my idea basically 
[11:29:03] <Amir1>	 A mix of 10.11 and 10.6
[11:29:09] <marostegui>	 mmmm
[11:29:17] <marostegui>	 Then maybe it is better to migrate them whenever we are ready for everything
[11:29:21] <marostegui>	 We'd not be gaining much now
[11:30:04] <Amir1>	 Sure
[12:06:53] <jynus>	 m2 backups are taking too much time (7h 17m) :-(
[14:14:39] <federico3>	 Amir1: what reboot automation are you working on?
[14:19:14] <Amir1>	 pc and ms hosts
[14:19:25] <Amir1>	 the depool is different for them
[14:21:18] <federico3>	 please let me know if there's anything I can do to integrate it into the existing MRs for the auto_schema and scripts repositories
[16:08:35] <Emperor>	 Anyone up for a cookbook change review, please? This make no functional change, it's just splitting out a function I want to use for other swift cookbooks (and I've checked that the cookbook I took it from still works) https://gerrit.wikimedia.org/r/c/operations/cookbooks/+/1145236
[16:12:00] <federico3>	 I can do it
[16:23:32] <Emperor>	 thanks
[16:51:14] <federico3>	 Emperor: want me to test it or something?
[18:07:19] <Emperor>	 federico3: thanks, but I already did (see comment I left on the CR)
[18:07:35] <federico3>	 ok
[21:25:00] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s2 on db1182 is CRITICAL: 42.8 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1182&var-port=9104
[21:26:00] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s2 on db1182 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1182&var-port=9104