[09:31:51] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s8 on db2181 is CRITICAL: 351.4 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2181&var-port=9104
[09:34:50] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s8 on db2181 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2181&var-port=9104
[10:55:03] <Amir1>	 marostegui: federico3: I have a schema change on old masters of codfw. T371742 is it free for me to do it?
[10:55:03] <stashbot>	 T371742: Change page.page_links_updated to fixed-length timestamp in wmf wikis - https://phabricator.wikimedia.org/T371742
[10:55:27] <Amir1>	 and T370903
[10:55:27] <stashbot>	 T370903: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903
[11:02:13] <marostegui>	 Amir1: from my side good to go
[11:06:06] <federico3>	 Amir1: do we want to take the opportunity to test an updated version of schema change?
[11:06:44] <federico3>	 Amir1: if so I can update https://gitlab.wikimedia.org/repos/data_persistence/dbtools/auto_schema/-/merge_requests/7 shortly
[11:06:44] <Amir1>	 ooh
[11:06:49] <Amir1>	 sounds good to me
[12:12:50] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on s8 on db2181 is CRITICAL: 710.8 ge 10 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2181&var-port=9104
[12:16:50] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on s8 on db2181 is OK: (C)10 ge (W)5 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2181&var-port=9104
[13:42:08] <federico3>	 shall we create a MariaDB or DBA subgroup on https://gitlab.wikimedia.org/repos/sre to keep things tidy?
[13:47:37] <Amir1>	 there should be a data persistence somewhere?
[13:47:58] <Amir1>	 ah, it's under repos directly https://gitlab.wikimedia.org/repos/data_persistence
[14:06:07] <federico3>	 Amir1: yes that's where Auto Schema lives but I don't know if we can reparent it under sre
[15:04:00] <Amir1>	 federico3: sorry I was in meeting. I think it should be possible but releng would know better
[15:04:44] <Emperor>	 the downside of moving repos around is that folk who've cloned it have to update their remotes (and any doc links)
[15:39:08] <federico3>	 Emperor: indeed, yet the repos under data_persistence are a handful so perhaps it's not too much work