[01:07:43] <icinga-wm>	 PROBLEM - MariaDB sustained replica lag on m1 on db2160 is CRITICAL: 2 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2160&var-port=13321
[01:09:07] <icinga-wm>	 RECOVERY - MariaDB sustained replica lag on m1 on db2160 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db2160&var-port=13321
[08:05:16] <marostegui>	 Amir1: Isn't the cloning script supposed to bring up also the host that was used as a clone source?
[08:06:06] <marostegui>	 oh it just did
[08:06:10] <marostegui>	 right after finishing everything
[08:11:28] <Amir1>	 😁😁
[08:11:49] <Amir1>	 Glad it works 😍
[12:25:42] <godog>	 Emperor: would you have time to look into https://gerrit.wikimedia.org/r/c/operations/puppet/+/956906 this week? thank you
[12:28:52] <Emperor>	 I'll try to
[18:31:07] <andrewbogott>	 Anyone have time to teach me a tiny lesson about database maintenance? clouddb1019.eqiad.wmnet is threatening to go oom and I only barely know how to diagnose.  I can restart mariadb and kill everyone's queries but perhaps there are more graceful options
[18:35:51] <marostegui>	 andrewbogott: probably the best way is to depool from the proxy let it drain and then restart 
[18:37:11] <andrewbogott>	 Thanks, let's see if I remember how to do that :)
[18:47:57] <andrewbogott>	 marostegui: https://gerrit.wikimedia.org/r/c/operations/puppet/+/959036 look right?
[18:49:08] <marostegui>	 I'm on my phone so I can't verify whether those ports are the ones 1019 has but if it does then it looks good
[18:49:24] <andrewbogott>	 ok :)  We'll see
[18:49:29] <andrewbogott>	 thank you!