[07:41:11] <marostegui>	 x1 is back to RBR
[11:22:32] <jinxer-wm>	 (MysqlReplicationLag) firing: (2) MySQL instance db2151:9104 has too large replication lag (8m 59s) - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&refresh=1m&var-job=All&var-server=db2151&var-port=9104 - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationLag
[11:22:38] <marostegui>	 ^ me
[11:27:32] <jinxer-wm>	 (MysqlReplicationLag) resolved: (2) MySQL instance db2151:9104 has too large replication lag (10m 50s) - https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting#Depooling_a_replica - https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&refresh=1m&var-job=All&var-server=db2151&var-port=9104 - https://alerts.wikimedia.org/?q=alertname%3DMysqlReplicationLag
[11:32:08] <Amir1>	 FYI. We merged all db logging channels to rdbms, so starting late next week, they all switch to that instead of DBQuery, DBError, DBTransaction, etc.
[11:33:16] <marostegui>	 mmm what?
[11:33:19] <marostegui>	 So what's the correct dashboard?
[11:40:14] <Amir1>	 marostegui: dashboards shouldn't change
[11:40:55] <Amir1>	 but e.g. https://logstash.wikimedia.org/app/dashboards#/view/87348b60-90dd-11e8-8687-73968bebd217 needs changing to have "Channel is one of.." to have "rdbms" 
[11:41:51] <Amir1>	 I can fix it, but we have time
[14:06:48] <marostegui>	 Amir1: what is centralnotice? centralauth? meta?
[14:07:42] <Amir1>	 Good question. I don't know but it's probably in metawiki
[14:07:45] <Amir1>	 S7
[14:08:19] <marostegui>	 yeah, looks so
[14:08:20] <marostegui>	 thanks!
[16:33:48] <marostegui>	 Amir1: is there a way with the schema change to only run the change on the dc master host?
[16:34:01] <marostegui>	 (other than editing and adding the host manually I mean)
[16:34:16] <Amir1>	 --dc-primaries ?
[16:34:34] <Amir1>	 it does it on both codfw and eqiad masters though
[16:34:35] <marostegui>	 oh ys
[16:34:37] <marostegui>	 I am an idiot
[16:34:40] <marostegui>	 I was checking the wrong host
[16:34:42] <marostegui>	 yeah yeah
[16:34:45] <marostegui>	 I was runnign: run_schema_change.py --run --include-masters  --dc codfw --dc-masters codfw
[16:34:58] <marostegui>	 And it did the right thing, but I was looking at the wrong hostname
[16:34:59] <marostegui>	 Nevermind!
[16:35:03] <marostegui>	 Thanks for opening my eyes
[16:35:04] <Amir1>	 aah
[16:35:11] <Amir1>	 ok, happy to be a rubber duck
[16:35:16] <marostegui>	 XDDDD
[16:35:27] <Amir1>	 python run_schema_change.py  --help should also give you list of options
[16:35:37] <Amir1>	 in case you're not sure of anything
[16:36:17] <marostegui>	 yeah, I was sure I ran this thing before, but I got very confused
[16:36:26] <marostegui>	 But it was because I was looking at the wrong section in orchestrator