[10:46:31] dcaro: hi [10:46:40] I think -operations is a bit busy [10:46:56] Wrong channel [10:47:20] RhinosF1: You are missing lots of context about that patch [10:47:59] We moved to PM [12:19:28] Amir1: I am doing the masters here https://phabricator.wikimedia.org/T300992 [12:19:54] marostegui: thanks. I was planning to do it this week. Thank you for reducing my work :D [12:20:44] Amir1: Ah, I saw it was moved to Blocked like 20 days ago and wasn't sure [12:20:58] https://phabricator.wikimedia.org/T300992#7757089 [12:21:16] yeah, After seeing your other ticket on linter, I changed my mind [12:21:35] Amir1: So I will stop now then, I just did s7, but the others are pending [12:22:15] s1 is unlikely to be able to get done but the others should be doable [12:22:23] marostegui: awesome, quick check, I just need to run it without replication? No set some magic value? [12:22:43] Amir1: this should work: set session sql_log_bin=0; SET SESSION innodb_lock_wait_timeout=1; SET SESSION lock_wait_timeout=30; [12:22:45] like "fail if it takes more than thirty seconds" [12:22:49] that's what I use for the masters [12:22:54] aaaaaaaah [12:23:05] Thanks. I should add that to auto_schema [12:23:10] for linter I reduced it to 15 anyways [12:24:04] I'm deploying the tl_target_id write both on all of s6 now btw [12:24:11] oki! [12:24:31] at first I deployed it to labswiki and intentionally did changes, no replication issues [12:24:43] so we know all of s6 has the column everywhere [12:25:09] yeah that's better [12:26:24] later I'm gonna pick one wiki from each section and enable it there (except s1, s3, and s4 where it's not done there yet) [12:26:31] for s4 it's gonna take a while [12:27:11] yes, it will take this whole week at least [12:27:18] making it 2 weeks since I started it [12:30:58] wow, frwiki already has 140 rows in linktarget table. They are fast [21:24:03] Hey all, cjming and I are planning to run some database updates over the next hour. We'll be updating approx 100 rows per wiki. [21:24:11] Will let you know more as we go through the process [22:30:21] We are finishing this up now. [23:42:40] PROBLEM - MariaDB sustained replica lag on m1 on db1117 is CRITICAL: 31.6 ge 2 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1117&var-port=13321 [23:43:48] RECOVERY - MariaDB sustained replica lag on m1 on db1117 is OK: (C)2 ge (W)1 ge 0 https://wikitech.wikimedia.org/wiki/MariaDB/troubleshooting%23Replication_lag https://grafana.wikimedia.org/d/000000273/mysql?orgId=1&var-server=db1117&var-port=13321