[13:48:27] marostegui: btw, we will drop revision_actor_temp in a week or two. it's 138GB on wikidata and 98GB in enwiki ^^ [13:48:37] oh cool! [13:48:44] finally! [13:49:26] I should send an announcement to the cloud [13:49:50] yeah [13:55:17] templatelinks clean ups (dropping the text column) will happen in a month or so btw [13:55:32] that'd be similar size [13:55:56] great [14:14:25] marostegui: are you done with s7? [14:14:48] maint map says you did stuff on it today [14:17:39] yeah [14:17:42] I am done with it [14:18:57] awesome [14:29:32] Amir1: a couple of questions... 1) I have a calendar invite pending for you for tomorrow, will you be able to make it? 2) I'm about to stop a replica server for a bit and could use advice about how to check replag before/after to make sure that I don't break syncing [14:29:56] regarding 1- yes. Sorry I forgot to RSVP is [14:30:20] regarding 2- let me grab it from code [14:30:23] thx [14:31:13] andrewbogott: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/software/+/master/dbtools/auto_schema/auto_schema/host.py#62 [14:31:17] Does that work for you? [14:31:24] I don't know if you have heartbeat there [14:31:26] Today's reimage will be a rehearsal for tomorrow. In theory it's just stopping a VM (which contains a DB server), rebuilding the hypervisor, and then restarting the VM -- your presence is requested in case something terrible happens, if all goes as planned it'll be boring. [14:31:38] oh and also if it's in core dbs, you can check https://orchestrator.wikimedia.org/ [14:31:47] we don't have pt-heartbeat on toolsdb [14:32:12] hmm, then you need to get "show slave status" [14:32:17] ok, maybe 'replag' is the wrong word then... I just want to make sure that the replica is still actually talking and replicating at all :) [14:32:35] which values are the interesting ones in 'slave status'? [14:32:51] log position [14:32:58] e.g. if it's advancing [14:33:15] s/log/binlog [14:33:26] Relay_Log_Pos: 103388389 ? [14:33:55] let me double check [14:34:04] that one is advancing at least :) [14:34:08] I don't think that's the one but let me double check [14:35:03] yup, that's not the one, it's Master_Log_File and Read_Master_Log_Pos [14:35:21] if it's not advancing, it can be simply because there is no write being done [14:35:37] I don't know how much toolsdb gets write [14:36:07] LOTS [14:36:21] looks good now, I'm going to do the rebuild and will check that again after. thank you! [15:19:40] marostegui: I completely missed this in the flood of schema change comments T298565#7860561 [15:19:40] T298565: Fix mismatching field type of user table for columns user_email_authenticated, user_email_token, user_email_token_expires, user_newpass_time, user_registration, user_token, user_touched, user_newpassword, user_password, user_email on wmf wikis - https://phabricator.wikimedia.org/T298565 [15:20:55] no problem! [15:21:40] I will check it once I'm done with the mess [15:22:12] tbh, that ticket should have been split into maybe five/six tickets [15:22:14] nbd [20:46:20] Amir1: no schema change running on s7 at the moment no? I'm doing the switchover in a few hours [20:46:41] marostegui: yeah, nothing on it [20:47:01] great thanks [20:47:04] I'm going to sleep bye