[06:47:13] uh... I'm the only one having issues with admin/data/data.yaml and vim? [06:47:47] vim is munching one CPU core and frozen.. lovely [07:12:54] vgutierrez: opens instantly for me, maybe some vim extension that doesn't like something? [07:13:16] volans: yeah.. opening it looks good.. probably I've triggered a git blame lol [07:13:22] ahahahah [07:13:25] good luck with that [07:13:55] alhoguh directly from git I get one in few seconds [07:41:19] Can anyone take a look at: [07:41:20] 7:24:59 ['/usr/bin/scap', 'pull', '--no-php-restart', '--no-update-l10n', '--exclude-wikiversions.php', 'mw1420.eqiad.wmnet', 'mw1366.eqiad.wmnet', 'deploy2002.codfw.wmnet', 'mw2300.codfw.wmnet', 'mw1404.eqiad.wmnet', 'mw2259.codfw.wmnet', 'mw2289.codfw.wmnet', 'mw1398.eqiad.wmnet', 'deploy1002.eqiad.wmnet', 'mw1486.eqiad.wmnet'] (ran as mwdeploy@mw2448.codfw.wmnet) returned [255]: ssh: connect to host mw2448.codfw.wmnet [07:41:20] port 22: Connection timed out [07:41:40] mw2448.codfw.wmnet seems down. [07:43:35] looking [07:46:24] kart_: there's nothing in hardware error logs and I can't connect, will open a task for DC ops. I've dropped mw2448 from the list of active servers, so you can proceed [07:46:52] Thanks moritzm [11:22:01] I have questions: I did a dry run of `sre.discovery.datacenter depool eqiad --all` in preparation for the DC switchover and it's not clear to me what to do with a few services, in particular: [11:23:27] jbond: puppetboard-next is supposed to be active/active but is only pooled in eqiad, what should happen with it? [11:24:38] Emperor: should swift-rw be moved or left as is? [11:24:52] kamila_: i thin k thats an oversight, i have made them both active now [11:25:04] ok, thanks a lot! [11:25:49] kamila_: not sure if in your list but Netbox should not be moved to codfw, last time caused issues and I don't think we've fixed that [11:26:47] volans: Already told them :) [11:26:51] Same for apt [11:27:05] thx :D [11:27:07] yup, thanks to both of you :-) [11:30:41] volans: should we look at fixing it? :) [11:31:07] kamila_: swift-rw should get moved (I don't think it's actually in use, but best to move it anyhow) [11:31:09] I forgot where we left it at [11:31:31] XioNoX: sure, but when? I don't think I'll have time before the switchover [11:31:43] fair [11:39:36] Emperor: OK, thanks! [11:42:18] (mw attempts to write all uploads to both swift clusters) [11:42:22] _attempts_ [12:04:20] claime: do is there a task or something for the apt one? i dont quite rember the issue there [12:04:40] jbond: It wasn't as much an issue per se as it requires an additional puppet change [12:04:53] It's documented in wikitech, but I don't think it needs to be part of the planned switchover [12:05:11] claime: ahh yes thanks i think this is probably the relevent ticket https://phabricator.wikimedia.org/T330849 [12:06:05] I'll add the task ref to the cookbook exclusion message [12:06:13] sgtm thanks [14:44:58] Heads up that the live test from today actually turned into an accidental depool/repool of codfw active/active discovery services [14:45:12] We'll clean it up, but tell us if there's any weirdness [14:45:51] (None expected, obviously) [16:00:54] rzl: brett: nothing to report from EU-OnCall [16:01:24] jbond: thanks! have a good night [16:02:53] thanks :)