[03:47:42] cdanis: while I enjoy the various funatic ways of pronouncing dcs, I did actually document a pronunciation guide on each DC based on a limited survey, to help with on boarding and not feeling weird about saying them [03:48:06] Each Dc page on wikitech has one in the lede [03:48:31] Hope that helps, denisse|m :) [03:50:16] Thanks Krinkle! ! It does help and it's a nice gesture from you. :D [05:50:50] <_joe_> taavi: amsterdam is northern europe :P [08:05:27] I'm late to the conversation, but location encoding systems is a fascinating topic, see https://github.com/google/open-location-code/wiki/Evaluation-of-Location-Encoding-Systems for example [08:28:37] our puppet mirror isn't getting updated? or is it me? [08:28:42] in github I mean [08:34:42] <_joe_> 301 hashar [08:35:47] marostegui: it's definitely missing everything today [08:46:02] i can see some repos working [10:12:12] ca someone ping me if they have an host to decom? I'd like to test something (not urgent), thanks! [10:46:40] XioNoX: I've got some coming up in the next ~few~ weeks, if that's not too far away [10:47:33] Emperor: ping me just in case [10:48:07] Emperor: but don't block it on me [10:52:09] OK [10:53:04] <_joe_> XioNoX: serviceops should have copious servers decommed for you soon(TM) [11:01:08] I have to go, but consider silencing the "High average GET latency for mw requests on * in codfw" [11:01:46] codfw load is so low, and it is based on a sliding window that with codfw test + pdu maintenance it will likly fire many times [11:02:02] <_joe_> jynus: yeah it's due to hte fact that we're reaching services on the other dc [11:02:38] just saying it may not be useful as it is "normal" to get spikes, and it is not based on absolute numbers [11:02:47] (for now) [13:59:39] moritzm: no news from the owner of that expired key in the pwstore repo? [14:08:35] <_joe_> dhinus: that is me :P [14:09:04] <_joe_> yeah today I kinda forgot, yesterday because of the maintenance I worked 14 hours and I forgot. Scusa ;/ [14:09:30] <_joe_> first coffee, then I'll fix it :P [14:11:21] lots of coffee [14:20:10] <_joe_> dhinus: {{done}} [14:21:16] <_joe_> dhinus: try no [14:21:20] <_joe_> *now [14:23:28] thanks _joe_ :) I think it's up to moritzm or mutante to update .users now, right? [14:24:06] <_joe_> dhinus: my key didn't change [14:24:09] <_joe_> but to add you, yes [14:52:16] fyi, codfw-ulsfo circuit maintenance in 10min, the link will go down [14:53:26] <_joe_> XioNoX: should we depool ulsfo? [14:53:54] _joe_: nah, I'm draining the circuit and the downtime should be brief [15:07:08] hi all during insperation week i did some work on beaker (acceptence testing for puppet) and have finnaly got around to creating aa wikitech article https://wikitech.wikimedia.org/wiki/Puppet/Beaker feel free to ping me if anything is unclear or missing [15:09:04] nice! [15:10:50] hi. can someone who has ops in this channel (I don't) please invite papaul here? thanks [15:11:17] i thought this was a public channle [15:11:22] it is [15:11:27] but i wanted to send an explicit /invite [15:11:28] <_joe_> it's public, anyone should be able to invite I think? [15:11:35] no, you still need ops to /invite [15:11:46] <_joe_> done [15:12:09] ahh ok and cheers jo.e [15:13:27] <_joe_> everyone, let's talk here about the maintenance [15:13:36] <_joe_> if papaul can't make it here, I'll rely his messages [15:14:42] thanks, picking one channel helps :) [15:14:56] +g allows anyone to invite anyone [15:16:36] <_joe_> (maintenance talk moved to #-dcops then) [15:16:49] #wikimedia-dcops to be clear [15:16:57] _joe_: here [15:17:07] <_joe_> papaul: ahah we were just moving everyone to dcops :* [15:17:11] ok [15:34:22] klausman: are you still around? we're looking for someone to shut down ml-serve2003 for PDU maintenance [18:19:29] Hello team, I need to connect the netmon1003 instance to the LibreNMS database. [18:19:29] Do you know where could I list the databases we currently have? I'd just like to confirm I'm connecting to the right database. [18:22:08] we have quite a few databases :-) is this https://wikitech.wikimedia.org/wiki/MariaDB#Sections_and_shards helpful? [18:22:47] Thanks a lot taavi , I'll take a look at it. :) [18:24:50] denisse|m: maybe I can help, most likely it will be a misc dv [18:24:53] *db [18:26:07] if it is the librenms db, you probably want to conect to the m1 proxy [18:38:27] * jynus wonders if wmf_auto_restart_prometheus-mysqld-exporter@s4.timer needs a wmf_auto_restart_wmf_auto_restart_prometheus-mysqld-exporter@s4.timer for when it fails, like it just did on db2099 [18:47:24] Thanks jynus , I'll write to you in private. [18:47:53] sure [22:00:25] Emperor: sorry out this week and next. I can take a look tomorrow [22:15:24] klausman: long handled if you mean the PDU maintenance [22:15:37] okey-dokey then :) [22:19:48] klausman: https://phabricator.wikimedia.org/T309956 is the master task for the remaining work. No idea if other ML servers will be affected. The 5th is now going to the 10th and the subtasks are accurate. [23:25:39] we are currently switching the gerrit-replica.wikimedia.org to a new machine. DNS switch. everything looks fine, can clone from it etc