[06:21:51] <XioNoX>	 https://www.bbc.com/news/business-57070318
[06:32:00] <_joe_>	 anyone fancy reviewing https://gerrit.wikimedia.org/r/c/operations/puppet/+/695870 ?
[06:32:06] <_joe_>	 it's simple enough I promise
[06:36:35] <XioNoX>	 _joe_: I had to re-read it several times, but lgtm, especially with the PCC output
[06:38:51] <_joe_>	 ty!
[07:23:34] <legoktm>	 marostegui: I updated https://meta.wikimedia.org/wiki/IRC/Bots/ircservserv - it's ready for people to try out the process and point out where the docs are inadequate
[07:24:28] <marostegui>	 legoktm: oh thanks! I will have a look!
[08:35:50] <kormat>	 jbond: i had another look at the acmechief/pontoon issue i was having. i think it probably always happened, it just doesn't matter, because we're not using idp or even ssl in pontoon. the service runs on loopback, and we connect via an ssh forward.
[08:35:58] <kormat>	 jbond: so thanks for the help, and sorry for the bother :)
[08:37:29] <jbond>	 kormat: no problem, ping me if yu want to try and get it working in cloud 
[08:37:40] <vgutierrez>	 why do you use acmechief for a localhost only service then?
[08:38:10] <vgutierrez>	 it would make more sense to get issued an "internal"/PKI/Puppet CA cert
[08:39:19] <jbond>	 vgutierrez: it uses acme chief in production.  the cloud instance uses the same puppet role and tries to use acme chief but fails
[08:39:33] <jbond>	 but thats fine cause its cloud
[08:42:04] <kormat>	 ^ exactly
[08:42:29] <kormat>	 vgutierrez: in prod, it sits behind idp (https://orchestrator.wikimedia.org/web/clusters/)
[08:45:35] <vgutierrez>	 hmm not in terms of TLS :)
[08:45:55] <vgutierrez>	 it just sits behind idp in terms of L7 authentication/authorization
[08:50:33] <vgutierrez>	 vgutierrez@carrot:~$ openssl s_client -connect orchestrator.wikimedia.org:443 < /dev/null |& openssl x509 -noout -subject 
[08:50:33] <vgutierrez>	 subject=CN = orchestrator.wikimedia.org
[09:08:16] <kormat>	 vgutierrez: ok, correction :) it uses the idp client profile in prod, which runs an apache2 on the public interface using the acmechief-supplied tls cert
[09:19:42] <vgutierrez>	 makes perfect sense then to use acme chief :)
[09:21:33] <kormat>	 😅
[10:42:01] <dcaro>	 godog: If you have a minute, I just enabled sending logs to kibana (using ecs_107) and need some help getting a dashboard to show them (I cas change the index when doing 'discover' but I don't see an option when creating a dashboard)
[10:45:34] <dcaro>	 I think I got it, I have to create a visualization first right?
[10:46:38] <godog>	 dcaro: yes that's correct, you create visualizations for your dashboard (or reuse existing visualizations)
[10:47:15] <dcaro>	 ack, thanks, let me know if there's something I should *not* do xd (playing around)
[10:48:09] <XioNoX>	 don't edit existing visualization :)
[10:48:35] <godog>	 lol, yeah what XioNoX said
[10:50:37] <XioNoX>	 saying that as it can happen when your dashboard has a mix of re-used and custom visualizations
[10:50:52] <dcaro>	 I'll keep it in mind
[10:59:20] <godog>	 the other thing I'd suggest for ECS is indeed reusing existing ecs visualizations, I'm sure there are some missing but should be pretty easy to add
[11:00:27] <dcaro>	 ack, I see not "top 10 hosts" for ECS, is there one? (maybe I don't know how to search yet xd)
[11:01:14] <godog>	 heheh I see a data table like https://logstash.wikimedia.org/app/visualize#/edit/c5dbd120-6c90-11eb-b024-07c11958a85f?_g=h@7bdacab&_a=h@1d8a162
[11:01:30] <godog>	 or https://logstash.wikimedia.org/app/visualize#/edit/b2ef57c0-71c8-11eb-8a1a-ad06303682fe?_g=h@7bdacab&_a=h@be5f77d
[11:01:46] <godog>	 if you search for the field name in https://logstash.wikimedia.org/app/visualize#/?_g=h@7bdacab it should DTRT
[11:02:18] <dcaro>	 DTRT?
[11:02:30] <godog>	 do the right thing :)
[11:02:36] <dcaro>	 👍
[13:12:06] <cdanis>	 _joe_: thank you for fixing T283762
[13:12:07] <stashbot>	 T283762: Icinga alerts mention the wrong data center - https://phabricator.wikimedia.org/T283762
[13:31:07] <kormat>	 hmm. who is coordinating the DC switchover this time?
[13:32:09] <sobanski>	 kormat: legoktm
[13:32:21] <kormat>	 ah, yes
[13:32:51] <kormat>	 legoktm: hey. has there been an official announcement yet of the DC switchover, dates/times etc? i haven't seen it, but i could have just not been paying attention
[13:34:08] <marostegui>	 moritzm: are you working with cumin2001? Should I ack its alerts?
[13:39:49] <moritzm>	 hmmh, let me check the alert
[13:40:13] <kormat>	 moritzm: the dpkg one is because cumin is flagged for removal in dpkg
[13:42:05] <moritzm>	 yeah, that was fallout from yesterday's change, both fixed now
[13:51:52] <kormat>	 moritzm: the keyholder alert is still firing
[13:52:12] <kormat>	 it's weird, because the keyholder _is_ armed (or at least i can use cumin just fine on cumin2001)
[13:54:31] <_joe_>	 cdanis: I felt guilty that an error of mine spread like that through wild copypasta :)
[13:59:28] <moritzm>	 maybe it's still checking for the homer keyholder, having a closer look in a bit
[14:11:07] <ottomata>	 am adding a user entry in data.yaml, following instrutions in README
[14:11:08] <ottomata>	 q
[14:11:14] <ottomata>	 how to pick the numeric uid?
[14:11:19] <ottomata>	 instructions say to search ldap
[14:11:29] <ottomata>	 and i do, but the uidNumber there does not match up with other users in data.yaml
[14:11:50] <ottomata>	 e.g. eric gardner's ldap uidNumber: 5776, but in data.yasml uid: 20909
[14:13:24] <rzl>	 rzl@mwmaint1002:~$ ldapsearch -x uid=egardner | grep uidNumber
[14:13:24] <rzl>	 uidNumber: 20909
[14:13:30] <rzl>	 ottomata: are you looking at the wrong ldap maybe?
[14:13:33] <ottomata>	 oh
[14:14:00] <ottomata>	 ahhh I AM!
[14:14:01] <ottomata>	 thank you
[17:39:52] <legoktm>	 kormat: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/message/MR4YNIEIWVJG2LCCTSMBMEASUZUNNPZE/ times will be the same as last time, but I haven't written that down yet. Fell a bit behind :/
[17:44:15] <_joe_>	 interestingly, june 29th is bank holiday here :P
[17:44:39] <legoktm>	 >.<
[17:44:50] <legoktm>	 every week in June is bad