[07:36:00] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: package_builder puppet tests failing - https://phabricator.wikimedia.org/T293912 (10jbond) 05Open→03Resolved This was caused by a pre_condition in the rspec test, fixed now
[08:35:29] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL )) - https://phabricator.wikimedia.org/T283582 (10cmooney) @papaul @dzahn I had a go at enumerating the iDrac firmware version on o...
[09:07:42] <XioNoX>	 From Telia:
[09:07:42] <XioNoX>	 Our DDoS portal launches on 28th Oct – and it’s great. I’m happy to give the customer a tour of MyCarrier in November should they wish – but it’s pretty user friendly and self-explanatory.
[09:07:43] <XioNoX>	 There are also API’s which can be used too.
[09:12:12] <jbond>	 nice :)
[10:40:31] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: Q1:(Need By: TBD) rack/setup/install cloudswift100[12] - https://phabricator.wikimedia.org/T289882 (10ayounsi) a:05ayounsi→03aborrero Thanks for the doc, some follow up questions to make sure I understand it properly.  > However, li...
[10:44:11] <topranks>	 Is there anything special about ssh auth to kubernetes hosts?  
[10:44:51] <topranks>	 When I try to connect it attempts to use the correct key (or at least one that works for other systems), but it fails and lands me at a password prompt.
[10:56:25] <volans>	 topranks: which hosts?
[10:56:52] <volans>	 the ssh as your user should be as usual, then kinit to get a kerberos ticket if you need one
[10:56:56] <topranks>	 tried kubernetes1001 and kubernetes2001 
[10:57:28] <volans>	 topranks: those are stretch hosts, maybe it's a problem of compatible ciphers?
[10:57:31] <volans>	 try ssh -vvv
[10:57:41] <volans>	 and check when offering the key what it says
[10:58:04] <volans>	 check for something like
[10:58:05] <volans>	 debug1: send_pubkey_test: no mutual signature algorithm
[10:59:34] <topranks>	 thanks.... it doesn't appear to be that.
[10:59:37] <topranks>	 don't see that msg.
[11:00:35] <topranks>	 but I need to read up on the kerberos stuff, I didn't do kinit, is that needed prior to the ssh connection?
[11:00:58] <volans>	 that's on the host after ssh-ing
[11:01:10] <volans>	 the ssh is normal
[11:01:44] <topranks>	 ok yeah makes sense
[11:01:46] <volans>	 if you want I can look at your ssh -vvv attempt if you send it over to me
[11:02:43] <topranks>	 thanks...  I'm sure it's something stupid on my side :(
[11:02:58] <jbond>	 volans: are you sure about kerberos on kubernetes* ?
[11:03:31] <volans>	 those are not even kerberized hosts... true
[11:04:11] <volans>	 jbond: was a pebcak ;)
[11:04:23] <jbond>	 :)
[11:06:33] <jbond>	 topranks: from auth.log it looks like you get in
[11:07:05] <topranks>	 yeah I'm good, was stupidly trying the iDRAC mgmt hostname not the actual server.
[11:07:12] <jbond>	 ahh ok :)
[11:07:26] <topranks>	 sorry for the interruption !
[14:32:38] <volans>	 FYI I'll pick sretest1001 for a spin (test reimage)
[16:42:48] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: Q1:(Need By: TBD) rack/setup/install cloudswift100[12] - https://phabricator.wikimedia.org/T289882 (10aborrero) I take it the main concern here is allocating a public IPv4 address, which is a scarce resource, no? It seems we have a rese...
[18:02:44] <wikibugs>	 10Packaging, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Disable man-db in pbuilder in package_builder on deneb - https://phabricator.wikimedia.org/T276632 (10Legoktm) a:03Legoktm
[18:27:48] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL )) - https://phabricator.wikimedia.org/T283582 (10Papaul)
[18:31:31] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL )) - https://phabricator.wikimedia.org/T283582 (10Dzahn) >>! In T283582#7447115, @cmooney wrote: > There are many more in eqiad, bu...
[18:32:50] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL )) - https://phabricator.wikimedia.org/T283582 (10Dzahn) @cmooney many thanks for the txt :)
[18:36:08] <wikibugs>	 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10ops-codfw: DRAC firmware upgrades codfw (was: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL )) - https://phabricator.wikimedia.org/T283582 (10Dzahn) So the ones alerting in eqiad are one case of 2.30.30.30 and one case of "...