[07:02:35] <elukey>	 hello folks, checking the ms-be nodes, those are new and not serving traffic afaics
[07:05:43] <elukey>	 seems the same issue that happened with the new ms-fe nodes
[07:06:17] <elukey>	 and the top or rack is again lsw1-e1
[07:08:12] <elukey>	 pinged people and acked the alerts
[07:11:46] <_joe_>	 elukey: <3
[09:03:29] <topranks>	 elukey: thanks for the heads up looking now.
[09:18:37] <Emperor>	 elukey: thanks - the new ms-be nodes are indeed not in service
[09:35:11] <topranks>	 Emporer: what's the urgency with bringing them online?
[09:35:23] <topranks>	 dammit I will learn to spell Emperor :)
[09:36:14] <topranks>	 To confirm the issue seems the same as with ms-fe1012, however it's slightly more concerning cos obviously it's a repeat, and it's also happened on several different devices.
[09:38:10] <topranks>	 Bouncing the primary interface fixed ms-be1068, and I suspect will do the others.
[09:38:17] <Emperor>	 topranks: it's not urgent; I am trying to get a different bit of fiddly work done first (swift ring management automation), which will make bringing these fully into service less work
[09:38:27] <topranks>	 But ideally if we had some time to leave one of them "broken" and raise a TAC case with Juniper it would help.
[09:38:34] <topranks>	 Ok.
[09:38:42] <Emperor>	 so if there's a possible fault that still needs properly bottoming out, do please do that :)
[09:38:50] <topranks>	 thanks, I'll dig a bit more for now and likely contact Juniper shortly
[09:39:14] <Emperor>	 if you can keep T299462 updated with progress, that'd be kind
[09:39:14] <stashbot>	 T299462: Q3:(Need By: TBD) rack/setup/install ms-be10[68-71] - https://phabricator.wikimedia.org/T299462
[09:39:17] <topranks>	 the previous one we could dismiss as a one-off bug but this is worrying.
[09:39:19] <topranks>	 will do.
[09:39:26] <Emperor>	 Grand, thank you :)
[09:48:27] <vgutierrez>	 elukey: how's your partman testing going? ;P
[09:50:02] <elukey>	 vgutierrez: it is not nice to put more misery on a colleague's shoulders when he is clearly questioning his whole life choices
[09:50:30] <elukey>	 vgutierrez: good I'd say :D
[09:55:38] <vgutierrez>	 ack, I need to run puppet on apt1001, let me know when that's feasible :)
[09:55:47] <vgutierrez>	 no pressure ;P
[09:57:10] <elukey>	 vgutierrez: go ahead :)
[09:57:35] <vgutierrez>	 <3
[09:57:54] <elukey>	 lemme know once finished so I can restart my joy with partman
[10:00:41] <vgutierrez>	 {done}
[10:01:25] <elukey>	 <3
[14:50:11] <XioNoX>	 akosiaris, jayme: https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=cr1-codfw&service=BGP+status and https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=cr2-codfw&service=BGP+status let me know if I should have a look
[14:50:57] <jayme>	 XioNoX: ah, sorry. Thats elukey
[14:51:09] <jayme>	 (and it's fine)
[14:51:48] <elukey>	 XioNoX: yes it is me fighting with partman, kubernetes2005 is not honoring its BGP duties, should be done in ~1h
[14:52:01] <XioNoX>	 no pb, as long as it's known :)
[14:53:19] <elukey>	 kormat is not here today but she would have said that it is known that I cause alarms
[14:55:53] <kormat>	 ^^^
[14:56:04] <kormat>	 elukey: 💜
[14:58:35] <elukey>	 kormat: <3
[14:59:16] <Emperor>	 as if by magic, a kor.mat appears :)