[06:19:32] <wikibugs>	 10Puppet, 10DBA, 10Data-Engineering, 10Infrastructure-Foundations, 10Patch-For-Review: Split mariadb::dbstore_multiinstance into 2 separate roles (backup sources and analytics) - https://phabricator.wikimedia.org/T296285 (10Marostegui) p:05Triage→03Medium
[07:55:11] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10serviceops: Kubernetes1018's eth negotiated speed is 10MB/s - https://phabricator.wikimedia.org/T296369 (10elukey)
[08:09:42] <jayme>	 Hi o/ do we have a trusted source for ssh host keys of core routers somethere?
[08:43:48] <XioNoX>	 jayme: nop :(
[10:17:52] <moritzm>	 XioNoX, topranks: I need to reboot ping2002 for https://phabricator.wikimedia.org/T294119, could either of you please temporarily redirect the ICMP traffic to it for the time of the reboot?
[10:19:39] <XioNoX>	 moritzm: yep
[10:19:50] <topranks>	 XioNoX: want to step me through this?
[10:20:12] <XioNoX>	 topranks: https://wikitech.wikimedia.org/wiki/Ping_offload#Temporarily_stop_the_ICMP_echo_redirect :)
[10:20:12] <topranks>	 just searching wikitech / looking at the config now, but maybe easier if you're doing it
[10:21:02] <topranks>	 ping2002 isn't in Netbox?  
[10:21:07] <topranks>	 moritzm: do you mean ping2001?
[10:22:28] <XioNoX>	 I guess so, it's 2001 in https://phabricator.wikimedia.org/T294119
[10:22:40] <topranks>	 yeah and that's the only IP configured on the CRs
[10:22:44] <XioNoX>	 that reminds me I need to do https://phabricator.wikimedia.org/T295767
[10:23:17] <topranks>	 ok thanks for the link, makes sense fire away XioNoX I'm comfortable I can do it next time if needed.
[10:23:39] <moritzm>	 sorry, meant ping2001, yes
[10:23:49] <topranks>	 np
[10:23:55] <topranks>	 Do we only redirect pings for IPv4?
[10:23:59] <XioNoX>	 topranks: yeah
[10:24:13] <XioNoX>	 it's the main icmp traffic we get
[10:24:20] <topranks>	 is there a rationale behind that or is it on the "to do" list?
[10:24:22] <topranks>	 ok yeah.
[10:24:42] <XioNoX>	 hopefully that setup will go away with the future L4LB
[10:25:45] <topranks>	 hmm ok yep.
[10:28:01] <XioNoX>	 moritzm: you're good to go
[10:28:18] <moritzm>	 thx, doing that now. I'll ping you when it's complete
[10:31:31] <volans>	 inception alert, you can't ping them, pings are redirected...
[10:31:38] * volans hides
[10:31:48] <moritzm>	 :-)
[10:35:31] <XioNoX>	 topranks: I see rpki2001 in that list, is rpki1001 back in service?
[10:37:22] <moritzm>	 XioNoX: ping2001 is incomplete, you can revert
[10:38:48] <XioNoX>	 moritzm: incomplete?
[10:39:28] <moritzm>	 sorry, I actually mean to write complete...
[10:41:08] <topranks>	 XioNoX:  which list is this.  rpki1001 was rebuilt with the same name, rpki2001 was replaced with rpki2002.
[10:41:19] <topranks>	 Both are in service and IPs have not changed so Juniper config remains as it was.
[10:41:51] <topranks>	 Moritz's list from above I'm sure... looking closer.
[10:42:33] <XioNoX>	 https://phabricator.wikimedia.org/T294119 it's 2002 indeed
[10:42:51] <topranks>	 I'm guessing your question relates to our redundancy during reboot?  We should be good yes.
[10:42:57] <moritzm>	 yeah, that list is the output of "gnt-instance list" from yesterday
[10:43:02] <topranks>	 cool
[10:43:19] <XioNoX>	 yep exactly
[11:26:49] <moritzm>	 I'll do rpki2002 next, then?
[11:48:00] <topranks>	 mortizm: only seen your msg, yes feel free to proceed
[11:48:08] <wikibugs>	 10puppet-compiler, 10Infrastructure-Foundations, 10Cloud-Services-Origin-Team, 10Cloud-Services-Worktype-Project, and 2 others: PCC: create the output index once the first host is finished - https://phabricator.wikimedia.org/T295029 (10dcaro)
[11:48:32] <wikibugs>	 10puppet-compiler, 10Infrastructure-Foundations, 10Cloud-Services-Worktype-Project, 10Patch-For-Review, 10User-dcaro: PCC: add a fast fail option to boil out when the first error is encountered - https://phabricator.wikimedia.org/T295028 (10dcaro)
[11:49:03] <wikibugs>	 10puppet-compiler, 10Infrastructure-Foundations, 10Cloud-Services-Origin-Team, 10Cloud-Services-Worktype-Project, and 2 others: PCC: add a fast fail option to boil out when the first error is encountered - https://phabricator.wikimedia.org/T295028 (10dcaro)
[11:54:33] <moritzm>	 topranks: ack, doing that now
[11:59:50] <moritzm>	 done
[13:12:51] <wikibugs>	 10Puppet, 10Infrastructure-Foundations, 10Testing-Roadblocks, 10User-jbond: Allow using WMCS hiera lookup order in Puppet rspec tests - https://phabricator.wikimedia.org/T296327 (10jbond) @Majavah I have had a think about this and i don't think that it will work as expected.  Currently the shared spec help...
[14:06:27] <jhathaway>	 win 6
[14:06:35] <volans>	 good morning jhathaway :D
[14:06:46] <jhathaway>	 wrong channl sorry,  morning still getting use to irssi
[14:07:37] <volans>	 don't worry at all! It happens to everyone all the time ;)
[14:10:43] <XioNoX>	 It stopped happening to me the day I gave up getting used to irssi ;)
[14:12:07] <volans>	 I usually click on the name on the right column
[14:13:11] * volans uses the adv_windowlist plugin
[14:13:58] <godog>	 I'm a happy go.pl user for irssi
[14:16:22] <godog>	 not what I wanted to say though: I got this warning on idp.w.o by chrome 96, known? https://people.wikimedia.org/~filippo/2021-11-22-120448_372x269_scrot.png
[14:21:34] <moritzm>	 godog: thanks, it's known. we still need to open a task for this and adapt planning, though
[14:25:19] <godog>	 moritzm: ack, thanks
[14:31:19] <wikibugs>	 10Puppet, 10Cloud-VPS, 10Infrastructure-Foundations, 10Cloud-Services-Origin-Team, and 5 others: Refactor puppet:base module to reduce unneeded shared code paths - https://phabricator.wikimedia.org/T289661 (10dcaro)
[14:31:25] <wikibugs>	 10Puppet, 10Cloud-VPS, 10Infrastructure-Foundations, 10Cloud-Services-Origin-Team, and 5 others: Refactor puppet:base module to reduce unneeded shared code paths - https://phabricator.wikimedia.org/T289661 (10dcaro)
[15:00:45] <wikibugs>	 10Puppet, 10DBA, 10Data-Engineering, 10Infrastructure-Foundations, 10Patch-For-Review: Split mariadb::dbstore_multiinstance into 2 separate roles (backup sources and analytics) - https://phabricator.wikimedia.org/T296285 (10jcrespo) Deployment went as expected- but now that I thought a bit, I think btull...
[15:59:51] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10cloud-services-team (Kanban): cloud: decide on general idea for having cloud-dedicated hardware provide service in the cloud realm & the internet - https://phabricator.wikimedia.org/T296411 (10aborrero)
[16:00:29] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, and 2 others: Q1:(Need By: TBD) rack/setup/install cloudswift100[12] - https://phabricator.wikimedia.org/T289882 (10aborrero) Moved debate into {T296411}
[16:41:39] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10cloud-services-team (Kanban): cloud: decide on general idea for having cloud-dedicated hardware provide service in the cloud realm & the internet - https://phabricator.wikimedia.org/T296411 (10cmooney) Thanks @arturo, I think that sums up the options we discussed.  A...
[17:07:54] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10cloud-services-team (Kanban): cloud: decide on general idea for having cloud-dedicated hardware provide service in the cloud realm & the internet - https://phabricator.wikimedia.org/T296411 (10ayounsi)
[17:34:13] <XioNoX>	 Still on the Netbox changelog, 2.11 has quite a few improvements useful to us:
[17:34:13] <XioNoX>	 - "The ObjectChange model (which is used to record the creation, modification, and deletion of NetBox objects) now explicitly records the pre-change and post-change state of each object, rather than only the post-change state." <- I'm wondering if this could help with rolling back changes (eg. having a netbox scripts that tries to undo a change based on the pre-change state")
[17:34:13] <XioNoX>	 - "NetBox now supports journaling for all primary objects. The journal is a collection of human-generated notes and comments about an object maintained for historical context. It supplements NetBox's change log to provide additional information about why changes have been made or to convey events which occur outside NetBox. Unlike the change log, in which records typically expire after some time, journal entries persist for 
[17:34:13] <XioNoX>	 the life of the associated object." <- this could be used to add the name of the person running the cookbook (seen as "sre_bot") to the change
[17:54:39] <volans>	 sure, why not
[23:28:44] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10RobH)
[23:57:23] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10RobH)
[23:58:37] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10RobH)
[23:58:46] <wikibugs>	 10netops, 10DC-Ops, 10Infrastructure-Foundations, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) Rows E/F network racking task - https://phabricator.wikimedia.org/T292095 (10RobH)