[07:17:35] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[07:17:59] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[07:23:49] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[07:24:16] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[07:50:17] <wikibugs>	 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE, 10Patch-For-Review: Undocumented IP on WMCS network - https://phabricator.wikimedia.org/T315955 (10taavi) >>! In T315955#8188684, @Andrew wrote: > This is good to know! I only recently changed those to CNAMEs, so I'll switch them back when...
[08:06:49] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[08:22:43] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 3 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[08:33:18] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 2 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[08:34:08] <wikibugs>	 10netbox, 10Infrastructure-Foundations, 10serviceops: Netbox and Redis - https://phabricator.wikimedia.org/T311385 (10ayounsi) Sounds like a great 1st step (if we want to test active active later on), if not final step (if we keep it as it).  So I guess onthe Netbox side we "just" need those config options (...
[08:35:47] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 2 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui)
[08:37:20] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 2 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui) All done
[08:37:29] <wikibugs>	 10SRE-tools, 10DBA, 10Infrastructure-Foundations, 10Recommendation-API, and 2 others: Switchover m2 master db1159 -> db1164 - https://phabricator.wikimedia.org/T316202 (10Marostegui) 05Open→03Resolved
[08:46:34] <wikibugs>	 10netbox, 10Infrastructure-Foundations, 10serviceops: Netbox and Redis - https://phabricator.wikimedia.org/T311385 (10Joe) Silly question: do we have an idea of the size of the cached dataset? if it's small, do we need to keep redis remote to the VM where netbox runs, or should we install it as a local sidecar?
[11:09:30] <wikibugs>	 10netbox, 10Infrastructure-Foundations, 10serviceops: Netbox and Redis - https://phabricator.wikimedia.org/T311385 (10ayounsi) >  do we have an idea of the size of the cached dataset? Good question! I guess this is small compared to https://grafana.wikimedia.org/d/000000174/redis?orgId=1&viewPanel=9 `name=ne...
[11:55:59] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade core routers to Junos 20+ - https://phabricator.wikimedia.org/T295690 (10ayounsi)
[12:29:42] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade core routers to Junos 20+ - https://phabricator.wikimedia.org/T295690 (10ayounsi)
[12:30:40] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade core routers to Junos 21+ - https://phabricator.wikimedia.org/T295690 (10ayounsi)
[12:46:55] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade management routers to Junos 21 - https://phabricator.wikimedia.org/T316529 (10ayounsi) p:05Triage→03Low
[13:03:40] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade POPs asw to Junos 21 - https://phabricator.wikimedia.org/T316532 (10ayounsi) p:05Triage→03Low
[13:17:25] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade management routers and switches to Junos 21 - https://phabricator.wikimedia.org/T316529 (10ayounsi)
[13:26:43] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade network devices - https://phabricator.wikimedia.org/T316539 (10ayounsi) p:05Triage→03Lowest
[13:27:49] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade network devices - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:27:53] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade POPs asw to Junos 21 - https://phabricator.wikimedia.org/T316532 (10ayounsi)
[13:27:59] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade management routers and switches to Junos 21 - https://phabricator.wikimedia.org/T316529 (10ayounsi)
[13:28:07] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade core routers to Junos 21+ - https://phabricator.wikimedia.org/T295690 (10ayounsi)
[13:28:17] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10fundraising-tech-ops: Upgrade pfw to Junos 20+ - https://phabricator.wikimedia.org/T295691 (10ayounsi)
[13:28:23] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade network devices - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:28:29] <wikibugs>	 10netops, 10Infrastructure-Foundations: Upgrade network devices to Junos 20+ - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:41:08] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10fundraising-tech-ops: Upgrade fasw to Junos 21 - https://phabricator.wikimedia.org/T316542 (10ayounsi) p:05Triage→03Low
[13:41:30] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10fundraising-tech-ops: Upgrade fasw to Junos 21 - https://phabricator.wikimedia.org/T316542 (10ayounsi)
[13:41:38] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade network devices to Junos 20+ - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:45:17] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: all network devices must run OpenSSH >= 7.2p1 but != 7.4p1 - https://phabricator.wikimedia.org/T254013 (10ayounsi)
[13:45:25] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade network devices to Junos 20+ - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:46:31] <wikibugs>	 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Upgrade cloudsw1-c8-eqiad and cloudsw1-d5-eqiad to Junos 20+ - https://phabricator.wikimedia.org/T316544 (10cmooney) p:05Triage→03Low
[13:47:35] <wikibugs>	 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Upgrade cloudsw1-c8-eqiad and cloudsw1-d5-eqiad to Junos 20+ - https://phabricator.wikimedia.org/T316544 (10ayounsi)
[13:47:43] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE: Upgrade network devices to Junos 20+ - https://phabricator.wikimedia.org/T316539 (10ayounsi)
[13:47:56] <wikibugs>	 10netops, 10Cloud-VPS, 10Infrastructure-Foundations, 10SRE, 10cloud-services-team (Kanban): Upgrade cloudsw1-c8-eqiad and cloudsw1-d5-eqiad to Junos 20+ - https://phabricator.wikimedia.org/T316544 (10taavi)
[14:39:45] <wikibugs>	 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Routing loop for unused WMCS IPs in 185.15.56.0/24 - https://phabricator.wikimedia.org/T315956 (10cmooney) a:03cmooney Yeah I hadn't considered that when I made the change originally.  Figured it was a simplification but the routing loop is...
[18:12:10] <wikibugs>	 10netops, 10Cloud-Services, 10Infrastructure-Foundations, 10SRE: Routing loop for unused WMCS IPs in 185.15.56.0/24 - https://phabricator.wikimedia.org/T315956 (10cmooney) I've made these changes now manually.  One change to note is the included ASNs of eBGP peers cloudsw1-e4 and cloudsw1-f4 on the aggrega...
[23:33:01] <wikibugs>	 10netops, 10Infrastructure-Foundations, 10SRE, 10fundraising-tech-ops: Upgrade fasw to Junos 21 - https://phabricator.wikimedia.org/T316542 (10Dwisehaupt) @ayounsi We have a maintenance week for frack scheduled for Sep 26-30. Would sometime that week be good for you? We could do fasw1-c-codfw before then i...