[02:03:55] FIRING: MaxConntrack: Max conntrack at 85.83% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [02:13:55] RESOLVED: MaxConntrack: Max conntrack at 85.05% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [03:11:04] FIRING: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [03:11:24] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11067093 (10Papaul) Please see below for the migration diagram. {F65720751} [04:11:04] RESOLVED: NetboxAccounting: Netbox - Accounting job failed - https://wikitech.wikimedia.org/wiki/Netbox#Report_Alert - https://netbox.wikimedia.org/core/jobs/ - https://alerts.wikimedia.org/?q=alertname%3DNetboxAccounting [04:39:55] FIRING: MaxConntrack: Max conntrack at 80.67% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [04:44:55] RESOLVED: MaxConntrack: Max conntrack at 85.77% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [10:30:31] 10netops, 06Infrastructure-Foundations, 06SRE: Allow read-only users to view logs on Juniper devices - https://phabricator.wikimedia.org/T401378 (10cmooney) 03NEW p:05Triage→03Low [11:57:00] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11067948 (10cmooney) Thanks @papaul for the info. For the most part the diagram looks ok, a few questions/notes: The new switches should be called //fasw1-f5a// and //fasw... [13:16:37] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11068239 (10Papaul) @cmooney please see answers and comments below. >>! In T401297#11067948, @cmooney wrote: > Thanks @papaul for the info. For the most part the diagram l... [14:04:19] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11068404 (10cmooney) > "I think we have two cluster control / HA ports on each unit? > em0 and em1 as reported by the box, labelled HA0 and HA1 on the front? > We should u... [15:27:20] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11068877 (10Papaul) I Still strongly disagree we need redundancy on the HA port. The reason being that if the port goes down, this will not have any impact. On the other ha... [17:18:55] FIRING: MaxConntrack: Max conntrack at 85.31% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [17:28:55] RESOLVED: MaxConntrack: Max conntrack at 85.29% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [18:31:55] FIRING: MaxConntrack: Max conntrack at 80.3% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack [18:36:55] RESOLVED: MaxConntrack: Max conntrack at 80.26% on krb1002:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_conntrack - https://grafana.wikimedia.org/d/oITUqwKIk/netfilter-connection-tracking - https://alerts.wikimedia.org/?q=alertname%3DMaxConntrack