[04:45:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [04:50:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [06:19:14] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: Migrate non-fundraising hosts out of eqiad D6 - https://phabricator.wikimedia.org/T390240#10754475 (10ayounsi) a:05ayounsi→03None [06:31:44] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: eqiad: determine second frack - https://phabricator.wikimedia.org/T392007#10754484 (10ayounsi) @robh, thanks for that task, well summed up ! One more point is that we need to account for expected overall growth of the... [06:52:13] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: eqiad: determine second frack - https://phabricator.wikimedia.org/T392007#10754495 (10wiki_willy) Hey @ayounsi - after some feedback from my staff meeting earlier today, I reached out to Equinix to see if there's any wa... [07:27:13] 10netops, 06Infrastructure-Foundations, 06SRE: Update network SSH keys to ssh-ed25519 - https://phabricator.wikimedia.org/T336769#10754513 (10ayounsi) @BBlack ping ? :) [08:30:11] 10netops, 06Infrastructure-Foundations, 06SRE: Netbox: Use LAG interface MAC address field to store LACP system-id for MC-LAG - https://phabricator.wikimedia.org/T392056#10754547 (10ayounsi) Some other thoughts : * There is also a netbox plugin, but that doesn't seen like a great way : https://github.com/pv... [08:36:22] 10netops, 06Infrastructure-Foundations, 06SRE: Create alerting for saturation on sub-rated interfaces - https://phabricator.wikimedia.org/T374614#10754568 (10ayounsi) For the record, as mentioned on IRC, if the description parsing doesn't work, we can also set it in the UI. For example in https://librenms.wi... [12:17:22] 06Traffic: Map ISPs in Maxmind db, used in turnilo/superset, to use in requestctl rule - https://phabricator.wikimedia.org/T392219#10754746 (10Fabfur) I'm currently performing some tests on HAProxy to check how much it takes for a lookup using a file map built from maxmind GeoIP-ISP database where the first colu... [14:49:45] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: eqiad: determine second frack - https://phabricator.wikimedia.org/T392007#10754936 (10RobH) a:05Jclark-ctr→03wiki_willy Assigning from John to Willy pending EQ update on pricing and build out of an additional rack i... [14:50:57] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: Migrate non-fundraising hosts out of eqiad D6 - https://phabricator.wikimedia.org/T390240#10754938 (10RobH) 05Open→03Invalid Please note that after further deliberation and investigation, it has been determined... [14:51:59] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: eqiad: second frack parent tracking task - https://phabricator.wikimedia.org/T392006#10754971 (10RobH) [14:52:02] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: Migrate non-fundraising hosts out of eqiad D6 - https://phabricator.wikimedia.org/T390240#10754972 (10RobH) [14:52:32] 10netops, 06DC-Ops, 10fundraising-tech-ops, 06Infrastructure-Foundations, and 2 others: eqiad: second frack parent tracking task - https://phabricator.wikimedia.org/T392006#10754975 (10RobH) >>! In T392006#10745097, @RobH wrote: > Please note I've tied original task T390240 to this for ease of tracking. I... [15:57:12] 06Traffic: Provide debian packages for libvmod-wmfuniq - https://phabricator.wikimedia.org/T392059#10755164 (10BCornwall) 05In progress→03Resolved [20:20:39] 06Traffic, 06Experimentation Lab, 13Patch-For-Review: Edge Uniques Production Cookie Deployment - https://phabricator.wikimedia.org/T391411#10755536 (10Milimetric) [20:20:46] 06Traffic, 06Experimentation Lab, 13Patch-For-Review: Edge Uniques Production Cookie Deployment - https://phabricator.wikimedia.org/T391411#10755537 (10Milimetric) [20:21:07] 06Traffic, 06Experimentation Lab, 13Patch-For-Review: SDS 2.4.4 Edge Uniques Production Cookie Deployment - https://phabricator.wikimedia.org/T391411#10755539 (10Milimetric) [20:22:00] 06Traffic, 06Experimentation Lab, 13Patch-For-Review: SDS 2.4.4 Edge Uniques Production Cookie Deployment - https://phabricator.wikimedia.org/T391411#10755543 (10Milimetric) (feel free to reject that name update, just trying to sync up phab with what we have in hypotheses and mirror the dependencies so I can... [20:32:54] 06Traffic, 06Experimentation Lab, 13Patch-For-Review: SDS 2.4.4 Edge Uniques Production Cookie Deployment - https://phabricator.wikimedia.org/T391411#10755614 (10Milimetric)