[01:03:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [01:18:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [02:27:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [02:52:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [03:13:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [03:18:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [05:27:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [05:37:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [05:40:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [05:50:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [07:07:09] FIRING: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [07:12:09] RESOLVED: LVSHighRX: Excessive RX traffic on lvs2013:9100 (eno12399np0) - https://bit.ly/wmf-lvsrx - https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=lvs2013 - https://alerts.wikimedia.org/?q=alertname%3DLVSHighRX [09:43:48] 10netops, 06Infrastructure-Foundations, 06SRE: Productionize gnmic network telemetry pipeline - https://phabricator.wikimedia.org/T369384#10491452 (10ops-monitoring-bot) Icinga downtime and Alertmanager silence (ID=43ff15dd-e256-46b3-aea6-882240b9fe64) set by cmooney@cumin1002 for 1:00:00 on 1 host(s) and th... [10:13:06] 06Traffic, 13Patch-For-Review: bring katran to liberica - https://phabricator.wikimedia.org/T380450#10491493 (10Vgutierrez) 05Open→03Resolved [13:01:38] 10netops, 06DC-Ops, 06Infrastructure-Foundations, 10ops-eqiad, 06SRE: Check link from msw1-eqiad et-0/1/0 to msw2-eqiad et-0/1/0 - https://phabricator.wikimedia.org/T384708 (10cmooney) 03NEW p:05Triage→03Low [14:31:42] 06Traffic, 06DC-Ops, 10ops-esams, 10ops-magru, 06SRE: CPU temperature issues in cp hosts - https://phabricator.wikimedia.org/T373993#10492346 (10RobH) >>! In T373993#10490884, @BCornwall wrote: > > The first dip on all the hosts was unrelated to anything I did - not sure what happened t... [17:33:35] * leila waves [17:39:41] In my public office hour, I just talked to a PhD student and her advisor from CMU who are in search to find a topic for the second chapter of the PhD student. They are looking at the broad space of equity and networking/system engineering/... . I briefly dug with them on a few fronts, and concluded that the kind of questions they want to look at is very closely related to the SRE space [17:39:43] (and likely to start Traffic). sukhe I'm thinking if you have time and interest, it would be great if you have a chat with them. or bblac? If someone on your end has time and interest to have an exploratory conversation with them, I'm happy to ask someone on the research end to join you if that's helpful. let me know. [17:40:07] the researchers are Justine Sherry (advisor) and Isabel Suizo (PhD student), in case you want to look them up. [17:40:46] and fwiw, I would love to find a way to engage them. They have a lot of energy and they are interested to focus their attention on WP which is great. [17:42:44] leila: hello! [17:42:50] nice to see you here [17:43:13] and yes please, very happy to talk about stuff; please feel free to ask them to get in touch [17:43:23] I am out for a week in Feb (see calendar) but available otherwise [17:44:01] one of the lovely netops folks should also be a good person to talk and I can connect Justine and Isabel with them if desired [17:44:40] and of course we can drag in bblack whenever we want :) [17:44:55] sukhe: lovely. what's the best way for them to reach out to you? do you prefer IRC or email or something else? [17:45:21] sukhe: do you need someone from research to be with you for that conversation? [17:45:25] email is better for the flow and perfectly fine to use [17:45:34] sukhe: noted re email. [17:45:35] leila: not really but I leave that your judgement. [17:46:07] sukhe: okay. I'd say no need for now to have a researcher with you then. if you see you need someone on our end, let me know afterwards (and I'll share this with the researchers as well) [17:46:24] thanks leila! [17:46:53] sukhe: I will then connect you all over email now. the netops contact you suggested sounds great to me. I'll leave it to you to mention in the email response to them. [17:47:12] exciiiiting. thanks for getting back to me swiftly. [17:47:41] thanks! and yeah, we have two netops folks and one is on leave so I will ask Cathal and then maybe Arzhel can also have a chat when he is back [17:48:08] :) [17:56:04] 10netops, 06Infrastructure-Foundations, 10observability, 06SRE: Prevent BGP alerts triggering when K8s host maintenance is being done - https://phabricator.wikimedia.org/T384731 (10cmooney) 03NEW p:05Triage→03Low [23:46:34] 06Traffic, 06DC-Ops, 10ops-esams, 10ops-magru, 06SRE: CPU temperature issues in cp hosts - https://phabricator.wikimedia.org/T373993#10494148 (10BCornwall) I did some more testing: (Rounded/eyeballed averages) | Profile | Offset | Fan RPS | CPU Temp (Celsius) | Default | None | 4k | 80 | Maximum Perform...