[00:12:58] (SystemdUnitFailed) firing: logrotate.service Failed on cp4052:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [04:12:58] (SystemdUnitFailed) firing: logrotate.service Failed on cp4052:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [07:38:01] 10netops, 10Infrastructure-Foundations, 10SRE, 10netbox: Netbox Juniper report - https://phabricator.wikimedia.org/T306238 (10SLyngshede-WMF) Rereading the answer for Juniper: > For OIDC we’ll need your IDToken which would look like below or the IDP Issuer URL (This URL must be publicly accessible). > S... [08:12:58] (SystemdUnitFailed) firing: logrotate.service Failed on cp4052:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [08:42:03] 10Traffic: VarnishKafka logrotate fails on bookworm - https://phabricator.wikimedia.org/T346602 (10Fabfur) [08:48:00] 10Traffic, 10SRE, 10Patch-For-Review: VarnishKafka logrotate fails on bookworm - https://phabricator.wikimedia.org/T346602 (10Vgutierrez) [08:48:10] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10Vgutierrez) [09:45:59] 10Traffic, 10SRE, 10Patch-For-Review: VarnishKafka logrotate fails on bookworm - https://phabricator.wikimedia.org/T346602 (10Fabfur) Cannot test the actual change with PCC but tested on two different hosts (cp4050 && cp4052) to check behavior. The new logrotate configuration actually seems to rotate correc... [10:47:07] 10Traffic, 10SRE: VarnishKafka logrotate fails on bookworm - https://phabricator.wikimedia.org/T346602 (10Vgutierrez) 05Open→03Resolved a:03Fabfur [10:47:14] 10Traffic, 10SRE, 10Patch-For-Review: Upgrade Traffic hosts to bookworm - https://phabricator.wikimedia.org/T342154 (10Vgutierrez) [10:47:42] (SystemdUnitFailed) resolved: logrotate.service Failed on cp4052:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed [11:16:53] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, 10Release-Engineering-Team (Seen): Serve production traffic via Kubernetes - https://phabricator.wikimedia.org/T290536 (10Clement_Goubert) [13:47:10] 10netops, 10Infrastructure-Foundations, 10Observability-Metrics: cr*-eqsin long poll times from librenms - https://phabricator.wikimedia.org/T346606 (10fgiunchedi) + netops for visibility since this can impact network devices [13:48:19] 10netops, 10Infrastructure-Foundations, 10Observability-Metrics: cr*-eqsin long poll times from librenms - https://phabricator.wikimedia.org/T346606 (10ayounsi) We had a quick chat on IRC. the `ports` and `bgp-peers` modules are the ones taking the most time, so no need to focus on the `snmp-max-oids` Libre... [14:00:24] 10netops, 10Infrastructure-Foundations, 10SRE: scrape ripe atlas data for a few anchors at other large networks - https://phabricator.wikimedia.org/T252890 (10CDanis) 05Open→03Declined >>! In T252890#9165519, @ayounsi wrote: > @CDanis Is that still needed now that we have NEL? It would be interesting t... [14:26:03] 10Traffic: Traffic cache daemon restart scripts need some rework - https://phabricator.wikimedia.org/T346640 (10BBlack) p:05Triage→03Medium [14:27:27] 10Traffic: Traffic cache daemon restart scripts need some rework - https://phabricator.wikimedia.org/T346640 (10BBlack) [14:58:58] 10Traffic, 10RESTBase, 10RESTBase-API, 10SRE: REST API not returning latest page when queried title is a redirect - https://phabricator.wikimedia.org/T346579 (10akosiaris) I 'll admit I am a bit stumped here. This is clearly not the CDN's fault as RESTBase exhibits the same behavior while also violating wh... [17:18:25] 10Traffic, 10Privacy Engineering, 10SRE: Create and document Wikidough's privacy policy - https://phabricator.wikimedia.org/T275409 (10ssingh) a:05ssingh→03None [17:18:36] 10Traffic, 10Privacy Engineering, 10SRE: Create and document Wikidough's privacy policy - https://phabricator.wikimedia.org/T275409 (10ssingh) a:03ssingh [18:45:36] 10Wikimedia-Apache-configuration, 10SRE, 10serviceops: Investigate and restore K.A.Z httpbb test - https://phabricator.wikimedia.org/T289022 (10RLazarus) a:05RLazarus→03None [18:45:48] 10Wikimedia-Apache-configuration, 10SRE, 10serviceops: Investigate and restore K.A.Z httpbb test - https://phabricator.wikimedia.org/T289022 (10RLazarus) a:03RLazarus