[00:46:24] 10fundraising-tech-ops: Fundraising systems Debian Trixie support - https://phabricator.wikimedia.org/T401847#12291938 (10Dwisehaupt) [02:57:39] FIRING: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [10.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [02:57:39] FIRING: HighPaymentSchemeFailures: Average payment method scheme failures are high [10.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [02:57:39] FIRING: HighPaymentMethodFailures: Average cc payment method failures are high [10.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [03:17:39] RESOLVED: HighPaymentMethodFailures: Average cc payment method failures are high [6.9] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [03:17:39] RESOLVED: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [6.9] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [03:17:39] RESOLVED: HighPaymentSchemeFailures: Average payment method scheme failures are high [6.9] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [07:07:22] FIRING: GnmiInterfaceCountersDrop: ... [07:07:23] fasw1-f5b-codfw is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=fasw1-f5b-codfw:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [07:12:23] RESOLVED: GnmiInterfaceCountersDrop: ... [07:12:23] fasw1-f5b-codfw is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=fasw1-f5b-codfw:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [08:23:39] FIRING: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [11.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [08:23:39] FIRING: HighPaymentMethodFailures: Average cc payment method failures are high [11.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [08:23:39] FIRING: HighPaymentSchemeFailures: Average payment method scheme failures are high [11.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [08:28:39] RESOLVED: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [5.7] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [08:28:39] RESOLVED: HighPaymentMethodFailures: Average cc payment method failures are high [5.7] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [08:28:39] RESOLVED: HighPaymentSchemeFailures: Average payment method scheme failures are high [5.7] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [08:35:39] FIRING: HighPaymentMethodFailures: Average cc payment method failures are high [7.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [08:35:39] FIRING: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [7.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [08:35:39] FIRING: HighPaymentSchemeFailures: Average payment method scheme failures are high [7.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [09:35:39] RESOLVED: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [15.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [09:35:39] RESOLVED: HighPaymentMethodFailures: Average cc payment method failures are high [15.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [09:35:39] RESOLVED: HighPaymentSchemeFailures: Average payment method scheme failures are high [15.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [13:04:29] (03CR) 10Winston Sung: [C:04-1] "That doesn't sounds like 'SmartRedirect' or 'InteligentRedirect' in the original text for me." [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1221948 (https://phabricator.wikimedia.org/T399285) (owner: 10Anterdc99) [14:10:23] FIRING: GnmiInterfaceCountersDrop: ... [14:10:23] pfw1-eqiad is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=pfw1-eqiad:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [14:15:22] RESOLVED: GnmiInterfaceCountersDrop: ... [14:15:23] pfw1-eqiad is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=pfw1-eqiad:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [15:09:23] FIRING: GnmiInterfaceCountersDrop: ... [15:09:23] pfw1-eqiad is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=pfw1-eqiad:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [15:14:22] RESOLVED: GnmiInterfaceCountersDrop: ... [15:14:23] pfw1-eqiad is exporting less than half the gNMI interface counters it had 24h ago - https://wikitech.wikimedia.org/wiki/Network_monitoring#GnmiInterfaceCountersDrop - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?orgId=1&from=now-24h&to=now&var-site=%24__all&var-instance=pfw1-eqiad:9804&viewPanel=panel-19 - https://alerts.wikimedia.org/?q=alertname%3DGnmiInterfaceCountersDrop [17:11:27] (03CR) 10Umherirrender: [C:03+2] "Resubmit after rebase" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1331694 (https://phabricator.wikimedia.org/T432974) (owner: 10Wfan) [17:14:10] (03CR) 10Umherirrender: "I have missed the abandoned "Depends-On" patch set, if this patch set is still relevant, remove that line or abandon this patchset as well" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1331694 (https://phabricator.wikimedia.org/T432974) (owner: 10Wfan)