[05:25:26] (PurgedHighEventLag) firing: High event process lag with purged on cp3069:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://grafana.wikimedia.org/d/RvscY1CZk/purged?var-datasource=esams%20prometheus/ops&var-instance=cp3069 - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [05:30:26] (PurgedHighEventLag) firing: (4) High event process lag with purged on cp1078:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [07:52:27] 10Traffic, 10DNS, 10SRE: DNS Update, Google Postmaster Tools - https://phabricator.wikimedia.org/T349942 (10Aklapper) Who is we? [09:30:26] (PurgedHighEventLag) firing: (4) High event process lag with purged on cp1078:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [13:30:26] (PurgedHighEventLag) firing: (4) High event process lag with purged on cp1078:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [17:30:42] (PurgedHighEventLag) firing: (4) High event process lag with purged on cp1078:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [19:50:27] (PurgedHighEventLag) firing: (5) High event process lag with purged on cp1078:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [20:20:26] (PurgedHighEventLag) firing: (4) High event process lag with purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [20:20:42] (PurgedHighEventLag) firing: (5) High event process lag with purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [20:24:57] (PurgedHighBacklogQueue) firing: Large backlog queue for purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://grafana.wikimedia.org/d/RvscY1CZk/purged?var-datasource=eqiad%20prometheus/ops&var-instance=cp1089 - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighBacklogQueue [20:25:27] (PurgedHighEventLag) resolved: (5) High event process lag with purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [20:29:57] (PurgedHighBacklogQueue) firing: (2) Large backlog queue for purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighBacklogQueue [20:34:14] (HAProxyRestarted) firing: HAProxy server restarted on cp3069:9100 - https://wikitech.wikimedia.org/wiki/HAProxy#HAProxy_for_edge_caching - https://grafana.wikimedia.org/d/gQblbjtnk/haproxy-drilldown?orgId=1&var-site=esams%20prometheus/ops&var-instance=cp3069&viewPanel=10 - https://alerts.wikimedia.org/?q=alertname%3DHAProxyRestarted [20:37:44] (VarnishChildRestarted) firing: varnish-text restarted on cp1089 - https://wikitech.wikimedia.org/wiki/Varnish - https://grafana.wikimedia.org/d/000000330/varnish-machine-stats?orgId=1&viewPanel=66&var-server=cp1089&datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DVarnishChildRestarted [20:39:14] (HAProxyRestarted) firing: (2) HAProxy server restarted on cp1089:9100 - https://wikitech.wikimedia.org/wiki/HAProxy#HAProxy_for_edge_caching - https://alerts.wikimedia.org/?q=alertname%3DHAProxyRestarted [20:39:57] (PurgedHighBacklogQueue) firing: (3) Large backlog queue for purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighBacklogQueue [20:44:57] (PurgedHighBacklogQueue) firing: (2) Large backlog queue for purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighBacklogQueue [20:51:02] (PurgedHighEventLag) firing: High event process lag with purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://grafana.wikimedia.org/d/RvscY1CZk/purged?var-datasource=eqiad%20prometheus/ops&var-instance=cp1089 - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [20:56:02] (PurgedHighEventLag) resolved: High event process lag with purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://grafana.wikimedia.org/d/RvscY1CZk/purged?var-datasource=eqiad%20prometheus/ops&var-instance=cp1089 - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighEventLag [21:14:14] (HAProxyRestarted) firing: (2) HAProxy server restarted on cp1089:9100 - https://wikitech.wikimedia.org/wiki/HAProxy#HAProxy_for_edge_caching - https://alerts.wikimedia.org/?q=alertname%3DHAProxyRestarted [21:14:29] (HAProxyRestarted) resolved: (2) HAProxy server restarted on cp1089:9100 - https://wikitech.wikimedia.org/wiki/HAProxy#HAProxy_for_edge_caching - https://alerts.wikimedia.org/?q=alertname%3DHAProxyRestarted [21:14:58] (PurgedHighBacklogQueue) resolved: (4) Large backlog queue for purged on cp1089:2112 - https://wikitech.wikimedia.org/wiki/Purged#Alerts - https://alerts.wikimedia.org/?q=alertname%3DPurgedHighBacklogQueue [21:17:44] (VarnishChildRestarted) resolved: varnish-text restarted on cp1089 - https://wikitech.wikimedia.org/wiki/Varnish - https://grafana.wikimedia.org/d/000000330/varnish-machine-stats?orgId=1&viewPanel=66&var-server=cp1089&datasource=eqiad%20prometheus/ops - https://alerts.wikimedia.org/?q=alertname%3DVarnishChildRestarted