[08:12:57] (VarnishTrafficDrop) firing: 62% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [08:22:57] (VarnishTrafficDrop) resolved: 69% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [12:04:43] 10netops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10Infrastructure-Foundations, 10serviceops: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL ) - https://phabricator.wikimedia.org/T283582 (10hashar) That is still happening from time to time. Any person or team I can raise th... [13:55:43] 10Traffic, 10SRE, 10vm-requests, 10Patch-For-Review: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10Dzahn) Ready to create Ganeti VM durum2001.codfw.wmnet in the ganeti01.svc.codfw.wmnet cluster on row C with 2 vCPUs, 4GB of RAM, 15GB of disk in the private network. Read... [15:32:17] 10Traffic, 10SRE, 10vm-requests, 10Patch-For-Review: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10Dzahn) @ssingh durum2001, durum2002 and durum3001, durum3002 are ready to use, OS is installed but no role assigned. You can go ahead there. Other VM creations are in pro... [15:32:43] 10Traffic, 10SRE, 10vm-requests, 10Patch-For-Review: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10Dzahn) [15:34:03] 10Traffic, 10SRE, 10vm-requests, 10Patch-For-Review: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10ssingh) Thank you for all the help, @Dzahn! And yes, I will take it from there, thanks. [15:42:57] (VarnishTrafficDrop) firing: 52% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [16:36:17] 10Traffic, 10SRE, 10Patch-For-Review: Low root disk space on multiple eqsin cp nodes - https://phabricator.wikimedia.org/T290305 (10ema) 05Open→03Resolved a:03ema trafficserver-tls is not writing to local syslog anymore, and all eqsin hosts have at least ~ 30% available disk space now. Closing. [16:52:57] (VarnishTrafficDrop) resolved: 68% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [18:33:57] (VarnishTrafficDrop) firing: 48% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [18:38:57] (VarnishTrafficDrop) resolved: 59% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [18:52:41] 10Traffic, 10Analytics, 10Data-Engineering, 10Growth-Team, and 4 others: Migrated Server-side EventLogging events recording http.client_ip as 127.0.0.1 - https://phabricator.wikimedia.org/T288853 (10nettrom_WMF) Hi all, coming back to this as I've been OoO. As mentioned in T288853#7288436, having this data...