[09:09:25] 10netops, 10Infrastructure-Foundations: Arista engagement - new vendor evaluation/testing. - https://phabricator.wikimedia.org/T290716 (10cmooney) [09:10:15] 10netops, 10Infrastructure-Foundations: Arista engagement - new vendor evaluation/testing. - https://phabricator.wikimedia.org/T290716 (10cmooney) p:05Triage→03Medium [10:57:13] 10Traffic, 10SRE, 10vm-requests: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10Dzahn) Hi @ssingh In T290599 for durum1002 you asked for only 4GB RAM but here it is 8GB RAM. It seems either we need to recreate durum1002 with more RAM or all of them only need 4? [11:04:29] 10Traffic, 10SRE, 10vm-requests: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10ssingh) Hi @Dzahn: Thank you for catching this mistake and confirming before starting the process! I have updated the ticket and we do indeed need 4 GB per VM. [11:04:38] 10Traffic, 10SRE, 10vm-requests: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10ssingh) [11:33:10] 10Traffic, 10Infrastructure-Foundations, 10SRE: OpenSSL < 1.1.0 compatibility issues with new LE issuance chain - https://phabricator.wikimedia.org/T283165 (10MoritzMuehlenhoff) [11:36:51] 10Traffic, 10Infrastructure-Foundations, 10SRE: OpenSSL < 1.1.0 compatibility issues with new LE issuance chain - https://phabricator.wikimedia.org/T283165 (10MoritzMuehlenhoff) >>! In T283165#7103576, @Vgutierrez wrote: > As mentioned on the issue description, debian backported the fix for OpenSSL as it ca... [11:49:57] 10Traffic, 10SRE, 10vm-requests: Please create Ganeti VMs for durum - https://phabricator.wikimedia.org/T290672 (10Dzahn) a:03Dzahn [12:24:20] 10netops, 10Infrastructure-Foundations, 10Observability-Metrics, 10SRE: replace check_ripe_atlas Python script with a check_prometheus backed by atlasexporter data - https://phabricator.wikimedia.org/T251155 (10fgiunchedi) This should be a prometheus-native alert in `alerts.git` nowadays [13:03:24] bumping an OpenSSL compatibility concerns task just for testing is pretty evil, especially on a Friday [13:09:06] 10netops, 10Infrastructure-Foundations: Mellanox engagement - new vendor evaluation/testing. - https://phabricator.wikimedia.org/T290732 (10cmooney) [13:23:50] vgutierrez: are we deploying openssl 3.0.0 on a friday? :D [13:23:57] * elukey runs away [13:24:05] hopefully nope ;P [14:39:04] 10Traffic, 10Infrastructure-Foundations, 10SRE: OpenSSL < 1.1.0 compatibility issues with new LE issuance chain - https://phabricator.wikimedia.org/T283165 (10akosiaris) p:05Triage→03Medium [14:40:57] (VarnishTrafficDrop) firing: 54% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [14:45:56] (VarnishTrafficDrop) firing: (2) 63% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [14:50:56] (VarnishTrafficDrop) resolved: (2) 64% GET drop in text@ during the past 30 minutes - https://grafana.wikimedia.org/d/000000180/varnish-http-requests?viewPanel=6 - https://alerts.wikimedia.org [15:00:00] 10Traffic, 10SRE, 10Patch-For-Review, 10Performance-Team (Radar): Test haproxy as a WMF's CDN TLS terminator with real traffic - https://phabricator.wikimedia.org/T290005 (10Vgutierrez) [22:47:34] I want to add 3 new LVS endpoints next week for shellbox, when would be a good time to sync with someone on that? [22:48:01] and, would it be preferred if we added one after the other, or if we did all 3 at the same time? [22:51:21] legoktm: I'm not traffic-team but I suspect all three at the same time would be better -- LVS restarts are mildly disruptive [22:54:59] ack, that's what I was thinking too [22:55:15] I'll structure my git patches in that way then...on Monday