[12:12:35] i want to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/990962 which removes a vlan interface from lvs1018/20, i think it will need a manual `ifdown` run and the interface will need to be removed from /e/n/i. is that a 'safe' thing to do anytime or does it need some coordination? [12:36:10] I don't know that we've ever executed such a patch against a live lvs that was previously using such an interface. at least not in recent memory. [12:36:40] so, yeah, it leaves some questions as to how it will go :) [12:38:20] I don't see any ipvsadm entries referencing backend hosts in 10.64.37.x, so that's a good start (I guess they were all removed previously) [12:40:10] taavi: sounds about right though: a puppet agent run, and then check if it's necessarily (likely!) to remove the vlan1119 interface from /e/n/i and/or manually delete it from runtime as well. [12:40:28] start with lvs1020 first [12:44:49] re: the runtime interface removal, I would go for just: "ip link delete vlan1119 type vlan" (after /e/n/i removal) [12:45:15] (because ifdown is a script, and I don't know what else it might do) [12:47:26] alright, I'll try that and report back, thanks bblack. I disabled puppet on 1018 just in case [12:54:15] seems like everything went ok with that, so doing the same on 1018 [12:58:46] all done [13:12:43] awesome [13:47:21] 10netops, 10Infrastructure-Foundations, 10cloud-services-team: Remove cloud-support1-c-eqiad VLAN - https://phabricator.wikimedia.org/T355115 (10taavi) 05Open→03Resolved a:03taavi [13:48:00] 10netops, 10Infrastructure-Foundations, 10cloud-services-team: Remove cloud-support1-c-eqiad VLAN - https://phabricator.wikimedia.org/T355115 (10taavi) [16:08:13] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, 10Release-Engineering-Team (Seen): ThumbnailRender job calls $wgImageMagickConvertCommand - https://phabricator.wikimedia.org/T355243 (10Clement_Goubert) [16:08:47] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Move MediaWiki jobs to mw-on-k8s - https://phabricator.wikimedia.org/T349796 (10Clement_Goubert) [16:47:27] 10Traffic, 10Data-Engineering, 10Movement-Insights, 10Patch-For-Review: Identify and label prefetch proxy data in our traffic - https://phabricator.wikimedia.org/T346463 (10fkaelin) Thanks for the updates @dr0ptp4kt, and nice that you are able to reproduce such a google proxy request. One thing that I am... [17:30:07] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Migrate internal traffic to k8s - https://phabricator.wikimedia.org/T333120 (10kamila) [17:30:17] 10Traffic, 10MW-on-K8s, 10SRE, 10serviceops, and 2 others: Migrate mobileapps to k8s - https://phabricator.wikimedia.org/T350846 (10kamila) 05Open→03Resolved All traffic is now going to k8s \o/ I will keep an eye on php workers saturation, but it should be fine, so I'm calling it resolved. [18:13:05] 10Traffic, 10Data-Engineering, 10Movement-Insights, 10Patch-For-Review: Identify and label prefetch proxy data in our traffic - https://phabricator.wikimedia.org/T346463 (10dr0ptp4kt) Thanks @fkaelin . Yes, those prefetches happened without clicking on them. It seems to occur both for searches originating... [18:17:16] 10Traffic, 10Data-Engineering, 10Movement-Insights, 10Patch-For-Review: Identify and label prefetch proxy data in our traffic - https://phabricator.wikimedia.org/T346463 (10dr0ptp4kt) Just to put something concrete (not saying this is **the** thing), here's an interesting unit test on the prefetch predicto... [19:25:55] bblack v.gutierrez would it be possible to get a code review / deployment on https://gerrit.wikimedia.org/r/c/operations/puppet/+/981352 ? bblack if i should instead ping v.gutierrez tomorrow LMK (i know you're probably catching up on things) [19:35:59] 10Traffic, 10Data-Engineering, 10Movement-Insights, 10Patch-For-Review: Identify and label prefetch proxy data in our traffic - https://phabricator.wikimedia.org/T346463 (10dr0ptp4kt) [19:38:39] 10Traffic, 10Data-Engineering, 10Movement-Insights, 10Patch-For-Review: Identify and label prefetch proxy data in our traffic - https://phabricator.wikimedia.org/T346463 (10dr0ptp4kt) @fkaelin `Sec-Purpose: prefetch;prerender` is mentioned for the omnibox use case at https://developer.chrome.com/docs/web-p... [19:44:09] dr0ptp4kt: yeah ping vg please, I'm staying out of it :) [19:44:51] :laugh: , will do