[00:05:50] razzi: I'm not familiar with the haproxy setup for our dbproxy but AFAIK haproxy in addition to the wen dashboard can have a runtime socket API usable via socat [00:06:28] for this specific case I would check with the data persistence folks [00:07:58] Indeed volans|off , there is an `echo "command" | socat /run/haproxy/haproxy.sock stdio` interface [00:08:21] I'll look around the command there, good tip! [10:24:09] <_joe_> I have an interesting problem - I was looking at cloud IP ranges for T302471 and azure only publishes a doc with 55k ranges, of which about 20k are /32 [10:27:56] <_joe_> does anyone know of a better source? [10:28:10] that seems ... unhelpful of them :( [10:30:55] <_joe_> Emperor: well it depends on what's your use case [10:31:10] <_joe_> given I need to build acls for varnish, that's not really feasible indeed [10:33:00] do they look like they can be automatically aggregated? [10:33:04] i think there are tools that can help with that [10:33:08] (although I've not personally used any) [10:34:17] <_joe_> question_mark: I was thinking of writing one :D [10:34:33] https://projects.horms.net/projects/aggregate/ maybe? [10:34:41] let's not NIH unless we need to ; [10:35:20] looks to be part of debian [10:35:38] https://packages.debian.org/bullseye/aggregate [10:36:36] <_joe_> question_mark: yeah I would've looked around before actually doing anything more than "write 50 lines of python for one-time usage" [10:37:05] <_joe_> I mostly wanted to be able to assess if that list of horrors can be reduce [10:37:06] <_joe_> *d [10:37:28] I'd expect so (and might be tempted to "round up" if not) [10:38:10] <_joe_> Emperor: yeah well long term I want the process to be automated, but first I need to see if it's feasible. [10:42:08] <_joe_> ok only counting the ipv4 addresses, aggregate spits 1k entries [10:42:13] <_joe_> still a lot [10:45:32] <_joe_> https://www.gstatic.com/ipranges/cloud.json OTOH is quite usable [10:46:06] where are you thinking of using them? [10:46:31] <_joe_> we have ip ranges for vaarious clouds we use for acls in varnish [10:46:50] ah varnish [10:47:15] <_joe_> given I want to copy them over to etcd from puppet, I was looking at how we could gather the IP ranges fo the various clouds systematically [10:47:55] so besides aggregating them (there are a bunch of tools), you could also explore storing them in an MMDB file [10:48:05] <_joe_> just a short side quest, I don't strictly need to do it now, but the problem seems ok [10:48:31] <_joe_> oh right, varnish can read from MMDB right? [10:48:43] you can use the standard libraries that we use for GeoIP, libmaxminddb etc. [10:50:54] on the same note... is Azure a separate ASN than 8075? [10:51:05] 8075 is Microsoft, which includes also Bing etc. [10:51:17] but if Azure is split off from that, then you could use the maxmind ASN database to figure out their ranges [10:52:02] <_joe_> yes, that's a potentially interesting option for IP blocks indeed [10:52:16] <_joe_> for now I'll just import what we currently have in puppet [10:52:54] <_joe_> but if we can build an mmdb file for all the clouds that's probably a better options for those than using explicit acl objects like we do now [10:53:50] now I wonder if we still have netmapper [10:53:55] but ok I digress :P [10:56:41] Azure provides something similar BTW: https://www.microsoft.com/en-us/download/confirmation.aspx?id=56519 [10:57:24] proper link: https://download.microsoft.com/download/7/1/D/71D86715-5596-4529-9B13-DA13A5DE5B63/ServiceTags_Public_20220221.json [12:04:48] _joe_: see also prior work on T270391 [12:04:49] T270391: varnish filtering: should we automatically update public_cloud_nets - https://phabricator.wikimedia.org/T270391 [12:10:55] <_joe_> volans|off: yes I am aware of that, I was trying to assess the size of the data for feasibility both in varnish and in the analytics pipeline [12:12:19] indeed, it's a *long* list, more clouds you add more will be large, and will grow with time [12:31:31] ^^ on this we should weigh up how bad the "collateral" damage would be on building the lists just based on ASN / announced public prefixes. [12:32:05] That would be a shorter list, be include *everything* google/microsoft/amazon have, not just the space used by their respective public clouds. [12:40:56] <_joe_> so [12:41:09] <_joe_> if we take the approach of being specific about what we ban [12:41:25] <_joe_> a broad cannon should be ok-is for everything but google [12:42:05] <_joe_> I think paravoid's suggestion is actually quite good, and I am wondering if mm actually already have that data [12:42:50] <_joe_> I think it's the only way to do fine-grained bans of specific clouds (or even specific AZs!) efficiently [13:29:20] FYI, I'm disabling puppet in codfw for ~5 minutes [13:34:45] now re-enabled [13:46:47] hello please hold on any netbox changes for a few minutes, we're restoring a backup after I clicked the wrong button [13:53:55] netbox backup has been restored, all looks good, it shoud be good to resume normal operations [13:54:54] <3 [14:04:29] Friday afternoon a fine time to test our backups ;-)