[06:07:59] <godog>	 greetings
[07:34:45] <dcaro>	 morning!
[08:11:50] <dhinus>	 hello from berlin!
[08:12:05] <dcaro>	 \o/
[08:12:21] <dcaro>	 dhinus: how was the trip?
[08:16:46] <dhinus>	 all good thanks! today I'm with thilp and aputhin in a coworking space, while thu+fri I'll be at https://signalsconf.io/
[08:22:11] <dcaro>	 nice, eager to hear the review of the talks xd
[08:37:35] <godog>	 dhinus: pics or didn't happen!!1!!1one
[08:38:21] <dcaro>	 xd
[12:42:04] <taavi>	 I did some digging at the haproxy issues from last night, seems like the problem was that the new haproxy setup didn't place limits on backend connection pool sizes. and if you combine that with dynamic backend selection (which is very unlike the toolforge setup with ~3 backends servers that handle traffic for ALL tools), that ends up eating the
[12:42:04] <taavi>	 backend connections slots and so runs out of the ephemeral ports on the host
[12:42:18] <taavi>	 https://gerrit.wikimedia.org/r/c/operations/puppet/+/1338185 sets up a limit on the max connection pool size
[12:42:45] <taavi>	 so if that limit is there, it should just close old idle connections to make room for new ones if the limit is hit
[13:01:51] <volans>	 thanks for looking into that taavi
[13:03:16] <volans>	 why http limit much higher than https? don't we redirect everything to https?
[13:03:51] <taavi>	 those are the backends, not frontends
[13:04:09] <taavi>	 a lot less people use the feature to use tls for the backend traffic than just use plaintext for the last hop from the proxy
[13:04:38] <volans>	 right right
[13:04:47] <volans>	 +1ed
[13:05:18] <taavi>	 thanks!
[13:14:53] <andrewbogott>	 taavi: think I should restart wsexport now? Or leave that for the project folks?
[13:15:35] <taavi>	 andrewbogott: already did! just still in the middle of typing a comment to the task
[13:15:50] <andrewbogott>	 ok!
[15:07:58] <komla>	 We need one more +1 on this task: T437117. 
[15:08:00] <stashbot>	 T437117: Quota increase request for project deployment-prep - https://phabricator.wikimedia.org/T437117
[15:11:29] <taavi>	 komla: there is a +1 in there already?
[15:12:36] <komla>	 taavi: Yes. They are requesting for more than double the current quota. We need an additional +1, right?
[15:12:43] <dhinus>	 komla: added my +1
[15:12:54] <komla>	 thanks!
[15:13:02] <taavi>	 duh, that's true
[15:13:29] <taavi>	 it's an unusual enough category to request that I somehow didn't think about that
[16:33:57] * dhinus off
[18:37:03] <andrewbogott>	 dduvall: are you the keeper of the cloud-runner-staging k8s cluster, or is that someone else?
[18:40:51] <dduvall>	 andrewbogott: de facto owner, yep
[18:41:25] <andrewbogott>	 ok, I'm about to make you an upgrade ticket. Since you already built one magnum cluster with the new driver, hopefully it won't be too awful...
[18:41:36] <dduvall>	 sounds good
[18:41:50] <andrewbogott>	 also I'm trying to take your advice and provide ready-made templates for this. If the templates that I already made don't hit the spot lmk and I can make another one
[18:42:05] <dduvall>	 awesome!
[18:42:26] <dduvall>	 yeah, that would be a big help in magnum use i think
[18:43:07] <dduvall>	 currently off in ST6.1 (SBOMs) land, but i can circle back to cloud-runner cluster on magnum work next quarter hopefully
[18:45:54] <andrewbogott>	 next quarter is fine, I'm just trying to get people notice since right now we can't politely upgrade Openstack without messing up old magnum clusters.
[18:47:14] <andrewbogott>	 https://phabricator.wikimedia.org/T437498