[12:13:24] test [12:13:38] dhinus: can we use https://wm-bot.wmcloud.org/logs/%23wikimedia-cloud-admin/ as the archive URL instead? [12:14:55] taavi: that's what I wanted to add but I somehow got the copy/paste wrong :D [12:16:29] ah I know why, I copied it from #wikimedia-sre :P [12:17:07] You know those are the first messages that will be archived forever and ever now ;) [12:20:18] I thought about something more memorable to write as the first message but I think "test" is a good one :D [12:20:19] haha like the oldest human writing is a list of how many cows each dude owned [12:20:29] rather than something profound about the ancients :) [12:20:54] * taavi hides [12:21:29] arturo: CI taking forever on that, will push as soon as it merges [12:21:43] topranks: thanks [12:28:23] arturo: ok merged now can you see any improvement? [12:32:52] seems better [12:32:56] https://www.irccloud.com/pastebin/bXJW8EVy/ [12:36:31] yep, looks good [12:43:37] topranks: excellent, thanks! [12:48:25] dcaro: do you have opinions about this? https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/19 [12:49:44] taavi: I think it's a bit premature to 'not recommend' using local storage [12:49:50] given that we have no real alternative [12:50:00] and we want tools that use it to move to it [12:50:34] maybe rephrasing like "we recommend not using local storage if possible" would be nicer [12:50:43] fair enough [12:51:17] or "if you don't need it" [12:51:52] what about the general idea of pushing tools that use nfs to explicitely specify that? [12:56:21] I updated the wording [13:03:42] aborrero@cloudlb1001:~ $ sudo ss -putan | grep 28774 | wc -l [13:03:42] 62 [13:03:42] aborrero@cloudlb1002:~ $ sudo ss -putan | grep 28774 | wc -l [13:03:42] 4 [13:04:49] taavi: hmm, maybe if we can double ask, as in "do you need NFS storage? [yN]" [13:04:49] xd [13:05:19] hmm, maybe we can ask if not specified, and force to use the option to avoid the prompt [13:06:41] I want to avoid users having their tools break "by default" [13:06:48] when migrating I mean [13:12:23] Anyone have opinions on https://toolsadmin.wikimedia.org/tools/membership/status/1543 I find it kind of sus [13:18:50] Rook: indeed, very odd.. [14:43:19] arturo: what do we need a third cloudlb server for? [14:43:45] taavi: I believe 3 is the magic redundancy number [14:44:36] yes if you're storing any state and need an odd number to avoid ties, but haproxy does not need that [14:55:07] arturo: those numbers of connections will vary based on the location of hosts connecting to them [14:55:38] significantly for traffic behind cloudnet you need to consider what cloudgw is active [14:56:04] currently thats cloudgw1002 in rack d5, so all those connections are going to cloudlb1002 as it's in the same rack [14:56:37] location as in number of hops? [14:57:04] (or first cloudlb that replies?) [15:07:44] taavi: in the case you are describing it is even a strong requirement. The thing is that if we lose 1 server I still would like to have redundancy [15:10:50] dcaro: the cloudsw will use a "directly connected" cloudlb first, after that it's down to best BGP route [15:11:01] so packet arriving in c8/d5 will use the local one [15:11:16] 👍 thanks for clarifying! [15:11:22] e4/f4 are both "1 hop" away from either, so they are load-balancing between the two [15:11:44] it's flow-based, so for a certain host it'll always take the same path, but for 2 separate hosts in say e4 they might connect via different LBs [15:12:12] an alternative we have here is to make the cloudlb's active/passive, so one is primary unless there is a problem [15:14:46] any objection to merging the security groups "toolsdb" and "toolsdb account creation"? I think the "account creation" rules can live inside the "toolsdb" sec group, using the "Description" field to explain their meaning [15:15:03] dhinus: go for it [15:15:24] (I was also annoyed by the fact "toolsdb account creation" had spaces in the name :P) [15:21:55] done, let me know if you spot any issues with maintain-dbusers [15:59:43] * dcaro off [16:05:21] * arturo offline [16:45:03] * dhinus off