[08:01:18] brb [09:57:09] moritzm: morning, do you know who maintains the package nginx-extras from the http://apt.wikimedia.org/wikimedia stretch-wikimedia/main repo? [09:59:54] good morning, this is a more recent version of nginx than what is in standard Stretch, the reason was that back in the past Nginx was used for TLS termination and this needed more recent crypto. the caches have moved away from Nginx and since then I've been backporting security fixes to that version (since other roles apart from the caches still use nginx on Stretch) [10:00:13] so kinda me I guess :-) what do you need? [11:00:42] jbond: wmf-certificates seems to be having some issues across the fleet [11:01:32] vgutierrez: see -ops should be fixed by now by mori.tz [11:01:44] ack <3 [13:38:51] volans: is it intentional that 'cumin' exits with 0 when zero hosts match? [13:47:34] kormat: it's clearly intentional in the code, I can't recall if there was a specific reason though, in a meeting, can check it later [13:47:50] hah ack, thanks :) [13:48:42] <_joe_> kormat: doesn't it ask you to confirm it's operating on 0 hosts? [13:48:53] * kormat giggles [14:25:42] kormat: so that logic is there since the first initial import with the first working version. [14:25:58] volans: ah, an Original Sin! [14:26:12] I guess I could add a dedicated exit code for no hosts found and let it exit with that [14:26:29] technically speaking everything worked, you did a query and no host matches [14:26:44] I guess there might be valid use cases for that [14:28:24] volans: i ran into this because i have a script that does, effectively `$(cumin -n "$@" 2>/dev/null)`, and weird stuff happened when nothing matched [14:29:12] kormat: are you automating the automation? [14:29:34] ema: i'm rubbing two sticks together and wondering why my hair catches fire [14:30:22] kormat: what does that script do? [14:30:27] maybe there are better ways [14:30:49] like if you conver that to python using cumin as a library you would have full control [14:31:19] nop. using python means having to run it via sudo, which requires passing a full path. it's all Pain. [14:31:52] the script was `nodeset -e -S'\n' $(sudo cumin -n "$@" 2> /dev/null)` [14:32:16] i.e. given this cumin pattern, give me a list of fqdns of matching hosts [14:32:25] you're using sudo here too :) [14:32:47] volans: yes. _inside the script_. which you can do with bash, and you cannot do with python. [14:33:35] sure, but no need for the full path, sudo ./foo works :-P [14:33:56] volans: if you're in the same dir, yes [14:34:01] but it won't search $PATH [14:34:33] so this script, `nodelist`, is used by a bunch of different scripts i have [14:34:38] and it's in ~kormat/bin [14:34:54] if it was a python script, they'd all need to do `sudo ~kormat/bin/nodelist.py` [14:35:00] as opposed to just `nodelist` [14:35:33] I think the current logic makes sense, given you can use cumin to identify all hosts matched by a certain globbing, not matching any also seems like a proper success case suitable for returning 0 [14:36:36] kormat: if you change it to: "$(sudo cumin -n "$@" 2> /dev/null)" [14:36:41] quotes around the subcommand [14:36:46] doesn't solve your weirdness? [14:36:59] $ nodeset -e -S'\n' "" [14:37:09] just gives a new line [14:37:40] volans: yeah. so a blank line, and no error. that's, uh, perfectly suboptimal [14:38:09] i worked around it, btw: https://phabricator.wikimedia.org/P17497 [14:38:17] you can check if what nodeset returns is empty or not in bash :D [14:40:03] moritzm: i'd argue that matching nothing == error is principal of least surprise. e.g. `ls filethatdoesntexist` [14:41:19] or, at the very least, if no command is provided, and the pattern matches nothing, exit with 1 [14:41:53] `ls emptydirectory` returns zero though [14:42:06] and `sudo cumin A:aliasthatdoesntexist` exits 99 [14:43:44] I don't think either approach is unreasonable tbh, but it's true that adding a nonzero status makes it easier to differentiate even if it's an OK outcome [14:47:37] it's not a big deal in any case, i've worked around it here, and just need to remember to do the same in any similar future case. [16:24:15] godog: hi! Got a bunch of emails about crons failing on pontoon-thanos-01, are you aware of it? (as in, need any help? are you already working/managing/etc. it?) [16:34:41] moritzm: can you ack this change? there was a cloudvirt1020 host that changed it's role, but I think it was uninntentional https://gerrit.wikimedia.org/r/c/operations/puppet/+/731781 [16:37:33] moritzm: it's breaking puppet runs on our side, so I'll merge, but let me know if it was intentional [16:39:54] dcaro: we are all in the SRE Monday meeting (and godog is presenting even!) so expect a delay in responses :) [16:41:02] paravoid: ack [16:43:35] dcaro: sorry for that, that was an unintentional c&p hunk which wasn't intended to be in that patch, thanks for fixing! [16:55:56] dcaro: thanks, yeah I'm aware, the cronspam has stopped and I'll fix it properly [17:00:31] godog: thanks! [17:00:33] moritzm: ack [20:11:29] anyone know how to run/test mediawiki-config locally? [20:52:17] do we really want hosts like "testvm", "foo-test" and "foo-dev" to have Icinga prod monitoring? I mean it's pretty easy to deactivate by role or regex in Hiera but we don't and instead react to alerts on dev/test hosts by.. disabling notifications. so then we looked, icinga spent the work and it still creates noise in the web UI, why not skip it in the first place [20:53:48] then we end up ACKing with reasons like "dev hosts with prod alerts" just to reduce Icinga noise so we can see the actual issues