[07:27:53] In 30 minutes we'll switch over phabricator db master [07:29:57] That's a sane time for you [07:30:11] quite late, I know [07:30:49] * RhinosF1 is very used to you being an early bird and doing it before I wake up [07:31:23] yeah I normally do, but this one isn't a super complex one (famous last words) [08:03:45] <_joe_> marostegui: so after that epic jinx, when is the phab database get switched to a server that will perish within 10 minutes? [08:03:49] <_joe_> like now? [08:03:54] _joe_: probably! [08:04:00] so if you need to write a task you've got 10 minutes [10:02:52] probably-obvious python question. sys.exit exits the program by raising SystemExit - so if you have it inside an except BaseException block (or equivalent), you may need to handle it. sys.excepthook() is a function called "When an exception is raised and uncaught". So why doesn't excepthook get called when SystemExit is raised? [10:03:34] Emperor: what are you trying to do? [10:04:10] [for a bonus point: where is this documented?] [10:04:31] volans: currently, trying to understand this edge case :) [longer answer incoming] [10:07:08] volans: I'm trying to fix swift-ring-builder's error handling (TL;DR exit 1 on warnings, 2 on exits; but doesn't handle some exceptions properly resulting in exit 1 when should be to) - https://review.opendev.org/c/openstack/swift/+/829053 ; Tim Burke suggested that sys.excepthook might be an alternative to try/except since you'd not have to think about SystemExit. My initial reaction was "well, surely you'd still have to handle [10:07:08] SystemExit", but I did some testing and it seems that (uniquely?) sys.excepthook doesn't get called on an unhandled SystemExit exception. It does get other BaseExceptions (e.g. KeyboardInterrupt) [10:08:13] Emperor: i would expect that SystemExit is _implicitly_ handled [10:10:28] Emperor: "When it is not handled, the Python interpreter exits; no stack traceback is printed" https://docs.python.org/3/library/exceptions.html#SystemExit [10:10:36] that's the normal behaviour for SystemExit [10:11:20] just exits, so seems normal to me that it doesn't call excepthook [10:12:00] yes, I agree it could be documented on sys.excepthook() too [10:12:40] My intuition is obviously wrong, but I would expect that behaviour to be implemented in sys.excepthook :) [10:14:07] Now, do I want to shave the necessary yaks to try and submit a docs patch to python upstream...? [10:16:11] Emperor: import sys; help(sys): excepthook -- called to handle any uncaught exception other than SystemExit [10:16:19] :D [10:17:23] WTF, why doesn't help(sys.excepthook) say that? 😿 [10:17:41] I got there from git grep on the source code tbh [10:18:27] https://docs.python.org/3/library/sys.html#sys.excepthook likewise lacking that helpful comment [10:18:55] we both know that your fix is a workaround, the proper one would be to fix the program itself and its status codes, but that ofc would break compatibility, so I guess you're trying the minimal change that does the same thing [10:20:16] volans: yeah, there are clearly still some edge cases where swift-ring-builder will exit 1 where it shouldn't, but those aren't really fixable without writing a wrapper in some other language. Having exit 1 be anything other than error in a python program was a mistake, but I think trying to change the error codes for s-r-b is a non-starter [10:21:05] yeah [10:22:31] This fix (which I will now re-do with an excepthook) will make s-r-b's behaviour much closer to correct than it was :) [10:26:42] if you're curious the code is in https://github.com/python/cpython/blob/main/Python/pythonrun.c#L777, the call to handle_system_exit() makes it bail out before calling the excepthook a bit down from there [10:33:11] Thanks :) [10:33:39] * Emperor has opened a bug report asking for the docs to be made a bit more consistent [10:33:54] ...this is obviously not as good as actually patching the docs, but there we are... [10:34:11] fair enough! it's clearly not explicitly stated and can be confusing [10:35:03] (figuring out how to patch python docs upstream is a yak-shave to far given I'm _meant_ to be getting my swift ring manager code to a deployable state, not fixing swift-ring-builder...) [10:36:35] Emperor: mandatory https://www.youtube.com/watch?v=AbSehcT19u0 ;) [10:41:30] lol [10:43:18] [I will admit, there is non-zero chance of me snapping and putting in a docs patch at some point in a week or two when it's still bugging me] [10:48:06] anyhow, thanks for the assistance :) [10:49:43] anytime! That's the same way I landed a patch to cpython, there was a bug I opened that was bugging me so I sent also the patch [11:45:37] Ah, yes, you have to sign a CLA to fix python's docs... [11:48:45] IIRC it's the same docs or code, I did it for the code [11:49:15] and as I have all setup already if you want I could send the patch for you :) [11:51:11] :) [is that a personal CLA or a corporate one?] [11:51:30] personal [11:51:37] the one I signed [11:52:05] was back in 2017, so rules might have changed [16:35:22] Hi, I'm not getting any output from a ganeti console, probably missing something obvious, my command is: `sudo gnt-instance console datahubsearch1001.eqiad.wmnet` [16:36:06] `gnt-instance show` reports "State: configured to be up, actual state is up" [16:36:40] Does anybody have any tips on troubleshooting this? [16:37:49] I think it's kinda known, moritzm should have more details [16:40:35] razzi: did you see comments in https://gerrit.wikimedia.org/r/c/operations/puppet/+/762850 ? [16:41:21] In the older icinga checks we often had two alert levels, one for warning and one for critical. In alertmanager it seems possible with templated conditionals but not seeing anything actually doing that, is there a preferenace for separate alert definitions, or any particular guidance there? [16:42:34] elukey: oops ok yes I was missing something obvious. [16:42:58] ebernhardson: We have a few such checks in the data-engineering team alert. I'll find a link. [16:44:23] This one, for instance: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/alerts/+/refs/heads/master/team-data-engineering/druid.yaml#5 [16:45:21] btullis: thanks! Interesting, i hadn't thought that you could name two alerts the same thing. But makes sense [16:46:04] Different severity levels match different routes, so for example we have 'page','critical','warning' and 'task': https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/alertmanager/templates/alertmanager.yml.erb#57 [16:46:50] A pleasure.