[10:32:28] GitLab needs a short maintenance in 30 minutes. [10:38:15] topranks: Can I merge your gnmi_telemetry change? [10:38:40] claime: was just about to ping you :) [10:38:42] yep please do [10:39:42] topranks: done [10:39:46] ty! [11:09:07] GitLab maintenance finished [11:32:45] Heads-up: I've been asked to increase the eventgate-analytics-external logging to trace on the canary deployment for a short while to support troubleshooting a UBN: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1050588 [11:34:01] I think it's safe to do for a short while, but feel free to intervene if you feel that I should not go ahead. [11:45:52] Deployed. [13:38:37] Now reverted. [14:17:36] jclark-ctr: am I safe to merge changes for hosts/krb1002.yaml in netbox? [14:21:29] yes [14:21:54] ok [15:16:28] urandom: speaking of lots of sessionstore roundtrips, https://trace.wikimedia.org/trace/ce72b185b0bfe6e5f7b1d29f67b28cc7 [15:36:46] cdanis: good grief [15:37:06] :) [15:37:14] and what's with that DELETE? [15:39:00] every DELETE I've ever seen takes 50ms+ [15:39:59] ok I stand corrected, here's one that only took 37ms https://trace.wikimedia.org/trace/3fac1d1f1694839239c835f8e70fe9e4 [15:40:15] if you want to play around urandom, example search: https://trace.wikimedia.org/search?end=1719589186932000&limit=1500&lookback=1h&maxDuration=45ms&minDuration&service=sessionstore&start=1719585586932000&tags=%7B%22http.method%22%3A%22DELETE%22%7D [15:40:33] it shows teh top-level trace name in the result list, but the search parameters apply to the matching sessionstore span [15:41:39] cdanis: DELETEs use a quorum in each DC [15:41:53] so you'll always have cross-DC latency there [15:42:16] yeah that makes sense, and is what I had guessed basically [15:42:33] oh but you meant, what's with the DELETE after we had just done GETs and POSTs [15:42:37] that I don't know [15:42:46] right [15:42:50] why do one? [15:42:58] https://trace.wikimedia.org/trace/ccdc9b7262019923e90e2317eb6b3c0d [15:43:06] I dunno but there's a lot of examples haha [15:48:03] indeed, yeesh [15:56:53] I bet it boils down to storage semantic limitations (i.e. storing the session as an opaque blob), and the need to reconcile/harmonize the two types of sessions [15:58:03] I once spent a little time trying to go through that code, and found it pretty impenetrable [16:04:42] I believe it, on both counts