[11:10:28] lunch [13:29:02] o/ [15:44:22] gehel (and anyone else who might interested): I assume we are cancelling triage this week in favor of the DPE staff meeting. After finishing up the ICU token repair stuff and while waiting for the big reindexing and post-reindex eval to come, I created a ticked for a smaller more interruptable harmonization task (dotted_I_fix) to work on instead of taking on a more complicated task. See T358495 for details. And if anyone has [15:44:22] questions about the task or the estimate (5), I'm happy to discuss here or in email. [15:44:22] T358495: Enable dotted_I_fix (almost?) everywhere - https://phabricator.wikimedia.org/T358495 [15:47:57] Yep, we'll cancel triage. I'll update the calendar. Thanks for the reminder! [16:00:34] \o [16:03:15] o/ [16:51:36] workout, back in ~40 [17:31:17] dinner [17:47:45] realized i'm not setting a unique pipeline.name for -backfill releases. What does that do? I suppose i'll make it unique but not clear what would go wrong [17:50:01] That's only relevant for failures [17:50:31] At least, that's where we explicitly reference this information [17:50:40] hmm, ok [17:51:13] Kafka consumer group might also be named based on it [17:54:18] That cloud cause unexpected start offsets if the application is started without state and fetches the offsets from Kafka directly [17:58:24] hmm, we have specific arguments to set the consumer group ids, but i wasn't setting those either. Would be nice if they were templated, but i was trying to not be the first person to introduce templated values files in this repo :) [18:03:44] back [18:50:49] lunch, back in ~50 [19:08:43] unless there are concerns, intending to move consumer-cloudelastic to 100% of public wikis later today, as we discusseed last week. Can turn off cirrus writes tomorrow [19:18:35] ebernhardson excellent. How's it going w/the backfill process? I dunno if I can contribute much but LMK if I can help [19:18:37] also back [19:19:06] inflatador: it should be ready to go, david merged the script earlier today. I've run it a few times and am just tweaking some dashboard bits now [19:19:52] awesome, thanks for the update [19:28:09] (r) ;p [19:50:30] small change for migrating an icinga check if anyone has time to review: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1006564 [20:01:55] Thanks for the +1 ;) [20:05:07] just merged the change, keeping eye out for alerts [20:18:59] andd...we're getting too many alerts. Will roll back and start again [20:35:45] ebernhardson: I set the ES response timeout to match the server side default: https://phabricator.wikimedia.org/T356933, but that could be configured already and would not need a new version. I also started working on an alternative ES sink but that is not ready yet. [20:41:12] pfischer: excellent! I would note it's not quite the cluster state we are waiting on, while that can be updated by write requests (auto index creation, dynamic mapping) we have that all turned off [20:41:58] i guess it is cluster state though, we are waiting for the index shard to become active in cluster state [21:21:01] cloudelastic writes from SUP are now all enabled, looks like everything is going fine. [21:25:08] {◕ ◡ ◕} [22:44:45] * ebernhardson wonders if "read-only" in cirrus should get a better name...it should perhaps mean "no data writes" while still allowing schema upgrades and such [23:08:06] ContentUpdatesDisabled?