[09:34:20] <ejoseph>	 dcausse: i'll be mins late
[09:34:32] <dcausse>	 ejoseph: np
[10:33:25] <dcausse>	 lunch
[13:01:01] <inflatador>	 greetings! We're getting those "April showers" I've heard so much about
[14:13:51] <i22>	 hi
[14:23:00] <dcausse>	 o/
[14:23:37] <inflatador>	 <o/
[14:27:09] <inflatador>	 dcausse was going to ask you, we are missing some packages for bullseye, packages that appear to have been built specifically for us (search team) but aren't hosted out of our elastic68 repo. (liblogstash-gelf-java-doc and elasticsearch-madvise). Do you think these should be moved to the elastic68 repo, or should we just ask to publish them in the main repo for bullseye (they are in the main repo on Stretch)
[14:52:30] <dcausse>	 inflatador: I wonder if we still use liblogstash-gelf-java-doc, for elasticsearch-madvise we should moved them to the elastic68 repo (not sure why it's bound to a particular elastic apt component tho)
[14:58:26] <ebernhardson>	 \o
[14:58:30] <dcausse>	 o/
[15:02:06] <dcausse>	 I don't think we start elastic with logstash-gelf-1.11.0.jar in its classpath so I think we can drop it
[15:02:46] <dcausse>	 oh and you said liblogstash-gelf-java-doc misread "-doc" part, this one we cetainly don't need
[15:03:07] <inflatador>	 oh yeah, that's my mistake
[15:03:33] <inflatador>	 both liblogstash-gelf-java-doc and liblogstash-gelf-java need to be addressed, but I think you have done that
[15:03:43] <inflatador>	 I created https://phabricator.wikimedia.org/T306907 with more details
[15:04:16] <dcausse>	 wmf-elasticsearch-search-plugins should definitely be there
[15:04:36] <dcausse>	 if it's not we missed something
[15:06:07] <inflatador>	 Yeah, we have published 6.8 version of that package for stretch, but not bullseye (yet). I should be able to do that by end of week, if not sooner
[15:11:07] <dcausse>	 my bad elastic is still started with logstash-gelf-1.11.0.jar in its classpath
[15:13:08] <dcausse>	 and we have: "appender.ship_to_logstash.type = Gelf" so I think it's still being used so we need to copy it too
[15:13:22] <inflatador>	 OK, I'll update that task
[15:13:49] <inflatador>	 I wonder if any other teams use that
[15:14:48] <inflatador>	 looks like it: https://wikitech.wikimedia.org/wiki/Analytics/Archive/Hadoop_-_Logstash
[15:15:14] <dcausse>	 o11y switched to something else: https://gerrit.wikimedia.org/r/c/operations/puppet/+/763844/5/modules/opensearch/templates/log4j2_1.properties.erb
[15:16:49] <dcausse>	 in hadoop I think they removed it https://gerrit.wikimedia.org/r/c/operations/puppet/cdh/+/507297
[15:17:30] <dcausse>	 so we might be the last team still using this log4j layout
[15:18:52] <inflatador>	 Ah, so that hadoop page is out of date, then?
[15:19:40] <dcausse>	 I guess so, grepping Gelf over puppet I only see elasticsearch
[15:20:32] <inflatador>	 OK. I updated the ticket. I think it's best to get it published for bullseye and maybe revisit when we move to Opensearch
[15:20:45] <inflatador>	 (but if you prefer a different approach let me know)
[15:21:20] <dcausse>	 sounds good to me
[15:25:46] <i44>	 hi'
[16:07:37] <inflatador>	 quick workout, back in ~30
[16:19:14] <ebernhardson>	 we need a config system for our config system, these profiles have so many parameters :P
[16:19:27] <ebernhardson>	 varying them means duplicating a bunch of stuff
[16:21:42] <dcausse>	 yes :/
[16:22:26] <dcausse>	 it's not impossible to move them inside elastic if we need to (with proper caching I think that might be acceptable)
[16:23:09] <ebernhardson>	 hmm, yea i suppose if they were placed programatically instead of in a git repo it wouldn't be duplication
[16:39:18] <inflatador>	 back
[16:39:36] <inflatador>	 also I have a ticket for that (kinda) but haven't worked on it yet https://phabricator.wikimedia.org/T303011
[16:53:06] <inflatador>	 lunch/errand, back in ~1h
[17:14:38] <dcausse>	 dinner
[17:40:10] <inflatador>	 stuck getting my windshield repaired, will be a little longer
[18:09:12] <inflatador>	 and back
[18:40:59] * ebernhardson wonders why we never turned on the retry_on_conflict quirk i implemented to silence some warnings
[18:52:52] <mpham>	 I didn't make it to planning yesterday, but would like to push up the priority on T306466 -- it somewhat blocks next steps and I want to make sure we make the most of Ricardo's time
[18:52:52] <stashbot>	 T306466: Create list of current functioning and reliable search instrumentation - https://phabricator.wikimedia.org/T306466
[19:16:34] <ebernhardson>	 mpham: i added some notes, but really if they want 'reliable and functioning' the answer is we don't have any
[19:19:53] <mpham>	 ebernhardson: thanks. This looks like we have less than I thought -- don't we have dwell time, etc to calculate search engagement (clickthrough + 10s dwell time)?
[19:20:24] <ebernhardson>	 mpham: not directly, that would have been something done inside the old dashboards websites
[19:20:37] <ebernhardson>	 mpham: it can be extracted from the existing data the same as they did
[19:21:57] <ebernhardson>	 i suppose you can say any refinement of the raw data used to happen inside the dashboard support code that analysts built for us, all we have these days is the old data collection that used to feed into it
[19:22:38] <mpham>	 ok thanks. let me check in with leila to see if this helps us figure out what we need to do next or not. I'm not sure i fully understand yet what we have or don't have
[21:49:26] <ebernhardson>	 hmm, cindy doesn't look too bad wrt the _doc/page change, but hard to tell because it's mixed in with a bunch of failures where a page is mis-rendered (unclear if order of operations, or some other problem)
[21:51:00] <ebernhardson>	 the mis-rendering is that a page which includes a template that says `pickles` somehow gets another pages content  injected for the template even though the template is constant and we never edit it.