[08:18:52] <pfischer>	 o/ dcausse: Hey David! Is there any task (at the cirrus-update-pipeline-end) you could use my help for? For AirFlow there’s currently not so much I can do until Erik comes online.
[08:19:18] <dcausse>	 pfischer: hey, lemme take a look
[08:22:37] <dcausse>	 pfischer: T326318 comes to mind, it would involve learning blubber and kokkuri but there are few questions to answer first
[08:22:38] <stashbot>	 T326318: Create docker images for the cirrus-streaming-updater flink jobs - https://phabricator.wikimedia.org/T326318
[08:23:02] <dcausse>	 or T332763 (not yet in ready for dev)
[08:23:02] <stashbot>	 T332763: The search update pipeline should support events compatible with the /mediawiki/page/change/1.0.0 schema - https://phabricator.wikimedia.org/T332763
[08:30:45] <pfischer>	 Okay, I’ll have a look. Thanks!
[09:51:26] <gehel>	 Lunch 
[10:28:17] <dcausse>	 lunch
[13:09:33] <inflatador>	 <o/
[13:14:21] <dcausse>	 o/
[13:38:37] <gehel>	 ryankemper, ebernhardson: would it be OK to cancel our 1:1 today? With the WDQS Office Hours, it's already going to be a pretty long day for me
[13:40:16] <gehel>	 @team: I've just forwarded the WDQS Office Hours to you all. But just as a fyi, no need to be there unless you want to.
[13:40:59] <gehel>	 There is another one tomorrow European morning, I've only forwarded to pfischer, I expect all US people to be asleep at that time
[14:19:22] <gehel>	 pfischer: I'm sending a few time proposal to talk to Ververica about Flink training. Our contact is based in SF, so some of them might be late for you.
[14:19:43] <gehel>	 Worst case, I'll be there and try to represent your needs as best as I can.
[14:34:58] <pfischer>	 dcausse: Regarding the second question on https://phabricator.wikimedia.org/T326318 (“Should we use a single repo and have a new image generated per patch or a separate repo with a release process?”) - Can’t we keep this close to the code of the flink-applications? I’d expect this to be an artifact of the build pipeline for consumer/producer. I’d assume that JARs we’re deploying throughout CD will hardly be 
[14:34:58] <pfischer>	 used by themselves anyways.
[14:39:21] <dcausse>	 pfischer: sure! that'd mean we move the repo to gitlab I guess
[14:44:10] <dcausse>	 I think there's "pipelinelib" on gerrit that does what kokkuri is doing in gitlab but I think releng wants to get rid of "pipelinelib" 
[14:45:22] <dcausse>	 errand
[14:45:43] <pfischer>	 Okay, so I’ll assume that we have the men-packaged jar available locally when building the docker image.
[14:46:15] <dcausse>	 pfischer: yes it should be all managed by blubber
[14:47:15] <dcausse>	 blubber will package the jar (run maven) and build the docker image
[15:02:46] <gehel>	 ebernhardson, dcausse: triage meeting: https://meet.google.com/eki-rafx-cxi
[15:17:40] <ryankemper>	 gehel: yeah cancelling 1:1 is fine
[16:00:58] <ebernhardson>	 gehel: cancelling is fine for me too
[16:02:28] <pfischer>	 o/ ebernhardson: would you have some time to walk me through your process of testing the migrated airflow tasks?
[16:02:35] <ebernhardson>	 pfischer: sure
[16:03:49] <pfischer>	 https://meet.google.com/wwk-imhj-wpk
[16:07:08] <inflatador>	 errands, back in ~40
[16:41:17] <dcausse>	 going to start a reindex of all the wikis from mwmaint2002
[16:44:21] <dcausse>	 wow using Erik's reindex functions, that's a lot of tmux panes :) 
[16:44:40] <ebernhardson>	 lol, yea it makes 9 separate things
[16:44:57] <ebernhardson>	 but they should all fit in one screen (at least on tmux with particular height and font settings)
[16:45:32] <dcausse>	 yes, I have ~6lines per pane 
[16:46:21] <ebernhardson>	 i suppose i should copy that file into cirrus's scripts/ dir as well so its recorded somewhere
[16:46:40] <dcausse>	 yes otherwize it might diverge over time
[17:00:07] <pfischer>	 ebernhardson: glent is released
[17:01:24] <ebernhardson>	 pfischer: thanks!
[17:36:05] <inflatador>	 Back
[18:03:21] <dcausse>	 dinner
[18:57:09] <ryankemper>	 lunch
[19:48:30] <ebernhardson>	 how wierd...not sure how we are managing to trigger this: User class threw exception: org.apache.spark.sql.AnalysisException: Cannot overwrite a path that is also being read from
[19:48:50] <ebernhardson>	 and in classic fashion, no mention of what the actual path is :P
[19:52:27] <inflatador>	 I forgot that Elastic is getting nasty about its client libraries
[19:52:29] <inflatador>	 "elasticsearch.exceptions.UnsupportedProductError: The client noticed that the server is not a supported distribution of Elasticsearch"
[19:53:07] <ebernhardson>	 yea thats a bit tedious
[19:53:29] <inflatador>	 I guess we have this to look forward to when we switch to Opensearch ;P
[19:54:31] <ebernhardson>	 not sure what solution we would take...could probably fake compatability, or fork the library and change it (depending on license)
[19:59:02] <inflatador>	 There is an opensearch-py client library, I guess we'd probably have to migrate to that library eventually
[19:59:17] <ebernhardson>	 ahh, yea probably
[20:00:20] <inflatador>	 I'm running into it now when naively using the latest version of elasticsearch py from pip...it refuses to talk to our current cluster
[20:00:44] <inflatador>	 rolling back to 7.10 version of the library fixes that
[20:27:25] <inflatador>	 OK, I got banning to work with the ES python client