[13:22:17] <inflatador_>	 <o/
[14:07:15] <pfischer>	 o/ inflatador: do you have anything to discuss for our pairing session?
[14:07:51] <inflatador>	 pfischer sorry I missed it, was talking with MrG and forgot
[14:08:18] <inflatador>	 Nothing to discuss, just a heads-up that the test application should have Zookeeper soon: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/951551/
[14:18:31] <pfischer>	 👍
[14:23:19] <pfischer>	 Now that we have the up-to-date Debian packages for the elastic search plugin: How can we trigger a build (+push) of https://gitlab.wikimedia.org/repos/releng/dev-images/-/tree/main/dockerfiles/elasticsearch?
[14:36:37] <inflatador>	 pfischer I think I can help with that...otto-mata showed me how to do it awhile back. Let me check my notes
[14:54:00] <ebernhardson>	 \o
[14:54:17] * ebernhardson is contiuously surprised people keep submitting PR's to a php project i haven't touched in a decade
[15:01:56] <ebernhardson>	 pfischer: meeting time
[15:27:45] <ebernhardson>	 do we have other wed mtg topics?
[15:28:13] <ebernhardson>	 if nothing else, i have questions about how deployment-charts and minikube play together (i suspect they do not)
[16:03:18] <inflatador>	 workout, back in ~40
[16:57:25] <inflatador>	 back
[17:49:54] <inflatador>	 ebernhardson I found an MR that might satisfy pfischer 's need to build a dev img with new plugin pkg: https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/32 . I don't have perms on this repo, any chance you could review and/or approve?
[17:56:42] <inflatador>	 lunch, back in ~40
[18:25:17] <ebernhardson>	 inflatador: lgtm, but i don't have access there either
[18:55:17] <inflatador>	 back
[18:55:35] <inflatador>	 ebernhardson I just pinged in releng, maybe click the access request button if you haven't already and we can both get access
[18:57:09] <ebernhardson>	 inflatador:  mine says "Withdraw Access Request" so i must have done that at some point :)
[18:57:17] <inflatador>	 ACK
[18:58:49] <ebernhardson>	 inflatador: random idea for the flink chart,  right now we have to provide the cli args as a list, and helm overrides lists between values so we have to define all arguments everywhere
[18:59:22] <ebernhardson>	 reasonable to add a conversion in flinkdeployment.yaml that accepts a map of cli args and re-formats into a list?  Or maybe helm already accepts a map there i should check...
[19:13:23] <inflatador>	 ebernhardson yeah, that should help
[19:15:22] <ebernhardson>	 i do wonder about templating within the values files as well, but i don't see anyone else using it
[19:15:44] <inflatador>	 All I see a bunch of templates rendering templates rendering templates ;P
[19:15:44] <ebernhardson>	 reason being, i'd like to define `--page-change-stream ${dc}.mediawiki.page_change.v1` once and have it in both dc's rather than duplicating
[19:16:45] <ebernhardson>	 there are certainly lots of templates :) but `grep '{{' */values*` suggests the values files  at least are untemplated
[19:17:00] <ebernhardson>	 from helmfile.d/services, go basically looking through the values files for lots of services
[19:21:56] <inflatador>	 oh yeah, I'm just being facetious
[19:22:25] <inflatador>	 golang templating + yaml doesn't make for the nicest-looking code ;P
[19:30:25] <inflatador>	 ebernhardson can you confirm I won't pollute the git history if I merge this? Sorry for my ignorance, been awhile since I've done anything but gerrit https://gitlab.wikimedia.org/repos/releng/dev-images/-/merge_requests/32
[19:38:35] <ebernhardson>	 inflatador: lgtm
[19:41:00] <inflatador>	 thanks, merged
[19:41:28] <ebernhardson>	 i'm also wondering now if we shuld be doing some sort of chart with flink-app's as subcharts, or how exactly we manage the 5 instances running two variants of flink-app
[19:42:59] <inflatador>	 As in maybe splitting producer and consumer?
[19:44:59] <ebernhardson>	 well they are already split afaict, iiuc we expect to have 5 helm releases running across the 2 dc's. They need different arguments, use different docker images, etc.
[19:46:02] <ebernhardson>	 i was wondering how we unify them together, or if unifying is wasted time and we should be having separate service definitions in helmfile
[19:58:34] <inflatador>	 oh yeah, the args are different. I guess I missed that the producer and consumer will have different images
[19:58:50] <inflatador>	 regardless, I get your point
[20:00:20] <ebernhardson>	 maybe we can vary it all in a single helmfile, still learning how this layering of values files works out.
[21:48:26] <pfischer>	 Do we always release them together? After all, we have one CI pipeline that builds them as part of the same maven reactor build. In that case, a single chart might work. We do have to make sure we have as many flink-step-1 and flink-step-2 instances as we have kafka brokers/partitions (5 at the moment). However, I don’t know if the event gateway takes care of putting messages on partitions based on an algorithm like 
[21:48:26] <pfischer>	 page_id % 5. That would be required for any of our flink instances to see all messages related to a page id it is responsible for.
[21:51:54] <pfischer>	 How do we distribute requests among the two/three instances of ES? Should every instance see all page updates? Then flick-step-2 could also sink to a configurable list of ES instances instead of just one.
[21:53:30] <pfischer>	 ebernhardson: here’s the draft for late fetch: https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/11