[07:52:52] o/ [07:53:17] happy 2023! :) [09:37:19] pfischer: I'll be at least 5' late for our meeting. Sorry :/ [09:38:20] gehel: no worries. [10:03:48] pfischer: and I'm there! [11:18:08] lunch [13:51:39] o/ [14:22:28] o/ [14:22:53] dcausse: I let the search update pipeline run for some hours yesterday and so far it ran without crashing. Updates take between 1-2 minutes door to door. [14:23:31] pfischer: nice, well done! [14:24:47] inflatador: Do we have a flink deployment on k8s, that is ready to accept flink applications? [14:25:57] pfischer unfortunately no. I'm going to try and get something up in deployment-prep, but we've hit a few snags as far as the official deployment ( https://phabricator.wikimedia.org/T324576#8455459 ) [14:26:40] pfischer for your test purposes, I imagine you don't need to worry about the network ingress/egress stuff? [14:34:46] inflatador: thanks, no ingest/egress would be more of a security concern. As long as this cluster/node/pod is not public we could deal with it later. All we’d need is a way to access the flink master service to get logs/insights. dcausse: What do you think? [14:35:55] We’d have to make sure, all 3rd parties our flink application talks to (testwiki, elasticsearch) can be reached from within the k8s cluster. [14:36:33] getting a flink job running in k8s will require some charts to be written, there are no ways to test things like you did in yarn (manually deploying and starting things) [14:37:00] so I guess we're stuck waiting for the flink operator to be merged to deployement-charts [14:38:55] and I'm not sure taht testing the app on a custom k8s cluster (e.g. minikube in deployment-prep) will bring much value as it'll be far enough from the prod setup that we'd have to test again on the production (or staging) k8s clusters [14:40:26] fair enough, I'll hold off on standing up another kube env [14:41:09] will write a couple of tasks that could be started once the operator is up and ready in deployment-prep [14:42:55] OK, so I guess I should work on getting it running again in DP? [14:43:13] (no worries either way, just wanna make sure it will help if I do that) [14:45:18] oops sorry I meant deployment-chart not deployment-prep :) [14:46:40] getting the app running in deployment-prep would be nice but I'm not sure what approach to take [14:47:04] I wish we had a k8s setup there similar to production [14:47:35] I wonder how existing mw services are being run tho... [15:01:36] Good question. I was just looking at the kafka streams [15:01:48] dcausse: I can ask Janis, I’ll see him later. You mean how they migrated them to k8s? [15:02:24] mw services in deployment-prep: we often use puppet to run a systemd unit from the service docker image. [15:02:29] so no k8s [15:02:31] pfischer: I mean how services like changeprop that we run in k8s when in production are run in the beta cluster [15:02:39] ah ok [15:02:57] e.g. https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate/Administration#Beta_/_deployment-prep [15:03:23] flink on k8s is my main priority, so i'm working on it! [15:03:27] mostly just details with reviews right now [15:03:35] ottomata: thanks! :) [15:03:41] which takes a long time because janis is in europe and i'm in US [15:03:49] so occasional mornings are our only overlap [15:06:44] no worries! [15:12:35] I’ll poke Janis. ;-) [16:03:10] Wednesday meeting is starting: https://meet.google.com/eki-rafx-cxi [17:14:17] workout, back in ~40 [17:53:24] back [17:57:59] dinner [18:57:07] lunch, back in ~45 [19:40:18] rsyslog [19:40:22] oops, I mean...back!