[13:22:13] <cdanis>	 why is the state of tooling around k8s events so bad
[13:22:25] <cdanis>	 like, in general
[13:22:37] <cdanis>	 the simple tools everyone usually talks about have archived repos
[14:18:28] <brouberol>	 it's Kubernetes for you. Move fast, write go, archive repos
[14:19:16] <cdanis>	 lol
[14:19:18] <cdanis>	 yeah but no one else has moved fast on it in the meanwhile
[14:19:48] <cdanis>	 at this point I genuinely believe the 'best' solution is `while sleep 1s ; kubectl get events -w --all-namespaces | ts` in a pod
[14:22:26] <jayme>	 "real time" events are odd indeed. For those that are a bit older I find the archived ones in logstash pretty complete
[14:22:54] <jayme>	 although I think they might have the wrong timestamp indexed as well
[14:24:28] <brouberol>	 cdanis: fair point. Then it's just "Write {{ $.Values.yaml | indent 16.5 }}, archive repos"
[14:50:31] <cdanis>	 jayme: maybe the Founders made the default readinessProbe timeout 1s because they didn't want you to have readiness probes that issue RPCs
[14:50:38] <cdanis>	 🤔
[14:50:47] <jayme>	 eheh
[14:50:50] <jayme>	 nice trick
[15:28:26] <jayme>	 There is still a couple of charts missing the migration to the external-services network policy system like flink apps, the operator, datahub, kserve-inference and mediawiki (duh') - https://phabricator.wikimedia.org/T359423
[15:29:00] <jayme>	 it would be nice to bring this over the finish line, so please take a look and help out if you can
[15:33:42] <ottomata>	 brouberol: i seem to (mis?) remember...did you have a nice way of DRYing up the copy/pasted kafka broker lists in values files? 
[15:33:43] <ottomata>	 e.g.
[15:33:44] <ottomata>	 https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1037870/2/helmfile.d/services/eventstreams/values-codfw.yaml
[15:34:02] <ottomata>	 its nice that the egress is abstracted, would be nice if the broker list was too :)
[15:36:37] <brouberol>	 so, that is feasible, but it requires a recent-enough kafka client. We can set the bootstrap client list to `kafka-main-codfw.external-services` (for services running in k8s) _but_ it requires the client to resolve the DNS, and then test each ip.  This is what `client.dns.lookup = use_all_ips` does in librdkafka, and wwhat
[15:36:37] <brouberol>	 https://cwiki.apache.org/confluence/display/KAFKA/KIP-302+-+Enable+Kafka+clients+to+use+all+DNS+resolved+IP+addresses is about 
[15:37:24] <brouberol>	 jayme: sorry, datahub was migrated, and I forgot to report
[15:37:46] <jayme>	 ah, great. Quick achievement :D
[15:38:00] <ottomata>	 riihggt!  cool.