[09:17:06] Amir1: here, gonna need another 5/10 min if that's okay ? [09:35:42] Amir1: LMK when you are around re: the metrics [09:46:35] godog: Hi, I just got around, give me a couple of minutes to merge the deploy patch, it'll take half an hour to actually get deployed [09:49:12] Amir1: ok! let me know when I should go on my end [09:49:33] I will ping you [09:49:45] sorry for the delay, I was working late last night for train blockers [09:50:44] sure no worries, it happens [10:29:13] Hi o/ - it looks as if ES stopped igesting logs from kuberntes staging in eqiad yesterday (https://logstash.wikimedia.org/goto/f7bfa6d711e45a2e70bf0a00a189bb50) any idea what's going wrong? [10:30:03] I've not yet checked anything tbh [10:35:18] mmhh not off the top my head jayme [10:36:13] I can take a look too after lunch [10:36:56] ditto [14:34:24] jayme: mmhh found this on kubestage, perhaps rsyslog never reconnected? [14:34:27] Jul 14 14:13:27 kubestage1001 rsyslogd[14221]: mmkubernetes: failed to connect to [https://kubestagemaster.svc.eqiad.wmnet:6443/api/v1/namespaces/kube-system/pods/coredns-6674d59bd8-2x6kf] - 7:Couldn't connect to server [v8.1901.0] [14:35:08] no actually it can't be I think, there are errors from today taht rsyslog can't poll some containers log files [14:38:56] yeah short of restarting rsyslog I'm not sure what it could be offhand