[08:13:24] <godog>	 thank you for taking a look topranks !
[14:02:43] <jinxer-wm>	 FIRING: BenthosKafkaConsumerLag: Too many messages in jumbo-eqiad for group benthos-webrequest-sampled-live-franz - TODO - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=jumbo-eqiad&var-datasource=eqiad%20prometheus/ops&var-consumer_group=benthos-webrequest-sampled-live-franz - https://alerts.wikimedia.org/?q=alertname%3DBenthosKafkaConsumerLag
[14:12:43] <jinxer-wm>	 RESOLVED: BenthosKafkaConsumerLag: Too many messages in jumbo-eqiad for group benthos-webrequest-sampled-live-franz - TODO - https://grafana.wikimedia.org/d/000000484/kafka-consumer-lag?var-cluster=jumbo-eqiad&var-datasource=eqiad%20prometheus/ops&var-consumer_group=benthos-webrequest-sampled-live-franz - https://alerts.wikimedia.org/?q=alertname%3DBenthosKafkaConsumerLag
[16:00:30] <lmata>	 thanks dcaro, enjoy your time off!
[16:01:04] <lmata>	 thank you too topranks!
[20:50:48] <cdanis>	 anyone know how I can verify if this particular log made it into the ECS-clean index?  https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-k8s-1-7.0.0-1-2025.01.30?id=98z4uJQB60NnMROHUFV2
[20:51:25] <cdanis>	 ah, it definitely didn't, there's no ecs.version: in that one heh
[20:56:23] <cdanis>	 ah. https://logstash.wikimedia.org/app/discover#/doc/19d32430-85d8-11eb-8ab2-63c7f3b019fc/dlq-default-1-1.0.0-1-2025.01.30?id=4c_8uJQB60NnMROHZMCZ
[21:45:02] <cwhite>	 Taking a guess, but that possibly means that http.response.body.content wasn't a string?
[21:46:27] <cdanis>	 no, it was an object
[21:46:36] <cdanis>	 okay, that makes sense
[21:47:00] <cdanis>	 is there a place to see error messages from opensearch?
[21:50:36] <cdanis>	 or even better, a jsonschema or similar I can validate against?
[21:51:26] <cwhite>	 There isn't a jsonschema spec, but we have docs: https://doc.wikimedia.org/ecs/#field-http-response-body-content
[21:51:53] <cwhite>	 Closest I can get to a machine spec is the index template: https://doc.wikimedia.org/ecs/#downloads
[21:51:55] <cdanis>	 contrary to appearances I did read those while I was writing the patch 😅
[21:51:58] <cdanis>	 okay.