[07:45:41] godog: we can also start killing cloud graphite and statsd today. I sent a couple of patches your way on friday [07:47:46] taavi: exciting! thank you, I'll take a look today after clinic duty duties [07:52:46] sounds good [16:51:02] Hello, my friends. I'm following up on a discussion a few weeks ago re: discrepancies in SLO reporting. A summary: https://paste.debian.net/plainh/b5af6f57 [16:51:41] Is there a way to clean these anomalies up? Is it a known Grafana rendering issue (I have run into such things before)? [17:14:29] brett: last time I looked at this I created https://grafana.wikimedia.org/d/Uk3M_9C4k/haproxy-sli-scratchpad?orgId=1, good and can see the values follow each other very closely and at times good exceeds total. I think a good next step would be a closer look at the mtail rules, perhaps adjusting that config to create an sli bad metric to use directly as opposed to total - good [17:30:28] brett: or for a shorter term band-aid maybe something like this https://gerrit.wikimedia.org/r/c/operations/grafana-grizzly/+/944287 [17:30:28] https://grafana.wikimedia.org/dashboard/snapshot/Lp5wp0vhvTG9wvQ3UUqPl1LIC8xnoaVI?orgId=1