[10:09:10] (ThanosRuleHighRuleEvaluationFailures) firing: (2) Thanos Rule is failing to evaluate rules. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/35da848f5f92b2dc612e0c3a0577b8a1/thanos-rule - https://alerts.wikimedia.org/?q=alertname%3DThanosRuleHighRuleEvaluationFailures [10:14:10] (ThanosRuleHighRuleEvaluationFailures) resolved: (2) Thanos Rule is failing to evaluate rules. - https://wikitech.wikimedia.org/wiki/Thanos#Alerts - https://grafana.wikimedia.org/d/35da848f5f92b2dc612e0c3a0577b8a1/thanos-rule - https://alerts.wikimedia.org/?q=alertname%3DThanosRuleHighRuleEvaluationFailures [10:28:35] that's me restarting prometheus ^ [13:46:49] Southparkfan: are you ok with me going ahead with https://gerrit.wikimedia.org/r/c/operations/puppet/+/979108 ? [14:11:43] godog: fine with me [14:12:09] I'm feel for you the bug still exists :-) [14:12:29] lolsob, thank you Southparkfan, appreciate it [14:13:22] no problem [14:13:34] I feel sorry* [14:57:59] godog: is now a decent time to try https://gerrit.wikimedia.org/r/c/operations/puppet/+/979163 ? [14:58:46] herron: SGTM, though please check if the train is running [14:59:54] ack thanks [17:05:03] herron: o/ I am checking grr preview on grafana1002 and I get a out of range error, but it doesn't seem related to my last commit [17:05:22] tried to reset my local dir to prev HEADs and same issue [17:05:32] have you already seen something similar? [17:06:59] I think it happens when called without args, should work like grr preview slo_dashboards.jsonnet [17:07:04] elukey: [17:09:11] herron: thank youuuu totally forgot [17:09:28] elukey: sweet, glad thats all it was [17:09:52] the new time range seems to work fine, going to test a few more things and I'll apply it [17:11:34] kk [17:13:22] done! [17:14:18] thank you!