[12:18:32] !puppetweb is https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/$1 [12:18:33] Sorry, you are not authorized to perform this [12:18:36] of course not [12:18:38] stupid bot [12:18:53] the doc is https://meta.wikimedia.org/wiki/Wm-bot#Infobot [12:19:00] inflatador: ^ [12:19:17] which eventually would let one do `!puppetweb manifests/site.pp` [12:20:04] looks like only urbanecm is trusted by the bot [12:35:38] I was summoned by hashar [12:35:48] If you want me to entrust someone else, happy to! [13:16:25] hashar nice, will take a look at the docs! [15:42:13] [cross-posting for vis]: I'm updating etcd-mirror on configcluster hosts during the second half of the upcoming MW infra deployment window (after 17:30 UTC) for T358636. [15:42:13] No change in behavior, but it does require a mirror restart on conf2005 (a brief interruption in replication of conftool data to codfw). Just posting for awareness in the unlikely event anything goes wrong. [15:42:14] T358636: etcdmirror does not recover from a cleared waitIndex - https://phabricator.wikimedia.org/T358636 [16:46:57] mutante: just seen your comment, I'll revert the patch [16:48:50] fabfur: [16:48:51] 12:47:46 < denisse> I think I spotted the issue with the patch, the label matcher {job=~"cache_.*"} is in the wrong place. [16:48:57] maybe this [16:49:04] fabfur: ACK, thanks, forwarding it :) [16:49:06] denisse: ^ [16:49:08] what sukhe said [16:49:11] ack [16:49:23] fabfur: Don't revert it, I'm working on a patch. :) [16:49:46] ah ok me too but probably you're faster than me in this :) [16:49:53] (not probably, for sure...) [16:51:23] the filter should be before `[2m]`, correct? [16:56:43] fabfur: Yes, exactly. [16:56:55] ok abandone mine and +1 your [16:57:04] when using the sum by aggregation operator, the label matchers (such as {job=~"cache_.*"}) should be inside the rate function, not outside. https://gerrit.wikimedia.org/r/c/operations/puppet/+/1018745 [16:57:40] denisse: nice catch! [16:58:23] thanks, +1ed [16:59:59] Thanks, I'm just waiting for PCCs output: https://integration.wikimedia.org/ci/job/operations-puppet-catalog-compiler/1848/ [17:09:58] That fixed the issue. [17:10:56] perfect [17:10:59] thanks again ! [17:12:34] fabfur: Prego. :) [17:14:20] :) [20:58:36] Dear future EU-TZ on-callers: The videoscalers continue to be under duress, the dashboards got scary (again), and pages ensued (again). We lowered job concurrency (even more), they got a little less scary, and the pages stopped (so far). The gdoc is up to date. <3 -- Americas-TZ on-callers