[07:44:39] I'm seeking a reviewer for a quick one: https://gerrit.wikimedia.org/r/c/operations/puppet/+/904677 [08:10:13] cheers Emperor [08:10:45] NP :) [08:41:09] I was wrong btw, still crashloops even with reuseaddr, debugging [08:48:35] :sadpanda: [10:18:17] jbond: may I merge your pending CRs? [10:18:59] alertmanager: also pages to sre for data-engineering (9e69319e96) [10:18:59] alertmanager: also pages to sre for data-engineering, releng and search (0a3c42330a) [10:19:59] yes please [10:20:09] sorry thought they where in the alerts repo [10:20:48] done [10:22:45] thanks [10:26:20] jbond: Can I just check, what's the intention behind these? If we fire an alert with the `data-engineering-page` then it will also go to `sre-batphone` in Victorops now, instead of just the `analytics` route. Is that right? [10:28:30] btullis: oh sorry i should have got a +1 and disccussed with data-engineering was just going through and finnishing of tasks from sprint week [10:29:17] but the intention is that things marked as page and data-engineering-page will also page victorops as well as going to the receiver: 'data-engineering-page' [10:29:27] if this is not desired i can revert [10:30:45] jbond: No worries. Just want to get it straight in my head, so that I don't accidentally wake anyone up. [10:35:30] We only have one or two page alerts hooked up in alertmanager so far, so here's one, critically low space on HDFS: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/alerts/+/refs/heads/master/team-data-engineering/hadoop-hdfs.yaml#15 [10:37:41] Would you expect that if this were to trigger, it would now alert the on-call person in core SRE, as well as o.ttomata and me? [10:40:09] bacula is temporarilly down for maintenance, please wait some minutes if you have to accidentally delete important files from production [10:41:09] Only deliberate destruction of vital production data? I can live with that ;-) [10:50:59] (for others this chat to o11y ) [11:02:46] I think I got it re: socat "bridge" v6 -> v4 udp here: https://gerrit.wikimedia.org/r/c/operations/puppet/+/904771/ can I get a kind soul to sanity check / +1 ? thank you [11:05:59] +1 (but then the previous version looked plausible, so...) [11:08:06] thank you Emperor! yeah it looked plausible to me too, socat disagreed [18:32:53] Hi Folks, on-call & clinic duty rotation updates are live (details in sre@)