[14:38:46] Quick question: We seem to be getting duplicate alerts in Karma recently. One for the default receiver and one for our team's receiver. e.g. https://alerts.wikimedia.org/?q=%40state%3Dactive&q=team%3Ddata-engineering [14:38:53] What should we do about this? [14:49:14] btullis: yeah that's a long standing karma bug, upstream issue is https://github.com/prymitive/karma/issues/5144 though it hasn't been getting traction [14:49:40] I'm not sure there's a straightforward fix unfortunately [14:50:53] perhaps making sure alerts have only one receiver and no fallback/default, though I can see how that can get to a more complex configuration as a result [14:52:17] I have to run now, will read up later [14:52:34] godog: OK, thanks for the info. I guess we won't worry about it for now. [14:53:15] btullis: yeah that's definitely the easiest fix for now [19:21:10] if I've changed an instance of monitoring::service, what is needed to apply that? I've done this once and my vague recollection was "run puppet on alert1001" [19:27:00] I think puppet needs to run on both the target host and the alerting host. The first to change the values in puppetdb, and the second to apply those values to icinga. [19:28:22] yes was about to say the same, first the affected hosts update the puppet exported resources and then puppet on the alert hosts will realize them [19:29:30] awesome; thanks!