[08:08:09] <godog>	 FYI I'll be simplifying a bit the statsd.eqiad.wmnet dns/puppet setup here: https://gerrit.wikimedia.org/r/c/operations/dns/+/904185 https://gerrit.wikimedia.org/r/c/operations/puppet/+/904186
[08:08:15] <godog>	 jayme akosiaris ^
[08:11:06] <claime>	 godog: \o/
[08:11:34] <godog>	 claime: \o/ indeed! I failed over graphite the other day and it was annoying me to no end
[08:11:40] <claime>	 Last time it was Page Previews that stopped sending data
[08:11:57] <godog>	 thank you, yeah I'll keep an eye on the v6 traffic
[08:12:00] <volans>	 having statsd.eqiad potentially point to codfw feels weird though, is that on purpose?
[08:12:12] <akosiaris>	 thanks for heads up
[08:12:19] <godog>	 volans: it is yeah
[08:44:54] <elukey>	 hi folks
[08:45:06] <elukey>	 something changed in logstash indexations yesterday at around 18:40 UTC
[08:45:07] <elukey>	 https://grafana.wikimedia.org/d/000000561/logstash?var-datasource=eqiad%20prometheus%2Fops&orgId=1&refresh=5m&from=now-24h&to=now&viewPanel=40
[08:45:17] <elukey>	 the alert is flapping
[08:46:18] <elukey>	 maybe https://sal.toolforge.org/log/xyCqLocB0IdT-khD18i4 ?
[08:49:03] <godog>	 "reason"=>"Limit of total fields [2048] has been exceeded"}} :(
[08:51:22] <godog>	 could be only a trigger for the condition though, i.e. other things have been accumulating fields
[09:02:52] <godog>	 this is https://phabricator.wikimedia.org/T180051 FWIW
[12:57:15] <godog>	 claime: the statsd change is working as expected \o/ I'm glad we closed the loop on that
[12:57:24] <claime>	 Awesome :D
[12:57:37] <claime>	 One less weird thing
[12:57:46] <godog>	 indeed
[12:58:04] * godog eyes the pile of weird things
[13:03:00] <volans>	 elukey, btullis: I'm disabling puppet on superset (an-tool1010) to merge and test the patch on superset-next and then re-enabling and running it on the prod one. Ok for you?
[13:03:48] <elukey>	 +1
[13:07:09] <volans>	 actually one last minute thing, I'll make the URL relative to work separately on next and prod
[13:16:29] <btullis>	 volans: Both fine by me.
[13:17:09] <volans>	 thx
[13:43:32] <volans>	 elukey, btullis: all done, puppet re-enabled everywhere, all yours. Thanks for the reviews and support
[17:07:16] <sukhe>	 Traffic is failing over lvs4008 to lvs4010 to test the bullseye upgrade
[17:07:26] <sukhe>	 we are keeping an eye out but if something is broken, please use the rooftop to shout :)