[09:29:17] <btullis>	 dcaro: Can I merge your puppet change?
[09:29:28] <dcaro>	 oh yes sorry
[09:29:37] <btullis>	 Thanks.
[09:29:41] <dcaro>	 my ssh is waiting for me to touch my yubikey xd
[09:29:55] <dcaro>	 (and got distracted)
[09:30:04] <btullis>	 No worries :-)
[09:35:36] <dcaro>	 thanks :)
[12:13:55] <Krinkle>	 vgutierrez: related cleanup, a beta only hiera patch: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1183275
[16:05:13] <elukey>	 I noticed elevated memcached errors for kube-dumps: https://grafana.wikimedia.org/d/000000438/mediawiki-exceptions-alerts?var-datasource=000000006&orgId=1&from=now-6h&to=now&timezone=utc&viewPanel=panel-19
[16:07:39] <elukey>	 it seems related to kube-dumps, but I don't find the same level of errors in the logstash dashboard
[16:12:00] <elukey>	 I found https://phabricator.wikimedia.org/T401425 that seems relevant, it happened before
[16:12:26] <elukey>	 anyway, nothing should be burning but if anybody has time please give it a check (I need to go in a few sorry)
[16:14:02] <hnowlan>	 corresponding logstash error level: https://logstash.wikimedia.org/goto/6d2db878d2babded9a2e9ef8e8b2476b
[16:15:06] <hnowlan>	 vast majority are for s8 
[16:18:02] <swfrench-wmf>	 elukey: hnowlan: sorry, missed the discussion here. yes, this is T401425.
[16:18:03] <stashbot>	 T401425: Investigate memcache errors during wikidata and commons dumps runs - https://phabricator.wikimedia.org/T401425
[17:40:49] <Krinkle>	 vgutierrez: awaitint pcc to confirm that I've got the escaping right, but - something like this: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1184886
[17:41:20] <Krinkle>	 it's not the most elegant place, but perhaps most pragmatic
[17:42:35] <taavi>	 Krinkle: that patch is adding a double $ in there
[17:42:45] <Krinkle>	 yep, just noticed in pcc 
[17:42:48] <Krinkle>	 https://puppet-compiler.wmflabs.org/output/1184886/4884/cp1102.eqiad.wmnet/index.html
[17:44:29] <Krinkle>	 thx
[18:10:49] <Krinkle>	 heh, `puppet$ git grep 'regex.*escape'` was surprisingly fruitful
[18:11:09] <Krinkle>	 steals logic from wikimedia_domains_regex
[18:11:18] <Krinkle>	 mayve that's better?
[18:58:22] <Krinkle>	 I'm done thinkering with it for now. It's working in beta and VTC is passing locally at both the refactor + wikitech/mediawiki enablement.