[03:41:44] (See icinga-wm alerts in #-operations) Just out of curiosity, what would cause the sort of spike in ES latency like this? https://w.wiki/4TiY [05:12:02] someone doing a bunch of regex searches or something? [05:19:24] geez, they can really have that much of an impact? [18:04:12] perryprog: definitely. regex searches are pretty much the most expensive operation you can do in an Elasticsearch instance. [18:05:57] There is no way to precompute an index for an arbitrary regex so they are functionally all table scan operations. [18:48:33] Huh. And is there any DOS-mitigation that allowing arbitrary regex searches require, besides just a timeout? [18:51:32] timeouts and ratelimits [18:58:12] https://grafana.wikimedia.org/d/qrOStmdGk/elasticsearch-pool-counters?orgId=1&from=1638054000000&to=1638075600000&var-pool_counter=Regex doesn't look like there were too many more than the norm \