[09:58:19] <dcausse>	 lunch
[13:28:29] <ebernhardson1>	 \o
[13:31:23] <dcausse>	 o/
[14:01:19] <inflatador>	 <o/
[14:04:49] <dcausse>	 o/
[15:13:35] <inflatador>	 sorry, lost Internet, joining mtg shortly
[17:37:37] <dcausse>	 dinner
[20:05:03] <ebernhardson1>	 looking into a problem in the highlighter, it checks for a leading wildcard and uses an optimization if found.  lucene 9 added an nfa query in https://issues.apache.org/jira/browse/LUCENE-10010 "It should be slower, but also makes those adversarial cases doable."
[20:06:20] <ebernhardson1>	 but that nfa query basically changed some details about the automaton, but makes we wonder if we should take advantage of nfa fallback instead of our custom optimizations
[20:18:26] <ebernhardson>	 the nfa support is fairly drop-in, and this would essentially get rid of too complex to determinize.  Not yet sure what replaces it, there must still be pathological cases.
[20:42:22] <pfischer>	 ebernhardson: Is that a problem with the lexical/keyword highlighter or the semantic highlighter?
[20:43:58] <ebernhardson>	 pfischer: with the cirrus highlighter migration to 3.8.0.
[20:44:23] <ebernhardson>	 it's not a big problem, but an interesting change in lucene that has implications for our regex support
[20:48:18] <pfischer>	 Do we have tests for those “too complex to determinize” cases, to check if the alternative NFA query would make them work?
[20:50:02] <ebernhardson>	 pfischer: yea, we could catch the too complex exception and then run it as an nfa.  My read is the nfa pathological case is instead explosion of memory size. Which makes it a little awkward in a different way
[20:50:58] <ebernhardson>	 the nfa doesn't determinize, so it can't have the failure. 
[20:52:38] <pfischer>	 Understood. Do we know how often “too complex” happens? Just to better understand if it’s worth building such routing.
[20:53:05] <ebernhardson>	 hmm, there is maybe something in grafana
[20:53:20] <pfischer>	 BTW: while working on the metrics (https://phabricator.wikimedia.org/T434975#12283038), Claude pointed out a potential bug: `Searcher::getPoolCounterType()` checks the syntax token ‘deepcat’, but `DeepcatFeature::getFeatureName()` records ‘deepcategory’ and nothing records ‘deepcat’. I ran a quick check and apparently `SearchContext::isSyntaxUsed` only checks array containment.
[20:54:45] <ebernhardson>	 but it's basically very fuzzy regexes, like /[^]]*alt=[^]\\|}]{80,}"/ or /[^]]*s[tabcse]{1,10}"/
[20:55:29] <ebernhardson>	 yea that sounds like a highly plausible bug. Yea we should fix that
[20:57:18] <ebernhardson>	 hmm i don't see them broken out in grafana, we have the failed category but we don't record the low level here
[21:02:07] <ebernhardson>	 school run, back in a bit