[07:38:11] the updater producer died in eqiad... [07:43:25] seems kafka related... [07:43:56] but surprisingly it stopped retrying to restart... [08:26:00] good morning! [08:26:24] the translate script was working all night btw https://phabricator.wikimedia.org/T425377#12005913 [08:26:40] atsukoito: hi! [08:27:27] we are at 82% by count [08:28:14] yes I remember something slow but I did not expect it to take that long, but that was more than 5 years ago that I ran this script [08:29:08] lessons learned: `--dblist` option is not useful with `ttmserver-export.php` script, because if it gets a single indexing error, it will return non-zero exit code and this will stop the progress [08:29:08] atsukoito: possibly something to consider to speed-up indexing (and perhaps querying) is adjusting the number of shards to something greater than 1 [08:29:45] atsukoito: I'll take a look at the dblist issue, it might need a cleanup in mw-config [08:29:46] we have codfw, will try [08:30:10] atsukoito: was it on collabwiki that it failed? [08:30:43] collabwiki and otrs_wikiwiki [08:30:51] ack [08:31:49] still, `metawiki !!! Some threads failed. Review the script output !!!` will stop execution. maybe this one should be an error condition [08:33:31] ptwikisource is also missing [08:34:33] sigh... [08:36:19] prod index have 5600825 docs (5.5gb/16.5gb) [08:54:27] atsukoito: so indeed the translate dblist is a superset of the wikis where translation memories (the ttmserver index) is required, it's at https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/28d41f67aabe457f3fbd7fe41757a00e53156072/wmf-config/InitialiseSettings.php#8631 [08:54:48] so a local dblist is required or we make another dblist in mw-config [08:55:21] metawiki should not have failed tho [08:56:46] metawiki has failed because there was a single connection failure for an article, it has finished re-indexing other articles but exited with non-zero code [08:56:58] ah ok [08:57:19] to clarify wmgUseTranslationMemory is only used when wmgUseTranslate is true (https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/28d41f67aabe457f3fbd7fe41757a00e53156072/wmf-config/InitialiseSettings.php#8558) [08:57:21] I'll make a note to the operations wiki about skipping those 4 wikis [08:57:28] sure [09:40:37] errand [12:31:30] \o [12:32:56] o/ [13:53:13] brouberol mentions that the maintenance script penalty might be from a tls penalty on write.. i'll try doing reindexing with services_proxy enabled [13:55:38] sure [14:05:47] hi, on this dashboard https://grafana.wikimedia.org/goto/afotivinyjuo0f?orgId=1 in "per-query type", can we somehow see ttmserver requests? [14:06:26] hmm, checking. [14:09:22] I don't think translate produces metric, it'd have to use some cirrus classes for that but I don't think it does [14:09:58] if they set the 'stats' key in queries we would have some data, but not seeing that [14:10:09] it would only be server-side data, but would be something [14:12:57] o/ [14:13:28] surprised Query doesn't have a specific method for those stats groups, in cirrus we use $query->addParam( 'stats', $fo ) [14:13:40] yes... [15:03:08] https://github.com/mattpocock/skills [15:05:17] Hi, I can't make it to the retro today, have some short term family obligations [15:09:12] dcausse: https://phabricator.wikimedia.org/P94086 [15:09:16] Trey314159: ^^ [16:05:29] dcausse: https://github.com/mattpocock/skills/tree/main/skills [16:11:50] thanks! [16:57:53] My blog post on second-try autocomplete searches was published this morning, for anyone who is interested: https://diff.wikimedia.org/2026/06/11/do-what-i-mean-second-try-suggestions-for-wrong-keyboard-and-transliterated-search/ [17:03:28] \o/ [17:04:43] indeed, that looks awesome! [17:07:17] Thanks! [17:08:53] if ya'll didn't notice in slack #product-tech-dept there are signups for an o'reilly account, which (iiuc) gets you access to e-book versions of most things o'reilly has published. I rarely use books, but might be some things there [17:13:16] thanks! [17:20:59] dinner [17:44:53] ooh, thanks for that