[12:55:23] \o [13:30:23] sometimes there are odd emails full of attachments that i don't open because well, that would be silly, but sometimes still curious :) [14:01:02] Apparently they released OpenSearch 3.0 in late June https://opensearch.org/blog/opensearch-3-0-what-to-expect/ [14:02:46] errr...that was 3.1 [14:06:39] ebernhardson based on that feature set, do you think it's worth using for the new Mutual OpenSearch cluster? ref https://docs.google.com/document/d/1G7OTmBmzl5GVwoanCrzQNHMMXIrAvROpmKumdWtZfuo/edit?tab=t.0#heading=h.iaitjr2h6yxd [14:11:06] inflatador: version choices are always a bit arbitrary, unless you have specific use cases for features only in the newer software [14:12:34] from david's comments it looks like at least ttmserver has some requirements on plugins we maintain [14:14:00] ebernhardson Ah, good catch. That means we'll either need to use 1.x or wait to migrate TTMserver until we have plugins on 2.x then [14:14:12] (I think?) LMK if not [14:14:41] yea, migrating the plugin probably isn't super hard but we don't currently have a story around how a single plugin repo can support multiple versions of opensearch [14:14:54] maybe we maintain specific branches or something...its possible but decisions have to be made :) [14:17:43] Nah, we don't need to do a lot of extra work for this cluster. I think the options are to use 2.x and worry about migrating ttmserver in the future, or use 1.x and be able to migrate immediately. I'd rather start with 2.x and wait to migrate, 1.x is 2 major versions behind already [15:22:39] hmm, retrying the cirrussearch-opensearch-image build today worked (with the old buildkit:0.16.0). Logs show this is the first time (out of 5 tries) it ran on gitlab-runner1002, but not sure why that would matter [15:23:55] How do you control the buildkit version? Is that somewhere in blubber conf? [15:24:14] it's a container reference in the top of the .pipeline/blubber.yaml file [15:24:50] there are newer versions, they are up to v1.3.0, but i've never been able to get the >= 1.0 versions to work. They all fail very early in the pipeline. This is a v1.3.0 build: https://gitlab.wikimedia.org/repos/search-platform/cirrussearch-opensearch-image/-/jobs/558584 [15:25:50] my best guess is that a dockerfile is generated, it reports transfering 2.24kB, but it's somehow invalid? Because it fails immediately after it says it transfered the dockerfile [15:28:32] I don't think Blubber uses dockerfiles? https://doc.wikimedia.org/releng/blubber/#usage . But I guess it could be a bad buildkit command, same concept [15:28:47] i think under the hood there is still a dockerfile, it's just that blubber generates the dockerfile for you [15:29:47] i suppose thats not required though, it's possible to build containers directly with `docker ...` commands and not using a dockerfile [16:00:25] If we have the Wednesday meeting now-ish, I'll be a few minutes late. [16:18:56] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/1142638 [16:34:23] ryankemper follow up from last week, I added some instructions to the package build playbook. LMK if you have any questions https://gitlab.wikimedia.org/repos/search-platform/sre/ansible-playbooks/wmf_opensearch_plugin_deploy [16:54:28] Trey314159: https://meta.wikimedia.org/wiki/Community_Wishlist/Wishes/Better_add-new-wikilink_searches [16:56:53] Trey314159: https://hr.wikipedia.org/w/index.php?search=Svetog+Rimskog+Carstva&title=Posebno%3ATra%C5%BEi&profile=advanced&fulltext=1&ns0=1 [16:58:08] Trey314159: https://hr.wikipedia.org/w/index.php?search=%D0%A1%D0%B2%D0%B5%D1%82%D0%BE+%D1%80%D0%B8%D0%BC%D1%81%D0%BA%D0%BE+%D1%86%D0%B0%D1%80%D1%81%D1%82%D0%B2%D0%BE&title=Posebno%3ATra%C5%BEi&ns0=1 [17:06:21] Trey314159: https://news.ycombinator.com/item?id=44484137 [17:18:09] I'm playing around with opensearch-cli. The profiles feature could be very useful for our purposes https://docs.opensearch.org/docs/latest/tools/cli/ [17:34:29] lol, of course now that i sit back down to work i remember the thing i forgot about before :P [17:35:01] Our expected-indices api call fails since the introduction of dns-disc, because expected-indices is trying to determine the indices expected on dns-disc and doesn't know, as it's a pseudo cluster [17:35:35] so the question is, how should expected-indices know which set of clusters to report on? We already have a "writable clusters" concept, but that's not quite what we want here [17:35:48] almost need another name that means "cirrus maintains index schemas, but not writing to it" [17:37:01] so in addition to $wgCirrusSearchWritableClusters, we could have $wgCirrusSearchMaintenanceClusters ? sounds awkward [17:38:39] wgCirrusSearchManangedClusters? [17:38:47] errr.."Managed" that is [17:39:32] hmm, Managed does sound a little better [17:55:07] lunch, back in ~40 [18:46:12] back [19:12:29] * ebernhardson realizes problems with the regex transform while looking back over it :P Not sure if worth fixing [19:12:45] err, well maybe not. hmm [19:14:56] Lunch