[08:08:27] <zpapierski>	 dcausse: do you have time perhaps to go over our SRE interview questions for this round?
[08:08:58] <dcausse>	 zpapierski: sure
[08:09:47] <zpapierski>	 https://meet.google.com/fbj-nyfd-afg?authuser=0
[08:09:59] <zpapierski>	 https://meet.google.com/fbj-nyfd-afg
[08:53:57] <zpapierski>	 dcausse: is there an easy way to get the image that was used for a particular cindy run?
[08:54:53] <dcausse>	 the image is not rebuild on every run
[08:55:05] <dcausse>	 cindy runs vagrant
[08:55:39] <zpapierski>	 so, unless something else runs and cleans up my stuff, I should just be able to go there and check, good
[08:56:09] <dcausse>	 yes, the machine remains intact until a new build is detected
[08:56:15] <zpapierski>	 thx!
[08:57:28] <dcausse>	 if you want you can stop cindy from polling changes, ssh to the machine, sudo -u cindy -i then tmux attach, then on the upper pane hit Ctrl-C that should prevent it from polling
[08:57:38] <zpapierski>	 ah, ok
[08:57:53] <zpapierski>	 you're doing something that might trigger the build?
[08:58:36] <dcausse>	 no but there are patches sometimes, but tbh I rarely stop cindy on purpose
[09:00:00] <dcausse>	 if we have the web proxy setup can even access mw from your web browser
[09:00:08] <dcausse>	 if we have the web proxy setup *you can even access mw from your web browser
[09:01:16] <zpapierski>	 cool, but I really need ES access
[09:02:04] <dcausse>	 for that you need to ssh there and do: cd /src/mediawiki-vagrant; vagrant ssh
[09:02:08] <dcausse>	 then you can curl
[09:02:20] <dcausse>	 to localhost:9200
[09:02:29] <zpapierski>	 thx
[09:02:35] <dcausse>	 s/src/srv
[09:04:15] <dcausse>	 https://cirrustest-cirrus-integ.wmflabs.org/wiki/Main_Page
[09:04:27] <dcausse>	 just in case, it's sometimes helpful
[09:05:23] <dcausse>	 there're commons-cirrus-integ.wmflabs.org and ru-cirrus-integ.wmflabs.org but the wiki named "wiki" is not exposed
[09:26:12] <zpapierski>	 dcausse: does cindy use some kind of different configuraton that local vagrant? I see a blank index name a reason for a failure
[09:27:10] <dcausse>	 it should be mostly the same, what do you mean by "blank index name" ?
[09:27:31] <zpapierski>	 I mean this:
[09:27:31] <zpapierski>	 021-06-21 09:21:19 cirrustestwiki CirrusSearch WARNING: Search backend error during query_comp_suggest search for 't' after 7: index_not_found_exception: no such index {"query":"t","queryType":"query_comp_suggest","index":"","elasticTookMs":0,"hitsTotal":null,"maxScore":0.0,"hitsReturned":0,"hitsOffset":0,"tookMs":7,"error_message":"index_not_found_exception: no such index"}
[09:27:38] <zpapierski>	 "index":""
[09:27:51] <zpapierski>	 I had that when I didn't load a proper profile
[09:29:03] <dcausse>	 you need to activate the profile you want, I think tests/jenkins/FullyFeaturedConfig.php should be a good place
[09:29:42] <zpapierski>	 why it works within my local vagrant?
[09:30:21] <dcausse>	 no clue, looking for your patch
[09:30:54] <zpapierski>	 https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/697934
[09:31:02] <zpapierski>	 it's not yet cleaned up, sorry for that
[09:31:34] <zpapierski>	 (I'm following a "make it work, make it fast, make it beautiful" order :)
[09:31:52] <dcausse>	 :)
[09:32:09] <dcausse>	 so in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/697934/8/includes/Profile/SearchProfileServiceFactory.php#270 it says use 'query_completion' if nothing else is configured
[09:32:43] <zpapierski>	 not sure what you mean?
[09:34:18] <zpapierski>	 profile itself is here: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/697934/8/profiles/QueryCompletionSuggesterProfiles.config.php
[09:34:40] <dcausse>	 calling: $service->registerDefaultProfile tells the system "When something requests the default profile for SearchProfileService::QUERY_COMPLETION_SUGGESTER in the context SearchProfileService::CONTEXT_DEFAULT) return 'query_completion'
[09:35:01] <dcausse>	 so if nothing is set in the config under CirrusSearchQueryCompletionSettings
[09:35:09] <dcausse>	 then 'query_completion' will be used
[09:35:17] <zpapierski>	 that's ok, right
[09:35:17] <zpapierski>	 ?
[09:35:29] <dcausse>	 which is indeed properly declared QueryCompletionSuggesterProfiles.config.php
[09:35:41] <dcausse>	 so the problem is perhaps not here
[09:36:02] <dcausse>	 but rather in the way the profile data is parsed
[09:36:16] <zpapierski>	 I'm going to pepper few wfDebugs in the code, I want to understand why the same code works locally
[09:36:38] <zpapierski>	 better yet, I need to verify this, but I think maintenance tool also works and it uses the same profile
[09:37:12] <dcausse>	 I think the index does not exist
[09:37:22] <zpapierski>	 yeah, I was mistaken
[09:37:27] <zpapierski>	 it's not there
[09:38:52] <zpapierski>	 weirdly, though, maintenance script don't fail outright
[09:44:51] <zpapierski>	 hmm, interesting - it's failing on index creation actually
[09:45:05] <zpapierski>	 I think I modified auto_create_index at some point for my vagrant
[09:45:15] <zpapierski>	 what's the policy on this one?
[09:45:25] <zpapierski>	 index: /querycomp-production/_doc caused no such index and [action.auto_create_index] contains [-*] which forbids automatic creation of the index [index: querycomp-production]
[09:46:22] <dcausse>	 for prod we own the pattern, currently it's auto_create_index: '+apifeatureusage-*,+glent_*,-*'
[09:46:35] <zpapierski>	 huh, maybe the log message is incorrect - I get index from a profile, not the connection, that's why it's blank, but the failure itself is because of missing index
[09:46:38] <dcausse>	 other patterns can be added
[09:46:53] <zpapierski>	 ok, I'll add mine then 
[09:47:20] <dcausse>	 yeah I think the way the data is passed to the logger does not match what it expects
[09:47:48] <zpapierski>	 all makes sense, I'll verify the practice
[09:48:05] <zpapierski>	 s/the practice/this in practive
[09:49:28] <dcausse>	 you can also create the index before-hand in the reset script
[09:50:06] <zpapierski>	 true, but I sorts of defeats the purpose of a template?
[09:51:45] <dcausse>	 well yes and no :)
[09:52:23] <zpapierski>	 why would I have a template, if I'd just be going off creating index by myself? wouldn't I just provide the mapping on creation?
[09:53:07] <dcausse>	 I haven't tested but I hope you create an index without passing the mapping and such and it will pick the template
[09:53:31] <dcausse>	 if I'm wrong then yes there's no reason to cre
[09:53:56] <dcausse>	 ate the index by hand esp if you have to provide the mapping again
[09:55:46] <dcausse>	 "When an index is created - either manually or through indexing a document - the template settings are used as a basis for creating the index."
[09:56:14] <dcausse>	 so your call, hack the reset script or add the pattern in https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/vagrant/+/refs/heads/master/puppet/modules/elasticsearch/files/elasticsearch.yml
[10:00:37] <zpapierski>	 decisions, decisions...
[10:00:48] <zpapierski>	 since I copied my approach from glent, I'll go all the way
[10:01:17] <zpapierski>	 it feels weird having a template that can't be used easily without explicitly creating the index
[10:02:11] <gehel>	 Lunch
[10:03:39] <dcausse>	 fine by me, I suggest to change the elasticsearch.yaml manually after pushing a patch to mediawiki-vagrant (instead of reprovisionning cindy)
[10:04:22] <dcausse>	 lunch 2
[10:05:18] <zpapierski>	 sure
[10:06:19] <zpapierski>	 I suppose the same change is needed for our actual puppet? this is a specific one for vagrant, I guess?
[10:28:44] <zpapierski>	 https://gerrit.wikimedia.org/r/c/mediawiki/vagrant/+/700497
[10:58:49] <zpapierski>	 break
[12:04:53] <gehel>	 dcausse: looking at https://gerrit.wikimedia.org/r/c/wikidata/query/deploy/+/699746, there is a comment on T284137 about that URL not working. Simple curl check on my side shows that there is a response (which looks reasonable to me). Should I be checking something else before merging that CR?
[12:07:26] <dcausse>	 zpapierski: correct for production it's in the pupppet repo in hieradata/role/common/elasticsearch/cirrus.yaml
[12:10:46] <dcausse>	 gehel: not really I think the service must support https://www.w3.org/TR/2013/REC-sparql11-protocol-20130321 (tl/dr GET or POST with a "query" param) 
[12:10:52] <gehel>	 zpapierski: https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/698567/ has been waiting for a review for some time. I don't feel confident to review this, could you?
[12:13:02] <gehel>	 ebernhardson / ryankemper T265547 seems to be completed (or at least it's not obvious to me what's left to be done). Could you move it to "needs reporting" or make it more explicit on what's left? Thx!
[12:13:03] <stashbot>	 T265547: Replace mjolnir venv deployment scheme in analytics - https://phabricator.wikimedia.org/T265547
[12:35:38] <gehel>	 ryankemper: I've pinged Willy on T281989: it's unclear where those servers are 
[12:35:39] <stashbot>	 T281989: (Need By: TBD) rack/setup/install elastic10[68-83].eqiad.wmnet - https://phabricator.wikimedia.org/T281989
[12:47:39] <zpapierski>	 gehel: sure
[12:55:34] <gehel>	 zpapierski: pitest should run correctly, but not bind to any lifecycle phase. Should be good enough for some testing: https://gerrit.wikimedia.org/r/c/wikimedia-event-utilities/+/699948
[12:56:24] <zpapierski>	 will do 
[12:57:15] <zpapierski>	 24 minutes of waiting to find out I made a typo in Cindy test...
[12:57:39] <zpapierski>	 (IDE did highlight it, so fault is all mine)
[13:59:43] <zpapierski>	 relocating
[14:45:55] <zpapierski>	 yay, cindy finally likes my patch
[14:48:15] <bearloga>	 hey folks! good news I made a python package for analyzing interleaved a/b tests (https://github.com/bearloga/interleaved-python). I'm gonna transfer it under /wikimedia after Neil reviews it (T285110); then Connie and anyone else on your team can use it
[14:48:17] <stashbot>	 T285110: Review code of interleaved-python - https://phabricator.wikimedia.org/T285110
[14:48:59] <bearloga>	 https://github.com/bearloga/interleaved-python/blob/main/docs/demo-2017.ipynb shows how easy it is and applies it to the very first interleaved test we did 4 years ago
[15:41:30] <ebernhardson>	 bearloga: cool! 
[17:04:03] <dcausse>	 dinner
[21:18:00] <ebernhardson>	 ryankemper: hows it goin this week
[22:04:23] <ryankemper>	 ebernhardson: working on elasticsearch cookbook refactor (+ maybe wdqs too, need to migrate to the new spicerack class api)