[00:03:58] <ebernhardson>	 ryankemper: alright, we should be good to turn it back on
[00:08:20] <ryankemper>	 cool, fixing the commit message length lint error real quick then will get merging
[00:08:33] <ryankemper>	 ebernhardson: can i haz +1 on https://gerrit.wikimedia.org/r/c/operations/puppet/+/1087598
[00:08:48] <ebernhardson>	 ryankemper: done
[00:12:17] <ebernhardson>	 ryankemper: i'm going to guess role(dumps::generation::worker::dumper) would be the appropriate set to run puppet on
[00:12:55] <ebernhardson>	 hmm, actually thats missing a couple. Essentially snapshot101[0-7]
[00:14:46] <ryankemper>	 Cool yeah I'll do it manually rather than the 3 different puppet roles
[00:14:50] <ryankemper>	 well, manually as in regex-ly
[00:16:05] <ryankemper>	 Meanwhile I ran on snapshot1011 to verify things are working, I see two new timer units
[00:16:08] <ryankemper>	 https://www.irccloud.com/pastebin/tZco8iqi/
[00:17:40] <ebernhardson>	 ryankemper: i suppose we should centralize, i'm also talking to dan on slack. i forwarded you the message on slack
[00:18:04] <ebernhardson>	 forwarding seems odd, but thats what slack suggested when i tried to send a message with only a ping
[00:18:19] <ebernhardson>	 s/message/thread/.  words are hard :P
[08:25:10] <gehel>	 dcausse: would you have a few minutes to brainstorm a blurb about essential work completed sofar this fiscal year
[08:26:15] <dcausse>	 gehel: sure, in 15min would that be ok?
[08:38:47] <dcausse>	 gehel: I'm around if you want
[08:42:26] <gehel>	 dcausse: meet.google.com/por-eknc-twu
[10:15:15] <gehel>	 Interesting question about redirects on https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Suggestion_to_improve_search_bar_results. Would dcausse, ebernhardson or Trey314159 have a some time to answer?
[10:20:28] <dcausse>	 we could explain what it does today but improving it might not be trivial, we do nothing to identify "good" vs "meh" redirects... I had considered using PV at some points but they get very few hits, the canonical page is almost always the one linked 
[10:20:45] <gehel>	 PV?
[10:20:49] <dcausse>	 page views
[10:20:52] <gehel>	 ofc
[10:22:04] <dcausse>	 redirects also have some templates attached to them to inform things like "redirect because of a common typo" but this is very wiki specific and might not be always tagged properly
[14:18:10] <inflatador>	 <o/
[14:23:21] <dcausse>	 o/
[14:27:55] <inflatador>	 dcausse : gehel has asked me to do a server HW forecast for the next 5 yrs...I started https://phabricator.wikimedia.org/T379079 for this, if you have any suggestions/feedback on this LMK 
[14:48:36] <ebernhardson>	 \o
[14:52:18] <dcausse>	 o/
[15:21:34] <gehel>	 dcausse: I'm not going to be able to join the WMDE / WMF sync. I have one point about rescheduling the WDQS forthnightly meeting to include Lydia. I can also follow up with her async
[15:27:45] <dcausse>	 gehel: np, I can talk about that but might be better for you to discuss with directly to find the right time
[16:35:57] <ebernhardson>	 random thought...whats the difference between a category and a weighted tag? Other than that it comes from wikitext
[16:53:39] <inflatador>	 workout, back in ~40
[17:00:36] <dcausse>	 dinner
[17:37:54] <inflatador>	 back
[17:42:19] <Trey314159>	 dcausse: (for tomorrow) https://www.wikidata.org/wiki/Wikidata:Request_a_query#Most_recently_edited_lexemes_for_a_language_in_WDQS
[18:18:31] <inflatador>	 ryankemper any progress on the WDQS hardware stuff? I'm done with the elastic stuff if you need help
[19:06:29] <ebernhardson>	 inflatador: if you're up for it at some point, i have the gitlab repo created now and can create the token to inject a CI_RELEASE_TOKEN env var into some jenkins jobs from puppet private
[19:07:14] <ebernhardson>	 in theory, everything else already works and this is the only change in the jenkins side, maybe :) but once the token is in can try a release from gerrit->gitlab
[19:33:20] <inflatador>	 ebernhardson sure, I'm around now if you wanna walk thru together
[19:35:22] <inflatador>	 https://gerrit.wikimedia.org/g/labs/private has the structure of the private repo, I'd start with a CR there with a dummy secret. If you're not sure where that token needs to go LMK, I can look at the real repo if needed
[19:37:47] <ebernhardson>	 i don't know where exactly, lemme see if anything relevant is in labs/private
[19:38:54] <ebernhardson>	 don't see anything obvious, tbh i'm not even 100% sure secrets are configured there for ci jobs just thats where things typically are
[19:44:27] <inflatador>	 That's where all the secrets are AFAIK. That being said, committing to private is optional and not everyone does it
[19:45:24] <inflatador>	 let me poke around the real repo, I can ask serviceops collab too
[19:45:42] <inflatador>	 I have to leave in 15m but I'll be back by 1:30 your time
[19:45:51] <ebernhardson>	 I suppose i mostly wonder because i see archiva using what looks like https://plugins.jenkins.io/config-file-provider/. Now that was configured a decade ago, so it might not be the current way to do it though
[19:46:49] <ebernhardson>	 kk, no hurry
[19:49:31] <inflatador>	 ah, nice! maybe that helps
[19:51:01] <ebernhardson>	 that would be done through the jenkins ui
[19:51:21] <inflatador>	 I went ahead and asked in releng, they're usually pretty helpful
[19:51:24] <ryankemper>	 inflatador: will have it finished by 1. got most of it but stepped out to tire the dog out and at lunch now
[19:55:25] <inflatador>	 ryankemper ACK, np. Stepping out but will be back in ~90
[21:26:48] <inflatador>	 back
[21:29:07] <ebernhardson>	 so, to guess at if the puppet private repo has jenkins secrets, or if they live in the jenkins ui
[21:30:48] <ebernhardson>	 from poking around in the jjb i seen two basic kinds of secrets, there is one configured by `credentials-binding` with id's like `SONAR_API_KEY` and there is `config-file-provider`, used only by archiva. So i'm guessing we are looking for where SONAR_API_KEY is stored and this goes in a similar place?
[21:35:04] <ryankemper>	 ebernhardson: my initial spelunk in the private repo has failed to find anything relevant
[21:36:36] <ryankemper>	 inflatador: unsure how much detail to go into, I mainly just did host counts and refresh costs amortized over 5 years for wdqs & wcqs respectively, https://phabricator.wikimedia.org/T379079#10297781
[21:36:55] <ebernhardson>	 ryankemper: oh well, thanks for looking. Perhaps i'm best off making a ticket
[21:37:40] <ryankemper>	 ebernhardson: specifically no matches for sonar or SONAR (besides a file listing captcha words which happens to contain sonar as a possible word, lol), and as far as jenkins I see a few references to a jenkins api key but that's about it
[21:37:58] <inflatador>	 ryankemper excellent, thanks
[21:43:28] <inflatador>	 ebernhardson is this the repo that has the JJB's you were looking at? https://gerrit.wikimedia.org/g/integration/config
[21:43:48] <ebernhardson>	 inflatador: yea, mostly poking around job-templates.yaml 
[21:46:40] <ebernhardson>	 i'm going to guess they live here, although i don't have access to see it: https://integration.wikimedia.org/ci/credentials/
[21:50:07] <inflatador>	 I poked in #security as well
[21:50:26] <inflatador>	 speaking of, we just got an alert for CirrusSearchPoolCounterRejectionTooHigh
[21:53:41] <ebernhardson>	 curious, i see it in the graphite metric but not prom in the pool counter dashboard. or i'm doing something wrong
[21:54:38] <inflatador>	 that's not good, hopefully I didn't goof up that panel :(
[21:55:50] <ebernhardson>	 inflatador: turns out it's becaues they are in the automated bucket, and we hide that. The curiosity then is that the automated bucket triggered the alert, those should be expected and normal
[22:02:21] <ebernhardson>	 inflatador: https://gerrit.wikimedia.org/r/c/operations/alerts/+/1087973
[22:03:10] <inflatador>	 ebernhardson you work fast, amigo ;P .  Guessing you are following the thread in #security?
[22:05:26] <ebernhardson>	 inflatador: yea i looked, it seems like there is a general problem around it, but because they happened to land in the `automated` bucket on our side they got a lot less resources than a bot that wasn't would
[22:05:37] <ebernhardson>	 at least, on the search side. They still use up all the php compute
[22:31:33] <ebernhardson>	 sometimes i wonder what the right thing to do is when creating subtasks, sometimes i remove people from the subscribe list because it seems like it would be noise, but other times i don't