[06:07:44] <duesen>	 Hi all! I want to deploy another set of changes for the rest gateway today. Configuration and documentation only: a regex fix, more cg-nat IPs, and experimental shadow-mode limits. I stacked the three patches: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1270767
[06:08:46] <duesen>	 I'm planning to do that during the 10:00 UTC window, but I have to go fetch my car from the garage first. I may not be back in time.
[06:09:04] <duesen>	 Would it be ok to do it later in the day as well?
[09:37:15] <claime>	 sure duesen 
[09:47:18] <duesen>	 claime: could you have a quick look at https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1270767/1 and its parents, and give a +1 if they look ok to you? They are all trivial, but I don't want to merge them without a vote from someone else.
[09:48:06] <claime>	 sure, some light reading with my coffee :P
[09:50:13] <duesen>	 i guess you can just look at the combined diff: https://integration.wikimedia.org/ci/job/helm-lint/31507/console
[09:52:38] <claime>	 lgtm
[09:57:49] <duesen>	 ok, i'll hit +2
[09:58:14] <claime>	 shipit
[10:13:58] <duesen>	 tests pass on staging. i'll apply to codfw
[10:14:14] <claime>	 nice
[10:14:30] <claime>	 brb bio but I have my phone with irc if there's an emergency
[10:23:08] <duesen>	 thanks. looking good, applying  to eqiad.
[10:32:12] <claime>	 back
[10:33:41] <duesen>	 ok done. all good.
[10:34:10] <duesen>	 well, nothing broke. I'll need to wait a couple of hours to confirm that the tweaks are working as intended.
[10:34:23] <claime>	 Nice :)
[10:34:42] <claime>	 duesen: I have the CRs up (more or less WIP) for liftwing
[10:34:58] <claime>	 Do you want the links so you can stand the rate limit policies on them?
[15:04:51] <duesen>	 claime: hey, sorry, didn't see your ping... sure, i'll have a look!
[15:05:15] <duesen>	 meanwhile i have two more patches up that I would like to deploy today... if possible, right now.
[15:05:42] <claime>	 duesen: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1269401 and followups (only the last two interest you)
[15:05:51] <claime>	 I've +1'd your two patches already
[15:06:05] <claime>	 You can deploy
[15:06:26] <duesen>	 claime: oh nice, thanks! I'll do that. I'll do rate limits for liftwing later today or tomorrow.
[15:07:09] <claime>	 duesen: That'd be great, I may merge these patches early tomorrow with the BYPASS class so I can start testing the routes are ok
[15:08:33] <duesen>	 Sounds good!
[15:09:56] <duesen>	 can you remind me how to apply the redioscope service? what do i have to tell helmfile?
[15:10:55] <duesen>	 oh i see - it'll be helmfile -e aux-k8s-eqiad apply
[15:12:44] <claime>	 yeah from within the aux-k8s/redioscope folder
[15:16:38] <duesen>	 ah crud. foating point numbers in xyamle getting processed by helm templates are... gah.
[15:16:47] <duesen>	 "0.9999": 1e-05
[15:16:51] <duesen>	 that will break.
[15:17:19] <duesen>	 maybe a string would work there. maybe not. I'll just undo that part of the patch...
[15:17:49] <duesen>	 i'll move the redioscope survey definition from the chart to the service too. makes more sense.
[15:18:41] <duesen>	 meanwhile i'm applying the gateway patch
[15:26:12] <duesen>	 claime: could you have a look again? this *should* be trivial, but I didn't test locally. I just moved stuff around, but I may have messed something up.... https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1270976
[15:28:11] <claime>	 CI says no 
[15:29:08] <duesen>	 *sigh*
[15:29:26] <duesen>	 I have a meeting now. I'll fix it after. redioscope is out of whack for now.
[15:30:01] <claime>	   12   │ {{ toYaml .Values.app.surveys.rest_gateway | indent 4 }}
[15:30:03] <claime>	 That's your issue
[15:30:10] <claime>	 charts/redioscope/templates/configmap.yaml
[15:48:28] <cdanis>	 Amir1:  https://trace.wikimedia.org/trace/d87b8ba17060b0a2c725624c9fbf7544?uiFind=b0389802610f33e5
[15:49:39] <cdanis>	 if it would be helpful we can add some spans for different phases of the operation in filemultibackend (or whatever its called)
[15:58:05] <Amir1>	 okay meeting over
[15:58:07] <Amir1>	 Awesome <#
[16:00:10] <duesen>	 claime: fixed it, CI is happy now
[16:01:07] <Amir1>	 cdanis: Isn't amazing we spend 0.5s just talking to the other side of continent during every upload? 
[16:01:31] <cdanis>	 indeed
[16:08:41] <duesen>	 claime: ah, good catch. that's what i get for coding during meetings...
[16:08:56] <claime>	 :D
[16:19:07] <duesen>	 claime:I#ll go ahead and deploy now. it's just an aux service, shouldn't interfere with anything.
[16:19:26] <claime>	 duesen: yep you're good to go
[19:00:15] <urandom>	 swfrench-wmf: are you available to deploy `admin_ng`? 
[19:00:37] <swfrench-wmf>	 o/
[19:00:40] <swfrench-wmf>	 yes, can do
[19:01:34] <urandom>	 thanks!
[19:01:48] <urandom>	 (one more after this one)