[10:04:16] <effie>	 I will remove a redis shard, except from some the possibility of alerts, I do not expect any major issues
[10:30:21] <akosiaris>	 o/
[10:46:19] <XioNoX>	 who knows stuff about urldownloader2002 ? it's running out of disk space
[10:46:34] <XioNoX>	 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=urldownloader2002&service=Disk+space
[10:48:02] <XioNoX>	 seems to be for service ops ^ akosiaris (welcome back), effie
[10:49:14] <akosiaris>	 I can have a look
[10:52:13] <akosiaris>	 hmmm
[10:52:20] <akosiaris>	 zotero seems to be doing something 
[10:52:37] <akosiaris>	 for the love of.... zotero on my first day back? what did I do?
[10:57:35] <akosiaris>	 yeah, someone trying to cite http://www.macmillan.org/
[10:58:23] <akosiaris>	 7k time in the last 10mins alone
[11:02:47] <akosiaris>	 I think it's somehow went into a loop. Cause it was 1 single pod and the moment I killed it it no longer spews logs lines at that speed
[11:02:50] <akosiaris>	 hmmm
[11:03:01] <akosiaris>	 task worthy probably
[11:03:07] <akosiaris>	 not sure how to reproduce though yet.
[12:03:30] <akosiaris>	 XioNoX: fixed. TIL more about squid, a subject I was not looking forward to learn more about
[12:03:53] <akosiaris>	 anyway, https://gerrit.wikimedia.org/r/c/operations/puppet/+/713843 should fix the symptom, the cause needs to be investigated more 
[12:05:35] <XioNoX>	 akosiaris: we now have an owner for Squid, thanks!
[12:07:07] <akosiaris>	 Why thanks!!! I was expecting for a volunteer. Great. Now, by the powers vested in me by castle Grayskull (https://en.wikipedia.org/wiki/Castle_Grayskull), I anoint you High Templar of the Squid
[14:15:04] <effie>	 I will be removing quite a number of shards from the redis cluster that mediawiki is using 
[14:19:41] <effie>	 apart from some alert and some unavoidable user impact (some unsaved states, some lost user actions) 
[14:19:52] <effie>	 I do not expect any issues 
[14:39:26] <dcaro>	 I guess the 'check_icinga' email was this? ^
[14:50:07] <l5m>	 akosiaris: welcome back :)
[14:50:33] <l5m>	 https://phabricator.wikimedia.org/T286525 was the task I had filed about urldownloader
[14:53:17] <dancy>	 akosiaris: Welcome back!
[15:03:20] <Amir1>	 !bash akosiaris> XioNoX: fixed. TIL more about squid, a subject I was not looking forward to learn more about
[15:03:20] <stashbot>	 Amir1: Stored quip at https://bash.toolforge.org/quip/jv3wXnsB8Fs0LHO5IFmc
[15:14:46] <akosiaris>	 dancy: ❤️
[15:15:20] <akosiaris>	 Amir1: ❤️‍🔥
[15:15:38] <Amir1>	 :D
[15:15:38] <kormat>	 oh nice, an emoji for heartburn
[15:16:06] <akosiaris>	 yeah pretty new stuff https://emojipedia.org/heart-on-fire/ 
[15:16:25] <akosiaris>	 I see it as a heart and and a fire in thelounge but whatevs
[18:21:31] <Krinkle>	 effie: I see a third pair of redis_misc hosts in Grafana, is that temporary or woudl it make sense to start describing a Pair 3 on wikitech:Redis?
[18:21:45] <Krinkle>	 (e.g. hardware refresh)
[18:22:08] <effie>	 I do not remember really, there was a refresh recently, but I wasn't involved
[18:22:48] <effie>	 but  I am sure we can figure it out :p
[18:24:10] <Krinkle>	 https://phabricator.wikimedia.org/T264353
[18:24:39] <Krinkle>	 looks like rdb 11-12 at least started as refresh of 5-6
[18:26:26] <Krinkle>	 and still is, I think?
[18:26:27] <Krinkle>	 https://codesearch.wmcloud.org/operations/?q=rdb1005&i=nope&files=hiera&excludeFiles=&repos=Wikimedia%20Puppet
[18:26:43] <Krinkle>	 looks like both parts of that are replicating from 5 so it's still on track to be phased out but maybe forgotten?
[18:40:39] <effie>	 could be