[07:06:29] <wikibugs>	 10serviceops, 10Commons, 10MediaWiki-File-management, 10MediaWiki-Uploading, and 2 others: 502 Server Hangup Error on esams for "Upload a new version of this file" on Special:Upload on Commons - https://phabricator.wikimedia.org/T247454 (10Aklapper) See also {T239382}
[07:06:55] <wikibugs>	 10serviceops, 10Commons, 10MediaWiki-File-management, 10MediaWiki-Uploading, and 2 others: 502 Server Hangup Error on esams for "Upload a new version of this file" on Special:Upload on Commons - https://phabricator.wikimedia.org/T247454 (10Aklapper)
[08:27:18] <wikibugs>	 10serviceops, 10SRE, 10docker-pkg, 10Patch-For-Review: Refresh all images in production-images - https://phabricator.wikimedia.org/T284431 (10Jelto) > So we have two options for a rebuild:    @Joe I implemented option 1 in https://gerrit.wikimedia.org/r/699752 Could you take a look and add your feedback?...
[10:16:22] <wikibugs>	 10serviceops, 10MW-on-K8s, 10SRE, 10Patch-For-Review: Add all redis and memcached backends to mw on k8s automatically - https://phabricator.wikimedia.org/T284420 (10hashar) For Beta that should be fixed via https://gerrit.wikimedia.org/r/c/operations/puppet/+/699912/ :)
[13:54:25] <elukey>	 hi everybody
[13:54:46] <elukey>	 I am planning to create a very simple `istioctl` deb package as outlined in https://phabricator.wikimedia.org/T278192#7157939
[13:54:53] <elukey>	 lemme know if you have anything against it
[13:55:27] <elukey>	 I already have a basic structure, it basically gets upstream releases and wrap them in a .deb (the  goal is to deploy it on the deployment node)
[13:55:56] <elukey>	 (if you are ok I'd also need that command to be executed since I don't have enough perms, otherwise I'll ask to releng)
[14:06:26] <joe>	 oh so you're following the pig-with-lipstick pattern 
[14:06:30] <joe>	 I love it
[14:12:44] <elukey>	 joe: the alternative is to checkout the repo, for every tag/version that we support, and then copy the binaries
[14:12:57] <elukey>	 it is feasible as well if better looking, no strong opinions
[14:21:27] <jayme>	 elukey: we do have such patterns and a bit of scripting around it in the helm packages for example
[14:22:01] <jayme>	 debian/repack that is
[14:22:28] <jayme>	 taking care of getting the right version from upstream, fetching vendor etc
[14:23:03] <elukey>	 yep I took a look to calico, what I have done is basically create script that fetches from github some versions (plus sha256 verification etc..) and package them
[14:23:24] <jayme>	 oh, so you're not even building it. Okay :)
[14:23:50] <elukey>	 I can do it too, if it is better, no issue!
[14:24:00] <elukey>	 I was wondering what was best and more flexible
[14:36:39] <jayme>	 AIUI we try to build everything inhouse if that's feasible 
[14:37:19] <jayme>	 we at least ensure having the source around that way as well 
[14:38:30] <elukey>	 the way that istio "builds" is also to download a lot of binaries from various places, but maybe I can find a way to build only istioctl
[14:39:52] <joe>	 elukey: to be clear, this is ok!
[15:06:29] <elukey>	 joe: perfect that would simplify it, I'll ask for the new repo and then I'll send a quick code review to explain my idea
[17:31:10] <wikibugs>	 10serviceops, 10Continuous-Integration-Infrastructure, 10DC-Ops, 10netops: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL ) - https://phabricator.wikimedia.org/T283582 (10Dzahn) I can confirm since a while these have been happening. The pattern is always:  - only mgmt - only codfw - ran...