[12:48:17] hey folks! [12:48:35] I just moved the /v2/releng.* prefix on the Docker registry to a new S3 backend [12:49:09] so all images with an URI starting with the above are now routed to a new backend, where I copied the images that were requested in the past month [12:49:26] in the best case scenario, we are good and no complain is raised [12:49:49] but we may get users complaining about CI jobs failing due to docker pull failures for some images [12:50:19] if it happens, please blame and ping me :D If I am not around, I wrote https://wikitech.wikimedia.org/wiki/Docker-registry#Storage_backends to guide the process to fix the issue [12:50:25] cc: topranks, fabfur [12:50:38] ack, thanks elukey [12:51:19] elukey: ok thanks! [12:57:54] hi folks - I'm around and will begin prep for the etcd switchover shortly. no disruption planned until 13:30 UTC and I'll follow up here before I start that phase of it :) cc: fabfur topranks [12:58:14] ok, best of luck with it Scott! [12:59:06] best of luck swfrench-wmf! [12:59:58] thank you both :) [13:02:56] ack [13:20:46] can someone tell me where to RTFM on how to get this patch deployed? https://gerrit.wikimedia.org/r/1314962 [13:20:59] it adds a cert and SVG to static/ [13:21:03] (mediawiki-config) [13:30:02] alright, starting the disruptive part momentarily :) [13:37:21] sukhe: should be like any other mediawiki configuration change, so https://wikitech.wikimedia.org/wiki/Backport_windows basically [13:38:27] taavi: ok thanks, I will find a deployer [13:44:45] etcd is R/W again :) restarting requestctl shortly to get it into a good state [13:44:55] nice! [13:46:03] alright, all done. there are a couple of cleanups for me to do, but otherwise no further excitement planned [13:46:15] <_joe_> oh no we like excitement [13:46:25] :) [13:46:34] only during business hours though [13:48:07] <_joe_> taavi: I know marostegui loves when master dbs crash at 4 am [13:48:37] I cannot believe urandom said I sounded sleepy! How should I sound at 4:30! [13:48:44] <_joe_> lmao [13:49:03] <_joe_> I mean I guess he felt guilty of waking you up [13:50:27] I did [13:51:36] marostegui: please sound more alert next time, to lessen my guilt 🙃 [13:51:44] XDDDDDDD [13:54:24] :) [13:54:46] * Emperor tempted to bash that [13:56:44] hahaha [15:45:45] topranks, fabfur: apologies for the explicit ping, but I'm going to do some upgrades in the sessionstore environment momentarily (unless someone objects, ofc). I'm upgrading Cassandra (a major version upgrade), and Java. It's gone flawlessly elsewhere, I don't expect any problems, and I will canary first (codfw, then eqiad) before moving forward, but I wanted to make you aware since it can be so impacting if things don't go well. [15:46:21] urandom: ok no problem at all better we know! [15:46:27] good luck with it :) [15:47:05] ack, tnx [16:22:37] ok, given the mass of the hammer I was swinging, the result was pretty anticlimactic; I'm done messing with sessionstore for today [16:51:41] nice :)