[03:52:50] Doing wdqs reboots today has reminded me of T274270. We should probably take another look at it next quarter and see if we can figure anything out [03:52:51] T274270: WDQS servers taking up to 30 minutes to reboot - https://phabricator.wikimedia.org/T274270 [10:02:31] I've moved T274270 to current work [10:02:31] T274270: WDQS servers taking up to 30 minutes to reboot - https://phabricator.wikimedia.org/T274270 [10:58:47] Lunch [11:22:37] o/ [11:27:54] o/ [11:36:03] lunch [13:41:43] dcausse / pfischer: minor code review if you have time: https://gerrit.wikimedia.org/r/c/wikimedia/discovery/discovery-parent-pom/+/869758 [13:42:34] dcausse: thanks! [14:39:22] gehel thanks for the update re: WDQS pools! [14:39:40] o/ [14:54:18] inflatador: like a lot of things, it is a bit of a mess... [15:01:02] You call it a mess, I call it "job security" ;) [18:21:29] PR for using rsync instead of NFS is up if anyone has time for a review https://gerrit.wikimedia.org/r/c/operations/puppet/+/869828 [18:43:57] lunch, back in time for pairing [19:25:53] dinner [19:27:18] back [23:26:44] inflatador: okay all the wdqs hosts have successfully rebooted. there's a few hosts assigned to us in the security ticket that are left to be done: `apifeatureusage1001.eqiad.wmnet,apifeatureusage2001.codfw.wmnet,search-loader1001.eqiad.wmnet,search-loader2001.codfw.wmnet` (as an aside I'm not sure if we actually own the apifeatureusage hosts themselves, might want to do some digging there) [23:32:53] ryankemper ACK, will take a look at the loaders tomorrow