[08:10:39] godog: FYI thanos-be1003 is cron-spamming every 5 minutes with: [08:10:40] Exception during recon-cron while accessing devices: [Errno 13] Permission denied: '/srv/swift-storage/sdf1' [08:10:56] (cross posting in -databases too) [08:11:30] volans: thank you, I'll take a look [08:12:32] thanks [08:31:55] volans: if you have a minute for a quick review of https://gerrit.wikimedia.org/r/c/operations/puppet/+/702320/ [08:32:12] tested on thanos-be1003 and DTRT [08:32:14] sure [08:33:41] godog: what "root" means in that context? [08:34:05] I'm not familiar with the puppetization there [08:34:19] volans: the root user [08:35:02] sory I meant mountpoint-root-* [08:36:01] ah yeah that's a bit of a nasty puppet trick, basically we flip permissions on the mountpoint depending whether the filesystem is mounted or not [08:36:17] so mountpoint-root vs mountpoint-swift [08:36:22] ah [08:36:23] ok [08:36:35] seems sane :) [08:36:37] +1ed [08:37:04] thank you! appreciate it [08:37:22] thanks for the fix! [08:37:41] np, that was easy enough [08:38:22] that disk failing the way it did at least uncovered a bunch of problems heh [08:38:29] including https://phabricator.wikimedia.org/T285835 [08:45:35] eh [08:45:45] rabbit holes multiplier