[07:21:35] <godog>	 greetings
[07:58:58] <dcaro>	 morning!
[07:59:05] <dcaro>	 thanks dhinu.s!
[08:23:18] <dhinus>	 morning
[09:05:13] <dcaro>	 dhinus: the fix for the filelog thing is not enough, I reopened T423828 to keep track there
[09:05:14] <stashbot>	 T423828: jobs-cli: stop resolving dynamic fields in the cli - https://phabricator.wikimedia.org/T423828
[09:07:13] <dhinus>	 dcaro: I think I linked my patch to that task by mistake, I wanted to link it to https://phabricator.wikimedia.org/T428354
[09:07:48] <dcaro>	 xd, okok, I'll re-close then
[10:46:02] * dcaro lunch
[11:46:03] * godog lunch too
[13:30:29] <godog>	 andrewbogott: good morning, what do you think of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1303962 ?
[13:32:17] <andrewbogott>	 If they're in the right racks then let's do it.
[13:32:38] <godog>	 correct yes, they are now
[13:32:53] <andrewbogott>	 Puppet should do everything but after nova is up and running there's a nother puppet change needed, adding profile::openstack::base::nova::compute::id to hiera
[13:33:12] <andrewbogott>	 that ID allows reimaging of cloudvirts without nova being confused about the hostname going away and coming back.
[13:33:25] <godog>	 ok thank you, I'll send that too right away
[13:34:03] <andrewbogott>	 The process is probably not documented anywhere :/  unless you already found some docs about this?
[13:34:38] <godog>	 I have not, no, though I may have missed it
[13:35:09] <godog>	 ok the ID patch will have to actually wait for nova to generate the id, is my understanding
[13:36:00] <andrewbogott>	 that's right
[13:36:08] <andrewbogott>	 and now I'm trying to remember where it comes from :)
[13:38:15] <andrewbogott>	 ...it comes from "openstack hypervisor show <fqdn>"
[13:43:03] <godog>	 ok!
[13:43:06] <andrewbogott>	 godog: here is a tiny scrap of docs that I just wrote for this:  https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Maintenance#new_cloudvirt_install
[13:43:53] <andrewbogott>	 hm, why no syntax highlighting
[13:44:28] <andrewbogott>	 because you need open tags as well as close tags, that's why
[13:45:33] <godog>	 nice, thank you
[13:54:07] <andrewbogott>	 Hm, on Thursday did someone volunteer to sweep up paws-nfs?
[14:18:04] <dhinus>	 I did not but I opened a task ("There's a Phab for that!" :P)
[14:18:40] <dhinus>	 I see it's slowly creeping up so we should probably do it this week. did anybody clean up that volume in the past?
[14:18:44] <dhinus>	 I have no idea what's filling up the space
[14:39:27] <dcaro>	 I removed some stuff when we were looking for non-allowed usage
[15:35:17] * andrewbogott remembers writing docs about how to clean up PAWS, but where is it?
[15:48:50] * andrewbogott finds it at https://wikitech.wikimedia.org/wiki/PAWS/Admin#NFS_cleanup
[15:53:52] <andrewbogott>	 902Gb in  files-to-delete!
[15:57:43] <dhinus>	 nice!
[15:59:03] <dhinus>	 should we delete them automatically after x days? :)
[15:59:36] <andrewbogott>	 yes, maybe
[15:59:48] <andrewbogott>	 Well, better would be deleting files in that directory which are older than x days
[15:59:51] <andrewbogott>	 so we retain a per-file grace period
[16:00:17] <andrewbogott>	 would be a one-line 'find' command
[16:01:27] <dhinus>	 makes sense. can you check if we can add that wiki as a runbook link in the alert?
[16:02:26] <dhinus>	 hmm maybe it's a generic alert, not specific to paws...
[16:03:11] <dhinus>	 we would need a generic runbook that links to the specific ones
[16:04:20] * dhinus has to log off
[18:28:17] <volans>	 andrewbogott: the backups retention restore: https://gerrit.wikimedia.org/r/c/operations/puppet/+/1304865 
[18:30:19] <andrewbogott>	 looks good
[18:31:54] * dcaro off
[18:31:56] <dcaro>	 cya tomorrow!
[18:35:16] <volans>	 thx