[00:36:10] !log wikitextexp Deleted instance parsing-qa-01 (T292264) [00:36:15] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikitextexp/SAL [00:36:15] T292264: Loss of access to parsing-qa-01.eqiad.wmflabs - https://phabricator.wikimedia.org/T292264 [00:43:39] chicovenancio: I found a fix which worked for us and may work for you! https://phabricator.wikimedia.org/T291387#7400771 [00:48:28] !log wikitextexp Remove self (BryanDavis) from project after helping resolve T292264 [00:48:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wikitextexp/SAL [00:48:34] T292264: Loss of access to parsing-qa-01.eqiad.wmflabs - https://phabricator.wikimedia.org/T292264 [09:39:36] !log admin [codfw1dev] cleaning up manila stuff from openstack (db, endpoints, tenant, VMs, and such) T291257 [09:39:40] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [09:39:40] T291257: Cloud: NFS: PoC: manila with generic driver using DHSS=true - https://phabricator.wikimedia.org/T291257 [11:59:58] hello folks [12:00:18] I can't login to deployment-cache-text06.deployment-prep.eqiad.wmflabs anymore for some reason [12:01:05] all good w/ other cloud project (eg: traffic) [12:01:29] ema: that instance has something wrong with it, it has done that before too (https://phabricator.wikimedia.org/T286502) [12:02:12] majavah: thanks but I think the instance in itself is probably ok given that it serves https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page just fine [12:02:49] see for example curl -v https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page 2>&1 | grep x-cache: [12:03:35] majavah: can you ssh into the instance? [12:03:47] !log root@deployment-cache-text06:~# systemctl restart sssd # T286502 [12:03:48] majavah: Unknown project "root@deployment-cache-text06:~#" [12:03:49] T286502: Figure out why deployment-cache-text06 keeps crashing - https://phabricator.wikimedia.org/T286502 [12:03:55] !log deployment-prep root@deployment-cache-text06:~# systemctl restart sssd # T286502 [12:04:01] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep/SAL [12:04:26] majavah: now I'm in [12:04:31] majavah: thanks very much! [12:04:45] I can with a key in `passwords::root::extra_keys`, but could not with my normal account before restarting sssd [12:24:00] !log deployment-cache-text06: upgrade varnish to 6.0.8-1wm1 T292290 [12:24:01] ema: Unknown project "deployment-cache-text06:" [12:24:01] T292290: Package and deploy Varnish 6.0.8 - https://phabricator.wikimedia.org/T292290 [12:27:44] hi, with ceph in cloud vps is it fair to say we can ditch '::profile::labs::lvm::srv' calls ? I'm looking at graphite [12:28:13] godog: do you know who is using it? [12:29:05] dcaro: not off the top of my head, but if someone is they should talk to me [12:29:32] I'm expecting new instances to be on ceph and thus not have/require lvm for /srv [12:29:35] afai can see, toolforge prometheus is using it [12:29:59] package builder too (same project) [12:30:18] sorry I wasn't clear, I want to remove ::profile::labs::lvm::srv from graphite only [12:30:25] safe to do so, I'm assuming [12:30:57] ah, let me recheck with that in mind xd [12:31:57] coludmetrics uses it [12:33:59] those are physical servers [12:34:06] ah, yeah that should be fine [12:34:18] ok I'll send a review as it seems fine to remove to me [12:34:30] existing instances won't be affected, and new instances will be using ceph anyways [12:35:05] can't wait to ditch all of this graphite stuff, one day! [12:35:27] why do you assume that new instances will use ceph? [12:36:19] (not that I don't think it would be a good idea, but curious as we have replacement for those physical machines) [12:36:27] *refresh [12:37:12] ah perhaps I'm misunderstanding, I thought all newly provisioned cloud vps VMs have ceph [12:37:40] yes, the VMs do [12:37:55] (through cinder) [12:38:32] let me rephrase xd, why do you assume that cloudmetrics is going to become a VM? [12:38:44] I don't think I said or assumed that [12:39:44] okok, my bad, I assumed that that was why it was ok to remove from profile::graphite::base [12:40:32] ah yeah I see, the ::require is only for labs $realm so we're fine [12:41:25] looking at the review in this case will be faster than explaining heh :) [12:41:34] oh, I did just grep so saw only one line :), that's great then! [12:44:23] hehe ok! thanks dcaro for your help [12:44:45] https://gerrit.wikimedia.org/r/c/operations/puppet/+/726614 is the related review [12:46:20] looking [13:05:14] cheers! [15:33:08] !log toolhub Updated demo server to 2950dc6 [15:33:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Toolhub/SAL