[01:02:43] !log tools.sal restarted webservice, was throwing HTTP 500 [01:02:45] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.sal/SAL [01:02:48] zabe: ^^ [08:45:16] hi all, a question. Given this message: https://phabricator.wikimedia.org/T306103 to solve the issue I can `do-release-upgrade` is that right? [08:47:03] CristianCantoro: preferrably you'd create a new instance with a newer debian version instead of upgrading the current one [08:49:27] taavi I see, I will discuss about it with the other project admins [08:59:23] legoktm: I wonder if it could have a probe that auto restarts if a 500 [09:07:00] ok, another couple of questions regarding upgrading the OS on a VPS instance: 1) I see in the instance there is a volume attached, but it does not show uo under "Volumes" on Horizon [09:07:17] also, I wanted to ask if a snapshot could be created [09:14:23] CristianCantoro: I don't see any volumes that exist in the wcdo project. The existing instance is using an older system where the instance itself had non-movable extra disk space [11:02:13] taavi: thanks so that's why I don't see volumes [11:02:37] I am wondering what's the best way to make a backup / snapshot [11:10:05] I mean, the root disk is 19GB, the second local disk (`/dev/mapper/vd-second--local--disk`) is 138GB and in total there is ~110GB of data on the two disks [11:11:11] so, copying them out will take quite a bit [11:18:14] also, one thing that I don't understand is: if I want to deploy a new instance, I need to backup the old one first, then delete it and then copy back all the files because I don't have any resource lefrt (CPU, RAM) in the project [11:18:20] is that correct? [12:53:12] CristianCantoro: you can request a temporary quota increase to give your project room to do the update. https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_Instances#Increase_quotas_for_projects [14:21:01] RhinosF1: definitely, want to file a task? [14:23:28] legoktm: yeah, in a taxi at moment but will do either when I get to hotel this evening or if train wifi works [14:35:20] legoktm: https://phabricator.wikimedia.org/T314053 [14:35:34] I worded it as might be generally useful [14:35:48] In case other tools want similar and it's best done as something anyone can do [14:48:00] pretty sure this can be done in k8s with a liveness probe: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/ [14:48:29] in the Wikidata Lexeme Forms tool I use a startup probe to make restarts seamless (only destroy the old container once the new one is ready), which seems to be working well: https://github.com/lucaswerkmeister/tool-lexeme-forms/blob/main/patch-add-startup-probe.yml [14:49:01] (the /healthz endpoint used there doesn’t do anything special, it’s just cheaper to render than the proper index page) [16:21:14] Lucas_WMDE: RhinosF1: yeah, I had a k8s liveness probe in mind too. Would be nice if webservice had some shorthand for it instead of needing to do yaml stuff [16:25:52] I'm willing to accept a `webservice` patch to add an arg/config option to add a probe on /healthz or some other convention [19:54:09] !log tools.pagepile-visual-filter deployed 8b0b90317d (automatically reload images on errors) [19:54:10] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pagepile-visual-filter/SAL [19:56:12] hey all so my bot was approved, where do I schedule the auto schedule and which documentation? I see kubernates and such. I am running a local install on toolforge so nothing gets broken if I were to use the shared