[00:38:34] OK, I'm about to head to bed since it's 01:37 local time, but I'm having "fun" with some cinder volumes at the moment. When I started, Horizon was reporting two of my volumes were attached to "/dev/sdb on accounts-mwoauth3, /dev/sdb on accounts-mwoauth3" (yes, twice). I managed to detach it once via Horizon, but it wouldn't detach the second time. [00:39:20] I figured deleting the instance they were attached to would force-detach them (since I'm rebuilding that instance anyway, but now they're reporting they're attached to a GUID (/dev/sdb on 37bb784c-136d-4345-b6cd-0789158932c8 ) [00:39:32] Any ideas on how I proceed to attach these volumes to a new instance? [00:39:48] (this is the account-creation-assistance project) [07:19:31] Is any database-god present? [07:19:50] MariaDB [dewiki_p]> select * from templatelinks limit 1; [07:19:56] ERROR 1356 (HY000): View 'dewiki_p.templatelinks' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them [07:29:56] !log admin Starting up all the osd daemons on cloudcephosd1025 (T314870) [07:30:00] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [07:30:00] T314870: Setup cloudcephosd10[25-34] into the ceph eqiad cluster - https://phabricator.wikimedia.org/T314870 [08:01:17] Amir1: Wurgl's report above seems related to your templatelinks work. [08:01:52] JJMC89: yup, on it https://phabricator.wikimedia.org/T312865#8164341 [08:02:17] we need to new wiki creation to finish, then I need to run maintain views [08:02:33] thx [08:04:21] Wurgl: Can you try again? It should be fixed now [08:04:35] Great! Works [12:25:37] stw: we force-detached the 2 volumes, can you try if they now work correctly? [12:26:36] Awesome, thanks - I'll give it a go when I'm back at my desk [14:43:54] !log tools removing some inactive projectadmins: rush, petrb, mdipietro, jeh, krenair [14:43:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [14:45:10] !log tools adding lucaswerkmeister as projectadmin (T314527) [14:45:12] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [14:45:12] T314527: Toolforge root for LucasWerkmeister - https://phabricator.wikimedia.org/T314527 [14:47:23] yayyyyy [14:56:39] dhinus: that worked, thanks again for your help :) [15:03:00] lucas is on a cycle tour I think, so it may be a while before he shows up and tells me what things I missed. [15:20:34] Hi. I am uploading documents to Commons via login.toolforge.org. Some of my bot scripts get “python3 upload.py” terminated by signal SIGKILL (Forced quit). Could anyone please help me find out why? [15:21:58] Is there limits on the server that restrict a user from running too many scripts concurrently? [15:28:31] andrewbogott: thanks \o/ I am indeed away from my PC for a few more days :) [15:29:27] Hoi: there are definitely limits to what you run on the bastion host; typically it's just used as a jumping-off point for things scheduled using the grid or kubernetes. [15:29:38] That said if you do few enough things at a time you can probably do what you need there. [15:29:50] but `sudo whoami` (root) and `sudo who am i` (lucaswerkmeister, pts blah) seem to work ^^ [15:30:38] that's a good start! [15:43:58] andrewbogtt: Got it. Thanks. [16:44:13] Hoi: I don't think the limits are actually higher, but we do appreciate folks using the dev.toolforge.org bastion instead of login.toolforge.org when they need to run things more intense than starting/stopping a job or web service. https://wikitech.wikimedia.org/wiki/Help:Toolforge/Rules#rule_4 is the "rule". This is really all about trying to keep the bastions functional for everyone's use. [16:46:54] running things from a webservice shell can also be a good option [16:49:10] Yes, AntiComposite has a very good point there ^. `webservice shell` is a fairly lightweight process on the bastion with the real work being done on a randomly selected Kubernetes worker node. This also spreads out the NFS i/o load if you are doing things that are disk intensive. [17:03:46] +1 from me on that too ^, it also helps you get familiar with the environment if at any point you want to promote the script/manual run to a cron/webservice [20:43:26] Hi, can someone restart https://sal.toolforge.org/ please? bd808, maybe? [20:43:32] (returns 500 consistently now) [21:02:45] urbanecm: looking. I'm going to try and see if there is a new cause before I restart it [21:05:54] !log tools.sal Restarted for 500 errors like "response not received, request sent" for lighttpd<->PHP. [21:05:56] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.sal/SAL [21:17:42] thanks bd808 [21:19:49] I think every time it has been lighttpd and the PHP fcgi container getting disconnected somehow. I imagine this is a load triggered issue, but I don't have local access logs enabled and have been too lazy to try and check the upstream proxy logs to verify. [21:21:33] urbanecm: would you have any interest in being a co-maintainer for the sal tool so you could help with things like these occasional restarts? [21:21:56] why not, but i can't promise much more than restarting it from time to time :) [21:55:36] !log tools.sal Added urbanecm as co-maintainer [21:55:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.sal/SAL [21:56:23] urbanecm: occasional restarts are more than enough! :) [21:56:38] okay then!