[00:07:07] I don't know if anyone is still online... I've deduced I need to install `certifi` *before* `requests`, to ensure the correct old version is installed. Unfortunately it seems to hang on `Downloading/unpacking certifi-2018.8.24` [00:07:35] this is my second attempt [00:07:58] it's still running, but it definitely shouldn't take this long, right? should I just leave it running and see what happens? [00:08:09] over 20 minutes now :/ [00:11:11] it seems to have worked for me [00:11:16] webservice python2 shell [00:11:20] python -m virtualenv tmpvenv [00:11:24] source tmpvenv/bin/activate [00:11:31] pip2 install certifi==2018.8.24 [00:13:57] hmm [00:14:09] maybe I should just install everything that way, instead of using a bootstrap script? [00:14:56] this is just a temporary thing, if you weren't aware https://phabricator.wikimedia.org/T306888#9616566 [00:15:17] musikanimal: manually installing from inside a `webservice python2 shell` session is a perfectly reasonable thing to do. [00:15:26] okay great :) [00:16:06] the main thing that is needed is to be in the proper runtime target container type when you make the venv. [00:16:51] and `webservice TYPE shell` does that functionally identically to a bootstrap script in a job [00:23:54] +1 [00:27:21] Lucas, you used `pip2` instead of `pip`. Why is that? [00:27:43] `pip` still seems to hang [00:27:58] (on `certifi-2018.8.24`) [00:29:35] `pip` gave me `ImportError: No module named 'pip'` [00:30:11] I’m not exactly sure why, I already deleted the tmpvenv again ^^ [00:31:16] okay, I see `pip2` is specifically for Python2. I'm trying that now [00:31:35] did you run the above on Toolforge? [00:31:39] yes [00:31:49] `pip2 install certifi-2018.8.24` seems to hang too for me :( [00:31:54] (there would be no `webservice` command anywhere else ;)) [00:31:56] :( [00:32:00] hehe right [00:32:30] maybe I'm hitting the resource limits for `eranbot` or something? [00:33:03] trying to think why it would work for you and not for me [00:34:57] it must be doing something wild. A simple ^C to kill it takes foreverrrrr [00:37:57] https://grafana.wmcloud.org/d/TJuKfnt4z/kubernetes-namespace?orgId=1&var-namespace=tool-eranbot&viewPanel=2 [00:39:46] thanks, so I guess that's not it [00:40:06] err, maybe it is... [00:41:25] well, I've spent enough time on this for today. I'll post my findings and what little progress I've made to T306888 and revisit later [00:41:26] T306888: Migrate ERANBOT project off of Grid Engine - https://phabricator.wikimedia.org/T306888 [00:41:29] thanks to all for the help! [00:50:54] bahhh I put `certifi-2018.8.25` instead of `certifi==2018.8.25`. Weird how that would make it time out [00:51:11] okay I'm getting somewhere! running the bot and discovering more packages that are missing. yay for not using requirements.txt [01:00:26] !log melos@tools-sgebastion-10 tools.stewardbots SULWatcher/manage.sh restart # SULWatchers disconnected [01:00:30] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [10:55:27] Good morning. I get a module "importlib.matadata" error in my pywikibot bot? What has happened these last few days that I am not aware of? https://paste.toolforge.org/view/8ce86114 [10:55:55] Thanks in advance [11:33:22] Is there anybody there? [11:34:26] What are you running on? [11:36:15] And what python version [11:41:42] !log lucaswerkmeister@tools-sgebastion-10 tools.bridgebot Double IRC messages to other bridges [11:41:46] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL [11:53:16] +1 to that question – the paste looks like you ran the command directly on the bastion, which might be fine for one-off debugging / investigating, but it’s not how you should normally run a bot (re @wmtelegram_bot: What are you running on?) [11:59:01] !log melos@tools-sgebastion-10 tools.stewardbots Restarted Stewardbot [11:59:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [12:09:51] I am a newbie, I am running python3 but I don't understand what does it mean "directly on the bastion" [12:11:01] It can be that I run my bot abnormally, but something happened because I was running the bot this way until yesterday [12:42:09] !log taavi@tools-sgebastion-11 tools.wikibugs toolforge jobs restart irc, gerrit [12:42:14] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL [12:48:14] !log tools hard reboot tools-sgebastion-10 due to stuck NFS procs [12:48:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [12:55:47] !log tools.wikiauthbot-discordbot kill bot running in tmux on the bastion. please use https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework instead [12:55:49] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikiauthbot-discordbot/SAL [13:24:22] could someone reboot FlickreviewR 2 please ? :) [15:25:19] Can somebody guide me, please? [16:06:03] !log paws increase number of worker nodes T359747 [16:06:07] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL [16:06:07] T359747: Increase paws nodes for outreachy demand - https://phabricator.wikimedia.org/T359747 [16:07:38] Pau can you describe in more detail how you were running the bot until yesterday? it’s still not 100% clear to me [16:08:01] Ok, thanks [16:11:45] In the case that you can see at https://paste.toolforge.org/view/8ce86114, I run manually this bot, which is a very simple python program. The code of the bot is the one you can see at https://ca.wikipedia.org/wiki/Usuari:Rebot/Catpet [16:13:32] I login to toolforge, become rebot (my tool) and after that, I simply wrote "python3 catpet.py" and the program started (until yesterday) [16:14:27] ok, and what do you normally do then? do you just leave the shell open until it finishes running? [16:16:25] yes [16:17:17] If I want to close the shell, I use "screen", but maybe there's a much better way to do that, I presume ... [16:17:48] I am myself a chemist, I do what I can with toolforge [16:18:21] okay, I thought it was a bot that ran continuously somewhere [16:24:03] the shared pywikibot installation seems to be broken in all the Python versions I tried o_O [16:24:16] in 3.7 it’s the error you pasted (No module named 'importlib_metadata'), in 3.9 and 3.11 it’s instead No module named 'packaging' [16:24:21] well, I had a cron programmed with some bots, but It stopped a month ago because of the migration to kubernetes [16:24:32] I have not been able to switch it [16:43:03] !log admin restarted nova-api on cloudcontrol1006 [16:43:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:54:13] I understand it's a wide problem, it does not only affect me (re @lucaswerkmeister: in 3.7 it’s the error you pasted (No module named 'importlib_metadata'), in 3.9 and 3.11 it’s instead No module named 'packagin...) [16:55:04] Do not run noninteractive processes on the bastion servers . [16:55:08] The new version of pywikibot (9.0) has new dependencies - importlib_metadata for python 3.7 and packaging for all python versions. [16:55:14] The documentation for the supported ways of running pywikibot in Toolforge are at and . [17:05:05] Thanks! I'll read these documents and try to figure out what to do [17:16:24] I was reading https://wikitech.wikimedia.org/wiki/Help:Toolforge/Rules#rule_4 and I think everything is related to my non-migration to Kubernetes. I have not done it because I dodn't understand what files to create to replace my ancient cron. [17:33:31] I was following https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts_(advanced) and got this error: https://paste.toolforge.org/view/c3bfa133 [17:35:39] can someone help me [17:35:54] ? [17:38:39] This is one of the most active bots in Catalan Wikipedia and did some seven or eight jobs that were very important for the community. It stopped three weeks ago and I have not been able to figure out how to get it to work again- [17:38:42] The job just ran for a long time but looks completed now [17:39:16] setup-venv.err would have any errors from the job [17:44:32] Shall I continue with the install? [17:51:46] I think so, yes [17:56:02] 👍 [17:59:42] How can I test if pywikibot is correctly setup? If before I used this command; "python3 /data/project/shared/pywikibot/core/scripts/templatecount.py -count -namespace:0 CC", now it would be something like "toolforge jobs run llista-reds --command "$HOME/pwbvenv/bin/python3 templatecount.py -count -namespace:0 CC" --image python3.11"? [18:01:46] I get an error of pkg_resources [18:02:42] if you’re following https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts_(advanced), I don’t think you should use `/data/project/shared/pywikibot` anymore (if I understand correctly) [18:03:42] oh wait, sorry, you wrote that was the old command [18:05:01] where do you get this error? `llista-reds.out` and `llista-reds.err` look like that specific job ran successfully [18:06:37] Maybe checking the talk page of the Wikitech Kubernetes migrate page might help you? Some months ago I migrated my bot and in that talk page is every step and experiment that was tried until it started to work again in Kubernetes. [18:07:29] For a TL;DR you can check how the script was initially and how it changed in the end to make it work on Kubernetes and try to replicate the same thing to your script as well. [18:09:47] https://wikitech.wikimedia.org/wiki/Help_talk:Toolforge/Jobs_framework#Help_moving_pywikibot_jobs_including_user-fixes.py_customization [18:09:49] [18:09:50] This is the link to that discussion. If it can be of any help. [18:11:00] these command has to show the template count in the bash. It was doing that but the way that I now run the job now, it does not dispaly the result (re @lucaswerkmeister: where do you get this error? llista-reds.out and llista-reds.err look like that specific job ran successfully) [18:11:21] Maybe the problem is with pkg_resources (whatever it is) [18:12:51] Thanks! (re @Klein: https://wikitech.wikimedia.org/wiki/Help_talk:Toolforge/Jobs_framework#Help_moving_pywikibot_jobs_including_user-fixes.py_custom...) [18:13:41] I see template counts in the file `llista-reds.out` (e.g. CC has 25 transclusions, apparently) (re @Pau: these command has to show the template count in the bash. It was doing that but the way that I now run the job now, it does not ...) [18:14:15] it's correct, but why it does not display it directly? [18:18:51] So, it seems I have pywikibot up and running with a strange behaviour (output goes to some out file) and some pkg_resources error. Is that correct? [18:23:52] the output going to a file would already have happened on the grid engine if you used the job system there (jstart / jsub) [18:25:07] So I was doing it wrong without jstart / jsub? [18:25:47] If it was, It was not on purpose, it was because of my evident lack of skills [18:37:20] Normally the output and the errors go on two different files, not on the bash itself. If you excuted the scripts directly there it brought problems with resources usage if I'm not wrong. And the execution time was usually slower (maybe again because of the same reason). [18:37:54] ok [18:38:06] thanks for letting me know [18:38:19] I was not aware of that [18:43:16] I'll try to follow the instructions to be able to plan all the jobs that were on my cron. If I'm not able to do it, I'll come to bother you again 😉 [18:43:26] Thanks, guys! [20:01:22] You are welcome! (re @Pau: Thanks, guys!)