[03:01:28] !log admin restarting mariadb on cloudcontrol1005, hoping to get Galera back in sync [03:01:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [06:58:04] Can anyone help with this? several people are facing this error: 'MaybeEncodingError' [07:07:22] you should provide more details (re @Sufi: Can anyone help with this? several people are facing this error: 'MaybeEncodingError') [07:09:58] https://www.mediawiki.org/wiki/How_to_report_a_bug [07:10:24] and https://wikitech-static.wikimedia.org/wiki/Reporting_a_connectivity_issue [07:11:35] who are several and what are they doing to get to this message? [07:17:07] Thanks, I will report it there (re @jeremy_b: https://www.mediawiki.org/wiki/How_to_report_a_bug) [07:20:22] I did not understand the meaning of the latter part of this message. Two of the people facing this issue are participants of 'Indic Oral Culture Project (https://meta.wikimedia.org/wiki/Grants:Project/MSIG/Indic_Oral_Culture_Project)', they are trying to upload videos they have created a part of the project. Two others who are not participants have mentioned this on this discussi [07:20:23] on page (https://commons.wikimedia.org/wiki/Commons_talk:Video2commons#MaybeEncodingError) (re @jeremy_b: who are several and what are they doing to get to this message?) [09:53:45] Hey everyone! I'm still on my adventure trying to migrate my tasks in Kubernetes. [09:53:45] [09:53:47] I'm using this command: [09:53:48] [09:53:50] toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb replace -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Gjuhë -fix:Data -always" [09:53:51] [09:53:53] But I get: [09:53:54] [09:53:56] family and mylang are not set. [09:53:57] Defaulting to family='wikipedia' and mylang='test' [09:53:59] ERROR: Incomplete command line pattern replacement pair. [09:54:00] [09:54:02] Which tells me my user-config.py and user-fixes.py files are not being considered at all when the job is running. I'm stuck at this point currently. :/ [15:51:43] @Klein: The tool-pywikibot/pywikibot-scripts-stable:latest image does not mount and use files like user-config.py from your tool's $HOME. Instead it expects things like `-family:...` and `-lang:...` to be passed as arguments in the `--command` option to `toolforge jobs run`. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts for slightly more information. [15:54:15] If you need deeper customization, one possibility is forking https://gitlab.wikimedia.org/toolforge-repos/pywikibot-buildservice and adding your own customizations. I had to do this for a bot that I operate because its target wiki does not support OAuth authentication (). [16:30:50] @bd808, I was following those suggestions (given by Taavi - thankful) when I reached the dead-end. I tried adding those family and language arguments (the error message should be changed to "lang" instead of "mylang" BTW) but I strangely get the same error message. I will try it again. Even if it works though, how do I manage to incorporate my fix files in the command if that ima [16:30:51] ge doesn't use my files at all? No way at all to do that currently? [16:34:48] you should say these things immediately, don't wait for me to ask. also sometimes screenshot isn't my favorite solution but sometimes a screenshot of the error can be helpful. (re @Sufi: I did not understand the meaning of the latter part of this message. Two of the people facing this issue are participants of 'In...) [16:35:00] @Klein: At the moment there is no way to customize the code or config used by tool-pywikibot/pywikibot-scripts-stable:latest. The alternatives are to make your own custom build service image or to setup and run your scripts from a venv virtual environment maintained in your tool's $HOME using one of the per-built python images. [16:35:27] *pre-built python images [16:36:05] We probably could use an "advanced tutorial" for doing that actually. [16:39:14] apparently from your last link this is the bug. https://github.com/toolforge/video2commons/issues/176 (re @Sufi: I did not understand the meaning of the latter part of this message. Two of the people facing this issue are participants of 'In...) [16:40:06] There used to be a pywikibot one for the venv method, but it was removed in favor of using the image or following the generic python venv one. [16:48:07] Hmm... I tried some days ago reading the instructions in Wikitech about creating venvs but I was stuck when it was asking me to put what packages I wanted to install and I didn't know what to choose. I assume creating a custom image is even harder. Should I wait for a sort of guide? Or maybe someone can give me some information to read meanwhile and try to make my way in those mu [16:48:08] rky waters? :P I'd take it slower but the deprecation deadline for the grid is close so... [16:51:58] !log admin puppet node deactivate cloudvirt1063.eqiad.wmnet T353406 [16:52:04] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:52:04] T353406: NodeDown cloudvirt1063 - https://phabricator.wikimedia.org/T353406 [17:18:43] Would T353406 explain why creating a new instance is stuck in the "Scheduling" state? [17:18:44] T353406: NodeDown cloudvirt1063 - https://phabricator.wikimedia.org/T353406 [17:19:18] (for 8 minutes so far) [17:19:38] dancy: possibly, but dhinus or andrewbogott might be able to tell you more with some details about the instance. [17:20:13] dancy: likely not related but can you delete and try again? [17:20:21] Sure [17:21:38] Deleted and recreated.. now in scheduling state. [17:21:43] https://horizon.wikimedia.org/project/instances/8544c8f0-7b93-4903-abcb-51d2c474469f/ [17:21:58] JJMC89: hmmmm... yeah. Where was the prior information that you and others had worked on for that? The "If you want to run a script that is not included with Pywikibot itself, follow the generic Python on Toolforge documentation.' on taavi's new page leaves a lot to the imagination. [17:22:56] https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Pywikibot [17:25:14] dancy: fyi, horizon links depend on your session data so don't work for anyone but you. Did the VM schedule and start this time? [17:25:29] Still in scheduling state. [17:25:57] It's gitlab-runner-1003 in the devtools project. [17:26:26] JJMC89: Thanks. I think we should bring that content back to the Help namespace. I can work on that later today if nobody beats me to it. I do think it should be done in a way that makes it clear that if you only want to run a default script the tool-pywikibot/pywikibot-scripts-stable:latest image should be the simplest path to success. [17:28:04] andrewbogott: In the project overview, volume storage is showing as maxed out. I'm not sure if that is the blocker (I didn't check what it looked like before attempting to create the instance). [17:29:43] dancy: ok If I confirm the resize for zuul-1001? Does that server work properly? [17:30:02] @Klein: https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Pywikibot is probably the best documentation we have right now on how to do custom things with pywikibot. [17:30:08] andrewbogott: Go for it [17:30:18] I thought I had done that myself. [17:30:31] but apparently not! [17:30:31] @Klein: I would also say that it is ok to wait a bit longer for better help if 1) you make a note on your tool's grid shutdown task about what you are stuck waiting on help with and 2) there is some related phab task about fixing the thing that has you stuck. [17:32:52] dancy: hm, I've now scheduled three different VMs in that project and they all launched immediately. So yours is cursed somehow, can you try again with a different name, for science? [17:33:07] sure [17:34:24] ...and that works just fine [17:34:29] Very interesting! [17:34:50] I have no theory. Move on to 1004 I guess if that's an OK workaround [17:35:01] haha ok I'll give that a shot [17:36:01] gitlab-runner-1004 working fine. Weird! [17:37:34] I notice that when I created gitlab-runner-1003, there was already puppet configuration for it. Not the case with gitlab-runner-1004. [17:39:37] you weren't getting as far as puppet but that suggests that a previous host existed with that name. Things should get cleaned up properly but maybe somehow didn't. [17:39:52] nod.. that's the impression I get. [17:43:12] and I found some evidence of a prior `gitlab-runner-1003` in `puppetmaster-1001.devtools` [18:10:40] !log wm-bot full restart of bot + xmlrcs following disconnection [18:10:43] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Wm-bot/SAL [20:53:13] andrewbogott: I have my new gitlab-runner-1005 going and I'm running some jobs through it and I'm finding it bumping up against the write iops limit of 500. Should I file a phab ticket to request a higher limit on this node and gitlab-runner-1002.devtools? 5000 iops would be nice (and infrequently used at that level). [21:04:08] Sufi `MaybeEncodingError` is an error indicating video2commons failed to encode the error message properly. It requires another (hopefully legitamate) error. Common causes are filenames rejected by commons or duplicate files. cc @jeremy_b [21:06:07] Unfortunately some library upgrade I've done in the past 6 months broke the exception import to properly encode error messages in video2commons. I have not been able to fix it thus far, my current plan is to rebuild the deployment system, improve observability, and hopefullly at some point the issue will become obvious. [21:51:39] Hi everyone, I haven't been able to find instructions on how to upload a forked pywikibot script to my tool. Do you know where I can find one? [22:08:56] @A: using custom pywikibot scripts requires that you do something like https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Pywikibot to run pywikibot from your own venv. [22:09:28] * bd808 will work on moving that page back to the Help namespace now [22:16:03] When I create the pwb_venv.sh on my computer and use touch $Home/pwb_venv.sh, I keep getting cannot touch pwb_venv.sh No such file or directory. Where am I supposed to create this file exactly ? [22:17:01] @A: in the $HOME of your tool when ssh'ed into login.toolforge.org or dev.toolforge.org. [22:18:22] okay I need an instruction page to do that. How can I do this? [22:18:49] I don't think that is explained anywhere [22:18:54] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Quickstart#Connect_to_Toolforge_servers_using_SSH [22:19:26] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Quickstart#Login_and_%22become%22_your_tool [22:19:51] I have done both of them. Now how to send the file? [22:21:12] If you want to upload a file from your local computer, see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Tool_Accounts#Manage_files_in_Toolforge -- Alternately you can create and edit files directly from your ssh session using one of the locally installed editors like vim, emacs, nano, etc. [22:25:09] I will give it a try. Thank you! I will also try to use emacs I hope this is it https://opensource.com/resources/what-emacs [22:28:13] @A: yes, the normal GNU Emacs program is installed on the bastion servers.