[03:27:17] I followed these instructions and setup a new tool to host a pywikibot: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts [03:27:17] But any script I try to execute - I get an error "No username has been defined in your user config file" - but it looks like it's pulling from an image and so I cannot setup a local user file. I did setup global env variable with the bot's username and login credentials. Any ideas? [03:36:08] Screenshot of exact output https://usercontent.irccloud-cdn.com/file/srhmc7pp/Screenshot%202024-01-24%20at%207.35.43%E2%80%AFPM.png [05:28:34] Looks like solution was doing an advanced install. All set for now. :) [09:13:28] varnent: that should have worked :/, just tested with wikitech and it worked for me, maybe you should pass the lang and family as parameters `-lang:something -family:something` [09:14:38] varnent: the pywikibot config embedded in the image needs to have all the wikis listed manually, maybe foundationwiki is missing from that. I'll have a look [10:26:17] I have an issue with my new app. [10:26:19] I suspect a limit on url length is the cause, can someone tell me what the limit imposed is? [10:26:20] https://github.com/dpriskorn/WikidataTopicCurator/issues/5 [10:31:19] the URL in your screenshot is not very long? [10:33:38] correct, in this case a long quickstatements url is generated and redirected to. [10:33:40] I can provide the url in question if you want. [10:33:49] I see plenty of messages in `uwsgi.log` [10:34:41] it is rather difficult to troubleshoot a problem with a particular URL without knowing that specific URL [10:37:52] you should also use MarkupSafe instead of https://github.com/dpriskorn/WikidataTopicCurator/blob/cbe453f062c07ae089a9849de69932978ef5095b/app.py#L19-L32 [10:37:56] to avoid XSS vulnerabilities like https://topic-curator.toolforge.org/?terms=%3Cscript%3Ealert(%27xss%27)%3C/script%3E [10:39:34] (MarkupSafe comes with Flask, you “just” need to use it) [10:43:52] Thanks for the suggestion 🤩 I'm new to webapps so I very much welcome feedback. [10:45:04] here is a url that results in a 502: https://paste.debian.net/1305243/ [10:45:05] (it has statements for 300 qids) [10:47:27] it's 2167 characters long [10:48:25] If someone here knows how to send QS commands to QuickStatements using POST, that would probably solve the issue without any changes needed on toolforge. [10:51:57] that URL loads just fine for me (https://phabricator.wikimedia.org/P55661), but I don't think it does what you expect as all of the data is in a query string parameter not sent to the server [10:53:09] there's also some API support it seems https://www.wikidata.org/wiki/Help:QuickStatements#Using_the_API_to_start_batches [10:53:43] I'm reading https://quickstatements.toolforge.org/#/user, but I'm unsure if it works without a token. (re @wmtelegram_bot: there's also some API support it seems https://www.wikidata.org/wiki/Help:QuickStatements#Using_the_API_to_start_batches) [10:55:17] dpriskorn: I think you need a token yes (from those docs: 'If you are logged in on QuickStatements, you can get a token on your user page') [10:57:20] I would rather like to avoid that. [10:57:20] I see AuthorDisambiguator uses Oauth and has code for the user to log in, it might be necessary to do the same to make this work well in toolforge. [11:01:14] When I created this app I was hoping to offload all the batch uploading to QS. [11:01:14] I could use WikibaseIntegrator instead, but then I have to learn how to make oauth work 🙈 [11:01:16] I'll look into Lexeme Forms and see if I can copy over what is needed [11:01:17] Thanks for the feedback [12:55:45] Hello [12:55:52] Hello [12:55:57] hello? [12:56:03] hi taavi [12:56:10] whats your name taavi ? [12:56:16] hello [12:56:23] do you have a question related to Wikimedia Cloud Services? [12:56:27] yes [12:56:34] please ask your question [12:56:42] ok i will ask my question [12:57:08] how do i create my VPN Wikimedia cloud virus ant service [12:57:12] im struggling [12:57:22] it would be of great assistants [12:58:25] hello ? [12:58:38] i am very old and need help [13:01:51] Considering winstanley.ac.uk is a 6th form college... [13:06:08] sup niggas [13:21:30] dcaro about the problem with loading the Mediawiki History Dump dataset in a Dask dataframe, I solved it (after a lot of trial and error), Basically the point was finding the right combination of options to pass to `read_csv` that matched the actual dataset so that it would be read correctly. The errors I waS seeing were due to the fact that for [13:21:30] some rows Dask would split things incorrectly. [13:21:55] i will add a working version of the notebook to the Meta page about the dataset, so it is documented [13:22:04] Thanks again for your help. [13:36:01] np, glad you got it working [13:51:26] Is there a way I can replicate toolforge's basic architecture locally ? [13:51:48] (Specifically the kubernetes tool aspect) [13:53:27] sohom_datta: you can use https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo, but not everything works yet (webservice for example does not work, jobs and builds work) [13:53:52] yes, lima-kilo exists but is rather heavy and mostly intended for those hacking on the infrastructure itself [13:53:56] what are you trying to do? [13:53:59] yep [13:54:49] I'm trying to understand/debug why my tool (link-dispenser) works slower with more worker threads on toolforge than locally [13:55:25] I was assuming the answer lay somewhere in Kubernetes land, but I'm not sure [13:57:34] most likely you're hitting the CPU limit, which is by default half a CPU core per webservice replica. you can either increase the CPU limit with -c or the number of replicas with -r (and of course decrease the number of workers per replica) [13:57:52] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Webservice_templates and https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes [14:01:02] Hmm, I don't think it's a CPU limit, https://grafana.wmcloud.org/d/TJuKfnt4z/kubernetes-namespace?orgId=1&var-cluster=prometheus-tools&var-namespace=tool-link-dispenser&from=now-12h&to=now shows very little CPU usage :( [14:02:14] to me that dashboard shows the pods hitting the .5 limit quite often [14:02:59] The default limit is 0.5 ? [14:04:13] as I just said, the default CPU limit allocated to webservices is 0.5 CPU csore per webservice replica, and can be increased with the -c flag. https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Requesting_additional_tool_memory [14:04:15] That might explain the issues, I'll try increasing the limits and seeing [14:13:36] I think that helped a bit, will come back if something crops up again [16:48:31] !log admin taavi just moved all rabbitmq traffic to cloudrabbit1003 as part of T345610 [16:48:41] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL [16:48:42] T345610: cloudrabbit: connect them via cloudsw and cloud-private - https://phabricator.wikimedia.org/T345610