[01:59:31] Is there a way to debug this further? It's a normal Django migrate command, and it worked beautifully until today: [01:59:32] tools.como-backend@tools-bastion-13:~$ toolforge jobs run --wait --image tool-como-backend/tool-como-backend:latest --command "migrate" migratejob [01:59:34] ERROR: job 'migratejob' failed: [01:59:35] +---------------+------------------------------------------------------------------+ [01:59:37] | Job name: | migratejob | [01:59:38] +---------------+------------------------------------------------------------------+ [01:59:40] | Command: | launcher migrate | [01:59:41] +---------------+------------------------------------------------------------------+ [01:59:43] | Job type: | one-off | [01:59:44] +---------------+------------------------------------------------------------------+ [01:59:46] | Image: | tool-como-backend/tool-como-backend:latest | [01:59:47] +---------------+------------------------------------------------------------------+ [01:59:49] | File log: | no | [01:59:50] +---------------+------------------------------------------------------------------+ [01:59:52] | Output log: | | [01:59:53] +---------------+------------------------------------------------------------------+ [01:59:55] | Error log: | | [01:59:56] +---------------+------------------------------------------------------------------+ [01:59:58] | Emails: | none | [01:59:59] +---------------+------------------------------------------------------------------+ [02:00:01] | Resources: | default | [02:00:02] +---------------+------------------------------------------------------------------+ [02:00:04] | Mounts: | none | [02:00:05] +---------------+------------------------------------------------------------------+ [02:00:07] | Retry: | no | [02:00:08] +---------------+------------------------------------------------------------------+ [02:00:10] | Health check: | none | [02:00:11] +---------------+------------------------------------------------------------------+ [02:00:13] | Status: | Failed | [02:00:14] +---------------+------------------------------------------------------------------+ [02:00:16] | Hints: | Last run at 2024-05-20T01:58:50Z. Pod in 'Failed' phase. State | [07:37:34] !log urbanecm@tools-bastion-13 tools.sal webservice restart [07:37:37] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.sal/SAL [10:34:50] ah, the logs are only available for the brief moment the job runs! That solved it :D (re @kristbaum: Is there a way to debug this further? It's a normal Django migrate command, and it worked beautifully until today: [10:34:50] tools.como-ba...) [13:32:10] !log lucaswerkmeister@tools-bastion-13 tools.wd-image-positions deployed 2965e37ca2 (l10n updates: fr, ia, it, ms, sr-ec, zh-hans, zh-hant) [13:32:13] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wd-image-positions/SAL [13:34:13] !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed 4d952df88b (l10n updates: ms) [13:34:14] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL [14:03:44] Hi, is there someone fluent in Flask here ? [14:03:45] I have a question about creating new files at runtime and then exposing them as "static" files - even though the fact that they may be created or deleted at any time doesnt make them *that* static. [14:06:29] I'm in the process of converting my tool to Flask (instead of PHP), and it currently writes SVG files in the public_html/svgfiles/ directory which are then accessible directly provided you know the file name. [14:06:30] But if all the requests are handled by Flask, then I guess it won't simply allow arbitrary paths on the server to be accessed through. [14:06:30] So, does that mean I need to write everything in flask's static/ directory ? [14:22:58] that sounds like the easiest solution to me [14:23:05] though it’s a bit unconventional to have those files dynamic [14:23:28] and I’m not sure if it would work in a build pack webservice (I don’t know if the source directory is read-write or read-only in that case) [14:24:00] btw you could also configure a different route, and put the files elsewhere, by using `flask.send_file()` manually (I assume) [14:24:01] https://flask.palletsprojects.com/en/2.3.x/api/#flask.send_file [14:24:25] or you could use https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Serving_static_files [14:36:08] ok, one thing is that there are two different static directories : the flask tutorials I found lead me to `(home)/www/python/src/static/`, whereas that last toolforge link specifies `(home)/www/static` [14:36:16] ok, one thing is that there are two different static directories : the flask tutorials I found lead me to `(home)/www/python/src/static/`, whereas that last toolforge link specifies `(home)/www/static/` [14:36:24] f, you can't edit a message [14:36:27] anyway [14:38:38] is that something I should specify in the unwsgi.ini file, that it should use the former ? [14:38:47] as the static directory [14:38:52] no, the link taavi posted is unrelated to Flask [14:39:13] it’s a way to serve static files regardless of which webservice type you use (you can even use it without any webservice) [14:39:29] since it’s served from tools-static.wmflabs.org rather than toolname.toolforge.org [14:40:05] aaaah so it would change the domain of the url leading to them [14:40:40] that seems unnecessary in my case, though it's an acceptable fallback solution [14:42:38] The send_file solution doesn't seem very appropriate for my use case since I would like these files to be integrated in my pages. Or are you saying that I need to put that send_file in a `@app.route("/svgfiles/")` function ? [14:43:46] I guess my question is, if I put files in `(home)/www/python/src/static/`, will flask allow direct links to mytool.toolforge.com/static/, or do I need to tell it to ? [15:14:55] “Or are you saying that” – that’s what I meant, yeah, essentially just building your own little /static/ handler [15:15:35] “I guess my question is” – if you do that you should be able to link to the files using `url_for('static', filename='foobar')` [15:20:35] Gouvernathor: flask automatically makes a `/static/` route to serve files from it's configured static directory. Where that directory appears on disk depends on how your app is built and deployed. The opinionated config for `webservice` and the shared python images would make the default location `(home)/www/python/src/static/`. [15:20:37] For a build service app things are a bit more complicated as $HOME is not a persistent volume, so you would need to use `--mount all` and configure flask to look somewhere under $TOOL_DATA_DIR. https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Using_NFS_shared_storage [15:34:40] andrewbogott: o/ [15:35:01] Is there a task to upgrade deployment-prep to Bullseye+? [15:35:37] I can take some VMs like the kafka ones, but I'd ask if it is possible to dist-upgrade them in place and then fix the VM's metadata (like reported OS etc..) [15:35:46] if so it would be way easier to upgrade [15:35:53] re-creating the VMs will take a while [15:38:22] Is there a way to request access to a tool if the maintainer is inactive ? [15:38:55] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Abandoned_tool_policy [15:41:06] How relevant is p.3 ? [15:41:27] elukey: T327742 and subtasks - T361382 is kafka [15:41:27] T327742: Migrate deployment-prep away from Debian Buster to Bullseye/Bookworm - https://phabricator.wikimedia.org/T327742 [15:41:28] T361382: Replace deployment-prep kafka hosts with Bullseye or Bookworm - https://phabricator.wikimedia.org/T361382 [15:41:53] The tool I'm trying to look at has only 1 maintainer (re @sohom_datta: How relevant is p.3 ?) [15:42:12] I wonder if it is going to be easier to just fork [15:42:39] (For context, this is the indic-ocr tool) [15:47:21] JJMC89: thanks! [15:58:40] Any web interface (like Quarry) that I can use for querying my user databases? [15:59:47] I don't think so, `sql tools` on the ssh session works tho (if you have access to the CLI) (re @Yetkin: Any web interface (like Quarry) that I can use for querying my user databases?) [16:00:33] sql tools? [16:03:02] Running `sql tools` in the command line will give you access to the tool/user databases [16:04:07] oh, I know that. What I need is a simple GUI [16:05:07] There was a task to add access to public tool databases to quarry, not sure what it's status is [16:06:02] [[phab:T348407]] [16:07:17] I believe that was T348407 which was deployed a few hours ago [16:07:18] T348407: Create db user for Quarry with readonly access to public ToolsDB databases - https://phabricator.wikimedia.org/T348407 [16:09:56] and here I was about to write that it probably doesn’t exist for good reason… [16:10:52] so at what point are tool developers going to be told that their _p databases are now public to the whole world and not just other tool developers? IMHO that’s a significant difference [16:10:58] and I don’t think I’ve heard about this task via cloud-announce yet [16:14:20] lucaswerkmeister: good question, there was some discussion in the task, but maybe we should discuss it more [16:14:44] the reason it has not been announced yet is because it was literally deployed today [16:15:02] I haven't even tested if it works :) [16:16:38] yes, but the announcement I would expect is “here’s what’s going to happen, if it’s not okay for you then start moving your data away now”, in advance of any deployment [16:16:54] not “we did this and I sure hope it was okay for all your data” [16:17:36] Initial tests seem negative https://quarry.wmcloud.org/query/82992 (it's a public DB for my wsstats tool) (re @wmtelegram_bot: I haven't even tested if it works :)) [16:19:06] lucaswerkmeister: I see your point, and I agree. I will send an announcement and disable the user grants for now. [16:20:36] (FYI, my thinking here is influenced by T224725, which would’ve been a much worse vulnerability if Quarry had had ToolsDB access at the time – though in that case the readability to other tool maintainers wasn’t okay either, so it’s not a perfect comparison) [16:21:00] (unfortunately the task, though long resolved, is PermanentlyPrivate – I can DM you a summary if you don’t have access) [16:21:49] I do have access thanks [16:23:14] What kind of data would be okay to be readable to 3p maintainers and not general users ? [16:25:26] Also, https://quarry.wmcloud.org/query/82992 appears to have borken quarry (I can't stop the query) [16:26:24] lucaswerkmeister: would you mind adding your concerns to the Phab task? (T348407) [16:26:27] T348407: Create db user for Quarry with readonly access to public ToolsDB databases - https://phabricator.wikimedia.org/T348407 [16:27:26] Thanks wm-bb bd808_, I was AFK but I read. [16:27:26] Thanks for the confirmation about where to put my files on the toolforge server ^^ [16:27:27] There's something I don't understand. As bd808 said, flask automatically does a static directory handler. If that's the case, why would I need to build my own handler to do the exact same thing ? ? [16:27:35] dhinus: can do [16:28:17] Sohom_Datta: why your query it's not working, I'm not sure. how does Quarry detect when it should connect to toolsdb vs. wikireplicas? probably using "use dbname" is not enough. [16:29:00] Gouvernathor: you don’t need to, really; I mentioned it because it sounded like you were ambivalent about the static/ directory and/or URL path, and this approach would let you customize the handler if you wanted [16:29:08] but maybe I misunderstood you. the built-in handler is probably good enough :) [16:29:12] oh great [16:29:22] That's a good point, but it should probably have errored out in that case, not gotten stuck :( [16:29:27] Rook: do you know how to select toolsdb in quarry now that the credentials are available? [16:29:41] well the NFS solution is interesting too, even though it's a little more complex [16:30:00] but I think I'll stick to the simpler version [16:30:19] dhinus: I do not know how, no [16:30:52] Time to look at Quarry code and file a few bugs (probably) [16:31:03] :P [16:32:09] I will restrict access to a single _p database for now, while we discuss the privacy implications [16:32:29] I'm hopeful we can enable this feature, but I agree we should at least announce it in advance [16:32:58] I have yet another question wm-bb if you know the answer : what will be the current working dorectory for my python ? in other words, considering that my file is at `$HOME/www/python/src/app.py`, if I do `os.listdir("svgfiles")`, will it look for `$HOME/svgfiles` or `$HOME/www/python/src/svgfiles` ? [16:33:41] Sohom_Datta: I will enable the quarry user to access only your db s55771__wsstats_p so you can use that for testing [16:33:54] Sounds good :) [16:41:44] Gouvernathor: if you’re using a normal webservice, I think the working directory is ~/www/python/src/ [16:41:51] (in a build pack it’s different) [16:42:02] ok, thanks ! [16:42:16] but I usually have my tool load their config from `config.yaml` and that seems to resolve to the config.yaml in the src directory [18:04:58] @lucaswerkmeister, dhinus: There is no harm in making folks aware of the change to Quarry, but I also think that the "everyone" in "Database names ending with _p are granted read access for everyone." means everyone and the lack of a stable web UI for this data has been a bug not a feature. [18:14:06] !log bd808@tools-bastion-13 tools.versions Update to 66607ec [18:14:08] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.versions/SAL [20:13:12] Is there a plan to support python 3.12 instead of just 3.11 ? [20:14:49] not quite… the webservice python versions are determined by which versions debian distributes [20:15:09] so next summer, there will (probably) be a new debian release, which will (probably) contain python 3.13, and at that point a python3.13 webservice type will be added [20:15:18] but there probably won’t be a python3.12 type in between [20:15:58] that's fine, 3.13 is good too [20:16:09] however, according to https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service you can already use python 3.12 in the build service (but I haven’t tried that) [20:16:29] yes, I'm kind of trying to avoid doing that [20:17:26] we talked about it before, it sounds way more tedious than just running an existing webservice kind [20:17:39] I didn't know 3.13 was released tho [20:17:59] I don’t think it has yet [20:18:18] nah, it won't until october [20:18:23] https://peps.python.org/pep-0719/ [20:18:25] but as far as I understand the debian and python release schedules, they’re synced up enough that each debian release is pretty likely to bump the python minor version by 2 [20:18:40] so we’ve had 3.7, 3.9, 3.11, and the next one will probably be 3.13 [20:19:03] sometime in summer 2025 (probably) [21:41:50] !log bsadowski1@tools-bastion-13 tools.stewardbots Restarted StewardBot/SULWatcher because of a connection loss [21:41:52] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL [23:56:48] [Question] How to go from the URL of an article in any Wikipedia language edition to the Wikidata item?