[00:21:12] !log tools restarting toolsdb again as it's again low in free mem T353093 [00:21:16] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [00:21:17] T353093: [toolsdb] MariaDB process is killed by OOM killer (December 2023) - https://phabricator.wikimedia.org/T353093 [05:32:06] !log restarting mariadb on toolsdb-1 because it's just about to go oom (or possibly just did) [05:32:07] andrewbogott: Unknown project "restarting" [05:32:15] !log tools restarting mariadb on toolsdb-1 because it's just about to go oom (or possibly just did) [05:32:17] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [05:33:29] I was 30 seconds too late [06:55:10] Can I use this for a toolforge project [06:55:11] https://pypi.org/project/nitter-scraper/ [06:55:12] [06:55:14] It needs docker engine [07:34:03] gyana111: you cannot use docker engine from toolforge, but you have two options: 1. use the alternative method described (importing the python functions), 2. use cloud-vps instead of toolforge, you can run docker engine in cloud-vps [08:14:19] !log tools restarting toolsdb with jemalloc [08:14:22] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [08:37:35] Hi all, just a question about the docs at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework [08:37:46] What exactly constitutes "failure"? A nonzero exit code? [08:38:00] I feel like this detail could be mentioned somewhere on the doc page [08:52:50] tto: I think that's correct, a non-zero exit code, but I want to double check that before updating the docs [13:21:33] Hi, I have a question regarding toolforge. [13:21:33] I would like to host a big sql or nosql database. [13:21:35] Size >10GB probably. [13:21:36] [13:21:38] I'm planning to make all sentences of all CC0 documents from the Swedish Parliament and government search and addressable. [13:21:39] [13:21:41] @abartov would like them to be searchable for his new Luthor tool [13:21:42] [13:21:44] I thought about uploading them to Wikisource also since all 160k of them are CC0, but I don't know if that's a good idea. [15:04:21] Hi, I'm a bit of a newbie when it comes to Lighttpd config. Could anyone help? I'm working on https://letaxobot.toolforge.org/ and feeling a bit lost with affecting variables like server.username, using mods such mod_redirect or mod_rewrite. https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd and [15:04:22] https://redmine.lighttpd.net/projects/lighttpd/wiki/TutorialConfiguration didnt help me much (pure newbie) :) [15:19:11] !help [15:19:11] If you don't get a response in 15-30 minutes, please create a phabricator task -- https://phabricator.wikimedia.org/maniphest/task/edit/form/1/?projects=wmcs-kanban [15:49:10] LDwp: hi, what's your actual question? [15:50:14] your tool will always run as the tool user, so setting server.username is not really possible [15:51:28] I trust: .*@user/bd808 (2admin), .*@wikimedia/majavah (2admin), .*@wikipedia/Legoktm (2admin), .*@wikimedia/BDavis-WMF (2admin), [15:51:28] @trusted [15:51:40] Hm, so server.X variables are irrelevant? Don't need to link the root from /mnt/ ...? [15:52:47] you will automatically get a default config that sets the correct port, sets the public_html directory as the docroot, etc. you can customize that following https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web/Lighttpd, but in most cases that's not necessary [15:53:01] hm, wm-bot still has my old cloak :/ [15:56:07] Ok, then url.redirect = ( "(.*)" => "/web/index.php" ) & url.rewrite-if-not-file += ( "(.*)" => "/web/index.php" ) are not working properly. While index.php does not exist at $HOME/public_html/ I get a 403 not a redirect towards $HOME/public_html/web/index.php [15:56:53] Is that safe to paste my conf on phab to let you look at it? ^^ [15:59:10] yeah, or I can just look at the tool home itself if you give the tool name [15:59:20] letaxobot [15:59:36] as you wish, help is appreciated :) [16:03:53] Generally, most scripts shouldn't be accessible to users, except for two: public_html/web/index.php and public_html/taxobot.php ; it's mostly designed for being used under cli or web but with one entry point and one rendering point if that makes sense. [16:03:53] Successfully added .*@mediawiki/taavi [16:03:53] @trustadd .*@mediawiki/taavi admin [16:04:25] thanks legoktm :-) [16:04:31] yw :) [16:06:00] LDwp: try removing everything except the custom url.* rules you want to add, the rest comes from the toolforge infrastructure [16:06:56] !help is If you don't get a response in 15-30 minutes, please email the cloud@ mailing list -- https://wikitech.wikimedia.org/wiki/Help:Cloud_Services_communication [16:06:56] This key already exists - remove it, if you want to change it [16:06:59] !help del [16:07:00] Successfully removed help [16:07:01] !help is If you don't get a response in 15-30 minutes, please email the cloud@ mailing list -- https://wikitech.wikimedia.org/wiki/Help:Cloud_Services_communication [16:07:01] Key was added [16:08:15] With that config I get 403 on https://letaxobot.toolforge.org/ no redirect to https://letaxobot.toolforge.org/web/index.php ^^ [16:08:54] for purpose I renamed public_html/index.php to public_html/index2.php since I don't want to handle redirect manually ^^ [16:14:01] which image are you using? php8.2? [16:14:31] yes [16:15:35] that image seems to have an embarassing typo, it's reading "$HOME/lighttpd.conf" and not "$HOME/.lighttpd.conf" which might explain why anything is not working for you :/ [16:17:39] I'll send a patch to fix that, but you may want to add a symlink "$HOME/lighttpd.conf" to "$HOME/.lighttpd.conf" in the mean time for the config file to have any effect [16:22:17] I tried to rename "$HOME/.lighttpd.conf" to "$HOME/lighttpd.conf" and adding a symlink but same results ^^ [16:22:55] did you restart the webservice afterwards? [16:25:24] nope I forgot (told you pure newbie) its almost working: now I get ERR_TOO_MANY_REDIRECTS even with a cookie purge [16:25:54] Pretty sure the regex isnt good then [16:26:40] yep, presumably you need to have it so that if you're on /web/index.php it won't try to redirect there again [16:29:06] Hm can fix the error with rewrite-once but I suppose I need to exclude src [16:33:53] Fine it seems fixed, I just needed url.rewrite-if-not-file [16:34:14] Thanks for your help [16:53:50] hello, can you hear me? it's me again [16:56:17] so, do you know how to manage built images? maybe delete them? cause the latest image built is up-to-date, but the one ran by the buildservice is not [16:57:08] cause, the buildservice is trying to use the start command: "npm install && npm astro check && npm astro build && node ./dist/server/entry.mjs" but this is not the current start command: ""start": "node ./dist/server/entry.mjs" [16:58:28] nvm, found it " delete  Delete a build" [16:59:13] I have two images labelled as "tools-harbor.wmcloud.org/tool-hitaden/tool-hitaden:latest", I think [17:00:08] simple solution: i erased everything, let's build a new image [17:00:55] (btw envvars are indeed passed to the image, it is just the new cloud native buildpack that doesn't use them) [17:01:33] "Ref: toolforge-buildservice Envvars: NPM_CONFIG_PRODUCTION=true" [17:06:15] There is something wrong, the built image is using nodejs 18.13.0 when the engine specified is `"node": "20.x"`. The start command is still an old one... [17:07:23] please tell me the webservice is not using the git repo cloned instead of the image [17:08:58] The Toolforge webservice command starts node.js web servers using convention rather than configuration. These conventions are expected by the Toolforge tooling: A $HOME/www/js/package.json file must exist as part of your tool's application code. [17:09:52] I think it is not a wrong idea to remember in the documentation that this convention takes over built images [17:10:34] Because the default are indeed ignored: `--buildservice-image BUILDSERVICE_IMAGE : Set image to use for buildservice type. Defaults to 'tool-hitaden/tool-hitaden:latest` [17:12:34] Lofhi: which command are you using to start the webservice? [17:12:43] I tried a lot [17:12:44] of things [17:12:53] because I had time to cry... [17:12:59] so I used [17:13:23] `toolforge webservice --backend kubernetes buildservice` [17:13:26] and [17:13:37] `toolforge webservice start --buildservice-image tools-harbor.wmcloud.org/tool-hitaden/tool-hitaden:latest` [17:13:38] and [17:13:46] `toolforge webservice start --buildservice-image tool-hitaden/tool-hitaden:latest` [17:14:22] Also, I have a service.template : `backend: kubernetes ; type: buildservice ; mount: none` [17:15:11] Also, I don't know if `toolforge build list` is just listing build process results, or the current builds available [17:15:29] Cause I had two images under the `tools-harbor.wmcloud.org/tool-hitaden/tool-hitaden:latest` destination [17:16:06] currently deleting /www/js/** [17:16:17] Will try to ran again with the latest built image [17:16:21] (I have only one build now) [17:16:58] the `toolforge webservice --backend kubernetes buildservice` should work correctly. but that tool is currently for whatever reason running a `node18` and not a `buildservice` webservice [17:17:14] yea [17:17:15] because [17:17:16] I think [17:17:31] that having a `/www/js` folder is taking the upper hand [17:18:01] I used this command three days ago : `webservice --backend=kubernetes node18` [17:18:15] When I was testing to run a webservice without containerimages [17:18:45] the existence of `~/www/js` doesn't affect anything here [17:18:57] are you sure? [17:19:08] what is the link again for the pastebin wmf thing [17:19:48] can you try completely shutting down the non-buildservice node18 service down (`toolforge webservice stop`), and then using `toolforge webservice buildservice start` start it up? [17:20:01] I restarted it, didn't try stop [17:20:03] yes, as long as you're using the buildservice type that directory does not affect anything. and you're looking for https://phabricator.wikimedia.org/paste/edit/form/14/ [17:21:51] stop & start worked: `2023-12-16T17:21:06+00:00 [hitaden-6c8f4bd4d4-r55kr] 17:21:06 [@astrojs/node] Server listening on http://127.0.0.1:8000` [17:22:00] but [17:22:03] https://hitaden.toolforge.org/ is 502 [17:22:27] `http://127.0.0.1:8000` seems like a problem, you need to bind on all interfaces and not on localhost [17:22:44] (╯°□°)╯︵ ┻━┻ [17:22:52] Working on it... [17:25:14] taavi what happens if I rebuild an image? The first image is not the latest, right? [17:26:14] the latest build will override the earlier ones [17:26:23] noice [17:29:04] I tried something: [17:29:07] tools.hitaden@tools-sgebastion-10:~/www$ toolforge build delete * [17:29:07] I'm going to delete 1 builds, continue? [y/N]: y [17:29:08] BuildClientError: user hitaden not allowed to act on build js [17:29:08] Please report this issue to the Toolforge admins if it persists: https://w.wiki/6Zuu [17:29:09] Deleted 0 builds [17:30:39] I don't think a star wildcard works there, the shell is picking up the name of the 'js' as a build name. I'll file a task to clarify the error message [17:30:56] sad [17:31:13] would "*" works? [17:31:20] to avoid shell globbing think [17:31:56] also, it is weird that restart has not the same effect than start/stop? dunno? [17:32:07] yes, but you'll likely get the exact same error [17:32:19] there's --all, but that's broken per T353519 [17:32:19] T353519: [builds-cli] delete --all gets 0 builds to delete - https://phabricator.wikimedia.org/T353519 [17:32:55] https://hitaden.toolforge.org/ [17:32:58] it works, yay [17:33:09] \o/ [17:33:11] T353583 [17:33:12] T353583: `build delete` gives a confusing error message on a non-existent build - https://phabricator.wikimedia.org/T353583 [17:33:18] Vue is broken and it is perma loading but we did progress [17:34:13] thanks for help! [17:54:05] is the WMF gitlab reserved for the WMF employees? [17:54:11] it looks like it: https://idm.wikimedia.org/signup/ [17:55:40] no, anyone can have an account [17:56:27] it's the same account system that toolforge uses :-P [17:56:35] ahhh [17:56:53] there are some CI/CD features to autobuild/deploy? [17:57:10] not yet, but planned [18:03:24] oki [18:30:32] Alright, two projects were migrated to Toolforge [18:30:43] One is waiting m2 cache before moving its Java project on Toolforge [18:31:08] I guess we can say that the Build Service/Web Service are enough stable [18:31:29] Devs refusing to try it are just lazy 😈 [19:30:26] alright, enough for this week, have a nice sunday [20:54:30] !log tools Rebuilding all containers to pick up lighttpd config fix and normal package updates (T293552) [20:54:34] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [20:54:35] T293552: Remove Python/webservice-runner from toolforge web containers - https://phabricator.wikimedia.org/T293552