[14:06:34] <wm-bb>	 <Yetkin> My toolforge job fails with the error
[14:06:34] <wm-bb>	 <Yetkin> PHP Fatal error:  Allowed memory size of 1048576000 bytes exhausted (tried to allocate 333447200 bytes) in
[14:06:36] <wm-bb>	 <Yetkin> despite fetching 1Gi when starting the job. This used to work fine until today
[16:49:56] <wm-bb>	 <Yetkin> Any ideas?
[16:52:41] <wm-bb>	 <lucaswerkmeister> I’m not sure what you mean by “despite fetching 1Gi”, that’s the limit it’s reporting
[16:52:58] <wm-bb>	 <lucaswerkmeister> (well, almost. for some reason it’s apparently 1024*1024*1000 rather than 1024*1024*1024 bytes 🤷)
[16:53:09] <wm-bb>	 <lucaswerkmeister> sounds like you might need more memory… or optimize the job somehow
[16:59:51] <wm-bb>	 <Yetkin> I mean, it fails when used like
[16:59:51] <wm-bb>	 <Yetkin> toolforge jobs run job-pageviews --command "php myscript.php" --image php7.4 --mem 1Gi --schedule "0 4 * * *"
[16:59:52] <wm-bb>	 <Yetkin> This was working before... (re @lucaswerkmeister: I’m not sure what you mean by “despite fetching 1Gi”, that’s the limit it’s reporting)
[17:12:03] <wm-bb>	 <jeremy_b> and? so your job is actually using more than it used to? (re @Yetkin: I mean, it fails when used like
[17:12:04] <wm-bb>	 <jeremy_b> toolforge jobs run job-pageviews --command "php myscript.php" --image php7.4 --mem 1Gi --schedu...)
[17:13:37] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed 15261afefb (change config keys to uppercase to work around T374780)
[17:14:44] <lucaswerkmeister>	 oh good, stashbot quit
[17:15:38] <wm-bb>	 <Yetkin> What do you mean? I am saying that the script that used to work fine fails with a memory error, discarding the memory allocated at the beginning (1 Gi) (re @jeremy_b: and? so your job is actually using more than it used to?)
[17:16:09] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed 15261afefb (change config keys to uppercase to work around T374780)
[17:16:14] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:17:39] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.stashbot bin/stashbot.sh restart
[17:17:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stashbot/SAL
[17:19:44] <wm-bb>	 <Yetkin> ? (re @wmtelegram_bot: <lucaswerkmeister> oh good, stashbot quit)
[17:20:41] <wm-bb>	 <lucaswerkmeister> stashbot, the bot responsible for processing !log messages, had quit the channel over on the IRC side; I restarted it (re @Yetkin: ?)
[17:22:44] <wm-bb>	 <Yetkin> Even though I start the shell with
[17:22:45] <wm-bb>	 <Yetkin> webservice shell --mem 2Gi
[17:22:46] <wm-bb>	 <Yetkin> the same mory error occurs
[17:40:16] <lucaswerkmeister>	 !log tools.lexeme-forms python3 -c 'import yaml; print(yaml.safe_dump(yaml.safe_load(open("config.yaml"))["OAUTH"]["CONSUMER_KEY"]))' | toolforge envvars create TOOL_OAUTH__CONSUMER_KEY
[17:40:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:40:35] <lucaswerkmeister>	 !log tools.lexeme-forms python3 -c 'import yaml; print(yaml.safe_dump(yaml.safe_load(open("config.yaml"))["OAUTH"]["CONSUMER_SECRET"]))' | toolforge envvars create TOOL_OAUTH__CONSUMER_SECRET
[17:40:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:41:17] <lucaswerkmeister>	 !log tools.lexeme-forms python3 -c 'import yaml; print(yaml.safe_dump(yaml.safe_load(open("config.yaml"))["SECRET_KEY"]))' | toolforge envvars create TOOL_SECRET_KEY
[17:41:19] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:41:55] <lucaswerkmeister>	 !log tools.lexeme-forms commented out config.yaml, should use envvars instead
[17:41:57] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:42:16] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed f8033bda4a (read config from envvars)
[17:42:18] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[17:43:15] <lucaswerkmeister>	 seems to work, yay
[18:01:06] <lucaswerkmeister>	 lol…
[18:01:18] <lucaswerkmeister>	 I added `mount: none` to my service.template file in preparation to moving it to the buildservice type
[18:01:24] <lucaswerkmeister>	 tools.lexeme-forms@tools-bastion-13:~/www/python/src$ webservice stop && webservice buildservice start
[18:01:24] <lucaswerkmeister>	 This --mount option is only supported for 'buildservice' type
[18:01:24] <lucaswerkmeister>	 Review the arguments passed and try again
[18:01:37] <lucaswerkmeister>	 so I guess I have to remove it again before I can stop the webservice?
[18:02:05] <lucaswerkmeister>	 ok, `webservice stop --mount=all && webservice buildservice start` worked 🤷
[18:04:19] <lucaswerkmeister>	 ok, but then I had to quickly revert to non-buildservice, and the *start* command apparently doesn’t support it at all
[18:04:46] <lucaswerkmeister>	 `webservice python3.11 start --mount=all` and `webservice python3.11 start` both gave “This --mount option is only supported for 'buildservice' type”
[18:04:53] <lucaswerkmeister>	 so I had to hastily hand-edit the file so I could bring my tool back up >.<
[18:09:38] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed 0bf37ac8d8 (tried but failed to migrate to build service [OSError: No username set in the environment], will try again later, for now running in python3.11 again)
[18:09:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[18:11:12] <lucaswerkmeister>	 (the fix for ^ will be https://phabricator.wikimedia.org/T381923#10850835 ftr but I’m already late for something else so that’ll have to wait ^^)
[18:15:34] <taavi>	 lucaswerkmeister: you need to set --mount before `start`, otherwise it interprets that as the command to run
[18:16:37] <lucaswerkmeister>	 o_O in webservice?
[18:18:01] <taavi>	 yeah, not all runtines support passing a command, but some do and the parser has no way of knowing that
[18:29:33] <lucaswerkmeister>	 I didn’t know any did, thanks
[20:47:17] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed e6d259028e (successful migration to buildservice)
[20:47:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[20:49:06] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms mv www www-unused-tool-now-runs-on-buildservice
[20:49:08] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[20:50:29] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms cp www-unused-tool-now-runs-on-buildservice/python/src/service.template .
[20:50:31] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[20:56:36] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed dd03e95876 (update documentation; no-op deployment, just to test the new buildservice procedure and put it in a single command)
[20:56:39] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[21:01:47] <wm-bot>	 !log lucaswerkmeister@tools-bastion-13 tools.lexeme-forms deployed 336dd318ca (upgrade to Python 3.13)
[21:01:50] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.lexeme-forms/SAL
[21:04:52] <lucaswerkmeister>	 (I’m planning to slowly move all my tools to the build service in the hope that push-to-deploy for webservices isn’t far away ^^)
[21:05:20] <lucaswerkmeister>	 (and also, while I’m at it, split the requirements into dev and non-dev everywhere, like I already do in lexeme-forms, for faster builds and smaller images – mypy is big!)