[11:03:42] <wm-bb>	 <jhsoby> seems like wikibugs is down, can someone restart it?
[11:32:11] <wm-bb>	 <jhsoby> and the bridge bot is double posting again 😅
[11:32:23] <wm-bb>	 <jhsoby> (wikibugs is back, thanks to whoever did that)
[12:38:32] <taavi>	 !log tools hard reboot tools-prometheus-6
[12:38:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[12:45:53] <taavi>	 TheresNoTime: is eranbot 'enwiki' job intentionally still running in SGE? your comment seems to indicate it should have ben moved
[12:46:52] <wm-bot>	 !log lucaswerkmeister@tools-sgebastion-10 tools.bridgebot Double IRC messages to other bridges
[12:46:55] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL
[13:12:35] <PotsdamLamb>	 Why am I having an issue running pip as https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia says to run
[13:12:54] <PotsdamLamb>	 I get "-bash: pip: command not found
[13:12:55] <PotsdamLamb>	 "
[13:13:01] <PotsdamLamb>	 "-bash: pip: command not found
[13:13:01] <PotsdamLamb>	 "
[13:14:12] <TheresNoTime>	 taavi: urgh I didn't kill the already running job
[13:15:19] <TheresNoTime>	 On lunch at the moment but I'll kill it when I get back (or feel free to in the meantime)
[13:15:58] <dcaro>	 PotsdamLamb: where are you running pip in? your laptop? it seems it's not installed/available
[13:16:36] <RhinosF1>	 try pip3
[13:23:09] <PotsdamLamb>	 @RhinosF1 tried that no luck, same error;@dcaro no on toolforge trying to set up my instance for a bot. Following the instructions, I get to the first pip command and it doesn't work
[13:24:44] <dcaro>	 PotsdamLamb: you can follow this on toolforge https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts
[13:25:03] <dcaro>	 if you want to run scripts
[13:26:25] <PotsdamLamb>	 @dcaro, so right at the top to get my tokens, it says to use pip
[13:26:26] <PotsdamLamb>	 https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia#Registering_your_bot_with_the_wiki_software
[13:26:52] <PotsdamLamb>	 pip install mwoauth
[13:27:59] <dcaro>	 ah, I see, that step is not needed (pip install + configure.py), you only need the part about registering the bot https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia#Registering_your_bot_with_the_wiki_software
[13:28:53] <dcaro>	 to get the tokens+secrets
[13:29:31] <PotsdamLamb>	 ah ok. The documentation should be updated. I even took forever to realize you have to use python3 instead of python as the manual says since the bot only works with >= 3
[13:58:15] <TheresNoTime>	 (killed the job ta/avi)
[13:59:07] <TheresNoTime>	 can `jobs.yaml` have `mem: 500Mi` (a la `--mem 500Mi`)?
[14:00:01] <taavi>	 I don't remember the exact syntax, but setting memory somehow is possible
[14:07:46] <TheresNoTime>	 https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-cli/-/blob/main/tjf_cli/cli.py?ref_type=heads suggests it is `mem` :D
[15:21:07] <wm-bot>	 !log anticomposite@tools-sgebastion-10 tools.stewardbots SULWatcher/manage.sh restart # SULWatchers disconnected
[15:21:10] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL
[15:46:06] <wm-bot>	 !log samtar@tools-sgebastion-10 tools.copypatrol webservice stop && webservice --mem 1G start
[15:46:09] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.copypatrol/SAL
[19:59:04] <wm-bot>	 !log bd808@tools-sgebastion-11 tools.wikibugs Update to a3bef6e (T128868, T140881, T90339, T281055, T180293)
[19:59:16] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
[21:25:33] <TimStarling>	 I am going to ask for ps to be available in toolforge images
[21:44:02] <bd808>	 TimStarling: In theory there should only be one process in a container. In theory
[21:45:39] <TimStarling>	 in theory all cows are spherical balls of uniform density
[21:47:07] <TimStarling>	 but that is not the world we live in
[21:52:59] <bd808>	 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/docker-images/toollabs-images/+/refs/heads/master/bookworm-sssd/Dockerfile.template would probably be the place to add it. That cascades to all of the newest images.
[21:53:10] <TimStarling>	 yeah done already
[21:53:26] <TimStarling>	 https://gerrit.wikimedia.org/r/c/operations/docker-images/toollabs-images/+/1010690
[22:14:20] <TimStarling>	 how do you feel about kubectl debug? I tried it but it is not allowed
[22:18:25] <bd808>	 I've never used it or thought about using it
[22:19:40] <TimStarling>	 it https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/
[22:22:09] <wm-bb>	 <lucaswerkmeister> would that be a way to keep debug utilities out of the main images (by putting them in a separate image for ephemeral containers instead)?
[22:22:38] <bd808>	 I have heard of ephemeral containers. serviceops blocked making a maintenance env for Toolhub on the idea that someday the main cluster will support ephemeral-containers/
[22:25:00] <TimStarling>	 lucaswerkmeister: yes, although I think procps is pretty harmless to install in the base image, and it's fine to provide multiple ways to do things
[22:26:25] <wm-bb>	 <lucaswerkmeister> ok, fair ^^
[22:27:43] <TimStarling>	 I haven't actually used it, I'm just reading the manual, but I gather the ephemeral container will share a process namespace (so you can run ps) but not a root filesystem (so you can't inspect the state of temporary files)
[22:29:26] <TimStarling>	 there are some hints in the manual that ephemeral containers are not as convenient as kubectl exec, but allow you to harden the production image
[22:29:39] <TimStarling>	 your main image can be distroless, without a shell
[22:32:24] <wm-bb>	 <lucaswerkmeister> to me it sounds similar to systemd-sysext, which also has “I’d like to have debug / dev tools now” as one of its use cases