|
2024-03-18 08:30:01
|
<wm-bot>
|
!log melos@tools-sgebastion-10 tools.itwiki deletionbot restarted
|
|
2024-03-18 08:30:04
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.itwiki/SAL
|
|
2024-03-18 08:50:34
|
<Sascha62>
|
Hi all, I'm trying to migrate my tool to be built with Toolforge Build Service. My tool consists of two binaries: a webserver, and a cronjob. Can I use the same buildpack (and the same git repository) for both? If so, how should the Procfile look like, and what toolforge command should I use to start webserver and cronjob? The examples in the
|
|
2024-03-18 08:50:34
|
<Sascha62>
|
documentation only seem to cover building a single binary. Sorry for my newbie question.
|
|
2024-03-18 08:56:13
|
<wm-bot>
|
!log melos@tools-sgebastion-10 tools.stewardbots ./stewardbots/StewardBot/manage.sh restart # RC reader not reading RC
|
|
2024-03-18 08:56:17
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.stewardbots/SAL
|
|
2024-03-18 09:13:30
|
<taavi>
|
Sascha79: yes, that's possible. you should add both commands to the procfile with different names (usually `web` and then something else). the `buildservice` web service type will always start the with the first command listed in the procfile, and for the cronjob the --command argument given to the jobs framework will select the command name in the
|
|
2024-03-18 09:13:30
|
<taavi>
|
Procfile
|
|
2024-03-18 09:16:51
|
<Sascha79>
|
Thank you, that's super helpful!
|
|
2024-03-18 09:17:24
|
<dcaro>
|
Sascha79: note though that depending on the language you use you might not be able to build the two binaries without some effort (ex. dotnet)
|
|
2024-03-18 09:27:31
|
<taavi>
|
!log tools deleted shutdown grid engine VMs T314664
|
|
2024-03-18 09:27:37
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
|
|
2024-03-18 09:27:38
|
<stashbot>
|
T314664: [infra] Decommission the Grid Engine infrastructure - https://phabricator.wikimedia.org/T314664
|
|
2024-03-18 10:14:08
|
<wm-bot>
|
!log taavi@tools-bastion-12 tools.wikibugs toolforge jobs restart irc
|
|
2024-03-18 10:14:11
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs/SAL
|
|
2024-03-18 10:31:48
|
<Sascha79>
|
Another build service question: How to run unit tests during the build? I've noticed that "toolforge build start https://github.com/brawer/wikidata-qrank"; will happily build an image even when the head revision of my git repo is breaking its unit tests. This is not a huge issue right now, but once Toolfoge has fully automated push-to-deploy, I'd
|
|
2024-03-18 10:31:48
|
<Sascha79>
|
obviously not want to not deploy releases with breaking unit tests.
|
|
2024-03-18 10:41:50
|
<wm-bot>
|
!log lucaswerkmeister@tools-sgebastion-10 tools.bridgebot Double IRC messages to other bridges
|
|
2024-03-18 10:41:53
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.bridgebot/SAL
|
|
2024-03-18 10:44:00
|
<arturo>
|
!status x
|
|
2024-03-18 10:45:25
|
<arturo>
|
there used to be a bot that could refresh the `Status:` in the topic of this channel?
|
|
2024-03-18 10:48:07
|
<wm-bb>
|
<lucaswerkmeister> Sascha79: someone else might correct me, but I suspect that’s not possible yet – as I understand it, buildpacks are intentionally limited about what you can do at build time, so I wouldn’t expect “run arbitrary test commands” to be supported
|
|
2024-03-18 10:48:26
|
<wm-bb>
|
<lucaswerkmeister> and if “run *specific, well-known* test commands (e.g. npm test)” was supported, you probably would’ve noticed it already
|
|
2024-03-18 10:48:52
|
<Sascha79>
|
"go test ./..." (the standard command for golang) would be enough
|
|
2024-03-18 10:48:57
|
<wm-bb>
|
<lucaswerkmeister> I agree it should be part of push-to-deploy, but I’d expect it to happen outside the build – that GitLab runs tests first and then builds and deploys a new image if they succeed
|
|
2024-03-18 10:49:06
|
<wm-bb>
|
<lucaswerkmeister> probably worth a phabricator task in any case
|
|
2024-03-18 10:49:25
|
<wm-bb>
|
<lucaswerkmeister> *looks if there already is one*
|
|
2024-03-18 10:49:35
|
<taavi>
|
i would agree that's not really in-scope for the build service.
|
|
2024-03-18 10:54:16
|
<wm-bot>
|
!log melos@tools-sgebastion-10 tools.itwiki toolforge-jobs restart itwiki-orphanizerbot
|
|
2024-03-18 10:54:18
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.itwiki/SAL
|
|
2024-03-18 11:00:23
|
<wm-bb>
|
<lucaswerkmeister> I left a comment in https://phabricator.wikimedia.org/T334587 that hopefully makes sense
|
|
2024-03-18 11:05:24
|
<Sascha79>
|
I filed a feature request in https://phabricator.wikimedia.org/T360295
|
|
2024-03-18 11:08:56
|
<wm-bb>
|
<lucaswerkmeister> 👍
|
|
2024-03-18 11:12:28
|
<dhinus>
|
arturo: I do find some examples of using "!status" in the logs for this channel, not sure why it didn't work this time
|
|
2024-03-18 11:13:39
|
<arturo>
|
🤷
|
|
2024-03-18 11:23:42
|
<taavi>
|
!log tools point tools-static proxy to tools-static-15 (bookworm) T311913
|
|
2024-03-18 11:23:45
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
|
|
2024-03-18 11:23:46
|
<stashbot>
|
T311913: Upgrade Toolforge static server (tools-static.wmflabs.org) to Debian Bullseye - https://phabricator.wikimedia.org/T311913
|
|
2024-03-18 13:13:18
|
<taavi>
|
!log tools restart harbor services after docker service restart
|
|
2024-03-18 13:13:22
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
|
|
2024-03-18 14:46:15
|
<bd808>
|
!status Ok
|
|
2024-03-18 14:48:01
|
<bd808>
|
arturo: wmopbot is the bot that does the !status trick. I don't actually remember what authn/authz it uses to decide who to listen to.
|
|
2024-03-18 14:50:42
|
<bd808>
|
Looks likely that it is tied to the channel permissions. We can probably do some tuning here. bstorm and balloons both have rights that arturo does not have.
|
|
2024-03-18 15:05:42
|
<bd808>
|
arturo & dhinus: you both have more irc superpowers in this channel now. I believe that you should both be able to use the "!status MESSAGE" feature of wmopbot as a result (as well as moderate the channel)
|
|
2024-03-18 15:07:01
|
<dhinus>
|
thanks bd808 :)
|
|
2024-03-18 15:13:56
|
<dcaro>
|
!status ok
|
|
2024-03-18 15:14:01
|
<dcaro>
|
I was already :)
|
|
2024-03-18 15:50:34
|
<balloons>
|
If you need me to give permissions for something let me know. I'm pretty sure I've been powers are spread around though
|
|
2024-03-18 16:04:10
|
<arturo>
|
!status ok
|
|
2024-03-18 16:05:24
|
<arturo>
|
!status ok
|
|
2024-03-18 16:05:58
|
<arturo>
|
thanks bd808, but I don't think it was enough
|
|
2024-03-18 16:15:56
|
<danilo>
|
the !status command is allowed to any user with wikimedia cloak, in any channel where wmopbot has +t flag in channel access list and the channel topic has "| [Ss]tatus: ... |"
|
|
2024-03-18 16:22:22
|
<dcaro>
|
ohh, maybe arturo lost the cloak
|
|
2024-03-18 16:23:32
|
<arturo>
|
I never had a wikimedia cloak
|
|
2024-03-18 16:24:17
|
<JJMC89>
|
if you want one: https://meta.wikimedia.org/wiki/IRC/Cloaks#Obtaining_a_cloak
|
|
2024-03-18 16:37:17
|
<bd808>
|
danilo: Thanks for the authoritative answer. :) Is there somewhere I should have been able to find that documented already?
|
|
2024-03-18 17:37:59
|
<danilo>
|
bd808: there is a documentation in https://wmopbot.toolforge.org/help (restricted to who have +o in #wikimedia-ops)
|
|
2024-03-18 17:47:39
|
<bd808>
|
danilo: all it says on that page about !status is "In channels where wmopbot has +t flag this command will change the status sction in the channel topic."
|
|
2024-03-18 17:49:40
|
<danilo>
|
I added now that any user with Wikimedia cloak can use the command
|
|
2024-03-18 17:51:30
|
<bd808>
|
thanks!
|
|
2024-03-18 18:45:54
|
<kindrobot>
|
Is it possible to have a wildcard web proxie? I.e. *.catalyst.wmcloud.org all routes to a k8s instance in our CloudVPS project.
|
|
2024-03-18 18:46:58
|
<kindrobot>
|
This will save us from having to make & clean up a web proxy for each environment on catalyst
|
|
2024-03-18 18:58:01
|
<taavi>
|
kindrobot: not at the moment. support for per-project subzones (e.g. allowing catalyst to use any subdomain catalyst.wmcloud.org) was added last week, and wildcard defaults in those subzones seems like a rather reasonable and not-that-complicated feature request.
|
|
2024-03-18 18:58:56
|
<kindrobot>
|
Oh nice taavi. Should I file a phab task/feature request?
|
|
2024-03-18 18:59:20
|
<kindrobot>
|
And is there documentation on the per-project web proxies?
|
|
2024-03-18 19:07:21
|
<taavi>
|
(sorry, my IRC bouncer froze for a second for some reason)
|
|
2024-03-18 19:07:24
|
<taavi>
|
yeah, please do.
|
|
2024-03-18 19:09:38
|
<taavi>
|
kindrobot: no docs yet. the tl;dr is that now a given dns zone managed by designate can be configured to be used via the web proxy system in the project that owns the zone. it needs an admin to configure it since we don't currently have an easy way to do self-service TLS cert generation.
|
|
2024-03-18 19:11:11
|
<kindrobot>
|
OK, I'd be very interested in setting that up, so that our environments don't pollute the main subdomain
|
|
2024-03-18 19:11:53
|
<taavi>
|
indeed your use case seems reasonable for this system. please file a task, I'll look into it tomorrow morning
|
|
2024-03-18 19:12:17
|
<kindrobot>
|
Is https://phabricator.wikimedia.org/project/view/474/ the best place to file both tasks?
|
|
2024-03-18 19:12:57
|
<taavi>
|
yes
|
|
2024-03-18 19:13:20
|
<kindrobot>
|
Thanks!
|
|
2024-03-18 21:13:34
|
<wm-bb>
|
<sohom_datta> I'm trying to run a pwb script using a custom buildservice container, user-config.py I get the following error `Skipped '/workspace/user-config.py': owned by someone else.` Any idea how I can resolve this ?
|
|
2024-03-18 21:14:09
|
<wm-bb>
|
<sohom_datta> https://gitlab.wikimedia.org/toolforge-repos/wsstats is the repo for the tool
|
|
2024-03-18 21:29:54
|
<wm-bb>
|
<sohom_datta> Wait, during the build process, is the user running the commands as root in the container ?
|
|
2024-03-18 21:42:30
|
<bd808>
|
@sohom_datta: quite likely, yes. It is most certainly not running as your tool's user.
|
|
2024-03-18 21:46:28
|
<bd808>
|
You might get some ideas about how to get the auth layer setup by looking at https://gitlab.wikimedia.org/toolforge-repos/pywikibot-buildservice
|
|
2024-03-18 22:30:39
|
<wm-bb>
|
<sohom_datta> I finally got it working :) (It's a bit jank, but I basically cat user-config.py from a different file that has the bad permissions)
|
|
2024-03-18 22:31:08
|
<wm-bb>
|
<sohom_datta> https://gitlab.wikimedia.org/toolforge-repos/wsstats/-/blob/main/scripts/runpwb.sh?ref_type=heads
|
|
2024-03-18 22:31:25
|
<wm-bb>
|
<sohom_datta> I should document this somewhere, else I will forget it :)
|
|
2024-03-18 22:51:13
|
<wm-bot>
|
!log bd808@tools-sgebastion-10 tools.wikibugs-testing Update and restart irc task to pick up dc8a19e2 in MR!15
|
|
2024-03-18 22:51:16
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs-testing/SAL
|
|
2024-03-18 23:08:53
|
<wm-bot>
|
!log bd808@tools-sgebastion-10 tools.wikibugs-testing Upgraded to redis==5.0.3
|
|
2024-03-18 23:08:56
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs-testing/SAL
|
|
2024-03-18 23:10:54
|
<wm-bb>
|
<sohom_datta> bd808: Do you know of any tool on that is using a celery backend ?
|
|
2024-03-18 23:11:22
|
<wm-bb>
|
<sohom_datta> link-dispenser has been a constant nightmare since the celery worker randomly looses connectivity with redis
|
|
2024-03-18 23:12:00
|
<wm-bb>
|
<sohom_datta> I haven't been able to figure out what I am doing wrong
|
|
2024-03-18 23:12:55
|
<bd808>
|
@sohom_datta: I think only whatever tool RoySmith has using it? That's the T318479 task you have commented on.
|
|
2024-03-18 23:12:56
|
<stashbot>
|
T318479: Intermittent redis connection timeouts in Toolforge - https://phabricator.wikimedia.org/T318479
|
|
2024-03-18 23:13:50
|
<bd808>
|
I did create T360378 today and will poke at that tomorrow if t.aavi doesn't tell me its the worst idea ever.
|
|
2024-03-18 23:13:50
|
<stashbot>
|
T360378: Provide a Redis container for use within a tool's namespace - https://phabricator.wikimedia.org/T360378
|
|
2024-03-18 23:15:43
|
<bd808>
|
I don't think anyone ever really considered the shared Redis being used for async task queuing. It was more intended to be a cache for expensive computations
|
|
2024-03-18 23:17:56
|
<taavi>
|
bd808: hmm. does the build service allow creating an image consisting purely of some apt packages? or would you also need some language runtime in a buildservice-managed redis image?
|
|
2024-03-18 23:19:04
|
<taavi>
|
(if yes, I'm wondering if someone registering tools.redis or similar and publishing a redis image for everyone to use would be better than a toollabs-images.git managed image)
|
|
2024-03-18 23:19:09
|
<bd808>
|
taavi: in a buildservice managed one you need something else too because the Apt pack is only an "add-on". I used Python in the https://gitlab.wikimedia.org/toolforge-repos/wikibugs2-znc image (which was needed for templating stuff anyway)
|
|
2024-03-18 23:19:52
|
<bd808>
|
I could make one via build-service but it will be a heavier image than just using the stuff we make the shared images with
|
|
2024-03-18 23:20:18
|
<taavi>
|
gotcha. in that case a prebuilt image sounds somewhat reasonable.
|
|
2024-03-18 23:20:25
|
<bd808>
|
The Apt build pack is pretty janky too, but that's a separate issue
|
|
2024-03-18 23:21:08
|
<taavi>
|
yep
|
|
2024-03-18 23:22:17
|
<taavi>
|
also, if you're poking around wikibugs today, mind giving tools-bastion-12.tools.eqiad1.wikimedia.cloud a go and testing it works as expected and isn't missing any packages you expect to be there?
|
|
2024-03-18 23:22:35
|
<wm-bot>
|
!log bd808@tools-sgebastion-10 tools.wikibugs-testing Update and restart irc task to pick up 3917e973 in MR!15
|
|
2024-03-18 23:22:38
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs-testing/SAL
|
|
2024-03-18 23:23:28
|
<bd808>
|
taavi: I'm about to stop for the day, but I'll try the new bastion when I pick things back up. :)
|
|
2024-03-18 23:24:26
|
<bd808>
|
taavi: first thing: "bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory"
|
|
2024-03-18 23:25:35
|
<taavi>
|
ugh. that's not the first weird locale issue I've seen in the last week
|
|
2024-03-18 23:26:30
|
<wm-bot>
|
!log bd808@tools-bastion-12 tools.wikibugs-testing Rolling back to dc8a19e2
|
|
2024-03-18 23:26:32
|
<stashbot>
|
Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikibugs-testing/SAL
|