[13:15:13] bumping this (from Dec 17) in case anyone’s online now who knows… (re @lucaswerkmeister: I have a question of my own, actually… is there any size limit on toolforge build pack container images?) [13:15:33] (this is currently blocking the migration of my tool to k8s) [13:17:14] in theory 'toolforge build quota' is supposed to tell you those limits, but that too currently returns an error at least for me [13:19:43] same here :/ [13:19:49] (but I didn’t know that command before, so thanks) [13:21:41] anyhow, I think the error you're getting is from one of the proxy layers in front of the registry complaining about the size of the individual layers, and not from the registry itself thinking it is more than the quota you have [13:25:27] more precisely, this: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/dynamicproxy/templates/domainproxy.conf#90 [13:31:50] filed T353698 [13:31:51] T353698: dynamicproxy client_max_body_size blocks large Harbor uploads - https://phabricator.wikimedia.org/T353698 [13:38:24] I don't see any problem in increasing it to 512, I +1d your patch [13:39:23] if we need more, it might be nice to increase it only for a specific host/path [13:40:43] @lucaswerkmeister: try now? [13:41:36] ok, running build… [13:43:31] looks like it worked \o/ thanks! [13:45:01] great [13:46:49] and the setup commands also worked fine until they hit a silly mistake I made ^^ [13:46:55] I’ll continue tinkering it with it later then [13:49:13] also, seems like the quota issue is only for tools that haven't had any builds complete yet. I filed T353701 for that [13:49:13] T353701: `build quota` fails if tool has no builds - https://phabricator.wikimedia.org/T353701 [14:24:37] ah, okay ^^ [14:24:51] (or that cleared out their builds, I guess) [15:39:49] !log tools restarting toolsdb to apply a config change T353093 [15:39:54] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [15:39:54] T353093: [toolsdb] MariaDB process is killed by OOM killer (December 2023) - https://phabricator.wikimedia.org/T353093 [17:03:31] Ohai. So; several years away and not really keeping up with the news and I find I am now maintainer of... 15 tools in addition to mine, most of which need to be migrated away from the old grid. Fun. Is there a quick and dirty guide to spin up instances of whatever the new thing is (k8s?). Most tools I should be able to migrate pretty seamlessly to containers I am guessing but I don't know [17:03:34] where to start. [17:05:05] hello Coren! [17:05:26] https://wikitech.wikimedia.org/wiki/News/Toolforge_Grid_Engine_deprecation is probably the "best" starting place we have right now. [17:06:17] Hey BD. Thanks for the pointer. [17:07:15] I was kinda burned out on the movement and anything to do with it but I'm not going to leave projects in a bind without their tools. [17:08:13] Coren: <3 [17:08:47] Plus, apparently, other maintainers who left decided I was the best person to dump their tools onto. [17:08:51] https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework is the replacement for old fashioned cron things [17:09:35] Yeah, I'm looking at the build service right now; once I have a container the jobs framework is the thing to schedule it? [17:09:59] I'll start with my own tools; those should be easy since I already know how they work. [17:10:45] yes, once you have built a custom container with https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service then you can use that container with either `webservice` or the jobs framework [17:11:45] or even a fully managed by you Kubernetes Deployment, but most tools shouldn't need to go that far into self-sufficience [17:12:14] I guess I /should/ have read those email instead of letting them pile up unread in a mailbox; the only reason I found out is that I had like a dozen people contact me offline about "omg the world is ended" [17:12:59] Nah, most of the tools are either small web services or bot jobs as far as I can tell. [17:13:14] heh. I guess some of the panicked canvasing against us is actually helping if it got you here. [17:13:41] wsexport is probably going to be a nightmare though. I dunno how I ended up with it on my plate, but that thing is critical to all the wikisources and is painfully complicated. [17:14:49] I thought Sam Wilson was running ws-export these days? [17:15:32] tools.wsexport:*:52561:jberkel,phe,tpt,tools.community-tech-tools,marc [17:15:36] And I thought it had moved out into a Cloud VPS project... /me looks [17:15:53] If so, it's broken atm. :-D [17:15:54] tools.community-tech-tools would be Sam's doorway there I think [17:16:13] https://ws-export.wmcloud.org/ [17:16:39] I haven't heard from phe or tpt in ages. [17:16:51] I... don't know who Sam is. :-D [17:17:41] https://meta.wikimedia.org/wiki/User:SWilson_(WMF) https://meta.wikimedia.org/wiki/User:Samwilson [17:17:47] Hmph. Okay, so I don't even know _which_ of the tools will need migration and tlc beyond the ones I wrote. I should make a backup of all of 'em though [17:18:03] Ah, he's my replacement. :-P [17:18:47] https://wikisource.org/wiki/Wikisource:WS_Export points to the Cloud VPS project. I wonder if the tool is anything other than a redirect at this point? [17:32:11] Possibly/probably. [17:32:54] I'm glad at least such a critical tool was taken over by a staffer; did the WMF suddenly remember that there are things other than enwp? [17:36:11] Coren: Sam was already a Wikisourcerer when we hired him into the Community Tech team, but yeah we have at least occasionally given attention to non-enwp projects. ;) [17:44:52] Hm. bookworm will be non-trivial. No love for perl. [17:52:11] Coren: there should be a base container that we created for T214343, but I don't think there is anything perl specific in the build service's buildpack stack. [17:52:11] T214343: Create a Perl Docker image for use on the Toolforge Kubernetes cluster - https://phabricator.wikimedia.org/T214343 [17:54:23] Bleh. I have 2FA enabled for phabricator. And last time I had to use it Leslie was still a staffer. [17:54:53] that was a few days ago... [17:55:33] disabling 2fa for a phab account requires a hat I don't have unfortunately [17:56:23] ... I don't think we're talking about the same Leslie. :-) [17:56:34] lcarr? [17:58:29] Aye. Last news I got from her she was working at some health place? Didn't she leave the wmf like in 2014? [18:01:31] yeah, she left about 6 months after I got here. LinkedIn reminds me that she is at a heathcare place again. I had remembered her going to an HR startup, but that apparently changed a year ago. [18:03:05] Okay, the good news is that the perl docker image works fine to run my bot. [18:03:23] excellent! [18:03:54] (I'm having it complete a run by hand now in a shell) [18:04:40] Now to figure out how to build an image from it with my code in it. :-) [18:05:06] (Also I probably need to figure out the canonical way of passing the DB creds into it) [18:05:20] if things work with the pre-built container you just use is and put stuff in the tool's $HOME [18:05:34] *use it [18:05:58] Ah, the tools homes aren't going away then? [18:07:10] not wholesale, no. Things using the build service to make a custom image mostly shouldn't use $HOME anymore, but things using pre-built images will still be able to mount $HOME from NFS. [18:11:33] So I use the toolforge jobs tool and I'm all set? [18:12:03] Or is that not also obsolescent? [18:17:54] Coren: yes, use https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework to setup a scheduled or continuous job for your bot. `toolforge jobs` is basically the new `jsub` + cron. [18:18:29] I figured as much; just wasn't sure whether that was deprecated in favor of the cloud thing. :-)