[07:40:51] blancadesal: we briefly discussed T316107 yesterday. I would like to work on that next. We should figure out a timeline and a list of steps/plan first [07:40:53] T316107: [infra,k8s] Upgrade Toolforge Kubernetes to version 1.25 - https://phabricator.wikimedia.org/T316107 [07:41:20] also Raymond_Ndibe is interested in shadowing this time in addition to the upgrade he was assigned [07:53:47] arturo: we could sync tomorrow during collab if that works for you [07:55:33] I wanted to do something today already. I can create a draft of the timeline and we can review it tomorrow together [08:01:46] arturo: ok, I don't mind doing it today [08:31:13] 👍 [08:40:01] arturo: my day is largely free of meetings, just put something down on my calendar whenever it suits you [08:43:21] 👍 [08:43:24] will ping you later [10:22:47] * dcaro lunch [11:29:44] ci is getting slow again for golanclint [11:29:55] will rebuild the ci image to pull the newer versions [11:53:35] 👍 [13:00:03] blancadesal, Raymond_Ndibe I created a bunch of tickets for the 1.25 upgrade, feel free to update them as you see fit, claim, etc [13:00:07] * arturo food time [13:00:23] arturo: thanks [13:02:00] don't forget to check if any components need upgrading [13:02:06] (ex. tekton) [13:26:34] 👍 [15:50:48] Is there a feature request in the Build Service backlog for custom tagging? I would like to be able to do something like `toolforge build tag bridgebot-buildpacks-pipelinerun-mqkrz 1.0.0` to create stable tags for stuff like the pywikibot and redis containers that tools are sharing with others. [15:56:52] I'm not aware of any [15:56:56] * arturo offline [15:57:29] there's not, you have the image name option, one of the things to keep in mind when using buildservice is that it's not aimed to build images, but just code (similar to heroku in that sense) [15:57:55] there might be a task somewhere to allow reverts and such instead of overwritting the latest tag all the time [15:57:59] * dcaro looking [16:00:14] might not have made it to a task :/ [16:00:58] bd808: what's the work flow you are thinking of? [16:02:17] I do think that having a 'stable/prod' image build and being able to test the newly built one would be useful [16:15:31] dcaro: Almost any ability to have a tag in addition to ":current" could be workable for me I think. I could see `toolforge build run --tag 1.2.3` or a post-build like `toolforge build tag bridgebot-buildpacks-pipelinerun-mqkrz 1.0.0` allowing a nicer container reuse experience. [16:16:06] s/run/start/ [16:16:57] As you pointed out, I could also do something like `toolforge build start --image-name foo-1.0.0` or something [16:17:08] for now the image name might be the fastest workaround yep [16:17:21] maybe have a foo-dev and foo-stable images [16:17:26] or similar [16:17:32] :nod: [16:18:09] feel free to open a task though with the usecase, I think others would benefit too [16:18:29] (even if it takes a bit to have the feature out) [16:20:17] dhinus: about ceph cookbooks erroring running ceph, I added a silly retry [16:20:20] https://www.irccloud.com/pastebin/j5eJz5mh/ [16:20:31] it seems the exit code is 2, but no output :/ [16:20:40] annoying :/ [16:21:17] I'll try running a while loop or similar on the cloudecephmon, see if the issues reproduces even from the host itself [16:21:27] ideas are welcome though :) [16:21:59] is maybe the output being hidden? I assume it does output something even when it works, but I don't see any output at all in your paste [16:23:31] that's one of the times where I would like Cumin debug logs to include the output of commands T347093 [16:23:32] T347093: [spicerack] Add remote command output to log file - https://phabricator.wikimedia.org/T347093 [16:48:13] dhinus: from the logs there's no output either, just the traceback from spicerack [16:50:38] https://www.irccloud.com/pastebin/4iDdVdJ2/ [16:50:55] ^ from a previous failed run (now it does not log there anything as it does not crash) [16:59:15] dcaro: the spicerack logs never include the output from commands unfortunately :/ [16:59:37] that's what I was suggesting to add in T347093 [16:59:40] T347093: [spicerack] Add remote command output to log file - https://phabricator.wikimedia.org/T347093 [17:00:50] in that task, Riccardo mentioned there was some work already underway to improve "return data" [17:01:20] and also that you have to capture the output manually if you are interested in logging/handling it in the cookbook [17:01:57] * dhinus off [17:05:17] Not sure who looks in the devex slack, but Tajh was looking for someone to submit a WMCS talk to the WikiCon NA conference in October, so I made https://wikiconference.org/wiki/Submissions:2024/What%27s_new_with_Wikimedia_Cloud_Services [17:06:12] If it gets accepted then I imagine y'all can help me make sure to cover the cool new stuff that has been packed into our platforms in the last couple of years that folks might have missed [17:06:47] l.egoktm has an interesting looking talk proposed too at https://wikiconference.org/wiki/Submissions:2024/10_years_of_Toolforge [17:08:22] Tricia also has this most excellent Toolhub edit-a-thon proposal in the submission queue-- https://wikiconference.org/wiki/Submissions:2024/Catalog-a-thon_for_GLAM_tools_in_Toolhub [17:10:03] bd808: <3 [17:49:19] * dcaro off