[04:00:01] (03PS1) 10Santhosh: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 [04:01:00] (03CR) 10CI reject: [V:04-1] major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (owner: 10Santhosh) [06:35:56] Good morning! [06:43:23] (03PS2) 10Santhosh: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 [07:00:06] (03PS3) 10Santhosh: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 [07:50:05] (03CR) 10Hashar: "recheck after having deployed https://gerrit.wikimedia.org/r/c/integration/config/+/1051838" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1051837 (https://phabricator.wikimedia.org/T364705) (owner: 10Kosta Harlan) [08:09:01] (03PS4) 10Santhosh: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 [08:20:11] (03PS5) 10Santhosh: major: modernize the codebase, keep only translation recommendations [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) [08:46:02] morning o/ [08:48:34] Morning everyone! [08:49:58] hi folks o/ [08:51:40] (03PS1) 10Kevin Bazira: logo_detection: migrate to src dir [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) [09:14:36] I have also figured out what my Internet trouble was caused by last week: contractors were drilling in the basement and nicked the power cable of the fiber optic endpoint thingy, which made it fault out intermittently. Will be fixed sometime today, so fingers crossed. [09:21:31] ouch! [09:21:58] a couple of years ago we were without internet for a month because of some road works [09:22:27] well we all know infrastructure is important :) [09:32:56] it's one of those "you don't quite know how much you'll miss it until it's gone" things [09:40:57] hello folks [09:41:06] I am going to deploy the api-gateway in a bit [09:41:26] nothing relevant except the new envoy image based on Bookworm [09:41:42] please let me know if you see anything weird [09:41:49] Roger! [09:46:51] o/ Luca, got it! [09:53:11] all done! [09:54:02] grazie! [09:55:07] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10AbuseFilter: [Spike] Investigate building a hook for abuse filter - https://phabricator.wikimedia.org/T123178#9960115 (10kostajh) I'm going to close this as a duplicate of {T364705}. [09:55:40] 06Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10AbuseFilter: [Spike] Investigate building a hook for abuse filter - https://phabricator.wikimedia.org/T123178#9960118 (10kostajh) →14Duplicate dup:03T364705 [10:05:07] (03CR) 10Ilias Sarantopoulos: [C:03+1] "LGTM!" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [10:12:49] 06Machine-Learning-Team, 07Kubernetes: Migrate ml-staging/ml-serve clusters off of Pod Security Policies - https://phabricator.wikimedia.org/T369493 (10JMeybohm) 03NEW [10:41:42] (03CR) 10Klausman: logo_detection: migrate to src dir (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [11:31:16] * klausman lunch (and passport stuff) [11:34:52] * isaranto lunch! [12:03:35] (03PS2) 10Kevin Bazira: logo_detection: migrate to src dir [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) [12:10:04] (03CR) 10Kevin Bazira: logo_detection: migrate to src dir (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [12:56:09] (03CR) 10Klausman: [C:03+1] logo_detection: migrate to src dir [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [12:56:24] (03CR) 10Klausman: [C:03+1] logo_detection: migrate to src dir (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [13:00:06] 06Machine-Learning-Team, 07Kubernetes, 13Patch-For-Review: Migrate ml-staging/ml-serve clusters off of Pod Security Policies - https://phabricator.wikimedia.org/T369493#9960707 (10klausman) a:03klausman [13:40:30] blubber moves to v1.0.0 https://gitlab.wikimedia.org/repos/releng/blubber/-/blob/main/CHANGELOG.md?ref_type=heads [13:41:09] I'm testing it in the hf image now that I'm working around PYTHONPATHs [13:45:01] well actually first I'm testing the v0.24.0 version which started using virtualenvs [14:40:54] 06Machine-Learning-Team, 10Observability-Metrics, 10SRE Observability (FY2024/2025-Q1): Gap in metrics rendered from Thanos Rules - https://phabricator.wikimedia.org/T352756#9961298 (10herron) It occurred to me that deploying absent metric alerts for the metrics where we're seeing gaps would be a reasonable... [14:59:10] finally figured out the issue with PYTHONPATHs. I had forgotten that it was removed in blubber v0.23.0 [15:01:39] 06Machine-Learning-Team: Update blubber version in docker images - https://phabricator.wikimedia.org/T367293#9961418 (10isarantopoulos) Something that affects our deployments and is mentioned in the [[ https://gitlab.wikimedia.org/repos/releng/blubber/-/merge_requests/83 | following MR that introduces v0.23.0 ]]... [15:27:27] 06Machine-Learning-Team: Simplify dependencies in hf image - https://phabricator.wikimedia.org/T369359#9961534 (10isarantopoulos) Unfortunately setting the PYTHONPATH in a new pytorch base image will not work as blubber has removed PYTHONPATH support https://phabricator.wikimedia.org/T367293#9961418. This means... [15:27:44] (03CR) 10Ilias Sarantopoulos: [C:03+1] logo_detection: migrate to src dir (031 comment) [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [15:33:23] (03CR) 10Kosta Harlan: "check phan" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1051837 (https://phabricator.wikimedia.org/T364705) (owner: 10Kosta Harlan) [15:34:20] I'm waiting for an image to be built and I'll open up the patch for the above first thing in the morning [15:34:29] logging off for the day folks, have a nice evening o/ [15:49:54] \o [16:04:16] bye Ilias! [16:24:33] 06Machine-Learning-Team, 06Structured-Data-Backlog: Host a logo detection model for Commons images - https://phabricator.wikimedia.org/T358676#9961889 (10AUgolnikova-WMF) [16:30:16] (03CR) 10Kevin Bazira: "recheck" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [16:31:11] (03CR) 10Hashar: "recheck after having deployed https://gerrit.wikimedia.org/r/c/integration/config/+/1052672" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [16:36:05] (03CR) 10Kevin Bazira: [C:03+2] "Thanks for the reviews :)" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [16:36:55] (03Merged) 10jenkins-bot: logo_detection: migrate to src dir [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1052674 (https://phabricator.wikimedia.org/T369344) (owner: 10Kevin Bazira) [19:30:07] (03CR) 10Isaac Johnson: "Excited to see this! I left some comments based on my testing via local virtualenv + gunicorn. Mainly focused on fixing up `mostpopular` e" [research/recommendation-api] - 10https://gerrit.wikimedia.org/r/1052445 (https://phabricator.wikimedia.org/T369484) (owner: 10Santhosh)