[00:01:28] <jinxer-wm>	 FIRING: [3x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[01:06:28] <jinxer-wm>	 FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[01:09:53] <wikibugs>	 (03PS1) 10Tim Starling: Allow API query hooks to work on watchlist [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1196783 (https://phabricator.wikimedia.org/T407087)
[01:23:02] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Allow API query hooks to work on watchlist [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1196783 (https://phabricator.wikimedia.org/T407087) (owner: 10Tim Starling)
[03:31:09] <wikibugs>	 (03PS2) 10Tim Starling: Allow API query hooks to work on watchlist [extensions/ORES] - 10https://gerrit.wikimedia.org/r/1196783 (https://phabricator.wikimedia.org/T407087)
[05:06:43] <jinxer-wm>	 FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[08:28:18] <ozge_>	 o/
[09:06:43] <jinxer-wm>	 FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[10:00:58] <klausman>	 Ah, my old nemesis :D
[10:30:17] <ozge_>	 should we deploy or revert the pending admin_ng changes  😇
[11:01:28] <jinxer-wm>	 FIRING: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[11:04:43] <wikibugs>	 06Machine-Learning-Team: Export retrained Tone-check model to an S3 bucket - https://phabricator.wikimedia.org/T406217#11284800 (10gkyziridis) Hey @brouberol, we are currently working on this task and there are some open questions. Here is the **idea**: We will build a `kokkuri` image which will include the logi...
[11:06:28] <jinxer-wm>	 RESOLVED: [6x] HelmfileAdminNGPendingChangesLiftWing: Pending admin_ng changes on ml-serve-codfw - https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service#Deploy_changes_to_helmfile.d%2Fadmin_ng  - https://alerts.wikimedia.org/?q=alertname%3DHelmfileAdminNGPendingChangesLiftWing
[13:01:53] <wikibugs>	 (03PS1) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196911
[13:02:21] <wikibugs>	 (03PS1) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196912
[13:03:04] <wikibugs>	 (03Abandoned) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196911 (owner: 10Ozge)
[13:10:59] <wikibugs>	 (03PS2) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196912 (https://phabricator.wikimedia.org/T400446)
[13:12:24] <ozge_>	 hello, I've another patch to upgrade article descriptions buildkit 1.x
[13:12:24] <ozge_>	  https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1196912
[13:12:24] <ozge_>	 Can you take a look when you're available? I'll deploy it to staging after the patch is merged. @georgekyz
[13:13:09] <wikibugs>	 06Machine-Learning-Team, 07Essential-Work, 13Patch-For-Review: Update blubber version in inference services images - https://phabricator.wikimedia.org/T400446#11285182 (10OKarakaya-WMF)
[13:55:01] <wikibugs>	 (03CR) 10Gkyziridis: "Thnx for updating the syntax in blubber." [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196912 (https://phabricator.wikimedia.org/T400446) (owner: 10Ozge)
[13:55:30] <georgekyz>	 ozge_: Thank you for tackling that one! I left a small comment,
[14:13:45] <wikibugs>	 (03PS1) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196922
[14:16:29] <wikibugs>	 (03Abandoned) 10Ozge: feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196912 (https://phabricator.wikimedia.org/T400446) (owner: 10Ozge)
[14:18:31] <wikibugs>	 (03CR) 10Ozge: "https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1196922" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196912 (https://phabricator.wikimedia.org/T400446) (owner: 10Ozge)
[14:19:10] <ozge_>	 thanks @heorgekyz I've mistakenly addressed your comment in a new patch :) https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1196922
[14:19:30] <ozge_>	 @https://gerrit.wikimedia.org/r/c/machinelearning/liftwing/inference-services/+/1196922
[14:19:46] <ozge_>	 @georgekyz
[14:21:18] <georgekyz>	 ozge_: no worries at all, have you tested with `se-system-site-packages: false` ??
[14:21:31] <ozge_>	 yes
[14:21:47] <wikibugs>	 (03CR) 10Gkyziridis: [C:03+1] "LGTM! Thank you" [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196922 (owner: 10Ozge)
[14:22:03] <wikibugs>	 (03CR) 10Ozge: [C:03+2] feat: upgrades article descriptions buildkit 1.x [machinelearning/liftwing/inference-services] - 10https://gerrit.wikimedia.org/r/1196922 (owner: 10Ozge)
[14:22:43] <georgekyz>	 ozge_: Thank you for closing the previous review-patch. 
[14:34:58] <wikibugs>	 06Machine-Learning-Team, 13Patch-For-Review: Export retrained Tone-check model to an S3 bucket - https://phabricator.wikimedia.org/T406217#11285444 (10gkyziridis)
[14:42:54] <ozge_>	 thanks @georgekyz . here is another patch for the staging deployment: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1196927
[14:44:03] <georgekyz>	 happy deploying!
[14:44:12] <wikibugs>	 06Machine-Learning-Team, 07Essential-Work: Update blubber version in inference services images - https://phabricator.wikimedia.org/T400446#11285492 (10OKarakaya-WMF)
[17:31:20] <wikibugs>	 06Machine-Learning-Team, 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11286092 (10achou) **Summary for yesterday's meeting** ([[ https://docs.google.c...