[08:48:15] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Automate unpublishing of add-a-link datasets - https://phabricator.wikimedia.org/T344799 (10kevinbazira) [09:56:52] Morning! [09:57:23] I'm doing the security-update reboots of the codfw workers (prod) today. While there will be plenty of churn for the pods, there should be no downtime. [09:57:27] (03PS10) 10Ilias Sarantopoulos: read thresholds numeric values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/948584 (https://phabricator.wikimedia.org/T343308) [09:58:55] morning klausman [09:58:58] ack! [10:00:14] (03PS11) 10Ilias Sarantopoulos: read thresholds numeric values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/948584 (https://phabricator.wikimedia.org/T343308) [10:06:22] (03CR) 10Ilias Sarantopoulos: "The patch is now ready for review. We are using only numeric values" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/948584 (https://phabricator.wikimedia.org/T343308) (owner: 10Ilias Sarantopoulos) [10:08:52] (03PS12) 10Ilias Sarantopoulos: read thresholds numeric values [extensions/ORES] - 10https://gerrit.wikimedia.org/r/948584 (https://phabricator.wikimedia.org/T343308) [10:14:06] 10Machine-Learning-Team, 10Patch-For-Review: fiwiki RC filters classify all edits as 'very likely bad faith' - https://phabricator.wikimedia.org/T343308 (10isarantopoulos) The patches are ready! https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ORES/+/948584 https://gerrit.wikimedia.org/r/c/operations/med... [10:36:04] klausman: FYI, the buster hosts (for ML ores*) are also ready to rebooted now: https://phabricator.wikimedia.org/T344671 [10:56:09] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team (Current Sprint), 10Patch-For-Review, 10User-notice: Deploy "add a link" to 13th round of wikis - https://phabricator.wikimedia.org/T308138 (10Sgs) [10:57:31] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team (Current Sprint), 10Patch-For-Review, 10User-notice: Deploy "add a link" to 13th round of wikis - https://phabricator.wikimedia.org/T308138 (10Sgs) All wikis have good amount of results except `pagwiki` which was showing [[ https://pag.wikipedia.org/w/in... [10:59:12] moritzm: roger! [11:00:28] cheers [11:08:38] * isaranto lunch! [11:19:51] same [12:18:39] Reboots for codfw LW workers are done, no issues [12:58:08] good morning all [13:00:33] Hi Chris o/ [13:19:17] 10Machine-Learning-Team: Model monitoring - https://phabricator.wikimedia.org/T344819 (10isarantopoulos) [15:35:51] 10Machine-Learning-Team: Model monitoring - https://phabricator.wikimedia.org/T344819 (10isarantopoulos) Some initial thoughts on this: My suggestion is to use WMF's existing stack (prometheus - grafana - alertmanager) for this kind of work. We could use [[ https://wikitech.wikimedia.org/wiki/Prometheus | Promet... [15:36:07] o/ I wrote some initial thoughts on the model monitoring task [15:50:19] 10Machine-Learning-Team, 10Add-Link, 10Growth-Team: Investigate why the add-a-link training pipeline concludes with missing datasets - https://phabricator.wikimedia.org/T344832 (10kevinbazira) [16:03:36] Got another request related to ORES and the recent changes filters https://wikitech.wikimedia.org/wiki/Talk:ORES [16:55:42] 10Machine-Learning-Team, 10Gerrit, 10Release-Engineering-Team (Radar): ML-Team will soon stop using LFS on Gerrit (for ORES deployment) - https://phabricator.wikimedia.org/T342765 (10thcipriani) 05Open→03Stalled Phabricator does the mirroring here so I think the things to fix on our side here: - [] Stop... [17:14:32] heading out now. talk to y'all tomorrow!