[06:49:50] good morning [06:50:05] opened https://github.com/kubeflow/kfserving/issues/1744 for their yaml files, let's see what they say about it [07:40:50] * elukey afk! bbiab [10:45:14] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10Ciell) Interesting suggestion in our Wikipedia:De Kroeg to turn around A-E classes and make A the minimum starters clas... [11:06:56] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10RonnieV) I already commented in [[ https://nl.wikipedia.org/w/index.php?title=Wikipedia:De_kroeg&oldid=59616493 | WP:DK... [14:15:04] so the webhook for kfserving seems to be inferenceservices.serving.kubeflow.org [14:15:07] and it needs a TLS cert [14:15:35] I was thinking to use inference-webhook.wikimedia.org, but it seems too generic (since we may have non kfserving webhooks too) [14:15:50] inference-kfserving-webhook.wikimedia.org? [15:54:31] elukey: yeah inference-kfserving-webhook.wikimedia.org is fine imo [16:00:35] accraze: ack perfect! I think I understand now how to tranfer the tls cert to the helm chart of kfserving, next we may have kfserving up [16:16:13] the editquality pipeline seems to be in good shape now, going to make some tasks today for building pipelines for the other model-servers [16:20:44] nice! [16:24:47] great to hear! [16:26:00] 10Lift-Wing, 10ML-Governance, 10Machine-Learning-Team (Active Tasks): Outlinks model card - https://phabricator.wikimedia.org/T287527 (10calbon) That is really the value we are going for. And if it is on a wiki it opens up community contributions to the page and discussions via a talk page [16:31:36] this is the trick [16:31:37] https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/700470/14/charts/kubeflow-kfserving/templates/secret.yaml [16:32:05] via puppet we deploy another helmfile config on deploy1002, that renders secrets like tls cert keys, passwords, etc.. [16:32:10] and we can reference them in the chart [16:32:34] for the caBundle, we have a public value [16:32:34] caBundle: |- [16:32:35] {{ .Values.puppet_ca_crt | b64enc | indent 6 }} [16:36:07] to reduce the excitement [16:36:08] Kubernetes 1.17 is the minimally recommended version, Knative Serving and Istio should be available on Kubernetes Cluster. [16:36:13] this is for kfserving [16:36:32] in 0.6.0 I just got a kubeyaml error for StatefulSet [16:39:21] * elukey cries in a corner [16:39:34] We are here for you elukey [17:10:46] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10Halfak) It should be OK to change the meaning of the current classes over time too. One nice thing about using an ML m... [17:11:48] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10Halfak) I should say, this pattern of retraining also works for //between classes// too. [17:13:10] false alarm, fixed, thanks for the support chrisalbon :D [17:13:57] https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/700470 is starting to become working [17:14:04] on monday I'll create the TLS certs [17:14:10] and ask for a review [17:14:27] awesome, feeling good about this [17:14:46] if I got the tls thing correctly it should be easy to add the istio/knative TLS cert for inference.wikimedia.org as well [17:15:05] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10Halfak) Sorry. One final thought. We could make the quality classes non-ordinal. E.g. call the lowest class Beginnet... [17:15:17] (great props to the work done by serviceops) [17:18:41] all right going afk for the weekend folks! Have a good weekend :) [17:19:03] have a good weekend elukey! [17:38:08] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks): Configure tox tests for editquality inference service pipelines - https://phabricator.wikimedia.org/T287053 (10ACraze) 05Open→03Resolved This is complete and seems to work well. Eventually we should add a pytest env that runs some integration tests, howe... [17:42:07] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10calbon) The interesting thing about that idea is that the lowest class and highest class likely 1) the easiest for a ma... [18:16:30] 10Lift-Wing, 10articlequality-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for articlequality model server - https://phabricator.wikimedia.org/T287781 (10ACraze) [18:19:31] 10Lift-Wing, 10draftquality-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for draftquality model server - https://phabricator.wikimedia.org/T287783 (10ACraze) [18:21:27] 10Lift-Wing, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for revscoring topic model server - https://phabricator.wikimedia.org/T287784 (10ACraze) [18:31:27] 10Lift-Wing, 10articlequality-modeling, 10Machine-Learning-Team (Active Tasks): Configure articlequality deployment pipeline - https://phabricator.wikimedia.org/T287786 (10ACraze) [18:32:18] 10Lift-Wing, 10draftquality-modeling, 10Machine-Learning-Team (Active Tasks): Configure draftquality deployment pipeline - https://phabricator.wikimedia.org/T287787 (10ACraze) [18:33:48] 10Lift-Wing, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): Configure revscoring topic deployment pipeline - https://phabricator.wikimedia.org/T287788 (10ACraze) [18:34:14] 10Lift-Wing, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): Configure revscoring topic deployment pipeline - https://phabricator.wikimedia.org/T287788 (10ACraze) [18:34:18] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Production images for ORES/revscoring models - https://phabricator.wikimedia.org/T279004 (10ACraze) [18:35:04] 10Lift-Wing, 10draftquality-modeling, 10Machine-Learning-Team (Active Tasks): Configure draftquality deployment pipeline - https://phabricator.wikimedia.org/T287787 (10ACraze) [18:35:07] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Production images for ORES/revscoring models - https://phabricator.wikimedia.org/T279004 (10ACraze) [18:35:33] 10Lift-Wing, 10articlequality-modeling, 10Machine-Learning-Team (Active Tasks): Configure articlequality deployment pipeline - https://phabricator.wikimedia.org/T287786 (10ACraze) [18:35:36] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Production images for ORES/revscoring models - https://phabricator.wikimedia.org/T279004 (10ACraze) [18:36:10] 10Lift-Wing, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for revscoring topic model server - https://phabricator.wikimedia.org/T287784 (10ACraze) [18:36:17] 10Lift-Wing, 10drafttopic-modeling, 10Machine-Learning-Team (Active Tasks): Configure revscoring topic deployment pipeline - https://phabricator.wikimedia.org/T287788 (10ACraze) [18:36:33] 10Lift-Wing, 10draftquality-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for draftquality model server - https://phabricator.wikimedia.org/T287783 (10ACraze) [18:36:35] 10Lift-Wing, 10draftquality-modeling, 10Machine-Learning-Team (Active Tasks): Configure draftquality deployment pipeline - https://phabricator.wikimedia.org/T287787 (10ACraze) [18:50:50] 10Lift-Wing, 10artificial-intelligence, 10articlequality-modeling, 10Machine-Learning-Team (Active Tasks): Create blubberfile for articlequality model server - https://phabricator.wikimedia.org/T287781 (10ACraze) [18:50:52] 10Lift-Wing, 10artificial-intelligence, 10articlequality-modeling, 10Machine-Learning-Team (Active Tasks): Configure articlequality deployment pipeline - https://phabricator.wikimedia.org/T287786 (10ACraze) [19:02:25] damn accraze lol [19:08:29] hah didn't mean for it to be so loud [19:08:33] just trying to keep track of all the moving pieces related to revscoring deployment pipelines! [19:08:38] no no it is good [19:08:48] I just came back to IRC to see a wall of tickets [19:08:52] LOL [19:11:29] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Production images for ORES/revscoring models - https://phabricator.wikimedia.org/T279004 (10ACraze) [19:12:59] 10Lift-Wing, 10Machine-Learning-Team (Active Tasks), 10Patch-For-Review: Production images for ORES/revscoring models - https://phabricator.wikimedia.org/T279004 (10ACraze) It seems like the editquality pipeline is working well. I went ahead and made sub-tasks to track building & configuring similar pipeline... [23:30:20] 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10RonnieV) I like the idea of class names above class labels. Some suggestions: Etalage Zeer goed Goed Redelijk Beginnetje [23:45:00] Alright, I’m out. Have a great weekend all