[08:22:23] <wikibugs>	 10Machine-Learning-Team, 10Add-Link, 10Growth-Scaling, 10Growth-Team (Current Sprint): Establish processes for running the dataset pipeline - https://phabricator.wikimedia.org/T276438 (10MGerlach)
[09:12:43] <elukey>	 hopefully https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/724448 will fix the problem with helm :)
[09:12:48] <elukey>	 (or a variation of it)
[10:20:02] * elukey lunch
[13:54:12] <elukey>	 ok the problem is fixed, more are coming :D
[14:02:25] <elukey>	 ok now the problem is
[14:02:25] <elukey>	 Error: namespaces is forbidden: User "revscoring-editquality-deploy" cannot create resource "namespaces" in API group "" at the cluster scope
[14:02:33] <elukey>	 why namespaces no idea
[14:15:24] <chrisalbon>	 Mooorning
[14:57:47] <elukey>	 o/
[15:05:54] <elukey>	 Revision "enwiki-goodfaith-predictor-default-7sbq5" failed with message: Unable to fetch image "docker-registry.discovery.wmnet/wikimedia/machinelearning-liftwing-inference-services-editquality:2021-09-01-140944-production": failed to resolve image to digest: Get "https://docker-registry.discovery.wmnet/v2/": dial tcp 10.2.1.44:443: i/o timeout.
[15:05:59] * elukey cries in a corner
[15:54:49] <accraze>	 o/
[16:02:48] <elukey>	 o/
[16:13:56] <elukey>	 really weird, can't get why knative behaves like that
[16:14:15] <elukey>	 trying to see if I added a typo or similar by mistake but it doesn't seem so
[16:16:23] <elukey>	 and docker pull on the host works fine, super fast, mmm
[16:17:13] <accraze>	 it's knative that cannot fetch the image??
[16:17:48] <elukey>	 yeah I see from the knative controller's logs the above error over and over
[16:18:34] <elukey>	 one of the thing that the controller does it to make sure that it has the digest for the image to use, so that it can be sure what image a revision needs
[16:18:50] <elukey>	 it seems that it can't contact the docker registry for some reason
[16:19:21] <accraze>	 is it psp related?
[16:19:44] <elukey>	 in theory no, we'd see some issues with permissions in logs etc.. 
[16:19:51] <elukey>	 (unable to do blabla(
[16:19:55] <accraze>	 ah good point... hmmm
[16:20:02] <accraze>	 it used to work tho
[16:20:08] <accraze>	 right?
[16:22:05] <elukey>	 the last one was docker-registry.wikimedia.org/wikimedia/machinelearning-liftwing-inference-services-editquality 
[16:22:19] <elukey>	 but it was for the global domain (the .w.o
[17:05:44] <elukey>	 accraze, chrisalbon, kevinbazira - meeting?
[17:05:59] <kevinbazira>	 jumping into the meeting now
[17:06:01] <accraze>	 omw sorry meeting ran over :)
[17:06:13] <elukey>	 ah I see slack now!
[17:54:42] * elukey afk!
[17:55:57] <accraze>	 see ya elukey!
[18:01:08] <groceryheist>	 thanks for having me folks
[18:02:16] <groceryheist>	 one thing that's exciting about having more focus on editor-users is that I feel that's the way to get your stuff institutionalized and valued by the community and lead to more motivation to do stuff like labeling campaigns. 
[18:03:08] <groceryheist>	 also exciting that you're working on building up experimental capacity!
[22:33:45] <accraze>	 groceryheist: thanks talking through your work, interesting stuff!
[23:07:08] <accraze>	 haha ok just added a project proposal for next week's DSE hackathon: generative neural audio mashups using random audio files on Wikimedia Commons
[23:25:38] <wikibugs>	 10Machine-Learning-Team, 10artificial-intelligence, 10Wikilabels, 10articlequality-modeling: Build article quality model for Dutch Wikipedia - https://phabricator.wikimedia.org/T223782 (10Halfak) Adds the nlwiki article quality scale form to Wikilabels: https://github.com/wikimedia/wikilabels-wmflabs-deplo...