[07:49:58] elukey: of course another kernel update landed. I'll do the upgrade-and-reboot dance for our machines [07:52:34] klausman: ack! Usually Moritz pings us when the upgrades are mandatory (mostly security fixes), but it is good anyway to keep kernels upgraded :) [07:53:06] Yeah, I figure now when it's 0 disruption is probably better than later, when Murphy's Law makes sure it will be at a super inconvenient time [08:01:37] yep yep [09:05:27] All done. [09:15:32] nice :) [09:43:33] I started https://wikitech.wikimedia.org/wiki/User:Elukey/MachineLearning/kfserving to brain dump what I am working on [09:44:04] not really complete but it is my understanding of the stack so far [09:44:19] I'll add more infos in the afternoon [09:51:07] * elukey errands + lunch! ttl [09:57:49] Will give it a read after lunch [11:37:18] 10Lift-Wing, 10artificial-intelligence, 10articlequality-modeling, 10revscoring, 10Machine-Learning-Team (Active Tasks): Create a KFServing model server for articlequality models - https://phabricator.wikimedia.org/T284678 (10kevinbazira) @ACraze, thank you for testing the articlequality inferenceservice... [14:50:12] ok finally merged in deployment charts the first part for istio configs! [14:50:28] tried to deploy it and it doesn't spin up pods, I think that some config is needed [14:51:04] the main downside is, for now, that istioctl throws a big warn sign saying "hello, kubernetes 1.16 is not supported, use it at your own risk! bye" [14:52:50] I chose 1.9.5 because it was the last version without CVes [14:52:56] *CVEs [15:40:44] 10Lift-Wing, 10API Platform (Product Roadmap), 10Epic: LiftWing API - https://phabricator.wikimedia.org/T285562 (10sdkim) [16:09:59] o/ [16:10:20] elukey: nice wikitech braindump on kfserving :) [16:28:53] accraze: thanks! really very high level and not super detailed, but I'll add more steps etc.. as we go [16:38:46] i've got a bunch of plain text notes on my machine related to building/debugging inference services and kubeflow, i should add them under my user page someday [16:42:04] been using org-roam to manage the notes, it's kinda like a local wiki, i'm fairly certain there is a way to export to mediawiki syntax too [16:59:05] I have been using wikitech during the past years as compromise, so many things that otherwise I forget [16:59:35] I also find this type of summaries great - https://wikitech.wikimedia.org/wiki/User:Elukey [17:01:58] Error creating: pods "istiod-7c49cdc6bd-" is forbidden: unable to validate against any pod security policy: [] [17:02:15] this is what I get for istio on ml-serve-eqiad, some more config is needed :) [17:02:27] we'll see next week if SRE can help in finding the right config [17:02:36] (the PSP part is still a mistery to me) [18:41:51] have a good weekend folks :) [21:18:40] 10Machine-Learning-Team, 10MediaWiki-extensions-ORES, 10CheckUser, 10Commons, and 30 others: WikiPage::doEditContent falls back to $wgUser - https://phabricator.wikimedia.org/T255507 (10DannyS712)