[05:58:53] <wikibugs>	 10serviceops, 10Phabricator, 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹️): move phabricator to new hardware generation - https://phabricator.wikimedia.org/T280597 (10Marostegui)
[05:59:11] <wikibugs>	 10serviceops, 10Phabricator, 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹️): sort out mysql privileges for phab1004/phab2002 - https://phabricator.wikimedia.org/T315713 (10Marostegui) 05Open→03Resolved The problem wasn't the grants from what I can see, as there's...
[07:45:42] <wikibugs>	 10serviceops, 10API Platform, 10Growth-Structured-Tasks, 10Image-Suggestions, and 7 others: GrowthExperiments\NewcomerTasks\AddImage\ServiceImageRecommendationProvider::get Unable to decode JSON response for page {title} upstream connect error or disconnect/reset b... - https://phabricator.wikimedia.org/T313973
[11:29:37] <wikibugs>	 10serviceops, 10Observability-Logging, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Kubernetes logs (container stderr,strout) do not show up in Elasticsearch/Kibana - https://phabricator.wikimedia.org/T289766 (10JMeybohm) New rsyslog & exporter rolled out to `A:wikikube-staging-master or A:wikiku...
[11:37:12] <btullis>	 Hello serviceops. I have a quick question. I'd like to deploy a 'hello world' app to the dse-k8s cluster to check that it's working: https://phabricator.wikimedia.org/T310180 
[11:38:49] <btullis>	 I've happened across echoserver, which looks ideal: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/docker-images/production-images/+/refs/heads/master/images/echoserver but how best to proceed? I don't see any history of it in the deployment-charts, so if you could give me a pointer, either here or on the ticket, that would be great. Thanks.
[11:57:19] <jayme>	 btullis: I think there is no chart for that. I more or less created it for debuggin/testin in the istio ingress area
[11:58:29] <jayme>	 you could just "kubectl run" it I suppose
[12:00:15] <claime>	 If there's the opportunity to do a "create a new service on a k8s cluster" tutorial I'd like to attend :p
[12:01:59] <btullis>	 jayme: Thanks. I'll give it a go. It's a bit of a box-ticking exercise I suppose, before we start putting any proper workload on this cluster. I think I've probably still got some work to do in the ingress side, but not urgent.
[12:05:23] <elukey>	 btullis: m2c - go for a custom Deployment/etc.. via kubectl apply, and then kubectl delete the same resources when you are done :)
[12:05:47] <elukey>	 *my2c
[12:10:57] <btullis>	 Thanks all.
[14:09:04] <wikibugs>	 10serviceops, 10Observability-Logging, 10Prod-Kubernetes, 10Kubernetes, 10Patch-For-Review: Kubernetes logs (container stderr,strout) do not show up in Elasticsearch/Kibana - https://phabricator.wikimedia.org/T289766 (10JMeybohm) 05Open→03Resolved Resolving this in a hopeful manner
[14:43:47] <wikibugs>	 10serviceops, 10Phabricator, 10serviceops-collab, 10Patch-For-Review, 10Release-Engineering-Team (Bonus Level 🕹️): sort out mysql privileges for phab1004/phab2002 - https://phabricator.wikimedia.org/T315713 (10Dzahn) Oh, I assumed that part (that it's not using a dbproxy in this case) was meant to be.  T...
[15:07:09] <rzl>	 claime: you might chat with mutante about that, I know he's run that workshop in the past
[15:07:26] <rzl>	 not sure if there are plans to do it again but I think https://wikitech.wikimedia.org/wiki/Kubernetes/Kubernetes_Workshop is the materials
[15:07:35] <claime>	 I've seen the wikitech page, haven't ran through it yet
[15:08:01] <claime>	 But yeah it'd be a good step in the onboarding process I think
[15:08:05] <rzl>	 👍
[15:45:56] <wikibugs>	 10serviceops, 10SRE-OnFire, 10Wikimedia-Incident: Add etcdmirror connection retry on etcd-tls-proxy unavailability - https://phabricator.wikimedia.org/T317535 (10Clement_Goubert) Logs at the moment of the incident point to an etcd.Client uncaught exception. ` Sep  8 15:17:23 conf2005 etcdmirror-conftool-eqia...