[07:46:20] <dcaro>	 morning!
[07:56:13] <godog>	 greetings
[08:32:49] <aputhin>	 good morning! 
[08:33:12] <aputhin>	 dcaro: is the process you ran yesterday for restoring PAWS described anywhere? 
[08:33:29] <dcaro>	 not that I know of, there's no alert or runbook either
[08:40:49] <dcaro>	 aputhin: should I create a task for that?
[08:41:08] <aputhin>	 not sure an alert is warranted (we can discuss) but I think a runbook would be nice 
[08:42:50] <dcaro>	 usually the runbook is reached through the alert
[08:43:16] <dcaro>	 we are trying to keep a policy of 1 alert<->1 runbook
[08:43:38] <dcaro>	 I can add just a section in the main page instead
[08:44:01] <dcaro>	 https://wikitech.wikimedia.org/wiki/PAWS/Admin  vs https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks
[08:47:33] <aputhin>	 ok makes sense
[08:52:38] <dcaro>	 https://wikitech.wikimedia.org/w/index.php?title=PAWS%2FAdmin&diff=2442897&oldid=2442891
[08:52:40] <dcaro>	 done
[09:16:39] <godog>	 I'll drain/reboot a couple of toolforge nfs workers re: dumps-nfs
[09:20:37] <dcaro>	 ack
[15:54:52] <dhinus>	 re: the discussion in the weekly meeting, we do show maintainer name for 404 pages, but not for 500 pages
[15:55:22] <dhinus>	 404 errors go through the four-oh-four tool, e.g. https://dcaro-test7.toolforge.org/
[15:55:45] <dhinus>	 500 are generated by Istio e.g. https://favicons.toolforge.org/
[15:56:54] <dcaro>	 that might be a regression? https://gitlab.wikimedia.org/toolforge-repos/fourohfour/-/blob/main/templates/502.html?ref_type=heads
[15:57:38] <dcaro>	 (or a limitation of the ingress change we did)
[15:58:58] <taavi>	 yeah, unlike what presumably llm-generated blog posts with very good SEO say, customizing the istio error pages is much more difficult than what it was with ingress-nginx
[16:16:39] <dhinus>	 "our LLM will make your SEO marketing 2x more convincing"
[16:17:56] <dcaro>	 lol
[16:59:31] <dcaro>	 fyi. taavi the jobs api delete times have gone down again
[16:59:34] <dcaro>	 https://usercontent.irccloud-cdn.com/file/6e2noeDS/image.png
[16:59:51] <dcaro>	 https://w.wiki/SuDH
[17:00:22] <dcaro>	  ̄\_(ツ)_/ ̄  oh well
[17:00:27] * dcaro off
[17:00:28] <taavi>	 neat but that's just making me even more curious what's behind that
[17:00:36] <dcaro>	 agree xd
[17:00:55] <dcaro>	 cya!