[03:48:01] <wm-bb>	 <nokibsarkar> I have my lighttpd.conf 
[03:48:02] <wm-bb>	 <nokibsarkar>  ```
[03:48:04] <wm-bb>	 <nokibsarkar> # Ensure required modules are loaded
[03:48:05] <wm-bb>	 <nokibsarkar> server.modules += (
[03:48:07] <wm-bb>	 <nokibsarkar>     "mod_proxy",
[03:48:08] <wm-bb>	 <nokibsarkar>     "mod_rewrite"
[03:48:10] <wm-bb>	 <nokibsarkar> )
[03:48:11] <wm-bb>	 <nokibsarkar> # 2. Proxy /api/* to the backend
[03:48:13] <wm-bb>	 <nokibsarkar> $HTTP["url"] =~ "^/api/" {
[03:48:14] <wm-bb>	 <nokibsarkar>     proxy.server = ( "" => ( ( 
[03:48:16] <wm-bb>	 <nokibsarkar>         "host" => "campwiz-backend-dev.toolforge.org",
[03:48:17] <wm-bb>	 <nokibsarkar>         "port" => 443 
[03:48:19] <wm-bb>	 <nokibsarkar>     ) ) )
[03:48:20] <wm-bb>	 <nokibsarkar>     proxy.header = ( "https-remap" => "enable" )
[03:48:22] <wm-bb>	 <nokibsarkar> }
[03:48:23] <wm-bb>	 <nokibsarkar> # 3. SPA Routing: Try files, else fallback to index.html
[03:48:25] <wm-bb>	 <nokibsarkar> # This excludes the /api/ path so it doesn't interfere with the proxy
[03:48:26] <wm-bb>	 <nokibsarkar> $HTTP["url"] !~ "^/api/" {
[03:48:28] <wm-bb>	 <nokibsarkar>     url.rewrite-if-not-file = (
[03:48:29] <wm-bb>	 <nokibsarkar>         "^/(.*)$" => "/index.html"
[03:48:31] <wm-bb>	 <nokibsarkar>     )
[03:48:32] <wm-bb>	 <nokibsarkar> }```
[03:48:34] <wm-bb>	 <nokibsarkar> but I am getting getting 500 even when i expect 404 at `/api/random string`.
[03:54:00] <wm-bb>	 <jeremy_b> I remind you please use a pastebin, your 1 message in telegram was 23 messages on irc.
[03:54:42] <wm-bb>	 <jeremy_b> I don't currently see any pins here. and somehow I don't have pinning rights. for the pastebin reminder
[04:13:14] <wm-bb>	 <nemoralis> https://tools-static.wmflabs.org/bridgebot/06b0cecc/file_79238.jpg
[04:14:16] <wm-bb>	 <jeremy_b> I must have somehow hidden them (for me) and idk how to unhide
[05:33:30] <wm-bb>	 <nokibsarkar> oops. I thought it was small enough to be pasted directly. (re @jeremy_b: I remind you please use a pastebin, your 1 message in telegram was 23 messages on irc.)
[05:34:56] <wm-bb>	 <nokibsarkar> My configuration is here: https://phabricator.wikimedia.org/P90295
[05:35:17] <wm-bb>	 <jeremy_b> even if that were all on line it's way more than could fit in a single IRC message. in this case every new line is a new message.
[05:36:01] <wm-bb>	 <jeremy_b> you could try some debugging. find an error message to go with the 500.
[05:36:15] <wm-bb>	 <nokibsarkar> i tried, but could not find any error log
[05:42:05] <wm-bb>	 <nokibsarkar> The configuration is located at `campwiz-dev` tool.
[08:16:21] <dcaro>	 nokibsarkar: lighttpd porxy seems not to support https backends "lighttpd mod_proxy is a reverse proxy. lighttpd mod_proxy does not currently support SSL/TLS connections to the backend server."
[08:16:27] <dcaro>	 from https://redmine.lighttpd.net/projects/lighttpd/wiki/Mod_proxy
[08:17:02] <dcaro>	 and you are forwarding traffic to port 443
[08:18:08] <dcaro>	 we do enforce https traffic at the proxy level (you just get a redirect to https), so you might not be able to use lighttpd to do the proxying for you
[08:29:25] <dcaro>	 you might be able to reach the other tool internally from k8s using http://campwiz-backend-dev.tool-campwiz-backend-dev.svc:8000
[10:23:32] <godog>	 !log trove switch mariatest1 to transient quorum queues - T421857
[10:23:40] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Trove/SAL
[10:23:40] <stashbot>	 T421857: Move trove DB instances to rabbitmq transient quorum queues - https://phabricator.wikimedia.org/T421857
[10:33:04] <Raine>	 o/, could someone from WMCS let me know whether hot-swapping a DB table under your feet is a problem? :D Context is https://phabricator.wikimedia.org/T419980 
[10:33:57] <Raine>	 and TLDR is that for the ICU upgrade, on some wikis we will temporarily duplicate the categorylinks table, write the new sort keys into it beforehand, and then swap the tables during the upgrade
[10:39:16] <dhinus>	 Raine: I can't think of a specific issue with that, worst case we'll have to recreate the views pointing to that table
[10:39:25] <dhinus>	 but it can be done quickly
[10:40:10] <Raine>	 dhinus: thanks, great. Would you (plural you, i.e. someone from WMCS) like to coordinate and be around for the swap in case the views need to be recreated?
[10:40:36] <dhinus>	 Raine: yes that would be ideal
[10:41:39] <Raine>	 alrighty, I'll ping you here and on task once we have a concrete date, which will hopefully be today
[10:41:41] <Raine>	 thank you!
[10:42:01] <dhinus>	 yw!
[12:51:29] <wm-bb>	 <divinely_dairylea> Looking for trusted service providers?DOSKA.CC brings together offers from various categories:hosting, proxies, services, software, and much more.✔️ Convenient catalog✔️ Fast search✔️ Seller reviews and ratingsVisit: DOSKA.CCNews Channel: https://t.me/+YxSY2OW6QvtlMjAx : https://tools-static.wmflabs.org/bridgebot/15ff0f89/file_79247.jpg
[13:40:00] <andrewbogott>	 !log admin upgrading spicerack on cloudcumin1001
[13:40:06] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Admin/SAL
[14:42:57] <andrewbogott>	 !log tools replacing etcd nodes with bookworm-based VMs
[14:43:01] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL
[15:32:02] <wm-bb>	 <nokibsarkar> I am getting the following error while running `toolforge webservice buildservice start --mount all`
[15:32:02] <wm-bb>	 <nokibsarkar> https://phabricator.wikimedia.org/P90317
[15:35:19] <wm-bb>	 <jeremy_b> please file a task (re @nokibsarkar: I am getting the following error while running toolforge webservice buildservice start --mount all
[15:35:20] <wm-bb>	 <jeremy_b> https://phabricator.wikimedia...)
[15:57:23] <wm-bot>	 !log difronzo@tools-bastion-15 tools.wikivisage deployed v0.7.6 (by DiFronzo on GitHub)
[15:57:25] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.wikivisage/SAL
[15:57:33] <wm-bb>	 <jeremy_b> and tell us the number (re @jeremy_b: please file a task)
[16:00:13] <dcaro>	 nokibsarkar does it happen with all commands, or only that one? that seems connection to k8s, so should affect everything
[17:12:39] <wm-bb>	 <nokibsarkar> apparently, it happened once, then `job is running`. to confirm, I stopped the job, then start command gave the error twice. then `job is running`.
[17:25:14] <wm-bb>	 <nokibsarkar> T422538
[17:27:28] <wm-bb>	 <nokibsarkar> @jeremy_b
[17:31:52] <Raine>	 dhinus: FYI, we will not be doing the table swap for the ICU upgrade after all, I will update https://phabricator.wikimedia.org/T419049 with the reasoning for the decision by end of day.
[17:32:09] <wm-bb>	 <nokibsarkar> Also, apparently I cannot connect with http://campwiz-backend-beta.tool-campwiz-backend-beta.svc:8000 using .lighttpd.conf . My configuration is below: P90319
[17:32:14] <Raine>	 (end of day in _some_ timezone for sure :D)
[18:18:29] <andrewbogott>	 nokibsarkar, I don't understand your question about  http://campwiz-backend-beta.tool-campwiz-backend-beta.svc:8000; that doesn't look like a valid url to me
[18:34:25] <dhinus>	 R.aine: ack, thanks for the update!
[18:34:59] <wm-bb>	 <jeremy_b> andrewbogott: hostname was suggested in this channel slightly over 10 hours ago (re @wmtelegram_bot: <andrewbogott> nokibsarkar, I don't understand your question about  http://campwiz-backend-beta.tool-campwiz-backend-beta.svc:80...)
[18:36:51] <andrewbogott>	 oh, I see, as an internal url
[18:37:08] <andrewbogott>	 in that case I don't have much to offer in terms of help :(
[19:58:14] <bd808>	 @nokibsarkar: I have something similar to your cross-namespace backend link in wikibugs and the hostname that works is "gitlab-webhooks.tool-gitlab-webhooks.svc.tools.local". The pattern there is `$TOOL.tool-$TOOL.svc.tools.local`. Try adding "tools.local" to your proxied hostname.
[20:00:35] <bd808>	 @nokibsarkar: Alternately, you could try using a reverse proxy container -- https://wikitech.wikimedia.org/wiki/Tool:Containers#Reverse_proxy_container
[21:11:47] <A_smart_kitten>	 q: out of interest, how easy would it be for Cloud VPS admins to e.g. look at mail-logs to see if emails are being successfully sent to specific email addresses from a specific Cloud VPS project/'from'-address?
[21:12:14] <A_smart_kitten>	 (the context is T422559. at first glance, at least to me, it's unclear whether there's an issue on the test-phabricator side of things or on the Cloud VPS side of things)
[21:12:15] <stashbot>	 T422559: The test Phabricator instance doesn't seem to be successfully sending emails to @wikimedia.org addresses - https://phabricator.wikimedia.org/T422559
[21:18:24] <taavi>	 A_smart_kitten: define 'successfully'
[21:19:03] <A_smart_kitten>	 taavi: the emails aren't getting through to the @wikimedia.org email accounts
[21:21:34] <taavi>	 in that case I can see https://phabricator.wikimedia.org/T422559#11796709
[21:22:21] <A_smart_kitten>	 thanks taavi :) appreciate you taking a look