[00:00:51] PROBLEM - wiki.sofurry.games - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:08:49] It's down again. [00:50:26] Miraheze is down once again [00:50:42] Yep. Error 503 error. [00:50:51] https://phabricator.miraheze.org/ is also down with a 502 error. [00:51:04] Known we are looking into the problem [00:51:41] ポーランドボールwikiを開くことができないのですが、何故ですか？ [00:52:08] Google Translate (Guest 66): Why can't I open the Polish Ball wiki? [00:53:13] The servers are currently down. The admins are investigating the outage. (Translated with Google translate) [00:53:13] 現在サーバーがダウンしております。 管理者は障害を調査中です。 (Google翻訳で翻訳しました) [00:55:36] The admins are aware of the Miraheze outage. They are investigating the issue, with no current ETA. [01:15:19] ポーランドボールwikiで開けるページと開くことのできないページがあります。 [01:30:41] hi [01:38:59] Hey guys, Miraheze is down for me, I get a "503 Backend fetch failed" whenever I access the website [01:39:32] This is known. The admins have been investigating this issue. [02:10:48] 503 backend fetch failure, been like this for a while [02:11:04] just checking whats going on with that [02:47:12] Does anyone know when the backend will get back online? [02:47:57] or Phabricator? [02:49:52] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+1/-0/±3] 13https://github.com/miraheze/puppet/commit/6baefd70b144 [02:49:55] [02miraheze/puppet] 07The-Voidwalker 036baefd7 - proxy pass through bastion servers [02:49:57] [02puppet] 07The-Voidwalker created branch 03patch-bastproxy - 13https://github.com/miraheze/puppet [02:50:02] [02puppet] 07The-Voidwalker opened pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [02:50:30] Miraheze isn't working [02:51:25] Nope there is no ETA [02:53:15] I see [03:04:44] FWIW, this may be a perfect time to work on status.miraheze.org, since it's not in a helpful state right now anyway,and this is a golden opportunity to work on ways of detecting outages while an outage is happening (assuming y'all are blocked by upstream providers to fix any problems) [03:06:22] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/6baefd70b144...e31da2a0bd11 [03:06:23] [02miraheze/puppet] 07The-Voidwalker 03e31da2a - use ssl in bastion proxies [03:06:24] [02puppet] 07The-Voidwalker synchronize pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:07:16] Whats going on site [03:07:21] It crashed [03:07:52] does anyone have a working invite to the discord server? or are new members purposely barred from joining? [03:08:57] there's no way to verify who is legit because of the outage, so there's not a good way of inviting new folks [03:09:49] i see, so its a closed server im guessing. i was planning on requesting a wiki but i see the site is currently down [03:15:12] Hey there, any status on the current site outage? Both my wiki and the Metawiki are responding with 503 errors. [03:16:06] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e31da2a0bd11...c7ecf615207a [03:16:09] [02miraheze/puppet] 07The-Voidwalker 03c7ecf61 - Updated mediawiki.conf.erb [03:16:10] [02puppet] 07The-Voidwalker synchronize pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:16:23] actually i think discord invites are just down in general [03:16:47] I'm not talking Discord, I mean the proper website. [03:16:59] yeah, i was just referring to my own message above [03:17:06] the site is experiencing an outage i believe [03:17:15] We're investigating the outage, but no ETA on when things are coming back. This seems to be an upstream connectivity issue that out of our hands. [03:18:39] Got it, thanks! [03:19:03] Voidwalker - gotta love when it's not you, but the guys upstairs, right? /s [03:20:03] yup [03:20:33] why is miraheze not working? [03:20:37] lmfao [03:20:40] Brokey brokey no workey workey [03:20:47] server outage [03:21:06] pensive [03:21:33] For all y'all theorizing the discord invite is down: have a go at clicking on the invite on the "down" page. 't Works just fine! [03:21:43] Though, you're not missing out on much. [03:22:37] miraheze.org has it under the "What can I do?" section. [03:22:55] But again, hardly anything to see there [03:26:31] Guest60 Welcome in ;) [03:27:44] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/c7ecf615207a...e409d181e322 [03:27:47] [02miraheze/puppet] 07The-Voidwalker 03e409d18 - fix tabs and spaces inconsistencies [03:27:50] [02puppet] 07The-Voidwalker synchronize pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:33:05] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e409d181e322...6f65d27c7b4c [03:33:07] [02miraheze/puppet] 07The-Voidwalker 036f65d27 - use server name [03:33:10] [02puppet] 07The-Voidwalker synchronize pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:35:09] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03patch-bastproxy [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6f65d27c7b4c...ec81d1e0c0e1 [03:35:11] [02miraheze/puppet] 07The-Voidwalker 03ec81d1e - whitespace [03:35:14] [02puppet] 07The-Voidwalker synchronize pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:42:01] [02puppet] 07The-Voidwalker closed pull request 03#3505: proxy pass through bastion servers - 13https://github.com/miraheze/puppet/pull/3505 [03:42:03] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03master [+1/-0/±3] 13https://github.com/miraheze/puppet/compare/1638039c0af3...fd479bcd0802 [03:42:04] [02miraheze/puppet] 07The-Voidwalker 03fd479bc - proxy pass through bastion servers (#3505) [03:42:06] [02miraheze/puppet] 07The-Voidwalker deleted branch 03patch-bastproxy [03:42:09] [02puppet] 07The-Voidwalker deleted branch 03patch-bastproxy - 13https://github.com/miraheze/puppet [04:02:28] Error 503 Backend fetch failed, forwarded for 2407:7000:a23b:3c00:c5b6:3b42:84a6:7563, 127.0.0.1 [04:02:28] (Varnish XID 368536642) via cp25.miraheze.org at Sat, 18 Nov 2023 04:01:48 GMT. [04:08:54] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-12 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/9d31d8057948 [04:08:55] [02miraheze/puppet] 07paladox 039d31d80 - mediawiki: Allow bastion to connect temporarily [04:08:56] [02puppet] 07paladox created branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [04:08:59] [02puppet] 07paladox opened pull request 03#3506: mediawiki: Allow bastion to connect temporarily - 13https://github.com/miraheze/puppet/pull/3506 [04:09:21] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fd479bcd0802...d5b4327b8ba7 [04:09:23] [02miraheze/puppet] 07The-Voidwalker 03d5b4327 - add bastion to mw firewall [04:09:40] [02puppet] 07paladox closed pull request 03#3506: mediawiki: Allow bastion to connect temporarily - 13https://github.com/miraheze/puppet/pull/3506 [04:09:42] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-12 [04:09:44] [02puppet] 07paladox deleted branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [04:12:23] the irc is working [04:13:07] Also why was I timed out on the discord??? [04:22:09] Back up now [04:22:51] hi why am i timed out on the miraheze discord?? [04:23:23] No idea, I'm not a discord user. [04:23:59] Zppix why am i timed out?? [04:41:33] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d5b4327b8ba7...4a18bedf7064 [04:41:35] [02miraheze/puppet] 07The-Voidwalker 034a18bed - fix problem with bastions upstream [04:42:04] why am i timed out on the discord server??? [04:43:31] also only some images are loading on my wiki [04:44:40] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/4a18bedf7064...2e164c8a7c1f [04:44:42] [02miraheze/puppet] 07The-Voidwalker 032e164c8 - remove bastproxy config [04:45:39] is anyone going to respond to me?? [04:46:21] Youre timed out for being unhelpful and disruptive in the discord [04:46:30] Now please stop disrupting this channel [04:47:59] I was not being disruptive on discord, all I did was ask some questions & make conversation while we were waiting, the only reason i keep messaging in this channel is because no one responded. [04:51:11] [02miraheze/mw-config] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3eed014c95fe...5413c6b2545a [04:51:12] [02miraheze/mw-config] 07The-Voidwalker 035413c6b - add bastion servers to $wgCdnServersNoPurge [04:51:57] [02miraheze/mw-config] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5413c6b2545a...d5a04fe4343d [04:52:00] miraheze/mw-config - The-Voidwalker the build has errored. [04:52:00] [02miraheze/mw-config] 07The-Voidwalker 03d5a04fe - fix [04:52:54] miraheze/mw-config - The-Voidwalker the build passed. [04:56:22] For IRC only people: We've deployed a temporary fix to allow users to access their wikis while our provider's Network Operations Team fixes their network. However, please be aware that wikis may be slower than usual, and images loaded from Wikimedia Commons will not load. [04:57:32] On my wiki some of the images from commons load, but the miraheze icons & logos will not. [05:06:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-12 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/bb40b49aac66 [05:06:33] [02miraheze/puppet] 07paladox 03bb40b49 - Fix [05:06:34] [02puppet] 07paladox created branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [05:06:35] [02puppet] 07paladox opened pull request 03#3507: Fix - 13https://github.com/miraheze/puppet/pull/3507 [12:30:43] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/52e661fc4f13...b0a9192a9576 [12:30:46] [02miraheze/ssl] 07MirahezeSSLBot 03b0a9192 - Bot: Update SSL cert for beamngmodding.com [13:08:37] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2e164c8a7c1f...ea9f25c1e0f8 [13:08:38] [02miraheze/puppet] 07The-Voidwalker 03ea9f25c - fix minor issue [20:17:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/b0a9192a9576...9cad55de734f [20:18:00] [02miraheze/ssl] 07MirahezeSSLBot 039cad55d - Bot: Add SSL cert for wiki.vot.gg [20:18:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/9cad55de734f...efc760be6651 [20:18:21] [02miraheze/ssl] 07MirahezeSSLBot 03efc760b - Bot: Add SSL cert for wiki.aclevo.com [20:19:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/efc760be6651...ba7d1316cd4c [20:19:10] [02miraheze/ssl] 07MirahezeSSLBot 03ba7d131 - Bot: Add SSL cert for de.berlinwiki.org [20:20:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ssl/compare/ba7d1316cd4c...2d6096c016fa [20:20:08] [02miraheze/ssl] 07MirahezeSSLBot 032d6096c - Bot: Add SSL cert for fid.koymi.net [20:20:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/2d6096c016fa...4d2418e1bb4e [20:20:30] [02miraheze/ssl] 07MirahezeSSLBot 034d2418e - Bot: Add SSL cert for www.gengbaike.top [20:20:57] miraheze/ssl - MirahezeSSLBot the build has errored. [20:21:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/4d2418e1bb4e...30c5f6580458 [20:21:17] [02miraheze/ssl] 07MirahezeSSLBot 0330c5f65 - Bot: Add SSL cert for progressbar95.wiki [20:21:19] miraheze/ssl - MirahezeSSLBot the build has errored. [20:21:55] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/30c5f6580458...8118b80f8448 [20:21:57] [02miraheze/ssl] 07MirahezeSSLBot 038118b80 - Bot: Add SSL cert for en.clockup.wiki [20:22:04] miraheze/ssl - MirahezeSSLBot the build has errored. [20:22:44] miraheze/ssl - MirahezeSSLBot the build has errored. [20:22:56] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/8118b80f8448...85925732b12b [20:22:57] [02miraheze/ssl] 07MirahezeSSLBot 038592573 - Bot: Add SSL cert for wiki.themarkside.com [20:23:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/85925732b12b...68489c37c06c [20:23:23] [02miraheze/ssl] 07MirahezeSSLBot 0368489c3 - Bot: Add SSL cert for wiki.rlab.org.uk [20:23:52] miraheze/ssl - MirahezeSSLBot the build has errored. [20:24:14] miraheze/ssl - MirahezeSSLBot the build has errored. [20:29:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/68489c37c06c...77013a2ee7a9 [20:29:38] [02miraheze/ssl] 07MirahezeSSLBot 0377013a2 - Bot: Update SSL cert for wiki.widwwa.co.uk [20:30:33] miraheze/ssl - MirahezeSSLBot the build has errored. [21:31:13] looks like there are no bots working (except relay) for some reason [21:31:47] meaning lsbot, icingabot, logbot, and ircrcbot [21:54:35] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/commit/ebec3ba1febe [21:54:37] [02miraheze/ssl] 07MacFan4000 03ebec3ba - Delete certificates/archives.themarkside.com.crt [21:54:40] [02ssl] 07MacFan4000 created branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [21:54:42] [02ssl] 07MacFan4000 opened pull request 03#737: Delete certificates/archives.themarkside.com.crt - 13https://github.com/miraheze/ssl/pull/737 [21:55:31] miraheze/ssl - MacFan4000 the build has errored. [21:55:54] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ebec3ba1febe...0331ebf408b9 [21:55:56] [02miraheze/ssl] 07MacFan4000 030331ebf - Update certs.yaml [21:55:57] [02ssl] 07MacFan4000 synchronize pull request 03#737: Delete certificates/archives.themarkside.com.crt - 13https://github.com/miraheze/ssl/pull/737 [21:56:47] miraheze/ssl - MacFan4000 the build has errored. [21:57:12] [02ssl] 07MacFan4000 closed pull request 03#737: Delete certificates/archives.themarkside.com.crt - 13https://github.com/miraheze/ssl/pull/737 [21:57:15] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/77013a2ee7a9...28c320470e96 [21:57:18] [02miraheze/ssl] 07MacFan4000 0328c3204 - Delete certificates/archives.themarkside.com.crt (#737) [21:57:21] [02miraheze/ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 [21:57:22] [02ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [21:58:07] miraheze/ssl - MacFan4000 the build has errored. [21:59:36] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/28c320470e96...1fcc3fde51d3 [21:59:38] [02miraheze/ssl] 07MacFan4000 031fcc3fd - rm duplicate [23:01:49] Nooooooooo [23:38:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1fcc3fde51d3...7363bb2ab407 [23:38:03] [02miraheze/ssl] 07MirahezeSSLBot 037363bb2 - Bot: Update SSL cert for osfirsttimer.wiki [23:41:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7363bb2ab407...6d5f4687d209 [23:41:31] [02miraheze/ssl] 07MirahezeSSLBot 036d5f468 - Bot: Update SSL cert for partopedia.ru [23:41:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6d5f4687d209...692b839c21de [23:41:44] [02miraheze/ssl] 07MirahezeSSLBot 03692b839 - Bot: Update SSL cert for www.widedata.org