[00:05:17] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.08, 19.19, 16.60 [00:06:04] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.83, 8.04, 6.13 [00:07:18] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.55, 8.21, 6.40 [00:07:19] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.16, 22.15, 17.95 [00:07:23] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.42, 8.71, 7.03 [00:09:17] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.95, 7.45, 6.35 [00:09:19] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.80, 21.83, 18.30 [00:09:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.06, 7.65, 6.83 [00:10:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.14, 7.40, 6.34 [00:11:18] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.67, 23.60, 19.38 [00:13:13] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.06, 6.74, 6.29 [00:13:17] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.75, 21.85, 19.26 [00:15:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.52, 6.18, 6.50 [00:16:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.13, 6.25, 6.20 [00:17:17] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.23, 22.63, 20.13 [00:20:09] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.71, 8.14, 6.93 [00:20:50] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.94, 4.05, 3.29 [00:21:18] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.50, 23.42, 21.07 [00:22:07] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.66, 7.08, 6.68 [00:22:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.20, 3.59, 3.20 [00:23:11] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.96, 7.14, 6.59 [00:24:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.40, 6.44, 6.49 [00:25:11] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.46, 6.36, 6.37 [00:25:17] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.35, 21.85, 20.83 [00:27:21] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.15, 20.85, 20.54 [00:28:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.89, 3.29, 3.21 [00:31:21] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.05, 19.40, 20.09 [00:35:10] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.76, 6.81, 6.38 [00:37:09] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.43, 6.33, 6.26 [00:40:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.28, 3.43, 3.32 [00:46:43] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.20, 3.69, 3.43 [00:48:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.51, 3.61, 3.43 [00:50:42] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.77, 4.07, 3.63 [00:52:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.95, 3.59, 3.50 [01:00:43] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.34, 2.90, 3.27 [01:05:01] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.58, 7.48, 6.39 [01:05:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.52, 7.31, 6.04 [01:06:59] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.50, 6.53, 6.16 [01:07:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.30, 6.15, 5.77 [02:10:42] Internal errors reported on polandballwiki on the forums. I was able to reproduce these on https://polandball.miraheze.org/wiki/Special:WikiForum/oh_wow_forums after refreshing a few times [02:10:43] [url] User is blocked - Polandball Wiki | polandball.miraheze.org [02:43:38] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.18, 3.53, 3.14 [02:45:36] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.85, 2.81, 2.92 [03:23:36] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.56, 3.59, 3.04 [03:25:35] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.44, 3.14, 2.94 [03:57:35] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.21, 3.44, 3.18 [03:59:35] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.29, 3.30, 3.16 [06:05:52] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[php7.3-apcu] [06:35:41] [02puppet] 07RhinosF1 reviewed pull request 03#1885 commit - 13https://git.io/JEvfO [06:35:47] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:47:36] PROBLEM - bots1.miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Mon 06 Sep 2021 10:42:56 GMT +0000). [10:50:15] PROBLEM - bots.miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Mon 06 Sep 2021 10:42:56 GMT +0000). [10:50:29] PROBLEM - db1.miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Mon 06 Sep 2021 10:42:56 GMT +0000). [10:51:11] PROBLEM - miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Mon 06 Sep 2021 10:42:56 GMT +0000). [10:51:22] PROBLEM - tools1.miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Mon 06 Sep 2021 10:42:56 GMT +0000). [11:10:52] Word of warning, just got a 429 Varnish Rate Limit Exceeded when trying to perform a search. It went away now, but not sure it should be happening at all? [11:54:39] Not unless your wiki loads a lot of js [11:55:47] It's fairly rare for a human to see them [12:35:12] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.69, 6.89, 5.30 [12:37:11] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.69, 6.72, 5.44 [13:05:20] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.18, 6.96, 5.39 [13:07:18] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.32, 6.28, 5.33 [13:38:02] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.16, 3.20, 2.89 [13:40:00] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.03, 2.75, 2.77 [13:51:59] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.69, 3.42, 3.13 [13:53:59] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.46, 3.16, 3.07 [13:54:40] I haven't added any js to the wiki myself. I have enabled some of the extensions provided in the managewiki, but that's all [13:55:07] I just got another odd one, about how my session had been lost (but on the same page it clearly showed me as logged in) [14:05:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.87, 5.82, 4.82 [14:07:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.18, 5.08, 4.66 [14:12:41] If it goes away on refresh don't worry [14:33:36] https://cdn.discordapp.com/attachments/808001911868489748/878648030008594512/image0.png [14:33:40] I can’t save my pages [14:37:50] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.26, 3.49, 3.26 [14:39:53] I'm not worried at all, just reporting in case it was important/related to the maintenance [14:41:49] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.80, 3.30, 3.25 [14:45:37] Please file a task re the cant save question [15:07:35] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.93, 4.00, 3.31 [15:09:34] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.31, 3.30, 3.13 [15:22:19] Bugs too? [15:51:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 8.00, 6.23, 5.18 [15:53:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.39, 6.25, 5.32 [15:54:36] That's definitely a bug [18:02:51] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.42, 1.87, 1.33 [18:04:51] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.25, 1.52, 1.26 [18:19:18] PROBLEM - tep.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:39:28] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-3 [+0/-0/±1] 13https://git.io/JEUVA [18:39:30] [02miraheze/mw-config] 07RhinosF1 033e83dc9 - [REVERT: EMERGENCY] remove emergency measure [18:39:31] [02mw-config] 07RhinosF1 created branch 03RhinosF1-patch-3 - 13https://git.io/vbvb3 [18:40:16] [02mw-config] 07RhinosF1 opened pull request 03#4058: [REVERT: EMERGENCY] remove emergency measure - 13https://git.io/JEUwG [18:40:28] [02mw-config] 07RhinosF1 assigned pull request 03#4058: [REVERT: EMERGENCY] remove emergency measure - 13https://git.io/JEUwG [18:41:12] miraheze/mw-config - RhinosF1 the build passed. [18:44:13] PROBLEM - tep.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - tep.wiki All nameservers failed to answer the query. [18:45:37] [02mw-config] 07RhinosF1 closed pull request 03#4058: [REVERT: EMERGENCY] remove emergency measure - 13https://git.io/JEUwG [18:45:38] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JEUrV [18:45:40] [02miraheze/mw-config] 07RhinosF1 0314b1cfa - [REVERT: EMERGENCY] remove emergency measure (#4058) [18:45:41] [02mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-3 - 13https://git.io/vbvb3 [18:45:43] [02miraheze/mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-3 [18:46:33] miraheze/mw-config - RhinosF1 the build passed. [18:51:15] RECOVERY - tep.wiki - reverse DNS on sslhost is OK: SSL OK - tep.wiki reverse DNS resolves to cp13.miraheze.org - CNAME OK [18:51:35] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.45, 3.22, 2.76 [18:53:34] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.65, 2.64, 2.60 [19:01:28] RECOVERY - tep.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'tep.wiki' will expire on Wed 20 Oct 2021 04:26:11 GMT +0000. [19:05:10] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.09, 6.83, 5.65 [19:05:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.81, 5.86, 5.09 [19:07:09] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.18, 6.18, 5.56 [19:07:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.19, 5.46, 5.03 [19:35:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.98, 6.31, 5.47 [19:37:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.85, 5.73, 5.35 [20:34:51] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.33, 6.64, 5.44 [20:36:49] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.96, 6.13, 5.39 [20:46:51] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.23, 1.78, 1.31 [20:48:51] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.18, 1.63, 1.30 [20:50:43] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 10.98, 6.94, 5.56 [20:52:03] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.83, 7.13, 5.64 [20:52:06] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 11.25, 7.82, 5.88 [20:53:59] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.05, 6.92, 5.74 [20:54:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.14, 7.60, 6.05 [20:54:11] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 9.13, 6.57, 5.21 [20:54:40] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.12, 6.69, 5.84 [20:55:53] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.94, 6.52, 5.73 [20:56:04] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.63, 6.74, 5.91 [20:56:10] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.10, 6.45, 5.34 [21:05:34] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.58, 7.70, 6.44 [21:05:35] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.16, 7.64, 6.54 [21:06:05] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 7.52, 8.06, 6.77 [21:07:28] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.09, 7.14, 6.39 [21:07:34] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.34, 6.76, 6.36 [21:08:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.98, 6.98, 6.54 [21:09:23] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.97, 7.63, 6.64 [21:10:04] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.96, 7.57, 6.79 [21:11:32] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.46, 7.82, 6.88 [21:13:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.62, 7.79, 6.97 [21:13:30] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.32, 7.17, 6.75 [21:14:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.10, 7.72, 7.06 [21:17:26] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.05, 1.54, 1.23 [21:17:29] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.70, 6.80, 6.75 [21:18:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.53, 6.28, 6.65 [21:19:19] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.63, 7.90, 7.23 [21:19:23] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.03, 1.34, 1.19 [21:21:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.80, 7.08, 7.02 [21:23:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.36, 6.20, 6.70 [21:35:21] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 8.00, 7.39, 6.72 [21:36:31] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.76, 6.89, 6.65 [21:37:20] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.38, 6.64, 6.52 [21:38:29] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.45, 6.06, 6.38 [22:05:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.10, 5.83, 5.31 [22:07:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.49, 5.23, 5.15