[00:00:11] PROBLEM - wiki.fbpml.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.fbpml.org All nameservers failed to answer the query. [00:00:30] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.84, 6.02, 6.08 [00:01:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 31.67, 24.81, 24.79 [00:02:17] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.80, 3.51, 3.07 [00:04:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.50, 3.12 [00:06:07] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.01, 3.62, 3.20 [00:07:08] PROBLEM - wiki.fbpml.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns3.hostnetbv.nl.', 'ns2.hostnetbv.com.', 'ns1.hostnet.nl.'], 'CNAME': 'fbpml.miraheze.org.'} [00:08:03] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.80, 3.65, 3.26 [00:11:52] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.86, 2.91, 3.06 [00:18:01] I keep experiencing exceptions when creating wikis [00:18:02] [210d64ab56ab85323a604918] 2021-11-25 00:17:04: Fatal exception of type "JobQueueError" [00:19:18] I got this error previously and got it just now [00:22:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.70, 3.33, 3.21 [00:24:22] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 6.21, 4.19, 3.53 [00:26:17] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.86, 3.83, 3.49 [00:37:47] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.93, 3.17, 3.33 [01:22:23] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.96, 3.42, 3.30 [01:23:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.91, 20.85, 23.85 [01:24:18] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.61, 3.75, 3.42 [01:26:13] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.98, 3.43, 3.35 [01:28:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.32, 4.37, 5.80 [01:30:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.16, 4.96, 5.83 [01:34:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.12, 5.19, 5.75 [01:37:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.04, 20.84, 21.67 [01:37:43] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.21, 3.90, 3.63 [01:38:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.96, 5.59, 5.74 [01:39:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.67, 19.28, 21.02 [01:40:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.62, 4.86, 5.46 [01:43:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.07, 18.31, 20.20 [01:43:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.62, 3.55, 3.61 [01:44:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.04, 4.08, 5.02 [01:45:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.36, 3.82, 3.69 [01:47:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.29, 3.64, 3.64 [01:48:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.68, 5.45, 5.36 [01:52:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.94, 5.39, 5.38 [02:00:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 11.91, 7.54, 6.16 [02:01:07] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.22, 20.87, 20.32 [02:03:01] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 19.78, 20.31, 20.17 [02:04:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.69, 5.97, 5.88 [02:07:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.70, 3.07, 3.35 [02:10:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.19, 5.28, 5.50 [02:11:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.13, 4.11, 3.70 [02:12:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.01, 4.86, 5.33 [02:14:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.65, 4.21, 5.04 [02:17:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.76, 3.89, 3.76 [02:22:54] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.84, 9.62, 6.67 [02:23:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.01, 4.53, 4.04 [02:24:52] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.26, 7.63, 6.29 [02:26:50] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.41, 6.48, 6.02 [02:29:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.47, 3.75, 3.85 [02:43:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.58, 3.66, 3.61 [02:45:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.30, 3.45, 3.54 [02:47:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.69, 3.98, 3.72 [02:49:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.49, 3.69, 3.64 [02:51:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.65, 4.41, 3.93 [02:51:58] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.85, 21.33, 18.43 [02:53:53] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.04, 19.92, 18.28 [02:55:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.49, 3.73, 3.77 [03:05:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.39, 3.47 [03:07:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.50, 3.50 [03:09:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.07, 3.79, 3.61 [03:11:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.05, 19.60, 17.62 [03:11:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.23, 3.74, 3.63 [03:12:13] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 6.00, 5.49, 4.76 [03:14:08] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.55, 6.37, 5.20 [03:15:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.95, 20.25, 18.67 [03:15:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.79, 2.99, 3.32 [03:16:04] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.67, 5.57, 5.03 [03:18:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.61, 4.87, 4.84 [03:31:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.60, 3.52, 3.35 [03:35:32] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.90, 3.05, 3.22 [03:39:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 4.00, 3.54, 3.37 [03:41:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.25, 3.17, 3.27 [03:46:28] PROBLEM - cp15 Current Load on cp15 is CRITICAL: CRITICAL - load average: 2.53, 1.69, 1.00 [03:47:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.53, 3.46, 3.40 [03:48:28] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.60, 1.26, 0.92 [03:49:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.99, 3.00, 3.24 [03:51:17] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.31, 6.97, 5.02 [03:51:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.28, 19.93, 16.98 [03:52:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.57, 4.66, 4.12 [03:53:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.25, 6.50, 5.08 [03:53:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.82, 20.07, 17.39 [03:54:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.71, 6.18, 4.75 [03:55:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.01, 21.32, 18.15 [03:57:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.79, 19.97, 18.04 [04:00:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.66, 5.93, 5.19 [04:00:29] PROBLEM - wiki.phgalaxy.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.phgalaxy.eu.org' expires in 15 day(s) (Sat 11 Dec 2021 03:53:47 GMT +0000). [04:01:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.85, 22.56, 19.44 [04:02:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.87, 6.53, 5.50 [04:02:39] PROBLEM - pwiki.drydraytonvillagehall.org.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pwiki.drydraytonvillagehall.org.uk' expires in 15 day(s) (Sat 11 Dec 2021 03:54:50 GMT +0000). [04:03:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.14, 21.94, 19.64 [04:04:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.28, 5.77, 5.36 [04:05:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.48, 19.65, 19.06 [04:08:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.73, 4.16, 4.79 [04:09:56] PROBLEM - usaha.click - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'usaha.click' expires in 15 day(s) (Sat 11 Dec 2021 04:03:56 GMT +0000). [04:12:23] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.58, 3.56, 3.39 [04:14:17] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.79, 3.26, 3.30 [04:20:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.42, 5.08, 4.72 [04:21:59] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.87, 3.62, 3.38 [04:22:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.01, 5.22, 4.81 [04:24:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.24, 5.43, 4.95 [04:26:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.11, 4.85, 4.79 [04:27:44] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.63, 2.90, 3.16 [05:11:20] hello! could anyone look up the exception with ID e6b0474c4b670fb2929467e4 ? This is on spcodex.wiki (dbname: solarawiki). The error happens when I attempt to create a new page via PageForms, i.e. at https://spcodex.wiki/wiki/Form:New_venue [05:11:21] [url] Form:New venue - SPCodex | spcodex.wiki [05:12:08] `[e6b0474c4b670fb2929467e4] 2021-11-25 05:09:44: Fatal exception of type "Error"` [05:13:15] some software update broke this, and it may only require simple fixing on my end on the wiki [05:14:27] obviously I can't tell from this message alone. I don't see any similar reports on mw:Extension talk:Page Forms [05:21:58] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.75, 3.64, 3.17 [05:22:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.67, 5.78, 4.61 [05:23:53] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.68, 3.23, 3.08 [05:24:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.85, 5.24, 4.58 [05:26:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.87, 4.47, 4.37 [05:32:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.93, 5.13, 4.68 [05:34:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.43, 4.98, 4.69 [05:35:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.03, 20.10, 18.04 [05:36:24] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.34, 3.41, 3.28 [05:37:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.75, 18.36, 17.63 [05:42:09] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.51, 3.73, 3.44 [05:44:04] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.44, 3.57, 3.42 [05:45:23] PROBLEM - wiki.konjitownmc.cf - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.konjitownmc.cf' expires in 7 day(s) (Fri 03 Dec 2021 05:41:46 GMT +0000). [05:45:26] PROBLEM - commons.polandballwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'commons.polandballwiki.com' expires in 7 day(s) (Fri 03 Dec 2021 05:39:44 GMT +0000). [05:45:59] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.38, 3.24, 3.32 [05:49:43] PROBLEM - wiki.digitalcandela.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.digitalcandela.com' expires in 7 day(s) (Fri 03 Dec 2021 05:43:28 GMT +0000). [05:53:41] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.63, 3.43, 3.35 [05:57:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.53, 3.22, 3.30 [05:58:10] musikanimal, can reproduce, will file Phabricator ticket [05:58:30] dmehus: thanks! I just filed https://phabricator.miraheze.org/T8337 hehe [05:58:31] [url] ⚓ T8337 PageForms no longer submit on spcodex.wiki | phabricator.miraheze.org [06:00:40] musikanimal, oh thanks :) [06:00:46] I'll note that I can reproduce [06:01:04] heh it's 8337...that's almost l33t speak :P [06:01:12] too bad it wasn't 7337 [06:06:35] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.83, 6.26, 4.01 [06:08:35] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 6.78, 6.57, 4.41 [06:14:04] RECOVERY - db12 Disk Space on db12 is OK: DISK OK - free space: / 50515 MB (11% inode=98%); [06:15:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.41, 3.09, 2.99 [06:17:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.15, 3.41, 3.12 [06:19:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.54, 2.95, 2.97 [06:20:04] PROBLEM - db12 Disk Space on db12 is WARNING: DISK WARNING - free space: / 46525 MB (10% inode=98%); [07:29:46] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.11, 5.98, 4.28 [07:30:01] PROBLEM - mw12 APT on mw12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:31:44] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.59, 5.71, 4.40 [07:31:57] RECOVERY - mw12 APT on mw12 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [07:52:04] RECOVERY - db12 Disk Space on db12 is OK: DISK OK - free space: / 51164 MB (11% inode=98%); [08:08:04] PROBLEM - db12 Disk Space on db12 is WARNING: DISK WARNING - free space: / 48510 MB (10% inode=98%); [08:31:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 2.96, 2.43 [08:33:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.64, 2.37, 2.28 [09:25:37] why did the Upload File link disappeared from the Sidebar of all wikis I've visited [09:34:59] PROBLEM - stardewmodding.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'stardewmodding.wiki' expires in 7 day(s) (Fri 03 Dec 2021 09:32:10 GMT +0000). [09:36:49] PROBLEM - www.stardewmodding.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'stardewmodding.wiki' expires in 7 day(s) (Fri 03 Dec 2021 09:32:10 GMT +0000). [09:45:14] ^ [11:29:39] PROBLEM - www.trollpasta.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'trollpasta.com' expires in 15 day(s) (Sat 11 Dec 2021 11:25:32 GMT +0000). [11:32:48] PROBLEM - trollpasta.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'trollpasta.com' expires in 15 day(s) (Sat 11 Dec 2021 11:25:32 GMT +0000). [11:42:13] PROBLEM - repository.archiopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'repository.archiopedia.org' expires in 15 day(s) (Sat 11 Dec 2021 11:35:47 GMT +0000). [12:09:20] [02miraheze/GlobalNewFiles] 07translatewiki pushed 031 commit to 03master [+0/-0/±4] 13https://git.io/JMI2l [12:09:22] [02miraheze/GlobalNewFiles] 07translatewiki 0353b69aa - Localisation updates from https://translatewiki.net. [12:09:23] [url] Main page - translatewiki.net | translatewiki.net [12:09:23] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±9] 13https://git.io/JMI28 [12:09:25] [02miraheze/DataDump] 07translatewiki 035230ffe - Localisation updates from https://translatewiki.net. [12:09:26] [url] Main page - translatewiki.net | translatewiki.net [12:09:26] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+1/-0/±42] 13https://git.io/JMI24 [12:09:28] [02miraheze/CreateWiki] 07translatewiki 033f42e44 - Localisation updates from https://translatewiki.net. [12:09:29] [url] Main page - translatewiki.net | translatewiki.net [12:09:29] [02miraheze/MatomoAnalytics] 07translatewiki pushed 031 commit to 03master [+0/-0/±14] 13https://git.io/JMI2B [12:09:31] [02miraheze/MatomoAnalytics] 07translatewiki 03eb1cbd4 - Localisation updates from https://translatewiki.net. [12:09:32] [url] Main page - translatewiki.net | translatewiki.net [12:09:32] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JMI2R [12:09:34] [02miraheze/WikiDiscover] 07translatewiki 03e04acca - Localisation updates from https://translatewiki.net. [12:09:35] [url] Main page - translatewiki.net | translatewiki.net [12:27:45] miraheze/DataDump - translatewiki the build passed. [12:28:23] miraheze/GlobalNewFiles - translatewiki the build passed. [12:29:20] miraheze/CreateWiki - translatewiki the build passed. [12:30:21] miraheze/WikiDiscover - translatewiki the build passed. [12:30:37] miraheze/MatomoAnalytics - translatewiki the build passed. [14:20:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.57, 7.44, 5.87 [14:22:46] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.97, 6.24, 5.62 [14:38:22] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.51, 2.95, 2.63 [14:38:54] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.49, 5.66, 4.13 [14:40:18] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.91, 2.96, 2.67 [14:42:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.06, 6.00, 4.67 [14:46:35] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.19, 5.97, 4.89 [14:48:30] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.86, 5.58, 4.88 [14:50:03] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.72, 19.41, 17.96 [14:50:26] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.57, 5.83, 5.05 [14:51:58] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.10, 18.82, 17.95 [14:52:21] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.33, 5.55, 5.03 [14:56:12] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.52, 5.06, 4.97 [15:08:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.30, 6.36, 5.20 [15:10:46] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.00, 5.60, 5.08 [16:23:15] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.22, 3.45, 3.19 [16:24:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.37, 5.35, 4.59 [16:25:10] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.35, 3.22, 3.14 [16:28:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.57, 4.28, 4.36 [16:29:01] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.58, 3.31 [16:32:51] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.47, 3.96, 3.49 [16:34:46] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.31, 3.37, 3.34 [16:38:37] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.88, 3.54, 3.41 [16:40:32] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.71, 4.05, 3.61 [16:42:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.54, 3.42, 3.43 [16:44:22] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.24, 3.34, 3.40 [16:50:12] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.26, 19.23, 17.93 [16:52:07] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 19.07, 19.27, 18.11 [16:52:50] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.18, 6.39, 5.32 [16:54:48] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.04, 5.72, 5.19 [16:58:46] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.64, 8.14, 6.25 [16:59:45] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 30.48, 22.80, 19.73 [17:01:40] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.54, 20.31, 19.20 [17:01:58] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 22.09, 20.06, 16.62 [17:02:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.67, 7.68, 6.66 [17:03:58] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 14.26, 17.62, 16.12 [17:04:46] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.02, 6.34, 6.28 [17:25:43] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.68, 2.97, 2.83 [17:27:38] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.88, 2.89, 2.82 [17:43:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.51, 3.40, 3.07 [17:45:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.63, 3.18, 3.04 [17:49:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.06, 3.81, 3.34 [17:51:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.05, 3.60, 3.33 [17:53:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.92, 3.27, 3.23 [18:07:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.94, 3.72, 3.40 [18:09:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.30, 3.59, 3.40 [18:11:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.91, 3.96, 3.55 [18:13:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.42, 3.43, 3.41 [18:15:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.23, 3.12, 3.31 [18:21:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.74, 18.30, 16.48 [18:23:27] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.63, 17.51, 16.39 [18:50:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.82, 4.82, 3.99 [18:54:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.83, 4.72, 4.17 [19:46:19] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.65, 4.98, 4.40 [19:47:31] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.10, 3.35, 3.00 [19:49:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.31, 3.22, 2.98 [19:49:32] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.68, 21.45, 18.86 [19:51:26] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.97, 20.67, 18.87 [19:53:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.36, 19.96, 18.84 [19:56:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.10, 6.36, 5.24 [19:57:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.06, 22.95, 20.20 [19:59:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.98, 23.24, 20.67 [19:59:27] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.92, 7.30, 6.16 [20:01:25] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.87, 6.27, 5.91 [20:02:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.87, 5.94, 5.43 [20:04:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.74, 6.37, 5.65 [20:06:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.24, 5.65, 5.48 [20:09:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.79, 19.87, 20.20 [20:11:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.64, 3.05, 2.85 [20:12:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.27, 6.14, 5.66 [20:13:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.50, 22.34, 21.25 [20:13:47] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.37, 2.81, 2.79 [20:14:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.14, 5.62, 5.52 [20:16:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 5.69, 6.02, 5.70 [20:18:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.65, 5.78, 5.65 [20:20:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.40, 6.64, 5.96 [20:21:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.97, 23.90, 22.15 [20:25:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.13, 22.49, 22.15 [20:27:44] PROBLEM - lcn.zfc.id.lv - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/d [20:27:45] ackages/dns/resolver.py", line 1002, in query raise NXDOMAIN(qnames=qnames_to_try, responses=nxdomain_responses)dns.resolver.NXDOMAIN: None of DNS query names exist: lcn.zfc.id.lv., lcn.zfc.id.lv. [20:29:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.15, 23.64, 22.72 [20:33:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.08, 22.35, 22.64 [20:34:35] RECOVERY - lcn.zfc.id.lv - reverse DNS on sslhost is OK: SSL OK - lcn.zfc.id.lv reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:49:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.98, 22.91, 22.12 [20:53:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.90, 22.04, 22.07 [20:57:16] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.13, 7.50, 5.96 [20:58:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.48, 7.35, 5.83 [20:59:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.24, 6.17, 5.66 [20:59:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.53, 21.67, 21.76 [21:00:46] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.99, 6.56, 5.73 [21:01:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 15.31, 19.53, 21.00 [21:05:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.56, 18.00, 20.14 [21:10:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.55, 4.79, 5.91 [21:18:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.68, 3.43, 4.83 [21:46:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.09, 5.39, 4.39 [21:48:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.83, 5.12, 4.41 [21:50:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.53, 4.94, 4.44 [21:56:14] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.77, 21.03, 18.40 [22:00:03] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.06, 20.10, 18.67 [22:03:40] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.94, 5.58, 5.03 [22:05:35] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.73, 5.98, 5.24 [22:07:30] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.37, 5.73, 5.23 [22:17:07] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.75, 6.20, 5.57 [22:19:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.98, 6.00, 5.56 [22:20:58] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.37, 6.30, 5.70 [22:23:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.80, 21.91, 19.79 [22:31:25] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 12.95, 19.34, 19.80 [22:36:21] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.87, 5.51, 5.90 [22:52:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.89, 4.54, 5.04 [22:56:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.17, 5.38, 5.31 [22:59:42] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.73, 6.38, 4.78 [23:00:08] !log [@mw11] starting deploy of {'l10nupdate': True} to all [23:00:09] !log [@test3] starting deploy of {'l10nupdate': True} to skip [23:00:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:01:40] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.71, 7.28, 5.28 [23:03:26] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 40.58, 26.38, 21.04 [23:03:51] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 14.86, 9.99, 6.04 [23:03:58] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 36.43, 24.25, 17.72 [23:03:59] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.10, 6.85, 4.82 [23:04:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.57, 5.71, 5.50 [23:04:21] PROBLEM - mw11 APT on mw11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:05:43] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 54.38.211.199/cpweb [23:05:58] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.84, 7.18, 5.22 [23:06:02] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 12.02, 10.25, 6.59 [23:06:07] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:06:19] RECOVERY - mw11 APT on mw11 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:06:30] !log [@mw11] finished deploy of {'l10nupdate': True} to all - FAIL: [35072, 0, 0, 0, 0, 0, 0, 0, 0] in 382s [23:06:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:07:16] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.25, 7.81, 5.54 [23:07:35] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.46, 7.62, 6.20 [23:07:37] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [23:07:58] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 17.62, 21.61, 18.29 [23:07:58] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.50, 6.58, 5.23 [23:09:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.74, 6.79, 5.45 [23:09:33] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.43, 7.69, 6.36 [23:09:49] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.33, 7.52, 6.32 [23:10:02] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.32, 7.26, 6.18 [23:11:06] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:11:58] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.43, 19.70, 18.25 [23:13:05] PROBLEM - mw9 APT on mw9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:13:16] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.73, 8.87, 6.58 [23:13:49] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 3.64, 5.76, 5.90 [23:14:02] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.26, 6.28, 6.03 [23:15:08] RECOVERY - mw9 APT on mw9 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:15:27] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.66, 8.00, 7.00 [23:16:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.78, 6.00, 5.89 [23:16:22] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.36, 2.96 [23:17:25] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.62, 6.46, 6.56 [23:18:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.49, 5.97, 5.88 [23:20:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.79, 5.92, 5.88 [23:20:12] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.50, 3.23, 3.03 [23:22:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.04, 5.85, 5.85 [23:23:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.76, 22.29, 23.50 [23:23:56] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.54, 7.45, 6.48 [23:24:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.51, 5.52, 5.71 [23:25:16] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.02, 7.09, 7.12 [23:25:56] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.02, 5.89, 6.02 [23:27:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.60, 5.85, 6.66 [23:30:21] !log [@test3] finished deploy of {'l10nupdate': True} to skip - SUCCESS in 1812s [23:30:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:32:01] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.64, 4.42, 5.05 [23:36:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.30, 5.60, 5.48 [23:39:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.94, 23.06, 22.05 [23:40:01] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.52, 5.54, 5.44 [23:41:16] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.80, 6.88, 6.48 [23:41:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.44, 22.76, 22.10 [23:42:02] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.09, 5.39, 5.40 [23:43:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 2.59, 5.40, 5.98 [23:43:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.45, 23.39, 22.39 [23:44:02] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.83, 6.78, 5.93 [23:45:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.62, 22.47, 22.18 [23:49:25] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.63, 23.71, 22.57