[00:00:27] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.22, 5.20, 4.97 [00:03:04] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.14, 22.88, 21.29 [00:03:07] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:04:27] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.73, 5.77, 5.31 [00:05:04] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.12, 22.32, 21.28 [00:06:28] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.50, 6.36, 5.57 [00:07:04] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.83, 24.74, 22.29 [00:10:38] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.52, 3.72, 3.28 [00:12:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.05, 3.54, 3.27 [00:14:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.72, 3.26, 3.20 [00:28:24] PROBLEM - graylog2 Current Load on graylog2 is WARNING: WARNING - load average: 3.84, 3.30, 2.66 [00:30:24] RECOVERY - graylog2 Current Load on graylog2 is OK: OK - load average: 3.28, 3.30, 2.74 [00:33:05] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:54:44] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.45, 3.65, 3.35 [00:58:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.46, 3.59, 3.40 [01:00:27] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.25, 4.71, 5.93 [01:00:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.86, 3.89, 3.52 [01:02:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.68, 3.64, 3.46 [01:03:04] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.05, 18.42, 23.06 [01:06:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.86, 3.18, 3.30 [01:08:28] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.10, 3.34, 4.84 [01:09:04] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.29, 15.33, 20.33 [01:10:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.91, 4.28, 3.75 [01:12:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.07, 3.91, 3.68 [01:20:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.72, 3.79, 3.64 [01:28:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.06, 3.98, 3.88 [01:30:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.31, 4.31, 4.02 [01:44:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.30, 3.66, 3.92 [01:50:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.22, 3.94, 3.91 [01:52:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.63, 3.77, 3.85 [01:59:11] PROBLEM - cp15 Current Load on cp15 is CRITICAL: CRITICAL - load average: 2.32, 2.14, 1.28 [02:02:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.56, 2.88, 3.34 [02:03:11] PROBLEM - cp15 Current Load on cp15 is WARNING: WARNING - load average: 1.55, 1.88, 1.39 [02:05:11] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.72, 1.45, 1.29 [02:19:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.65, 3.53, 3.29 [02:21:42] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.75, 3.40 [02:29:16] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.78, 3.72, 3.59 [02:33:03] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.31, 3.67, 3.57 [02:34:57] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.76, 3.33, 3.46 [02:36:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.93, 3.13, 3.36 [02:40:38] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.53, 3.41, 3.43 [02:42:32] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.33, 3.68, 3.52 [02:44:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.70, 3.75, 3.57 [02:48:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.93, 3.15, 3.34 [02:52:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.47, 3.33, 3.36 [02:54:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.74, 3.17, 3.31 [03:08:00] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.07, 3.46, 3.38 [03:09:54] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.46, 3.22, 3.31 [03:19:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.77, 4.00, 3.59 [03:28:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.83, 3.81 [03:32:36] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.17, 3.91, 3.84 [03:34:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.00, 3.40, 3.65 [03:46:30] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.99, 2.91, 3.32 [05:20:51] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.98, 5.68, 3.13 [05:21:04] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.29, 17.90, 16.04 [05:21:11] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.43, 5.12, 3.56 [05:23:06] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 30.19, 22.64, 18.01 [05:25:04] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.37, 21.20, 18.01 [05:25:05] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.96, 5.70, 4.21 [05:30:51] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 0.61, 1.29, 1.95 [05:31:04] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.92, 19.96, 18.59 [05:32:54] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.86, 5.02, 4.58 [05:34:51] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.56, 0.88, 1.62 [05:44:11] PROBLEM - wiki.konjitownmc.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.konjitownmc.cf' expires in 15 day(s) (Fri 03 Dec 2021 05:41:46 GMT +0000). [05:46:29] PROBLEM - wiki.digitalcandela.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.digitalcandela.com' expires in 15 day(s) (Fri 03 Dec 2021 05:43:28 GMT +0000). [05:48:35] PROBLEM - commons.polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'commons.polandballwiki.com' expires in 15 day(s) (Fri 03 Dec 2021 05:39:44 GMT +0000). [06:00:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.40, 19.13, 17.90 [06:02:34] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.97, 16.91, 17.22 [06:04:23] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 9.04, 6.58, 3.89 [06:13:49] PROBLEM - db13 Current Load on db13 is CRITICAL: CRITICAL - load average: 11.04, 11.02, 6.34 [06:17:41] RECOVERY - db13 Current Load on db13 is OK: OK - load average: 2.38, 6.31, 5.44 [06:20:23] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 5.40, 7.90, 7.22 [06:26:45] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.86, 5.97, 4.68 [06:28:45] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.58, 6.09, 4.88 [06:34:23] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.13, 6.35, 6.74 [06:44:18] [02puppet] 07Universal-Omega reviewed pull request 03#2125 commit - 13https://git.io/J18JN [07:10:03] wow that's a lot of commits on #2125 for RhinosF1 [07:10:13] that looks like a CosmicAlpha PR :P [07:30:04] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.87, 20.25, 17.10 [07:32:02] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.36, 19.33, 17.16 [07:36:27] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.86, 4.93, 3.49 [07:38:27] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.65, 4.66, 3.54 [07:40:27] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.51, 4.83, 3.76 [07:41:48] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.36, 20.83, 18.94 [07:45:44] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.47, 18.54, 18.62 [07:53:29] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.06, 6.82, 6.31 [07:57:26] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 4.36, 6.05, 6.14 [08:08:54] [02ssl] 07Reception123 closed pull request 03#438: Fix mwcosmos.com redirect + icinga warning - 13https://git.io/J1WIp [08:08:55] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J18zP [08:08:57] [02miraheze/ssl] 07Universal-Omega 033b08491 - Fix mwcosmos.com redirect + icinga warning (#438) [08:21:10] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.88, 6.43, 6.17 [08:23:05] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.35, 6.03, 6.05 [08:23:14] dmehus: hey yeah, and actually I'm probably going to stop making such large PRs, and start using git more to push changes at once rather than so many PR commits. Guess we'll see if I actually do though. I do PRs from mobile often also, so in which case git is not an option. [08:23:21] [02dns] 07Reception123 closed pull request 03#228: Add www to famedata.org zone - 13https://git.io/J1Wq1 [08:23:23] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J18wS [08:23:24] [02miraheze/dns] 07Universal-Omega 037afe4e7 - Add www to famedata.org zone (#228) [08:25:24] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [08:28:58] RECOVERY - www.famedata.org - reverse DNS on sslhost is OK: SSL OK - www.famedata.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [08:30:51] PROBLEM - cp13 Puppet on cp13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [08:37:51] PROBLEM - www.famedata.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for www.famedata.org could not be found [08:39:34] PROBLEM - cp14 Puppet on cp14 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [08:41:15] PROBLEM - cp15 Puppet on cp15 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [08:44:45] RECOVERY - www.famedata.org - reverse DNS on sslhost is OK: SSL OK - www.famedata.org reverse DNS resolves to cp14.miraheze.org - NS RECORDS OK [08:52:51] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [08:58:53] RECOVERY - cp13 Puppet on cp13 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [09:07:27] RECOVERY - cp14 Puppet on cp14 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [09:10:18] RECOVERY - cp15 Puppet on cp15 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:15:31] PROBLEM - wiki.qadrishattari.xyz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.qadrishattari.xyz' expires in 7 day(s) (Thu 25 Nov 2021 09:09:05 GMT +0000). [09:36:26] PROBLEM - www.stardewmodding.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'stardewmodding.wiki' expires in 15 day(s) (Fri 03 Dec 2021 09:32:10 GMT +0000). [09:39:02] PROBLEM - stardewmodding.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'stardewmodding.wiki' expires in 15 day(s) (Fri 03 Dec 2021 09:32:10 GMT +0000). [09:56:37] PROBLEM - www.hoolehistoryheritagesociety.org.uk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'www.hoolehistoryheritagesociety.org.uk' expires in 7 day(s) (Thu 25 Nov 2021 09:52:00 GMT +0000). [10:44:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.12, 19.25, 15.92 [10:50:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.42, 22.28, 18.11 [10:52:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.75, 21.06, 18.20 [10:54:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.38, 23.35, 19.33 [10:55:38] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.39, 6.77, 5.00 [10:56:12] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.71, 6.74, 5.07 [10:56:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.25, 22.01, 19.39 [10:57:32] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.45, 6.27, 4.70 [10:57:38] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.14, 6.88, 5.23 [10:58:13] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.12, 6.44, 5.17 [10:59:32] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.79, 5.89, 4.75 [11:00:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.20, 22.67, 20.16 [11:02:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.56, 21.55, 20.07 [11:03:38] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.10, 6.95, 6.01 [11:04:33] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.08, 19.68, 19.55 [11:05:38] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.10, 6.50, 5.96 [11:32:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.63, 19.32, 18.08 [11:34:33] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.43, 17.50, 17.58 [11:59:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.10, 6.65, 5.68 [12:01:38] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.21, 5.69, 5.44 [12:13:39] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.37, 6.28, 5.19 [12:15:36] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.10, 5.98, 5.20 [12:23:13] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.77, 8.10, 5.92 [12:24:01] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.49, 6.40, 5.21 [12:25:19] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.34, 7.62, 6.10 [12:27:13] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.09, 7.52, 6.22 [12:28:01] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.27, 6.09, 5.43 [12:29:13] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.11, 6.76, 6.15 [12:29:13] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.36, 6.49, 5.98 [12:39:04] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [12:39:35] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [12:40:04] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [12:40:35] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [12:40:44] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [12:43:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:45:56] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [12:46:12] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [12:48:25] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [12:59:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:01:45] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [13:01:57] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:04:11] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [13:09:12] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:11:13] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [13:11:24] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:13:28] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [13:15:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:17:51] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [13:27:14] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:29:28] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [13:36:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.22, 2.88 [13:38:10] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.15, 3.21, 2.92 [13:42:27] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:44:29] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [13:45:01] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.44, 3.51, 3.15 [13:48:48] PROBLEM - wiki.fbpml.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.fbpml.org All nameservers failed to answer the query. [13:49:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:50:56] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.67, 3.27, 3.19 [13:51:45] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [13:51:57] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:54:08] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [13:55:45] PROBLEM - wiki.fbpml.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns1.hostnet.nl.', 'ns3.hostnetbv.nl.', 'ns2.hostnetbv.com.'], 'CNAME': 'fbpml.miraheze.org.'} [13:56:20] Atom not working? https://www.irccloud.com/pastebin/hce4qX8B/XML [13:56:20] [tell] Bongo-Cat: 2021-11-17 - 05:51:12UTC tell Bongo-Cat done and responded via MemoServ :) [14:01:14] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:03:27] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [14:04:54] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:06:58] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [14:12:12] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:14:13] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [14:17:12] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:18:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.74, 21.20, 17.98 [14:19:24] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [14:19:46] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.85, 5.47, 4.23 [14:20:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.89, 22.21, 18.76 [14:21:47] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 1.61, 4.10, 3.88 [14:22:33] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.72, 20.35, 18.52 [14:23:42] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:25:44] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [14:35:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.33, 2.92 [14:39:27] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:39:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:41:29] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [14:41:37] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.38, 3.32, 3.09 [14:41:56] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [14:44:27] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:44:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.46, 21.13, 19.51 [14:45:24] Hmm on test3wiki there seems to be errors connecting to the db [14:46:28] (Cannot access the database: Cannot access the database: Unknown database 'test3wiki' (db11) (db11)) [14:46:40] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [14:48:57] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:49:04] And now it’s back [14:50:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.90, 23.81, 21.14 [14:51:07] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [14:52:26] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.46, 3.56, 3.27 [14:54:24] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.79, 4.09, 3.50 [14:54:33] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.36, 5.92, 4.86 [14:56:22] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.72, 3.89, 3.49 [14:56:31] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.08, 4.98, 4.65 [14:56:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.15, 23.47, 22.26 [14:58:12] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [14:58:21] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.81, 4.20, 3.65 [15:00:19] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.83, 3.83, 3.57 [15:00:26] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [15:04:16] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.21, 3.22, 3.40 [15:08:57] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:10:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.91, 22.57, 21.74 [15:10:59] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [15:12:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.79, 21.23, 21.31 [15:14:33] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.15, 24.24, 22.43 [15:16:11] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.52, 5.12, 4.72 [15:18:03] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.41, 3.54, 3.40 [15:18:09] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.34, 4.86, 4.67 [15:18:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.97, 22.34, 22.00 [15:20:01] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.89, 3.72, 3.46 [15:20:27] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:22:40] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [15:27:54] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.32, 3.94, 3.77 [15:29:53] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.44, 4.12, 3.86 [15:31:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.24, 3.80, 3.77 [15:32:12] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:33:49] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.74, 3.99, 3.84 [15:34:25] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [15:35:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.15, 3.67, 3.74 [15:36:33] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.42, 17.07, 19.47 [15:37:46] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.19, 4.02, 3.84 [15:39:44] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.08, 3.72, 3.76 [15:47:38] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.08, 2.79, 3.30 [15:50:29] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.30, 6.14, 5.44 [15:52:24] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 18.14, 10.49, 7.10 [15:52:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.14, 20.19, 19.09 [15:52:51] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.60, 8.06, 6.53 [15:53:32] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.47, 6.54, 5.62 [15:54:32] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.36, 22.24, 19.95 [15:54:43] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:54:47] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.84, 7.77, 6.62 [15:55:32] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.84, 5.88, 5.49 [15:56:18] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 3.55, 7.27, 6.59 [15:56:28] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:56:33] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.21, 21.27, 19.88 [15:56:38] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.77, 4.73, 4.10 [15:56:43] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.28, 6.80, 6.39 [15:56:46] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [15:58:13] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.90, 6.53, 6.39 [15:58:33] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.24, 19.81, 19.51 [15:58:36] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.98, 3.87, 3.85 [15:58:42] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [15:58:44] PROBLEM - cp15 Current Load on cp15 is WARNING: WARNING - load average: 1.52, 1.83, 1.17 [16:00:38] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.04, 7.11, 6.64 [16:00:44] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.66, 1.41, 1.09 [16:01:25] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:03:31] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [16:07:13] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.86, 6.64, 5.53 [16:08:13] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:10:14] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.89, 3.55, 3.27 [16:10:15] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [16:12:13] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.03, 3.69, 3.35 [16:13:13] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.33, 6.31, 5.73 [16:14:11] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.19, 3.37, 3.27 [16:17:59] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.15, 8.31, 7.38 [16:18:06] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.17, 3.67, 3.40 [16:19:09] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.38, 8.62, 6.78 [16:19:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.43, 21.81, 20.32 [16:20:04] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.45, 3.37, 3.31 [16:20:13] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.41, 7.91, 6.91 [16:21:09] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.45, 7.59, 6.64 [16:21:15] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:21:50] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.79, 7.54, 7.27 [16:23:15] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [16:24:01] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.73, 3.24, 3.28 [16:24:13] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.36, 7.35, 6.94 [16:26:13] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.44, 6.39, 6.64 [16:26:14] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:27:09] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.28, 6.74, 6.55 [16:28:26] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [16:30:52] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.08, 19.02, 20.01 [16:35:38] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.82, 5.70, 6.60 [16:44:13] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:45:58] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:46:16] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [16:48:12] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [16:48:56] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:51:00] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [16:53:13] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:55:25] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [17:12:43] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:13:13] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:14:43] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [17:15:15] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [17:21:43] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:23:15] https://phabricator.miraheze.org/T8302 [17:23:16] [url] ⚓ T8302 Bacula Databases dbx on bacula2 is flapping between CRIT&WARN | phabricator.miraheze.org [17:23:57] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [17:28:24] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:30:29] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [17:37:28] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:37:44] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:39:29] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [17:39:45] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [17:44:13] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:46:24] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [17:48:58] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:51:00] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [18:07:38] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.67, 3.31, 2.77 [18:08:14] !log added ipv6 address to bacula2 and rebooted [18:08:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:11:44] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:11:58] RECOVERY - ping6 on bacula2 is OK: PING OK - Packet loss = 0%, RTA = 78.70 ms [18:13:32] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.70, 3.08, 2.85 [18:16:08] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:16:50] PROBLEM - bacula2 Bacula Databases db13 on bacula2 is WARNING: WARNING: Full, 70911 files, 26.39GB, 2021-11-07 00:59:00 (1.5 weeks ago) [18:18:21] PROBLEM - bacula2 Bacula Databases db12 on bacula2 is WARNING: WARNING: Full, 70416 files, 28.15GB, 2021-11-07 00:36:00 (1.5 weeks ago) [18:18:50] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:20:46] PROBLEM - bacula2 Bacula Databases db11 on bacula2 is WARNING: WARNING: Full, 81266 files, 13.66GB, 2021-11-07 00:13:00 (1.5 weeks ago) [18:21:19] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.85, 3.59, 3.06 [18:23:17] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.39, 3.11, 2.95 [18:25:11] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.11, 4.48, 3.60 [18:27:07] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.72, 4.12, 3.57 [19:09:30] [02ssl] 07Universal-Omega opened pull request 03#439: Remove nocyclo.tk - 13https://git.io/J10u3 [19:09:42] [02dns] 07Universal-Omega opened pull request 03#229: Remove nocyclo.tk - 13https://git.io/J10uG [19:13:11] PROBLEM - ping4 on cp15 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 78.29 ms [19:15:08] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.05, 3.35, 3.10 [19:15:14] RECOVERY - ping4 on cp15 is OK: PING OK - Packet loss = 0%, RTA = 79.11 ms [19:17:05] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.59, 3.01, 3.00 [19:20:57] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.75, 3.91, 3.37 [19:22:14] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.89, 19.53, 17.71 [19:24:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.23, 17.23, 17.10 [19:24:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.36, 3.51, 3.37 [19:26:52] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.09, 3.31, 3.30 [19:48:22] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.64, 20.57, 18.76 [19:50:17] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 19.82, 20.21, 18.83 [19:53:25] [02puppet] 07Universal-Omega opened pull request 03#2127: Handle CORS preflight requests - 13https://git.io/J10Ko [19:53:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.81, 3.42, 3.13 [19:54:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.39, 20.65, 19.50 [19:55:31] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.97, 3.31, 3.13 [19:56:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.97, 18.91, 18.99 [20:09:08] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.26, 3.50, 3.28 [20:11:05] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.81, 3.20, 3.20 [20:14:04] PROBLEM - graylog2 Disk Space on graylog2 is CRITICAL: DISK CRITICAL - free space: / 42904 MB (5% inode=99%); [20:14:18] paladox: ^ [20:14:25] Can you clear anything [20:20:04] PROBLEM - graylog2 Disk Space on graylog2 is WARNING: DISK WARNING - free space: / 43363 MB (6% inode=99%); [20:22:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.31, 5.18, 4.05 [20:22:50] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.09, 20.95, 19.35 [20:24:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 5.04, 5.08, 4.15 [20:24:45] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.53, 19.52, 19.02 [20:28:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.94, 20.74, 19.60 [20:32:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.53, 5.18, 4.55 [20:34:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.85, 4.69, 4.44 [20:36:16] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.98, 18.56, 19.27 [20:46:04] PROBLEM - graylog2 Disk Space on graylog2 is CRITICAL: DISK CRITICAL - free space: / 42596 MB (5% inode=99%); [20:46:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.06, 19.93, 18.88 [20:48:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.64, 18.47, 18.46 [20:52:04] PROBLEM - graylog2 Disk Space on graylog2 is WARNING: DISK WARNING - free space: / 43206 MB (6% inode=99%); [20:56:04] PROBLEM - graylog2 Disk Space on graylog2 is CRITICAL: DISK CRITICAL - free space: / 42524 MB (5% inode=99%); [21:08:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.34, 5.22, 4.43 [21:10:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.24, 4.69, 4.32 [21:13:20] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.72, 24.44, 21.25 [21:15:15] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.45, 23.57, 21.30 [21:21:34] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.34, 5.12, 4.74 [21:22:56] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 34.36, 27.15, 23.32 [21:23:30] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.81, 5.65, 4.96 [21:25:26] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.71, 5.48, 4.99 [21:26:18] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.89, 3.52, 3.00 [21:28:16] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.50, 2.74, 2.78 [21:29:17] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.56, 5.10, 4.96 [21:29:36] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.14, 6.45, 4.85 [21:31:36] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.71, 5.23, 4.59 [21:32:31] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 17.43, 23.44, 23.90 [21:36:04] PROBLEM - graylog2 Disk Space on graylog2 is WARNING: DISK WARNING - free space: / 44426 MB (6% inode=99%); [21:37:58] PROBLEM - graylog2 Current Load on graylog2 is CRITICAL: CRITICAL - load average: 9.79, 5.36, 3.03 [21:42:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.20, 17.05, 20.27 [21:46:57] !log delete 5 old indices on graylog to save space [21:47:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:47:10] RhinosF1: the MW team will need to reduce the amount of logs sent to graylog. [21:47:56] paladox: I assume it's the introduction of api-request causing issues [21:48:04] RECOVERY - graylog2 Disk Space on graylog2 is OK: DISK OK - free space: / 139515 MB (19% inode=99%); [21:48:04] CosmicAlpha: can we sample it [21:48:11] well there's been many changes to the channels config [21:48:17] paladox: how much by? [21:48:33] well only the logs that are really really needed [21:48:44] we don't have a ton of space like wikimedia [21:50:19] PROBLEM - files.pornwiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:50:46] paladox: well api-request was asked by spf [21:50:46] PROBLEM - cp15 Current Load on cp15 is WARNING: WARNING - load average: 1.31, 1.95, 1.19 [21:50:51] But it's nice to have [21:50:59] The rest are very useful [21:51:16] I think we should sample api-request [21:51:26] I'd imagine the DB issues aren't helping [21:52:16] RhinosF1: it was John who asked for api requests, not SPF. [21:52:32] PROBLEM - en.pornwiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:52:46] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.64, 1.49, 1.11 [21:53:09] As for sampling that would defeat the purpose I think. [21:53:23] PROBLEM - www.pornwiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:53:24] or we reduce how many data we store [21:53:29] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.69, 4.66, 4.04 [21:53:32] e.g. 15 days rather then 30 (or longer) [21:54:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.18, 20.42, 19.91 [21:55:14] PROBLEM - pornwiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:55:25] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.90, 4.46, 4.03 [21:56:13] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 33.67, 25.21, 21.68 [21:56:55] Hmm [21:56:58] Ty CosmicAlpha [21:59:19] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.91, 6.45, 4.99 [22:01:19] paladox: can we adjust retention per channel or is it global? [22:02:00] it's global i think [22:08:58] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.89, 5.99, 5.79 [22:10:54] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.73, 7.22, 6.27 [22:11:57] PROBLEM - graylog2 Current Load on graylog2 is WARNING: WARNING - load average: 2.92, 2.68, 3.92 [22:17:57] RECOVERY - graylog2 Current Load on graylog2 is OK: OK - load average: 0.60, 1.87, 3.22 [22:28:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.51, 3.32, 2.91 [22:29:36] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.96, 7.21, 5.35 [22:29:40] PROBLEM - en.pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [22:30:52] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.15, 3.09, 2.86 [22:31:36] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.76, 5.83, 5.07 [22:32:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.95, 5.16, 5.82 [22:34:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.61, 5.53, 5.86 [22:36:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.73, 5.12, 5.66 [22:38:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.90, 19.81, 23.72 [22:42:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.45, 5.83, 5.68 [22:43:31] RECOVERY - en.pornwiki.org - reverse DNS on sslhost is OK: SSL OK - en.pornwiki.org reverse DNS resolves to cp14.miraheze.org - NS RECORDS OK [22:46:12] [02puppet] 07Universal-Omega synchronize pull request 03#2127: Handle CORS preflight requests - 13https://git.io/J10Ko [22:46:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.77, 5.55, 5.68 [22:47:20] paladox: do you mind deploying https://github.com/miraheze/puppet/pull/2127 when you have time, and of course only if you think it'll work. [22:47:21] [url] Handle CORS preflight requests by Universal-Omega · Pull Request #2127 · miraheze/puppet · GitHub | github.com [22:50:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.65, 5.65, 5.64 [22:52:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.61, 5.25, 5.49 [22:56:09] PROBLEM - pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [22:56:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.92, 3.99, 4.93 [23:00:02] !log [@test3] starting deploy of {'l10nupdate': True} to skip [23:00:04] !log [@mw11] starting deploy of {'l10nupdate': True} to all [23:00:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.22, 17.62, 19.91 [23:00:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:04:06] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 11.93, 9.11, 6.32 [23:04:24] PROBLEM - mw11 APT on mw11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:05:51] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.77, 7.09, 6.04 [23:06:15] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.81, 21.76, 20.95 [23:06:24] RECOVERY - mw11 APT on mw11 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:06:26] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.95, 7.41, 5.96 [23:06:31] PROBLEM - mw12 APT on mw12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [23:06:48] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.73, 6.82, 5.19 [23:07:45] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.56, 7.96, 6.49 [23:08:13] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.42, 23.46, 21.64 [23:08:20] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.58, 7.90, 6.27 [23:08:31] RECOVERY - mw12 APT on mw12 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [23:08:48] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.79, 7.76, 5.68 [23:09:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.27, 3.10 [23:09:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.05, 7.18, 6.37 [23:10:05] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.71, 7.95, 6.92 [23:10:09] PROBLEM - pornwiki.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns102.registrar-servers.com.', 'dns2.registrar-servers.com.', 'dns101.registrar-servers.com.', 'dns1.registrar-servers.com.'], 'CNAME': None} [23:10:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.47, 22.55, 21.51 [23:10:15] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.26, 6.96, 6.13 [23:10:48] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.64, 7.52, 5.88 [23:11:08] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.05, 3.20, 3.09 [23:12:09] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.54, 6.43, 6.02 [23:12:13] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 31.32, 25.67, 22.78 [23:12:48] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.96, 6.24, 5.61 [23:13:36] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.87, 6.32, 6.25 [23:14:05] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.27, 6.25, 6.48 [23:14:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.70, 6.19, 5.02 [23:17:09] PROBLEM - pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [23:18:46] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.25, 5.90, 5.19 [23:18:52] !log [@mw11] finished deploy of {'l10nupdate': True} to all - FAIL: [35072, 0, 0, 0, 0, 0, 0, 0, 0] in 1127s [23:18:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:20:10] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 25.24, 24.10, 18.96 [23:20:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.26, 6.15, 5.35 [23:22:07] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.28, 7.54, 6.35 [23:22:09] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 14.58, 20.49, 18.27 [23:22:44] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 2.96, 5.17, 5.11 [23:24:07] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.92, 6.69, 6.20 [23:24:09] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 14.81, 18.39, 17.75 [23:24:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.73, 4.76, 4.96 [23:25:32] https://phabricator.miraheze.org/T8285 [23:25:33] [url] ⚓ T8285 Custom Domain - Hypothetical Hurricanes Wiki | phabricator.miraheze.org [23:25:52] despite reception adding the cert, it seems to not be taking effect [23:26:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 15.59, 20.92, 22.99 [23:27:40] MacFan4000: I can remove the custom domain from server in ManageWiki for now. If seems it didn't work because certs.yaml change seemed not to take effect. It looks like Reception123 made a mistake and added the cert to certs.yaml separately from the .crt file. Also it seems a change I made to redirects.yaml yesterday is still not taking effect. The cert its trying to use for that wiki is the miraheze.org wildcard, which is a [23:27:40] mismatch. I assume because wildcard is default if not in certs.yaml. [23:27:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.57, 3.36, 3.09 [23:28:31] paladox: ^ any ideas as to the latest changes to ssl aren't taking effect? [23:28:50] as to why* [23:29:40] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.16, 3.23, 3.07 [23:30:44] !log [@test3] finished deploy of {'l10nupdate': True} to skip - SUCCESS in 1841s [23:30:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:35:30] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.70, 3.61, 3.22 [23:38:08] RECOVERY - pornwiki.org - reverse DNS on sslhost is OK: SSL OK - pornwiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [23:39:24] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.29, 3.35, 3.23 [23:40:13] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.23, 21.09, 21.42 [23:41:36] PROBLEM - files.pornwiki.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns101.registrar-servers.com.', 'dns102.registrar-servers.com.', 'dns1.registrar-servers.com.', 'dns2.registrar-servers.com.'], 'CNAME': None} [23:42:16] ^ that domain is expired [23:44:44] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.44, 6.15, 4.97 [23:46:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 15.18, 21.65, 22.18 [23:46:16] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_item.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:46:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:46:44] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.19, 4.80, 4.63 [23:46:54] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_item.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:46:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:12] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_mediawiki.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:47:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:28] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_mediawiki.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:47:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:42] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_module.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:47:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:48] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_module.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:47:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:53] PROBLEM - en.pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [23:48:20] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_property.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:48:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:54] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_property.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:49:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:50:20] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_template.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:50:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:51:42] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_template.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:51:59] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_translations.xml --username-prefix=famedatawiki --report 1 --no-updates (START) [23:52:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:52:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:52:13] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.58, 21.99, 21.82 [23:52:25] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gratisdatawiki /home/universalomega/gratisdatawiki_translations.xml --username-prefix=famedatawiki --report 1 --no-updates (END - exit=0) [23:52:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:54:13] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 13.80, 18.77, 20.67 [23:54:19] !log [universalomega@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildrecentchanges.php --wiki=gratisdatawiki (END - exit=0) [23:54:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:54:49] PROBLEM - en.pornwiki.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns102.registrar-servers.com.', 'dns2.registrar-servers.com.', 'dns101.registrar-servers.com.', 'dns1.registrar-servers.com.'], 'CNAME': None} [23:55:17] PROBLEM - www.pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [23:55:27] PROBLEM - files.pornwiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - files.pornwiki.org reverse DNS resolves to a51062ecadbb5a26e.awsglobalaccelerator.com [23:56:13] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.12, 17.80, 20.13