[00:01:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.04, 4.03, 3.84 [00:06:42] RECOVERY - cp33 Disk Space on cp33 is OK: DISK OK - free space: / 14448 MB (19% inode=98%); [00:09:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.65, 3.72, 3.82 [00:11:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.29, 4.79, 4.18 [00:27:10] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.01841965318 secs [00:27:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.78, 3.42, 3.89 [00:30:51] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.15, 3.33, 2.70 [00:31:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.66, 3.97, 3.97 [00:32:49] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.42, 2.88, 2.61 [00:33:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.30, 3.66, 3.86 [00:35:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.13, 4.62, 4.15 [00:37:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.97, 3.79, 3.90 [00:43:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.87, 3.62, 3.70 [00:47:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.52, 3.86, 3.84 [00:51:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.84, 3.90, 3.79 [00:55:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.89, 3.68, 3.77 [00:59:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.29, 3.66, 3.70 [01:03:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.62, 3.74 [01:07:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.29, 2.88, 3.40 [01:11:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.57, 4.40, 3.86 [01:17:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.31, 3.72, 3.80 [01:26:55] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.49, 3.63, 3.25 [01:27:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.86, 2.88, 3.32 [01:28:53] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.68, 3.33, 3.18 [01:49:29] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.31, 4.03, 3.49 [01:51:26] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.87, 3.64, 3.42 [01:53:23] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.09, 5.22, 4.07 [03:01:40] RECOVERY - db101 Backups SQL on db101 is OK: FILE_AGE OK: /var/log/sql-backup.log is 92 seconds old and 1899 bytes [04:33:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.49, 3.50, 3.98 [04:41:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.11, 3.47, 3.71 [04:43:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.85, 3.41, 3.65 [04:45:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.86, 3.88, 3.79 [04:51:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.24, 3.80, 3.89 [04:53:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.69, 4.25, 4.03 [04:57:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.13, 3.60, 3.85 [05:01:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.26, 4.73, 4.16 [05:05:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.69, 3.84, 3.93 [05:07:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.38, 3.87, 3.91 [05:12:55] PROBLEM - puppet141 Current Load on puppet141 is CRITICAL: CRITICAL - load average: 8.03, 7.57, 4.19 [05:13:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.81, 3.64, 3.87 [05:13:54] PROBLEM - mail121 Puppet on mail121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:14:21] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:14:28] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 442 failures. Last run 2 minutes ago with 442 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/nexus.wiki.crt],File[/etc/ssl/private/nexus.wiki.key],File[/etc/ssl/localcerts/slymods.info.crt],File[/etc/ssl/private/slymods.info.key] [05:14:35] PROBLEM - swiftproxy111 Puppet on swiftproxy111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:14:48] PROBLEM - mw131 Puppet on mw131 is CRITICAL: CRITICAL: Puppet has 1104 failures. Last run 2 minutes ago with 1104 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[/etc/rsyslog.d],File[/etc/rsyslog.conf] [05:14:50] RECOVERY - puppet141 Current Load on puppet141 is OK: OK - load average: 2.61, 5.59, 3.85 [05:15:18] PROBLEM - mem141 Puppet on mem141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:23:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.76, 2.60, 3.25 [05:28:21] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [05:33:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.20, 8.21, 5.55 [05:40:24] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [05:42:35] RECOVERY - swiftproxy111 Puppet on swiftproxy111 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [05:43:18] RECOVERY - mem141 Puppet on mem141 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [05:43:29] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:43:54] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:44:21] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:45:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.62, 3.06, 3.85 [05:55:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.12, 2.90, 3.37 [06:26:42] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 13.29, 6.94, 4.56 [06:28:06] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [06:32:33] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.40, 4.00, 3.94 [06:36:27] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.25, 2.57, 3.37 [07:47:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.85, 3.22, 2.67 [07:49:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.39, 3.07, 2.67 [08:06:04] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.99, 2.67, 1.70 [08:08:04] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.43, 2.46, 1.74 [08:33:50] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [09:02:40] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:03:06] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7948 MB (5% inode=99%); [09:08:37] PROBLEM - liberhood.su - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for liberhood.su could not be found [09:33:50] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [09:44:49] PROBLEM - liberhood.su - LetsEncrypt on sslhost is CRITICAL: connect to address liberhood.su and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:10:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [10:31:50] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [10:36:36] RECOVERY - liberhood.su - reverse DNS on sslhost is OK: SSL OK - liberhood.su reverse DNS resolves to cp22.miraheze.org - CNAME FLAT [10:40:57] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [11:38:37] PROBLEM - liberhood.su - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for liberhood.su could not be found [12:55:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.41, 2.74, 2.23 [12:56:30] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [12:57:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.96, 2.38, 2.15 [13:26:22] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [14:24:08] PROBLEM - kalons-reverie.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'kalons-reverie.com' expires in 15 day(s) (Fri 11 Aug 2023 13:54:16 GMT +0000). [14:28:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e09fd2f7aa62...3f0d9e7f2efd [14:28:20] [02miraheze/ssl] 07MirahezeSSLBot 033f0d9e7 - Bot: Update SSL cert for kalons-reverie.com [14:36:13] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [14:40:13] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [14:57:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.11, 3.88, 3.22 [15:03:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.19, 3.67, 3.46 [15:07:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.21, 3.20, 3.30 [15:23:23] RECOVERY - kalons-reverie.com - LetsEncrypt on sslhost is OK: OK - Certificate 'kalons-reverie.com' will expire on Tue 24 Oct 2023 13:28:10 GMT +0000. [15:29:32] [02ssl] 07ajhalili2006 opened pull request 03#662: Redirect recaptimewiki to squad.lorebooks.eu.org - 13https://github.com/miraheze/ssl/pull/662 [15:47:17] PROBLEM - www.hebammenwiki.de - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.hebammenwiki.de' expires in 15 day(s) (Fri 11 Aug 2023 15:38:31 GMT +0000). [15:50:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3f0d9e7f2efd...5262950d575d [15:50:31] [02miraheze/ssl] 07MirahezeSSLBot 035262950 - Bot: Update SSL cert for www.hebammenwiki.de [15:55:01] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.98, 3.86, 2.94 [15:55:45] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [15:56:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.54, 3.32, 2.85 [16:03:53] PROBLEM - bushcraftwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'bushcraftwiki.com' expires in 15 day(s) (Fri 11 Aug 2023 15:40:36 GMT +0000). [16:04:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5262950d575d...5d9361a0ac06 [16:04:35] [02miraheze/ssl] 07MirahezeSSLBot 035d9361a - Bot: Update SSL cert for bushcraftwiki.com [16:16:03] RECOVERY - www.hebammenwiki.de - LetsEncrypt on sslhost is OK: OK - Certificate 'www.hebammenwiki.de' will expire on Tue 24 Oct 2023 14:50:22 GMT +0000. [16:21:24] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.99, 3.27, 2.87 [16:23:21] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.68, 3.77, 3.10 [16:25:38] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [16:31:08] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.76, 3.84, 3.54 [16:32:46] RECOVERY - bushcraftwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'bushcraftwiki.com' will expire on Tue 24 Oct 2023 15:04:26 GMT +0000. [16:33:51] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:35:02] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.11, 3.18, 3.36 [16:42:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.54, 3.44, 3.46 [16:44:50] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.84, 3.21, 3.37 [16:52:25] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.28, 3.42, 3.17 [16:54:23] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.33, 3.37, 3.18 [17:02:41] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [17:07:21] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.13, 4.07, 3.47 [17:09:18] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.35, 3.06, 3.17 [17:24:59] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 4.00, 3.50, 3.18 [17:26:56] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.96, 3.34, 3.16 [17:35:35] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.46, 2.95, 2.85 [17:37:33] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.34, 2.65, 2.75 [17:47:10] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.126550734 secs [17:59:10] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.08388340473 secs [18:03:10] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset 0.2908316553 secs [18:11:42] PROBLEM - db142 Puppet on db142 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:12:24] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:12:24] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:12:33] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:12:35] PROBLEM - mw131 Puppet on mw131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:12:51] PROBLEM - puppet141 Puppet on puppet141 is CRITICAL: CRITICAL: Puppet has 7 failures. Last run 3 minutes ago with 7 failures. Failed resources (up to 3 shown): File[/etc/puppetlabs/puppet/fileserver.conf],File[/etc/puppetlabs/puppet/hiera.yaml],File[/etc/sysctl.d],File[/etc/logrotate.d/rsyslog] [18:18:40] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/efeabb191051...eccee522f583 [18:18:42] [02miraheze/mediawiki] 07paladox 03eccee52 - Update ApprovedRevs [18:19:30] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d62356e1e2cd...1f9d009188ae [18:19:32] [02miraheze/mediawiki] 07paladox 031f9d009 - Update ApprovedRevs [18:19:55] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [18:20:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:20:10] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [18:20:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:21:59] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.82, 3.13, 2.32 [18:23:59] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.59, 3.53, 2.56 [18:25:59] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.20, 3.40, 2.63 [18:26:20] !log [paladox@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 384s [18:26:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:26:52] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 401s [18:26:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:28:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5afb98b50531...ad2e40cfe660 [18:28:36] [02miraheze/puppet] 07paladox 03ad2e40c - php: increase pm.max_requests to 7k [18:29:35] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.36, 3.66, 3.25 [18:31:31] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.95, 3.11, 3.10 [18:32:14] PROBLEM - db131 Puppet on db131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [18:32:58] PROBLEM - mw133 Puppet on mw133 is CRITICAL: CRITICAL: Puppet has 807 failures. Last run 2 minutes ago with 807 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/www.rothwell-leeds.co.uk.key],File[/etc/ssl/localcerts/wiki.cyberfurs.org.crt],File[/etc/ssl/private/wiki.cyberfurs.org.key],File[/etc/ssl/localcerts/wikiru.wildterra2.com.crt] [18:33:10] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset 0.07017022371 secs [18:34:24] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [18:34:54] RECOVERY - mw133 Puppet on mw133 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [18:36:36] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:39:09] RECOVERY - puppet141 Puppet on puppet141 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [18:40:24] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:40:33] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:41:42] RECOVERY - db142 Puppet on db142 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:42:19] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.19, 3.96, 3.45 [18:43:59] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 3.62, 3.21 [18:44:16] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.30, 3.24, 3.24 [18:45:59] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.80, 3.32, 3.15 [18:47:10] PROBLEM - cp33 NTP time on cp33 is WARNING: NTP WARNING: Offset -0.1178198457 secs [18:49:10] RECOVERY - cp33 NTP time on cp33 is OK: NTP OK: Offset -0.08975589275 secs [19:00:14] RECOVERY - db131 Puppet on db131 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [19:38:42] PROBLEM - cp33 Disk Space on cp33 is WARNING: DISK WARNING - free space: / 8211 MB (10% inode=98%); [19:55:13] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.27, 4.42, 3.23 [19:59:07] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.74, 3.70, 3.22 [20:01:04] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 3.13, 3.06 [20:06:57] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.83, 3.60, 3.31 [20:10:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.61, 3.23, 3.23 [20:10:57] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [20:20:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 3.72, 4.27, 3.63 [20:22:36] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.73, 3.73, 3.50 [20:24:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.12, 3.31, 3.39 [20:40:59] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [20:44:11] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.38, 3.52, 3.43 [20:46:08] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.17, 3.05, 3.27 [20:53:58] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.03, 5.32, 4.06 [21:03:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.89, 3.77, 3.84 [21:09:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 3.52, 3.60 [21:11:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.00, 3.45, 3.58 [21:13:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.99, 4.71, 3.99 [21:17:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.89, 3.35, 3.62 [21:19:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.45, 4.08, 3.84 [22:23:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.16, 3.35, 3.92 [22:29:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.07, 3.87, 3.92 [22:31:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.87, 3.16, 3.66 [22:37:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.75, 2.61, 3.27 [22:41:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.18, 3.82, 3.63 [22:43:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.15, 3.55, 3.56 [22:47:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.26, 3.79, 3.65 [22:49:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.27, 3.55, 3.58 [22:55:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.53, 3.10, 3.38 [23:06:48] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 4.00, 3.86, 3.49 [23:10:41] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.53, 3.64, 3.45 [23:12:38] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.26, 3.57, 3.45 [23:14:35] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.66, 4.31, 3.75 [23:18:29] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.10, 3.89, 3.73 [23:20:26] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.16, 3.89, 3.74 [23:25:23] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [23:32:07] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.54, 3.50, 3.90 [23:37:59] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.38, 4.94, 4.21 [23:43:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.23, 3.49, 3.82 [23:51:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 4.09, 3.91 [23:55:22] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [23:57:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.94, 3.34, 3.75