[00:02:05] RECOVERY - cp3 Disk Space on cp3 is OK: DISK OK - free space: / 5142 MB (21% inode=94%); [00:02:21] PROBLEM - cp12 Disk Space on cp12 is CRITICAL: DISK CRITICAL - free space: / 2305 MB (5% inode=96%); [00:04:00] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown) [00:04:02] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.28, 7.60, 6.00 [00:04:24] RECOVERY - cp12 Disk Space on cp12 is OK: DISK OK - free space: / 7681 MB (19% inode=96%); [00:05:22] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 32.01, 23.06, 18.16 [00:05:55] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.38, 7.91, 6.24 [00:07:53] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.52, 7.14, 6.14 [00:08:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.87, 7.63, 6.49 [00:09:17] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.79, 23.89, 19.64 [00:10:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.07, 8.72, 7.02 [00:11:53] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.49, 8.81, 7.00 [00:19:20] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.09, 25.40, 22.15 [00:21:17] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.08, 23.99, 22.00 [00:25:53] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.89, 7.26, 7.50 [00:28:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.57, 7.21, 7.80 [00:31:16] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.14, 17.87, 19.79 [00:31:59] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:33:53] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.50, 7.53, 7.34 [00:34:02] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 14.63, 9.05, 8.13 [00:37:53] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.47, 7.71, 7.49 [00:40:03] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.55, 7.32, 7.74 [00:45:52] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.74, 5.87, 6.65 [00:50:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.85, 7.01, 7.22 [00:51:14] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [00:51:14] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003798007965088 seconds [00:52:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.33, 6.32, 6.94 [00:56:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.70, 7.94, 7.46 [00:58:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.58, 6.83, 7.12 [01:02:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.70, 5.60, 6.55 [01:12:59] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [03:04:03] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.66, 6.50, 5.26 [03:06:03] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.83, 6.22, 5.32 [03:25:44] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.14, 3.35, 2.86 [03:27:42] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.37, 3.34, 2.91 [03:38:39] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.qadrishattari.xyz could not be found [03:50:44] PROBLEM - diamowiki.ga - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:57:34] RECOVERY - diamowiki.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'diamowiki.ga' will expire on Sat 18 Sep 2021 14:34:47 GMT +0000. [04:12:25] RECOVERY - wiki.qadrishattari.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'sni.cloudflaressl.com' will expire on Sun 07 Aug 2022 23:59:59 GMT +0000. [04:58:25] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [04:58:25] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.004056215286255 seconds [05:10:31] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JRl09 [05:10:32] [02miraheze/mediawiki] 07Universal-Omega 0358837d3 - Update Cosmos [05:10:33] !log [universalomega@test3] starting deploy of {'config': True, 'world': True, 'l10n': True} to skip [05:10:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:13:07] RECOVERY - test3 Puppet on test3 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [05:14:35] !log [universalomega@test3] finished deploy of {'config': True, 'world': True, 'l10n': True} to skip - SUCCESS [05:14:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:21:09] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [05:31:59] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [05:32:00] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000368356704712 seconds [05:37:42] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.83, 19.23, 15.70 [05:39:43] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.14, 22.70, 17.41 [05:40:58] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:41:24] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:41:41] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.53, 19.48, 16.86 [05:42:02] PROBLEM - jobrunner3 Puppet on jobrunner3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:42:38] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [05:42:48] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [06:04:04] RECOVERY - jobrunner3 Puppet on jobrunner3 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [06:04:37] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [06:04:48] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [06:04:58] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:05:21] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:08:51] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [07:05:39] [02mw-config] 07R4356th opened pull request 03#4043: Update WVUI search configuration for snapwikiwiki for Vector and Snap! - 13https://git.io/JRldI [07:06:35] miraheze/mw-config - R4356th the build passed. [07:07:52] [02mw-config] 07Universal-Omega closed pull request 03#4043: Update WVUI search configuration for snapwikiwiki for Vector and Snap! - 13https://git.io/JRldI [07:07:53] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JRlFL [07:07:55] [02miraheze/mw-config] 07R4356th 033b5d1cf - Update WVUI search configuration for snapwikiwiki for Vector and Snap! (#4043) [07:08:50] miraheze/mw-config - Universal-Omega the build passed. [07:33:44] !log [@test3] starting deploy of {'config': True} to skip [07:33:45] !log [@test3] finished deploy of {'config': True} to skip - FAIL: [256] [07:33:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:33:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:36:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.14, 6.34, 5.12 [07:38:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.29, 6.28, 5.25 [08:05:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.84, 7.58, 6.34 [08:07:07] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.36, 6.41, 6.06 [08:30:47] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [08:30:47] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003508806228638 seconds [08:34:51] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.45, 7.22, 5.88 [08:36:49] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.15, 6.39, 5.74 [08:38:03] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [09:02:50] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [09:02:50] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003446340560913 seconds [09:17:53] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [09:35:53] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [09:35:53] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.002553462982178 seconds [09:50:11] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [10:15:30] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [10:15:30] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.0030779838562 seconds [10:30:12] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [10:48:41] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.21, 7.05, 6.03 [10:50:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.88, 7.23, 6.21 [10:54:24] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.93, 6.75, 6.25 [10:57:13] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [10:57:13] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00093102455139 seconds [11:04:58] PROBLEM - diamowiki.ga - reverse DNS on sslhost is UNKNOWN: [11:20:03] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [11:30:41] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [11:30:41] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.002067804336548 seconds [11:52:48] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [12:13:24] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [12:13:24] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00054621696472 seconds [12:35:25] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [12:35:40] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.23, 5.99, 5.20 [12:37:38] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.53, 5.46, 5.10 [13:00:41] PROBLEM - patternarchive.online - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'patternarchive.online' expires in 15 day(s) (Tue 24 Aug 2021 12:55:41 GMT +0000). [13:02:50] PROBLEM - www.puzzles.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.puzzles.wiki' expires in 15 day(s) (Tue 24 Aug 2021 12:59:54 GMT +0000). [13:03:35] PROBLEM - puzzles.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.puzzles.wiki' expires in 15 day(s) (Tue 24 Aug 2021 12:59:54 GMT +0000). [13:03:49] PROBLEM - diamowiki.ga - reverse DNS on sslhost is UNKNOWN: [13:05:51] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR40D [13:05:52] [02miraheze/ssl] 07MirahezeSSLBot 0371dedf9 - Bot: Update SSL cert for patternarchive.online [13:08:07] PROBLEM - electrowiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'electrowiki.org' expires in 15 day(s) (Tue 24 Aug 2021 13:04:00 GMT +0000). [13:11:04] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.query(hostname, [13:11:04] 0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 900, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000067472457886 seconds [13:12:24] PROBLEM - www.electrowiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'electrowiki.org' expires in 15 day(s) (Tue 24 Aug 2021 13:04:00 GMT +0000). [13:13:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4u9 [13:13:21] [02miraheze/ssl] 07MirahezeSSLBot 035ecf4b4 - Bot: Update SSL cert for www.puzzles.wiki [13:16:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4zE [13:16:46] [02miraheze/ssl] 07MirahezeSSLBot 03862e04a - Bot: Update SSL cert for electrowiki.org [13:18:49] PROBLEM - diamowiki.ga - reverse DNS on sslhost is UNKNOWN: [13:26:24] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [13:29:04] RECOVERY - electrowiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'electrowiki.org' will expire on Sat 06 Nov 2021 12:16:39 GMT +0000. [13:30:27] RECOVERY - www.puzzles.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.puzzles.wiki' will expire on Sat 06 Nov 2021 12:13:14 GMT +0000. [13:31:23] RECOVERY - puzzles.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.puzzles.wiki' will expire on Sat 06 Nov 2021 12:13:14 GMT +0000. [13:32:40] RECOVERY - www.electrowiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'electrowiki.org' will expire on Sat 06 Nov 2021 12:16:39 GMT +0000. [13:34:57] RECOVERY - patternarchive.online - LetsEncrypt on sslhost is OK: OK - Certificate 'patternarchive.online' will expire on Sat 06 Nov 2021 12:05:42 GMT +0000. [13:37:09] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [13:37:09] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003413677215576 seconds [13:37:16] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.30, 18.83, 15.24 [13:39:16] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.43, 20.01, 16.07 [13:41:16] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.91, 18.23, 15.89 [13:42:16] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.22, 6.87, 5.83 [13:44:24] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.28, 6.83, 5.94 [13:48:21] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.65, 6.80, 6.14 [14:00:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.19, 6.70, 6.22 [14:01:11] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.75, 7.59, 6.65 [14:02:02] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.35, 6.71, 6.29 [14:06:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.25, 7.70, 6.89 [14:08:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.48, 8.03, 7.10 [14:09:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.01, 7.87, 7.44 [14:14:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.41, 7.89, 7.48 [14:19:09] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.57, 6.14, 6.79 [14:22:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.13, 6.19, 6.79 [14:29:10] PROBLEM - cp13 Current Load on cp13 is WARNING: WARNING - load average: 1.95, 1.26, 0.81 [14:31:11] RECOVERY - cp13 Current Load on cp13 is OK: OK - load average: 0.74, 1.04, 0.79 [14:32:34] PROBLEM - fc.songcontests.eu - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for fc.songcontests.eu could not be found [14:32:42] PROBLEM - translate.petrawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for translate.petrawiki.org could not be found [14:35:11] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.17, 7.97, 6.93 [14:36:05] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.66, 7.74, 7.03 [14:36:28] PROBLEM - dreamsit.com.br - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'dreamsit.com.br' expires in 15 day(s) (Tue 24 Aug 2021 14:34:15 GMT +0000). [14:36:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4dz [14:36:59] [02miraheze/ssl] 07MirahezeSSLBot 03f98cbb2 - Bot: Update SSL cert for dreamsit.com.br [14:37:11] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.93, 7.50, 6.88 [14:38:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.46, 7.93, 7.18 [14:38:10] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.32, 6.53, 5.87 [14:39:16] RECOVERY - fc.songcontests.eu - reverse DNS on sslhost is OK: SSL OK - fc.songcontests.eu reverse DNS resolves to cp13.miraheze.org - CNAME OK [14:39:37] RECOVERY - translate.petrawiki.org - reverse DNS on sslhost is OK: SSL OK - translate.petrawiki.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [14:40:02] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.13, 7.68, 7.18 [14:40:11] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.24, 7.02, 6.12 [14:41:09] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.62, 6.52, 6.68 [14:42:11] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.77, 6.89, 6.19 [14:43:37] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [14:44:01] PROBLEM - www.burnout.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.burnout.wiki' expires in 15 day(s) (Tue 24 Aug 2021 14:40:44 GMT +0000). [14:44:13] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.66, 6.09, 5.99 [14:46:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.91, 6.08, 6.69 [14:49:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4NF [14:49:34] [02miraheze/ssl] 07MirahezeSSLBot 03a2dd947 - Bot: Update SSL cert for www.burnout.wiki [14:50:19] PROBLEM - reviwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'reviwiki.info' expires in 15 day(s) (Tue 24 Aug 2021 14:44:14 GMT +0000). [14:52:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4A8 [14:52:09] [02miraheze/ssl] 07MirahezeSSLBot 037522429 - Bot: Update SSL cert for reviwiki.info [14:54:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.85, 6.49, 6.61 [14:54:44] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [14:54:44] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000481367111206 seconds [14:56:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.47, 6.67, 6.68 [14:56:46] RECOVERY - dreamsit.com.br - LetsEncrypt on sslhost is OK: OK - Certificate 'dreamsit.com.br' will expire on Sat 06 Nov 2021 13:36:51 GMT +0000. [14:58:18] PROBLEM - meta.gyaanipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'meta.gyaanipedia.com' expires in 15 day(s) (Tue 24 Aug 2021 14:52:54 GMT +0000). [15:00:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JR4hg [15:00:31] [02miraheze/ssl] 07MirahezeSSLBot 03401c024 - Bot: Update SSL cert for meta.gyaanipedia.com [15:01:59] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [15:04:35] PROBLEM - mr.gyaanipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mr.gyaanipedia.com' expires in 15 day(s) (Tue 24 Aug 2021 14:55:43 GMT +0000). [15:05:36] PROBLEM - hi.gyaanipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'hi.gyaanipedia.com' expires in 15 day(s) (Tue 24 Aug 2021 14:59:07 GMT +0000). [15:06:00] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JRBeI [15:06:02] [02miraheze/ssl] 07MirahezeSSLBot 033451177 - Bot: Update SSL cert for mr.gyaanipedia.com [15:06:21] PROBLEM - pa.gyaanipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pa.gyaanipedia.com' expires in 15 day(s) (Tue 24 Aug 2021 15:03:56 GMT +0000). [15:08:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JRBvJ [15:08:31] [02miraheze/ssl] 07MirahezeSSLBot 0313d026e - Bot: Update SSL cert for hi.gyaanipedia.com [15:09:27] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.62, 3.03, 2.61 [15:11:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.46, 2.73, 2.55 [15:11:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JRBvy [15:11:25] [02miraheze/ssl] 07MirahezeSSLBot 03e91873b - Bot: Update SSL cert for pa.gyaanipedia.com [15:12:32] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [15:12:32] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.004047393798828 seconds [15:18:11] PROBLEM - commons.gyaanipedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'commons.gyaanipedia.com' expires in 15 day(s) (Tue 24 Aug 2021 15:11:00 GMT +0000). [15:20:06] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [15:21:25] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.58, 7.19, 6.48 [15:23:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JRBTu [15:23:24] [02miraheze/ssl] 07MirahezeSSLBot 03107398f - Bot: Update SSL cert for commons.gyaanipedia.com [15:25:14] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.32, 6.51, 6.44 [15:32:06] RECOVERY - reviwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'reviwiki.info' will expire on Sat 06 Nov 2021 13:52:02 GMT +0000. [15:32:10] RECOVERY - mr.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mr.gyaanipedia.com' will expire on Sat 06 Nov 2021 14:05:54 GMT +0000. [15:32:55] RECOVERY - meta.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'meta.gyaanipedia.com' will expire on Sat 06 Nov 2021 14:00:24 GMT +0000. [15:32:59] RECOVERY - www.burnout.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.burnout.wiki' will expire on Sat 06 Nov 2021 13:49:27 GMT +0000. [15:33:04] RECOVERY - hi.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'hi.gyaanipedia.com' will expire on Sat 06 Nov 2021 14:08:24 GMT +0000. [15:34:00] RECOVERY - pa.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'pa.gyaanipedia.com' will expire on Sat 06 Nov 2021 14:11:17 GMT +0000. [15:55:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.42, 1.88, 1.38 [15:57:13] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.57, 1.38, 1.25 [15:59:15] RECOVERY - commons.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'commons.gyaanipedia.com' will expire on Sat 06 Nov 2021 14:23:16 GMT +0000. [16:32:23] PROBLEM - diamowiki.ga - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:35:23] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.11, 6.71, 5.47 [16:35:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.34, 8.38, 6.74 [16:36:51] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.57, 7.03, 6.07 [16:37:18] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.70, 6.49, 5.54 [16:37:27] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.17, 7.52, 6.61 [16:38:45] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.23, 6.50, 5.99 [16:39:14] RECOVERY - diamowiki.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'diamowiki.ga' will expire on Sat 18 Sep 2021 14:34:47 GMT +0000. [16:41:26] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.18, 6.61, 6.44 [17:05:10] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.07, 7.71, 6.54 [17:06:02] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.83, 7.98, 6.47 [17:07:09] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.51, 6.69, 6.30 [17:08:01] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.38, 7.31, 6.40 [17:14:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.04, 6.24, 6.26 [17:35:01] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.69, 7.81, 6.45 [17:36:01] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.54, 7.09, 5.98 [17:37:00] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.42, 7.26, 6.41 [17:38:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.57, 6.63, 5.96 [17:42:58] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.79, 6.52, 6.36 [18:07:23] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [18:07:23] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00057029724121 seconds [18:14:39] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [18:20:32] PROBLEM - diamowiki.ga - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:27:22] RECOVERY - diamowiki.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'diamowiki.ga' will expire on Sat 18 Sep 2021 14:34:47 GMT +0000. [18:44:49] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [18:44:49] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000921487808228 seconds [18:46:47] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.42, 7.01, 6.27 [18:48:42] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.81, 6.53, 6.18 [18:52:37] PROBLEM - diamowiki.ga - reverse DNS on sslhost is UNKNOWN: [18:54:21] PROBLEM - cp12 Disk Space on cp12 is WARNING: DISK WARNING - free space: / 4174 MB (10% inode=96%); [19:00:13] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [19:05:54] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.04, 20.10, 16.88 [19:07:52] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.94, 19.49, 17.08 [19:07:59] PROBLEM - test2.miraheze.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for test2.miraheze.org could not be found [19:13:03] PROBLEM - cp3 Disk Space on cp3 is WARNING: DISK WARNING - free space: / 2604 MB (10% inode=94%); [19:14:53] RECOVERY - test2.miraheze.org - reverse DNS on sslhost is OK: SSL OK - test2.miraheze.org reverse DNS resolves to cp13.miraheze.org - NS RECORDS OK [19:33:30] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.38, 7.66, 6.76 [19:36:04] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.69, 7.40, 6.71 [19:37:27] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.44, 7.40, 6.99 [19:39:26] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.67, 6.55, 6.73 [19:40:01] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.31, 6.46, 6.50 [20:11:09] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.45, 7.19, 6.37 [20:13:09] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.96, 6.29, 6.13 [20:33:33] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.34, 8.61, 7.02 [20:35:10] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.13, 7.55, 6.66 [20:35:18] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 37.45, 26.50, 19.22 [20:36:18] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.85, 6.02, 3.67 [20:36:22] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2400:6180:0:d0::403:f001/cpweb [20:37:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.29, 7.55, 6.79 [20:38:14] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.32, 5.41, 3.73 [20:38:20] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [20:38:57] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [20:38:57] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.001589059829712 seconds [20:39:13] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 7.99, 8.01, 7.06 [20:41:11] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.55, 7.66, 7.05 [20:42:14] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.20, 4.63, 3.83 [20:43:11] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.11, 8.44, 7.42 [20:43:16] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.75, 23.20, 20.93 [20:46:00] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [20:49:21] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.20, 7.31, 6.18 [20:51:17] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.64, 20.10, 20.25 [20:52:53] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.43, 7.28, 7.78 [20:54:47] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.85, 7.98, 7.97 [20:56:41] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.72, 7.75, 7.91 [20:57:11] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.06, 7.26, 7.81 [20:59:05] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.11, 7.96, 7.01 [20:59:10] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.56, 7.62, 7.86 [21:00:33] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.83, 8.24, 8.00 [21:01:03] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.84, 7.40, 6.91 [21:01:11] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.40, 7.62, 7.84 [21:02:29] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.35, 7.75, 7.87 [21:02:59] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.33, 6.62, 6.68 [21:05:41] PROBLEM - cp13 HTTPS on cp13 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 344 bytes in 0.006 second response time [21:05:49] PROBLEM - cp13 HTTP 4xx/5xx ERROR Rate on cp13 is CRITICAL: CRITICAL - NGINX Error Rate is 99% [21:06:04] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 51.38.69.175/cpweb, 2001:41d0:801:2000::58af/cpweb [21:06:22] PROBLEM - cp13 Varnish Backends on cp13 is WARNING: No backends detected. If this is an error, see readme.txt [21:06:29] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.38.69.175/cpweb, 2001:41d0:801:2000::58af/cpweb [21:07:17] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 3.55, 2.23, 1.43 [21:09:14] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.54, 4.88, 6.49 [21:10:04] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JRRmb [21:10:05] [02miraheze/mediawiki] 07Universal-Omega 03f75e3d4 - Update Cosmos [21:10:11] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.03, 5.34, 6.65 [21:11:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.53, 1.85, 1.46 [21:11:16] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.96, 21.14, 20.15 [21:13:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 3.10, 2.34, 1.68 [21:13:17] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.89, 22.77, 20.82 [21:14:35] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.36, 6.51, 4.92 [21:15:16] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.19, 22.59, 21.06 [21:17:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.21, 1.82, 1.63 [21:17:16] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.18, 23.12, 21.45 [21:19:18] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.34, 22.13, 21.29 [21:20:05] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.query(hostname, [21:20:05] 0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000380039215088 seconds [21:20:28] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.20, 5.35, 5.00 [21:21:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 3.40, 2.09, 1.75 [21:22:23] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.04, 4.68, 4.79 [21:23:16] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.15, 17.83, 19.68 [21:27:16] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.39, 1.88, 1.79 [21:33:18] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 1.99, 2.01, 1.85 [21:34:53] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [21:35:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.76, 1.87, 1.81 [21:39:16] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.35, 1.55, 1.70 [21:40:05] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.80, 7.04, 6.17 [21:41:28] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:42:38] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:43:00] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:43:25] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_MediaWiki core] [21:44:03] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.82, 6.14, 6.01 [21:45:54] PROBLEM - diamowiki.ga - reverse DNS on sslhost is UNKNOWN: [21:53:31] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [21:53:31] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000723838806152 seconds [21:53:52] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.83, 2.23, 1.92 [21:55:47] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.30, 1.92, 1.84 [22:00:41] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK [22:04:58] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:05:21] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:05:22] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.19, 1.45, 1.66 [22:05:28] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:06:38] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [22:09:17] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.79, 1.77, 1.75 [22:13:14] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.02, 1.91, 1.80 [22:15:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.72, 1.83, 1.79 [22:17:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.53, 2.10, 1.89 [22:19:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.50, 1.85, 1.82 [22:23:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 1.85, 2.13, 1.95 [22:27:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.46, 1.88, 1.89 [22:33:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.30, 2.05, 1.96 [22:35:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.73, 1.94, 1.93 [22:41:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.10, 1.93, 1.93 [22:47:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.31, 1.87, 1.94 [22:51:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.18, 2.01, 1.98 [22:53:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.71, 1.93, 1.96 [23:01:14] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.28, 1.77, 1.83 [23:03:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.46, 1.64, 1.77 [23:11:13] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.81, 2.54, 2.08 [23:15:16] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 0.81, 1.61, 1.80 [23:21:13] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.04, 1.38, 1.66 [23:25:18] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.90, 1.74, 1.74 [23:27:14] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.26, 1.62, 1.70 [23:35:13] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.76, 1.97, 1.82 [23:36:07] Funny you mentioned that @dmehus. We've had to deal with a couple of impersonations of ERoss (WMF) and Stryn, which they were created in less than 48 hours. [23:36:21] Whoops, wrong channel. [23:39:15] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.94, 1.50, 1.67 [23:40:52] 🤗 [23:52:37] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [23:52:38] ackages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.001276969909668 seconds [23:54:04] PROBLEM - cp12 Current Load on cp12 is CRITICAL: CRITICAL - load average: 2.51, 1.96, 1.72 [23:55:59] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.26, 1.69, 1.65 [23:59:42] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp13.miraheze.org - CNAME OK