[00:09:09] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.47, 3.88, 3.23 [00:14:14] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.76, 2.10, 1.51 [00:15:07] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.38, 3.59, 3.40 [00:18:11] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.04, 1.63, 1.45 [00:21:06] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.58, 4.15, 3.63 [00:37:54] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.11, 2.54, 1.70 [00:39:01] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.64, 3.81, 3.87 [00:39:52] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.02, 1.97, 1.60 [00:41:00] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.56, 4.23, 4.02 [00:41:49] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.64, 1.53, 1.48 [01:18:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.18, 3.59, 3.97 [01:20:48] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.52, 4.35, 4.19 [01:24:47] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.98, 3.46, 3.87 [01:32:45] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.03, 3.45, 3.66 [01:34:44] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.83, 3.41, 3.61 [01:40:42] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.05, 3.34, 3.47 [01:42:42] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.33, 3.33, 3.45 [01:44:41] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.02, 3.20, 3.39 [01:52:38] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.95, 4.12, 3.64 [01:58:36] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.51, 3.51, 3.57 [02:02:35] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.20, 3.72, 3.62 [02:04:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.20, 3.51, 3.55 [02:10:33] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.60, 4.51, 3.95 [02:15:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.69, 7.05, 5.95 [02:17:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.51, 6.46, 5.86 [02:20:13] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.26, 7.00, 5.68 [02:21:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.09, 8.18, 6.71 [02:22:11] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.12, 6.70, 5.74 [02:22:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 6.77, 8.47, 6.50 [02:23:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.62, 6.94, 6.44 [02:24:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.06, 6.99, 6.20 [02:25:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.60, 6.55, 6.36 [02:26:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.00, 6.05, 5.96 [02:29:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.79, 7.50, 6.76 [02:31:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.06, 6.61, 6.53 [02:57:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.16, 6.99, 6.03 [02:59:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.38, 6.00, 5.78 [03:04:18] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.89, 3.69, 3.99 [03:22:13] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.43, 3.92, 3.73 [03:22:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.11, 6.97, 5.46 [03:24:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.23, 6.27, 5.39 [03:26:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.60, 3.85, 3.75 [03:28:12] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 6.19, 4.37, 3.93 [03:30:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.36, 3.87, 3.80 [03:46:12] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.35, 3.65, 3.63 [03:48:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.72, 3.88, 3.74 [03:58:34] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.85, 6.52, 5.25 [04:00:30] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.69, 6.07, 5.23 [04:03:12] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.64, 6.50, 4.88 [04:03:49] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.02, 6.57, 4.96 [04:04:12] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.79, 3.21, 3.40 [04:05:49] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.46, 5.72, 4.84 [04:07:12] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 3.97, 5.36, 4.80 [04:17:09] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.66, 3.55, 3.43 [04:21:19] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.59, 6.53, 5.42 [04:23:13] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.56, 5.32, 5.10 [04:27:06] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.95, 3.84, 3.58 [04:29:06] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.80, 3.42, 3.45 [04:31:05] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.07, 3.27, 3.39 [04:44:59] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.99, 3.32, 3.29 [04:46:59] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.91, 3.30, 3.29 [05:21:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.93, 3.54, 3.17 [05:25:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.44, 3.08, 3.07 [05:31:48] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.41, 3.59, 3.23 [05:33:47] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.90, 3.47, 3.24 [05:35:47] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.62, 3.32 [05:37:46] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.10, 3.23, 3.20 [06:03:45] RECOVERY - db13 APT on db13 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:08:50] RECOVERY - mem2 APT on mem2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:09:51] RECOVERY - mail2 APT on mail2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:13:44] RECOVERY - ns2 APT on ns2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:14:20] RECOVERY - gluster3 APT on gluster3 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:22:31] RECOVERY - puppet3 APT on puppet3 is OK: APT OK: 4 packages available for upgrade (0 critical updates). [06:27:16] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:28:27] RECOVERY - bacula2 APT on bacula2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:28:30] RECOVERY - graylog2 APT on graylog2 is OK: APT OK: 4 packages available for upgrade (0 critical updates). [06:35:39] RECOVERY - ns1 APT on ns1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:37:09] RECOVERY - mem1 APT on mem1 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:41:58] RECOVERY - db11 APT on db11 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [06:42:59] RECOVERY - phab2 APT on phab2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [06:45:43] RECOVERY - gluster4 APT on gluster4 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [06:51:13] RECOVERY - mon2 APT on mon2 is OK: APT OK: 19 packages available for upgrade (0 critical updates). [06:53:20] RECOVERY - ldap2 APT on ldap2 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [07:00:36] RECOVERY - db12 APT on db12 is OK: APT OK: 9 packages available for upgrade (0 critical updates). [07:08:31] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 9.30, 11.04, 6.76 [07:10:30] PROBLEM - db12 Current Load on db12 is WARNING: WARNING - load average: 1.79, 7.64, 6.03 [07:12:29] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 1.63, 5.69, 5.51 [07:21:31] PROBLEM - mw9 APT on mw9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [07:22:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 12.73, 8.92, 5.51 [07:23:27] RECOVERY - mw9 APT on mw9 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [07:24:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.33, 7.42, 5.38 [07:26:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.63, 6.32, 5.21 [07:29:24] !log [rhinos@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/purgeList.php --wiki=gratisdatawiki --all-namespaces --db-touch --delay=1 (END - exit=2) [07:29:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:29:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.64, 6.66, 4.96 [07:31:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.69, 5.89, 4.89 [07:34:28] [02puppet] 07RhinosF1 opened pull request 03#2138: mwscript: mark purgeList.php long - 13https://git.io/JMV9n [07:34:38] Reception123: ^ [07:38:37] Why does Mobile.css not work here on Miraheze? [07:45:02] There's a task somewhere [07:45:33] [02puppet] 07Reception123 closed pull request 03#2138: mwscript: mark purgeList.php long - 13https://git.io/JMV9n [07:45:35] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JMVQy [07:45:36] [02miraheze/puppet] 07RhinosF1 03b1fad71 - mwscript: mark purgeList.php long (#2138) [07:56:08] RhinosF1 🥲 Okay [07:56:21] It really a pain in the ass [10:01:44] !log [rhinos@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/purgeList.php --wiki=gratisdatawiki --all-namespaces --db-touch --delay=1 --verbose (END - exit=0) [10:01:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:06:52] .in 9hours do 09:44:00 !log [urbanecm@mwmaint1002 ~]$ foreachwiki extensions/CheckUser/maintenance/fixTrailingSpacesInLogs.php for mh [10:06:52] RhinosF1: Okay, will remind at 2021-12-01 - 19:06:52GMT [10:48:25] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thedev.gq All nameservers failed to answer the query. [10:49:55] PROBLEM - wikidiffs.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [10:49:55] ckages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00318169593811 seconds [10:50:07] PROBLEM - otcg.ml - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/dist-p [10:50:07] es/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query otcg.ml. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [10:50:14] PROBLEM - wiki.apadotech.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/pyth [10:50:14] ist-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query apadotech.ga. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [10:50:17] PROBLEM - matttest.mtwiki.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - matttest.mtwiki.cf All nameservers failed to answer the query. [10:51:25] PROBLEM - miraheze.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.ml All nameservers failed to answer the query. [10:51:39] PROBLEM - wikileague.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikileague.cf All nameservers failed to answer the query. [10:51:47] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [10:51:53] PROBLEM - miraheze.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.gq All nameservers failed to answer the query. [10:53:45] PROBLEM - wiki-asterix.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki-asterix.cf All nameservers failed to answer the query. [10:54:09] PROBLEM - miraheze.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.ga All nameservers failed to answer the query. [10:54:15] PROBLEM - ircwiki.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/dis [10:54:15] kages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query ircwiki.cf. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [10:56:48] PROBLEM - wikidiffs.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikidiffs.ga All nameservers failed to answer the query. [10:56:57] PROBLEM - otcg.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - otcg.ml All nameservers failed to answer the query. [10:58:37] PROBLEM - diamowiki.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/d [10:58:37] ackages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query diamowiki.ga. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [10:58:57] PROBLEM - wikileague.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/ [10:58:57] packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wikileague.cf. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:00:51] PROBLEM - wiki-asterix.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python [11:00:51] t-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wiki-asterix.cf. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:01:17] PROBLEM - miraheze.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/di [11:01:17] ckages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query miraheze.ga. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:01:37] PROBLEM - pj-masks-info.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/pytho [11:01:37] st-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query pj-masks-info.cf. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:02:15] RECOVERY - wiki.thedev.gq - reverse DNS on sslhost is OK: SSL OK - wiki.thedev.gq reverse DNS resolves to cp21.miraheze.org - CNAME OK [11:02:31] PROBLEM - storytime.jdstroy.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - storytime.jdstroy.cf All nameservers failed to answer the query. [11:03:05] PROBLEM - wiki.konjitownmc.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.konjitownmc.cf All nameservers failed to answer the query. [11:03:15] PROBLEM - linkwiki.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - linkwiki.cf All nameservers failed to answer the query. [11:03:50] RECOVERY - otcg.ml - reverse DNS on sslhost is OK: SSL OK - otcg.ml reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [11:04:44] PROBLEM - memipedia.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - memipedia.ga All nameservers failed to answer the query. [11:04:46] PROBLEM - matttest.mtwiki.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/pyt [11:04:46] dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00181770324707 seconds [11:05:03] PROBLEM - miraheze.ml - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/di [11:05:04] ckages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query miraheze.ml. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:05:37] RECOVERY - diamowiki.ga - reverse DNS on sslhost is OK: SSL OK - diamowiki.ga reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:06:34] PROBLEM - lakehub.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - lakehub.ga All nameservers failed to answer the query. [11:07:47] PROBLEM - wiki-asterix.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki-asterix.cf All nameservers failed to answer the query. [11:08:22] PROBLEM - miraheze.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.ga All nameservers failed to answer the query. [11:08:49] PROBLEM - pj-masks-info.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - pj-masks-info.cf All nameservers failed to answer the query. [11:10:06] PROBLEM - wiki.konjitownmc.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/pyt [11:10:06] dist-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wiki.konjitownmc.cf. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS [11:10:06] ation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:10:29] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ripto.gq All nameservers failed to answer the query. [11:10:32] RECOVERY - wikidiffs.ga - reverse DNS on sslhost is OK: SSL OK - wikidiffs.ga reverse DNS resolves to cp20.miraheze.org - CNAME FLAT [11:11:23] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thedev.gq All nameservers failed to answer the query. [11:11:47] PROBLEM - data.memipedia.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/pytho [11:11:47] st-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query data.memipedia.ga. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:11:51] PROBLEM - miraheze.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - miraheze.ml All nameservers failed to answer the query. [11:11:57] PROBLEM - wiki.apadotech.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.apadotech.ga All nameservers failed to answer the query. [11:12:48] PROBLEM - otcg.ml - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - otcg.ml All nameservers failed to answer the query. [11:13:56] PROBLEM - lakehub.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.query(hostname, 'A [11:13:56] ) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003780126571655 seconds [11:15:00] PROBLEM - ircwiki.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ircwiki.cf All nameservers failed to answer the query. [11:15:22] PROBLEM - diamowiki.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - diamowiki.ga All nameservers failed to answer the query. [11:15:31] PROBLEM - miraheze.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/di [11:15:31] ckages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query miraheze.ga. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:15:42] PROBLEM - wiki.ct777.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ct777.cf All nameservers failed to answer the query. [11:16:56] PROBLEM - wiki.konjitownmc.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.konjitownmc.cf All nameservers failed to answer the query. [11:17:16] PROBLEM - linkwiki.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/dis [11:17:16] kages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query linkwiki.cf. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:17:29] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/ [11:17:29] packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query ripto.gq. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed ou [11:17:29] erver 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:17:48] paladox: ^ [11:17:58] They are all freenom domains [11:18:37] RECOVERY - wiki.thedev.gq - reverse DNS on sslhost is OK: SSL OK - wiki.thedev.gq reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:18:52] PROBLEM - miraheze.ml - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/di [11:18:52] ckages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query miraheze.ml. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:19:10] RECOVERY - memipedia.ga - reverse DNS on sslhost is OK: SSL OK - memipedia.ga reverse DNS resolves to cp21.miraheze.org - CNAME FLAT [11:19:12] PROBLEM - matttest.mtwiki.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - matttest.mtwiki.cf All nameservers failed to answer the query. [11:19:28] PROBLEM - gp.ct777.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - gp.ct777.cf All nameservers failed to answer the query. [11:19:42] RECOVERY - miraheze.gq - reverse DNS on sslhost is OK: SSL OK - miraheze.gq reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [11:19:42] PROBLEM - wikidiffs.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikidiffs.ga All nameservers failed to answer the query. [11:20:52] PROBLEM - lakehub.ga - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - lakehub.ga All nameservers failed to answer the query. [11:22:22] !log downtime sslhost for until 15:21 [11:22:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:22:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.46, 7.94, 5.53 [11:24:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.32, 6.51, 5.29 [11:26:50] PROBLEM - wikidiffs.ga - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/di [11:26:51] ckages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wikidiffs.ga. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation tim [11:26:51] t.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:31:24] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: SSL OK - wiki.ripto.gq reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:40:19] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.41, 7.45, 5.51 [11:40:45] RECOVERY - wikidiffs.ga - reverse DNS on sslhost is OK: SSL OK - wikidiffs.ga reverse DNS resolves to cp20.miraheze.org - CNAME FLAT [11:40:50] RECOVERY - wikileague.cf - reverse DNS on sslhost is OK: SSL OK - wikileague.cf reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [11:44:04] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thedev.gq All nameservers failed to answer the query. [11:46:08] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.12, 7.84, 6.35 [11:49:51] PROBLEM - wikileague.cf - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikileague.cf All nameservers failed to answer the query. [11:50:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 12.24, 8.48, 6.85 [11:50:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 23.43, 12.71, 7.93 [11:50:38] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/d [11:50:38] ackages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wiki.ripto.gq. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:51:10] RECOVERY - wiki.thedev.gq - reverse DNS on sslhost is OK: SSL OK - wiki.thedev.gq reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:54:00] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.82, 7.61, 6.87 [11:55:56] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 13.91, 10.96, 8.23 [11:56:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.66, 7.68, 7.22 [11:56:54] PROBLEM - wikileague.cf - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/python3/ [11:56:54] packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wikileague.cf. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [11:57:40] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ripto.gq All nameservers failed to answer the query. [12:00:27] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.thedev.gq All nameservers failed to answer the query. [12:01:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.76, 7.72, 7.58 [12:02:59] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.18, 3.02, 2.59 [12:04:43] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: SSL OK - wiki.ripto.gq reverse DNS resolves to cp20.miraheze.org - CNAME OK [12:04:58] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.07, 2.94, 2.61 [12:07:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.71, 5.73, 6.74 [12:08:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 6.05, 6.42, 6.74 [12:13:12] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.73, 8.08, 5.67 [12:14:13] PROBLEM - wiki.ripto.gq - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ripto.gq All nameservers failed to answer the query. [12:14:31] PROBLEM - wiki.thedev.gq - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 79, in check_records cname = str(dns_resolver.query(hostname, 'CNAME')[0]) File "/usr/lib/python3/ [12:14:31] packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query wiki.thedev.gq. IN CNAME: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [12:14:39] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.07, 6.03, 4.89 [12:15:12] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.06, 7.24, 5.66 [12:16:34] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.17, 6.08, 5.05 [12:19:12] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.40, 6.08, 5.57 [12:20:56] RECOVERY - wiki.ripto.gq - reverse DNS on sslhost is OK: SSL OK - wiki.ripto.gq reverse DNS resolves to cp21.miraheze.org - CNAME OK [12:21:13] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 15.85, 12.57, 8.99 [12:21:18] RECOVERY - wiki.thedev.gq - reverse DNS on sslhost is OK: SSL OK - wiki.thedev.gq reverse DNS resolves to cp21.miraheze.org - CNAME OK [12:21:27] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 13.74, 9.34, 7.50 [12:25:21] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.13, 7.64, 7.24 [12:25:32] RECOVERY - wikileague.cf - reverse DNS on sslhost is OK: SSL OK - wikileague.cf reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [12:27:18] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 10.22, 8.24, 7.48 [12:28:42] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.12, 6.75, 5.88 [12:29:15] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.28, 7.89, 7.47 [12:30:39] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.55, 6.66, 5.95 [12:32:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.17, 6.24, 7.57 [12:33:08] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.02, 5.71, 6.67 [12:38:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.01, 4.83, 6.52 [12:52:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.12, 9.55, 7.65 [12:54:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.20, 7.86, 7.26 [12:58:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.32, 6.72, 5.80 [13:00:28] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.11, 6.46, 5.82 [13:01:43] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.95, 1.86, 1.25 [13:02:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.04, 5.90, 6.69 [13:03:40] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.37, 1.32, 1.12 [13:19:52] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.81, 7.51, 6.48 [13:20:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 11.99, 8.36, 6.83 [13:22:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.32, 6.88, 6.13 [13:22:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.54, 7.90, 6.85 [13:24:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.04, 6.71, 6.53 [13:25:42] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.71, 7.89, 7.11 [13:26:28] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.73, 6.17, 6.05 [13:27:38] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.43, 8.19, 7.32 [13:27:47] .op [13:27:48] Attempting to OP... [13:29:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.69, 7.11, 7.02 [13:31:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.10, 6.41, 6.77 [13:34:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.10, 6.75, 6.46 [13:43:17] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.21, 7.66, 6.57 [13:43:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 10.91, 8.93, 7.64 [13:44:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.42, 8.00, 7.64 [13:45:14] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.14, 7.14, 6.53 [13:47:12] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.52, 6.30, 6.30 [13:50:36] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 16.38, 10.59, 8.61 [13:51:05] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.97, 8.05, 6.94 [13:53:02] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 5.77, 7.40, 6.86 [13:53:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.45, 7.90, 7.95 [13:55:00] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.36, 6.51, 6.59 [13:55:40] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.90, 8.12, 8.00 [13:56:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.45, 7.51, 7.95 [14:00:25] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.82, 3.44, 3.16 [14:01:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.24, 7.58, 7.95 [14:02:24] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.30, 3.27, 3.12 [14:07:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.65, 8.24, 7.92 [14:10:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.35, 5.79, 6.69 [14:10:38] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.17, 6.92, 6.46 [14:11:49] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.05, 6.76, 5.57 [14:12:36] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 6.70, 6.75, 6.45 [14:13:12] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.11, 7.35, 6.25 [14:13:49] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.65, 5.62, 5.30 [14:15:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.65, 7.01, 7.67 [14:16:29] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.44, 6.87, 6.58 [14:17:12] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 5.53, 6.54, 6.19 [14:17:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.71, 7.31, 7.65 [14:18:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.04, 7.25, 6.74 [14:19:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.93, 6.66, 7.37 [14:20:05] PROBLEM - mw9 APT on mw9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [14:21:17] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.89, 10.04, 8.15 [14:21:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 12.61, 8.73, 8.03 [14:22:01] RECOVERY - mw9 APT on mw9 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [14:22:28] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.74, 7.83, 7.11 [14:26:04] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.74, 6.44, 5.60 [14:26:28] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.21, 8.24, 7.40 [14:28:00] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.61, 6.12, 5.59 [14:28:53] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.40, 19.51, 18.05 [14:30:45] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.26, 7.88, 7.94 [14:30:52] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.06, 18.59, 17.90 [14:32:28] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.90, 7.49, 7.54 [14:34:12] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.61, 3.73, 3.41 [14:35:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.59, 6.90, 7.92 [14:36:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.69, 5.29, 6.77 [14:38:28] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.27, 5.39, 6.60 [14:40:12] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.78, 2.98, 3.21 [14:41:38] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.21, 5.11, 6.72 [14:46:12] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.58, 3.61, 3.36 [14:48:12] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.81, 3.26, 3.25 [14:52:14] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.57, 3.85, 3.47 [14:54:13] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.73, 3.39, 3.35 [15:11:10] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.96, 3.41, 3.27 [15:21:01] RECOVERY - lakehub.ga - reverse DNS on sslhost is OK: SSL OK - lakehub.ga reverse DNS resolves to cp21.miraheze.org - NS RECORDS OK [15:21:39] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns1.dreamhost.com.', 'ns3.dreamhost.com.', 'ns2.dreamhost.com.'], 'CNAME': None} PROBLEM - wiki.3805.co.uk - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ultra104.uk2.net.', 'ultra103.uk2.net.'], 'CNAME': '3805.miraheze.org.'} [15:21:39] PROBLEM - dreamsit.com.br - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns82.domaincontrol.com.', 'ns81.domaincontrol.com.'], 'CNAME': None} [15:21:40] PROBLEM - wiki.minkyu.kim - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns3.wordpress.com.', 'ns2.wordpress.com.', 'ns1.wordpress.com.'], 'CNAME': 'mk.miraheze.org.'} [15:21:40] PROBLEM - fr.gyaanipedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for fr.gyaanipedia.com could not be found [15:21:40] PROBLEM - en.famepedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'en.famepedia.org' expires in 6 day(s) (Wed 08 Dec 2021 04:12:21 GMT +0000). PROBLEM - ambient.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'ambient.wiki' expires in 3 day(s) (Sun 05 Dec 2021 08:49:56 GMT +0000). PROBLEM - pa.gyaanipedia.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for pa.gyaanipedia.com c [15:24:36] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 3.25, 6.91, 5.92 [15:25:06] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.17, 3.75, 3.51 [15:26:36] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 2.68, 5.54, 5.54 [15:31:04] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.83, 3.55, 3.53 [15:37:02] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.90, 2.84, 3.24 [15:44:28] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.49, 6.31, 4.96 [15:45:49] RhinosF1: https://phabricator.miraheze.org/T8364 what else is left to do on the task? [15:45:51] [url] ⚓ T8364 DNS failures for all freenom registered domains | phabricator.miraheze.org [15:46:28] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.52, 5.71, 4.89 [15:48:41] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.01, 1.86, 1.16 [15:50:41] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.82, 1.48, 1.10 [15:50:57] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.53, 3.89, 3.52 [15:52:56] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.53, 3.83, 3.55 [15:54:56] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.85, 3.59 [15:55:11] JohnLewis: remove my downtime [15:55:20] And reset topic [15:55:47] Luckily no users complained [15:56:31] If anyone did, they can be directed to freenom :P [15:56:55] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.25, 3.79, 3.61 [15:57:39] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.79, 7.51, 6.23 [15:58:43] JohnLewis: https://phabricator.miraheze.org/T8364#168916 [15:58:44] [url] ⚓ T8364 DNS failures for all freenom registered domains | phabricator.miraheze.org [15:58:55] No idea what happened but not us [15:59:22] tbh, I don't think a task really achieved much anyway, I wouldn't have even opened a task [16:00:40] Tracking, a notice we're aware, central place we can send dupe tasks if one was made, reference for icinga history [16:00:54] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.96, 2.93, 3.30 [16:01:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.60, 6.77, 6.32 [16:04:44] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.56, 3.53, 3.47 [16:08:33] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.10, 3.31, 3.40 [16:11:07] mkay, I still wouldn't have made one regardless anyway [16:13:06] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.03, 6.06, 5.47 [16:15:06] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.37, 5.87, 5.48 [16:17:10] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.88, 3.72, 3.53 [16:18:55] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 10.82, 7.73, 6.52 [16:19:05] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.14, 3.24, 3.38 [16:19:50] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 14.55, 8.56, 6.78 [16:21:06] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 13.51, 9.47, 6.99 [16:22:37] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:22:41] PROBLEM - mw9 APT on mw9 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [16:24:44] RECOVERY - mw9 APT on mw9 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [16:27:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:28:38] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.85, 4.22, 3.68 [16:40:05] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.58, 3.66, 3.82 [16:40:55] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.46, 6.74, 5.58 [16:41:07] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.17, 7.47, 7.93 [16:41:31] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.04, 7.85, 6.66 [16:42:35] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 11.95, 7.95, 6.37 [16:43:29] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.26, 7.41, 6.63 [16:43:54] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.16, 3.87, 3.86 [16:45:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.03, 3.59, 3.77 [16:46:55] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.65, 7.22, 6.29 [16:47:25] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.55, 6.69, 6.54 [16:48:33] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.71, 7.81, 6.95 [16:48:36] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:48:55] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.97, 6.68, 6.19 [16:49:06] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 12.12, 9.16, 8.42 [16:52:33] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.39, 6.02, 6.42 [16:53:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [16:53:48] !log delete 5 old indices on graylog [16:53:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:54:46]