[00:02:18] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.26, 20.28, 18.04 [00:02:38] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 10.10, 5.72, 4.40 [00:04:38] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.91, 5.40, 4.44 [00:06:38] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.68, 5.82, 4.71 [00:06:57] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.67, 3.90, 3.23 [00:12:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.78, 3.82, 3.46 [00:14:38] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.63, 5.84, 5.47 [00:14:51] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.49, 4.01, 3.57 [00:16:38] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.57, 6.91, 5.90 [00:16:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.29, 3.81, 3.55 [00:20:38] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.89, 5.90, 5.74 [00:20:51] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.56, 3.48 [00:22:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.22, 3.46, 3.46 [00:23:55] PROBLEM - www.pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.pornwiki.org' expires in 15 day(s) (Thu 18 Nov 2021 00:20:12 GMT +0000). [00:24:38] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 10.38, 7.24, 6.24 [00:24:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.28, 3.08, 3.32 [00:29:25] PROBLEM - cp15 Current Load on cp15 is WARNING: WARNING - load average: 1.47, 1.71, 1.19 [00:31:21] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.98, 1.51, 1.18 [00:52:17] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 13.30, 16.87, 23.02 [00:52:38] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.07, 4.33, 5.84 [00:58:17] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.52, 15.68, 20.32 [00:58:38] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.19, 3.37, 4.91 [01:08:20] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.43, 3.87, 3.33 [01:10:16] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.84, 3.89, 3.41 [01:14:10] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.33, 3.23, 3.25 [01:21:54] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.13, 3.78, 3.44 [01:25:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.13, 3.34, 3.36 [01:29:39] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 4.00, 3.57, 3.45 [01:33:32] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.37, 3.67, 3.51 [01:37:26] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.35, 3.67, 3.56 [01:39:23] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.10, 3.89, 3.66 [01:41:20] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.11, 3.63, 3.60 [01:42:46] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [01:49:07] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.30, 3.58, 3.53 [01:51:04] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.75, 3.64, 3.56 [01:53:01] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.56, 3.98, 3.69 [01:56:54] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.60, 3.88, 3.72 [02:06:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.72, 3.14, 3.38 [02:37:01] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.57, 3.32, 3.06 [02:38:57] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.23, 3.61, 3.19 [02:40:54] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.01, 3.43, 3.18 [02:42:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.63, 2.90, 3.02 [02:52:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.14, 3.03 [02:54:51] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.44, 3.34, 3.10 [02:56:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.69, 3.07, 3.03 [03:02:51] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.23, 3.60, 3.25 [03:04:51] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.47, 3.42, 3.23 [03:06:51] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.86, 2.92, 3.07 [03:17:09] [02CreateWiki] 07Universal-Omega opened pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:18:19] [02CreateWiki] 07Universal-Omega commented on pull request 03#233: Cleanup Echo notifications code and fix Echo email notifications - 13https://git.io/JP1fS [03:18:21] [02CreateWiki] 07Universal-Omega closed pull request 03#233: Cleanup Echo notifications code and fix Echo email notifications - 13https://git.io/JBSa3 [03:18:23] [02miraheze/CreateWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [03:18:24] [02CreateWiki] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vpJTL [03:19:32] miraheze/CreateWiki - Universal-Omega the build has errored. [03:20:10] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:23:31] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:23:47] miraheze/CreateWiki - Universal-Omega the build has errored. [03:24:47] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:29:06] miraheze/CreateWiki - Universal-Omega the build has errored. [03:33:46] miraheze/CreateWiki - Universal-Omega the build passed. [03:36:20] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.34, 3.96, 3.40 [03:36:33] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:38:17] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.37, 3.74, 3.38 [03:42:47] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [03:44:08] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.19, 3.76, 3.46 [03:45:58] miraheze/CreateWiki - Universal-Omega the build passed. [03:46:04] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.44, 3.37, 3.36 [03:49:41] miraheze/CreateWiki - Universal-Omega the build passed. [04:03:38] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [04:03:38] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00015425682068 seconds [04:10:36] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [04:29:20] Should users be concerned about stability given there's no budget to address stuff currently? (from what I'm reading here that is) [04:43:54] There is a budget, the issue is that it's close to the time the next budget will get approved thus giving SRE no time to look for another server from what I read a little more below that. [05:21:23] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.01, 3.21, 2.71 [05:23:20] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.11, 3.12, 2.73 [05:35:04] Thanks for clarifying. So, perhaps the stability issues might go on for a bit then, given how much time it might take to get budget freed and set it up? [05:35:57] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:36:19] [02CreateWiki] 07Universal-Omega labeled pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:36:20] [02CreateWiki] 07Universal-Omega labeled pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:36:22] [02CreateWiki] 07Universal-Omega labeled pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:36:48] [02CreateWiki] 07Universal-Omega assigned pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:37:03] [02CreateWiki] 07Universal-Omega unassigned pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [05:42:04] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [05:42:04] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.001231908798218 seconds [05:49:08] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [06:05:31] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.10, 5.48, 3.47 [06:07:31] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.40, 6.41, 4.05 [06:09:31] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.56, 6.92, 4.54 [06:11:31] PROBLEM - db11 Current Load on db11 is CRITICAL: CRITICAL - load average: 8.11, 7.21, 4.93 [06:11:55] !log renamed reviwikiwiki to reviwiki [06:12:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:13:31] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.06, 7.30, 5.25 [06:15:31] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.38, 6.72, 5.30 [06:16:43] !log renamed supermariowikiwiki to mariowiki. [06:16:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:17:39] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [06:17:39] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.002336263656616 seconds [06:19:31] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 6.77, 7.11, 5.81 [06:23:31] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 5.82, 6.75, 5.97 [06:24:39] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [06:26:06] !log [reception@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteArchivedRevisions.php --wiki=quyranesswiki --delete (END - exit=0) [06:26:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:30:21] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.88, 7.29, 6.46 [06:32:00] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [06:32:00] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.006808519363403 seconds [06:34:11] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 6.18, 6.51, 6.32 [06:46:33] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [06:54:40] [02ssl] 07Reception123 closed pull request 03#437: Remove reviwiki from the SSL - 13https://git.io/JPKOR [06:54:42] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-1/±1] 13https://git.io/JP1is [06:54:43] [02miraheze/ssl] 07revi 039bcecfb - Remove reviwiki from the SSL (#437) [06:57:38] thx [06:59:38] it probably should also remove/suppress private.revi.wiki icinga alert (should be fine to remove if it doesn't) [07:04:21] PROBLEM - db11 Current Load on db11 is WARNING: WARNING - load average: 7.15, 6.62, 6.19 [07:08:28] RECOVERY - private.revi.wiki - Sectigo on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [07:10:05] RECOVERY - db11 Current Load on db11 is OK: OK - load average: 6.72, 6.74, 6.39 [07:56:48] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [07:56:48] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00226330757141 seconds [08:03:44] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [08:15:44] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 7.78, 8.22, 6.01 [08:17:39] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 2.00, 5.88, 5.41 [09:07:10] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [09:07:11] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.000688076019287 seconds [09:21:30] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [09:35:40] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [09:35:40] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.001935482025146 seconds [09:42:35] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [10:11:05] PROBLEM - staratlaswiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'staratlaswiki.org' expires in 7 day(s) (Wed 10 Nov 2021 10:07:11 GMT +0000). [10:25:06] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [10:25:06] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00271511077881 seconds [10:30:18] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.75, 1.87, 1.11 [10:32:01] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [10:32:18] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 1.10, 1.69, 1.14 [10:41:31] PROBLEM - cp15 Current Load on cp15 is CRITICAL: CRITICAL - load average: 1.90, 2.79, 1.52 [10:43:31] PROBLEM - cp15 Current Load on cp15 is WARNING: WARNING - load average: 0.39, 1.96, 1.37 [10:45:31] RECOVERY - cp15 Current Load on cp15 is OK: OK - load average: 0.42, 1.47, 1.26 [11:25:51] If I understand correctly, wasn't adding a new server part of the upcoming budget as well? I might have gotten the wrong idea though. [11:31:37] The current plan is an Infrastructure overhaul, not just adding one - adding one was part of the previous budget but that seemingly never went anywhere [11:34:53] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [11:34:53] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00871515274048 seconds [11:35:13] Ah, I see. [11:41:39] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [11:48:56] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [11:48:56] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.01104211807251 seconds [11:55:42] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [12:17:04] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [12:17:04] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.001848459243774 seconds [12:24:00] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [13:45:07] PROBLEM - wiki.aridia.space - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.aridia.space' expires in 15 day(s) (Thu 18 Nov 2021 13:36:19 GMT +0000). [13:48:57] PROBLEM - wiki.wilderyogi.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.wilderyogi.eu' expires in 15 day(s) (Thu 18 Nov 2021 13:46:36 GMT +0000). [14:01:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.50, 3.05, 2.76 [14:02:47] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [14:02:47] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.004289150238037 seconds [14:09:40] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [14:15:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.74, 3.31, 3.23 [14:51:26] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [14:51:26] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003538370132446 seconds [14:51:42] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.21, 3.68, 3.14 [14:53:42] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.81, 3.20, 3.02 [14:58:25] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [15:03:29] sre got a 504 error a few times [15:03:54] Is it continuous right now? [15:04:05] ok now its working [15:30:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.80, 3.10, 2.96 [15:34:27] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.38, 3.03, 2.99 [15:38:25] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.45, 3.45, 3.19 [15:42:24] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.43, 3.02, 3.08 [15:48:24] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [15:48:24] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00268816947937 seconds [15:55:20] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [16:02:36] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [16:02:37] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003488302230835 seconds [16:04:15] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.72, 3.80, 3.40 [16:06:15] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.74, 3.82, 3.46 [16:09:36] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [16:10:14] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.26, 3.39, 3.35 [16:14:12] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.52, 4.09, 3.66 [16:16:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.55, 3.68, 3.57 [16:22:09] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.80, 3.79, 3.58 [16:24:09] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.41, 3.79, 3.62 [16:36:06] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.47, 2.92, 3.36 [16:56:58] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.78, 3.29, 3.16 [16:58:58] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.28, 3.32, 3.19 [17:19:50] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.07, 3.81, 3.36 [17:27:19] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [17:27:20] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.0068941116333 seconds [17:27:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.61, 3.64, 3.58 [17:34:35] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [17:39:21] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPysM [17:39:23] [02miraheze/mw-config] 07Universal-Omega 033164028 - Reorder [17:40:50] miraheze/mw-config - Universal-Omega the build passed. [17:41:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.73, 3.20, 3.38 [17:44:07] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPyGQ [17:44:09] [02miraheze/mw-config] 07Universal-Omega 039446a06 - Add `$wgRPRatingAllowedNamespaces` to ManageWikiNamespaces [17:44:10] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [17:44:12] [02mw-config] 07Universal-Omega opened pull request 03#4198: Add `$wgRPRatingAllowedNamespaces` to ManageWikiNamespaces - 13https://git.io/JPyG7 [17:44:48] [02mw-config] 07Universal-Omega edited pull request 03#4198: T8192: add `$wgRPRatingAllowedNamespaces` to ManageWikiNamespaces - 13https://git.io/JPyG7 [17:45:16] miraheze/mw-config - Universal-Omega the build passed. [17:47:16] [02mw-config] 07Universal-Omega closed pull request 03#4198: T8192: add `$wgRPRatingAllowedNamespaces` to ManageWikiNamespaces - 13https://git.io/JPyG7 [17:47:17] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPyZ8 [17:47:19] [02miraheze/mw-config] 07Universal-Omega 031eb3cab - T8192: add `$wgRPRatingAllowedNamespaces` to ManageWikiNamespaces [17:47:20] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [17:47:22] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [17:48:15] miraheze/mw-config - Universal-Omega the build passed. [17:48:46] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [17:48:46] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00380802154541 seconds [17:53:23] PROBLEM - cp14 Current Load on cp14 is CRITICAL: CRITICAL - load average: 1.65, 2.28, 1.32 [17:55:22] RECOVERY - cp14 Current Load on cp14 is OK: OK - load average: 0.65, 1.67, 1.21 [17:55:42] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [17:59:44] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.49, 3.37 [18:01:43] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.05, 3.04, 3.23 [18:05:09] !log [@mw11] starting deploy of {'config': True} to all [18:05:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:05:27] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 17s [18:05:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:59] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPylL [18:09:01] [02miraheze/mw-config] 07Universal-Omega 03eb89df2 - Update ManageWikiSettings.php [18:09:02] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [18:09:26] [02mw-config] 07Universal-Omega opened pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:10:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPyl4 [18:10:18] [02miraheze/mw-config] 07Universal-Omega 03edc05e2 - Update ManageWikiSettings.php [18:10:19] [02mw-config] 07Universal-Omega synchronize pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:10:31] miraheze/mw-config - Universal-Omega the build passed. [18:11:21] miraheze/mw-config - Universal-Omega the build passed. [18:12:54] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPylF [18:12:55] [02miraheze/mw-config] 07Universal-Omega 03176a3c2 - Update LocalSettings.php [18:12:56] [02mw-config] 07Universal-Omega synchronize pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:13:58] miraheze/mw-config - Universal-Omega the build passed. [18:14:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPy8s [18:14:56] [02miraheze/mw-config] 07Universal-Omega 03e43120c - Update Defines.php [18:14:58] [02mw-config] 07Universal-Omega synchronize pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:16:01] miraheze/mw-config - Universal-Omega the build passed. [18:17:20] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JPy8A [18:17:22] [02miraheze/mw-config] 07Universal-Omega 0333e31fa - Update LocalSettings.php [18:17:23] [02mw-config] 07Universal-Omega synchronize pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:18:36] miraheze/mw-config - Universal-Omega the build passed. [18:22:36] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.06, 3.60, 3.26 [18:24:03] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [18:24:03] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00290584564209 seconds [18:24:35] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.77, 3.78, 3.38 [18:26:34] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.58, 3.29, 3.24 [18:26:36] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [18:26:38] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [18:26:39] [02mw-config] 07Universal-Omega closed pull request 03#4199: T8192: add RatePage configuration to ManageWiki - 13https://git.io/JPylZ [18:26:41] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://git.io/JPyBK [18:26:42] [02miraheze/mw-config] 07Universal-Omega 03b3549e8 - T8192: add RatePage configuration to ManageWiki (#4199) [18:27:53] miraheze/mw-config - Universal-Omega the build passed. [18:30:59] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [18:34:28] !log [@mw11] starting deploy of {'config': True} to all [18:34:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:34:40] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 12s [18:34:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:40:30] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.84, 3.06, 2.94 [18:42:29] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.20, 2.77, 2.85 [19:06:44] PROBLEM - celeste.ink - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'celeste.ink' expires in 15 day(s) (Thu 18 Nov 2021 19:02:04 GMT +0000). [19:21:15] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.26, 3.48, 3.11 [19:23:15] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.18, 3.68, 3.22 [19:25:14] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.90, 3.27, 3.12 [19:33:32] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [19:33:32] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.00232219696045 seconds [19:34:25] JohnLewis: Captcha seems to be not doing its job anymore. Thoughts? [19:35:05] I don't have access to any of the data so I can't comment [19:40:28] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [20:11:55] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:12:26] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:13:13] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:14:05] [02CreateWiki] 07Universal-Omega synchronize pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:14:30] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add new CreateWikiNotificationsManager service - 13https://git.io/JP1fg [20:14:49] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add new CreateWikiNotificationsManager service - 13https://git.io/JP1fg [20:14:58] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add CreateWikiNotificationsManager service - 13https://git.io/JP1fg [20:15:16] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [20:15:16] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003939151763916 seconds [20:22:31] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [20:23:59] miraheze/CreateWiki - Universal-Omega the build has errored. [20:24:08] miraheze/CreateWiki - Universal-Omega the build passed. [20:24:18] miraheze/CreateWiki - Universal-Omega the build has errored. [20:25:17] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:25:28] [02CreateWiki] 07Universal-Omega edited pull request 03#256: Fix notifications and add NotificationsManager service - 13https://git.io/JP1fg [20:27:16] miraheze/CreateWiki - Universal-Omega the build passed. [21:01:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy5M [21:01:48] [02miraheze/puppet] 07paladox 03fdb2b91 - randomise mw* puppet times [21:01:48] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.29, 3.95, 3.34 [21:01:49] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:01:51] [02puppet] 07paladox opened pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:02:04] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy5Q [21:02:05] [02miraheze/puppet] 07paladox 0313bd9a2 - Update mw11.yaml [21:02:07] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:02:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy55 [21:02:17] [02miraheze/puppet] 07paladox 035020f9f - Update mw10.yaml [21:02:19] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:02:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy5F [21:02:28] [02miraheze/puppet] 07paladox 03937e26c - Update mw12.yaml [21:02:29] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:02:38] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy5A [21:02:40] [02miraheze/puppet] 07paladox 03e959e72 - Update mw13.yaml [21:02:41] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:02:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPy5j [21:02:50] [02miraheze/puppet] 07paladox 03a585c30 - Update mw9.yaml [21:02:51] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:03:17] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JPydv [21:03:18] [02miraheze/puppet] 07paladox 0366be567 - Update test3.yaml [21:03:20] [02puppet] 07paladox synchronize pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:03:29] [02puppet] 07paladox closed pull request 03#2103: randomise mw* puppet times - 13https://git.io/JPy5D [21:03:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±7] 13https://git.io/JPydT [21:03:32] [02miraheze/puppet] 07paladox 035b57328 - randomise mw* puppet times (#2103) [21:03:34] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [21:03:35] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [21:05:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.31, 3.86, 3.46 [21:09:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.36, 3.16, 3.27 [21:36:36] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.90, 7.70, 5.12 [21:38:34] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.41, 6.63, 5.04 [21:53:35] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [21:53:35] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.003427505493164 seconds [22:00:25] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [22:07:41] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [22:07:42] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.002787113189697 seconds [22:14:37] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [22:21:25] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.21, 3.41, 2.91 [22:23:25] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.38, 2.96 [22:25:24] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.38, 2.71, 2.77 [22:28:53] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 116, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 101, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.quer [22:28:53] tname, 'A')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 992, in query timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 799, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 30.002741813659668 seconds [22:36:09] PROBLEM - zw.fontainebleau-avon.fr - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - zw.fontainebleau-avon.fr All nameservers failed to answer the query. [22:39:57] umm... the wiki lets encrypt certificate has expired? [22:40:37] Which wiki [22:40:45] SEC_ERROR_EXPIRED_CERTIFICATE from 4 august to 2 November www.portalsofphereon.com [22:42:52] Can see them [22:42:59] Apologies, I've filed https://phabricator.miraheze.org/T8240 [22:43:00] [url] ⚓ T8240 Numerous LE certs have expired | phabricator.miraheze.org [22:43:13] Reception123: can we please make sure these are kept on top off? [22:43:42] no problem, thanks [23:05:51] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.23, 19.33, 17.54 [23:15:37] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.82, 18.96, 18.65 [23:18:02] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPStH [23:18:04] [02miraheze/ssl] 07paladox 03bb7d875 - Update pornwiki.org.crt [23:19:25] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPStj [23:19:27] [02miraheze/ssl] 07paladox 0313f1874 - Update www.pornwiki.org.crt [23:20:23] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSqt [23:20:25] [02miraheze/ssl] 07paladox 03cd95bcd - Update files.pornwiki.org.crt [23:21:22] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSq3 [23:21:23] [02miraheze/ssl] 07paladox 034b10fa3 - Update en.pornwiki.org.crt [23:21:25] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.57, 20.84, 19.38 [23:22:48] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.88, 7.16, 5.42 [23:23:22] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.07, 17.86, 18.47 [23:23:47] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSq2 [23:23:48] [02miraheze/ssl] 07paladox 037019045 - Update sdiy.info.crt [23:24:15] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:24:40] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:24:49] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.40, 5.84, 5.15 [23:24:57] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSqX [23:24:58] [02miraheze/ssl] 07paladox 03f4b2c32 - Update wiki.meregos.com.crt [23:26:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSq7 [23:26:06] [02miraheze/ssl] 07paladox 0308a1b30 - Update wiki.worldsofweary.com.crt [23:26:12] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:27:18] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSqh [23:27:19] [02miraheze/ssl] 07paladox 03af6e707 - Update wiki.meeusen.net.crt [23:29:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSmm [23:29:06] [02miraheze/ssl] 07paladox 037d116bc - Update wiki.vinesh.eu.org.crt [23:30:16] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSmc [23:30:18] [02miraheze/ssl] 07paladox 033f15608 - Update www.baharna.org.crt [23:30:57] PROBLEM - cp13 Puppet on cp13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:31:36] PROBLEM - mw12 Puppet on mw12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:32:08] RECOVERY - wiki.meeusen.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.meeusen.net' will expire on Mon 31 Jan 2022 22:26:57 GMT +0000. [23:32:22] RECOVERY - baharna.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.baharna.org' will expire on Mon 31 Jan 2022 22:29:54 GMT +0000. [23:32:29] RECOVERY - sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Mon 31 Jan 2022 22:23:27 GMT +0000. [23:32:32] RECOVERY - www.sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Mon 31 Jan 2022 22:23:27 GMT +0000. [23:32:36] RECOVERY - wiki.meregos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.meregos.com' will expire on Mon 31 Jan 2022 22:24:31 GMT +0000. [23:32:40] RECOVERY - files.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'files.pornwiki.org' will expire on Mon 31 Jan 2022 22:19:53 GMT +0000. [23:32:45] RECOVERY - en.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.pornwiki.org' will expire on Mon 31 Jan 2022 22:21:00 GMT +0000. [23:32:55] RECOVERY - cp13 Puppet on cp13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:33:27] RECOVERY - wiki.vinesh.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.vinesh.eu.org' will expire on Mon 31 Jan 2022 22:28:33 GMT +0000. [23:33:55] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSm1 [23:33:56] [02miraheze/ssl] 07paladox 03ff38fbe - Update www.portalsofphereon.com.crt [23:34:11] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:34:15] RECOVERY - wiki.worldsofweary.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.worldsofweary.com' will expire on Mon 31 Jan 2022 22:25:48 GMT +0000. [23:34:42] RECOVERY - www.baharna.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.baharna.org' will expire on Mon 31 Jan 2022 22:29:54 GMT +0000. [23:36:12] RECOVERY - portalsofphereon.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.portalsofphereon.com' will expire on Mon 31 Jan 2022 22:31:57 GMT +0000. [23:37:06] RECOVERY - pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'pornwiki.org' will expire on Mon 31 Jan 2022 22:17:30 GMT +0000. [23:37:09] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.86, 4.94, 4.08 [23:37:15] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.28, 21.00, 19.48 [23:37:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSmN [23:37:39] [02miraheze/ssl] 07paladox 036f7a350 - Update wiki.autocountsoft.com.crt [23:37:44] RECOVERY - www.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.pornwiki.org' will expire on Mon 31 Jan 2022 22:18:29 GMT +0000. [23:38:39] RECOVERY - www.portalsofphereon.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.portalsofphereon.com' will expire on Mon 31 Jan 2022 22:31:57 GMT +0000. [23:38:54] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSYf [23:38:55] [02miraheze/ssl] 07paladox 038dc9bd5 - Update wiki.gesamtschule-nordkirchen.de.crt [23:39:06] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.27, 4.42, 3.98 [23:39:15] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.67, 19.31, 19.01 [23:40:01] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSYm [23:40:03] [02miraheze/ssl] 07paladox 03020a3e3 - Update www.christipedia.nl.crt [23:41:22] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSYc [23:41:23] [02miraheze/ssl] 07paladox 038f166ef - Update www.johanloopmans.nl.crt [23:43:32] PROBLEM - mwtask1 Puppet on mwtask1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:43:44] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSYa [23:43:45] [02miraheze/ssl] 07paladox 030993a15 - Update www.johanloopmans.nl.crt [23:44:12] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:44:38] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSYD [23:44:40] [02miraheze/ssl] 07paladox 039cd59c7 - Update www.rothwell-leeds.co.uk.crt [23:45:15] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.64, 21.13, 20.01 [23:45:53] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:46:27] RECOVERY - rothwell-leeds.co.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'www.rothwell-leeds.co.uk' will expire on Mon 31 Jan 2022 22:44:11 GMT +0000. [23:46:33] RECOVERY - wiki.autocountsoft.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.autocountsoft.com' will expire on Mon 31 Jan 2022 22:36:14 GMT +0000. [23:46:35] RECOVERY - wiki.gesamtschule-nordkirchen.de - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.gesamtschule-nordkirchen.de' will expire on Mon 31 Jan 2022 22:38:22 GMT +0000. [23:46:38] RECOVERY - johanloopmans.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'www.johanloopmans.nl' will expire on Mon 31 Jan 2022 22:42:56 GMT +0000. [23:46:39] RECOVERY - www.christipedia.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'www.christipedia.nl' will expire on Mon 31 Jan 2022 22:39:36 GMT +0000. [23:46:42] RECOVERY - www.johanloopmans.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'www.johanloopmans.nl' will expire on Mon 31 Jan 2022 22:42:56 GMT +0000. [23:46:45] RECOVERY - christipedia.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'www.christipedia.nl' will expire on Mon 31 Jan 2022 22:39:36 GMT +0000. [23:46:47] RECOVERY - www.rothwell-leeds.co.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'www.rothwell-leeds.co.uk' will expire on Mon 31 Jan 2022 22:44:11 GMT +0000. [23:48:08] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSOq [23:48:09] [02miraheze/ssl] 07paladox 0370a6a09 - Update thestarsareright.org.crt [23:49:15] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.55, 18.94, 19.37 [23:49:28] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSOW [23:49:30] [02miraheze/ssl] 07paladox 03f76fe1a - Update wiki.hrznstudio.com.crt [23:50:14] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [23:50:30] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSOg [23:50:32] [02miraheze/ssl] 07paladox 031508b74 - Update www.erikapedia.com.crt [23:51:43] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSO6 [23:51:44] [02miraheze/ssl] 07paladox 03224bbd0 - Update www.iceria.org.crt [23:51:46] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.01, 6.00, 4.84 [23:52:54] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPSOF [23:52:56] [02miraheze/ssl] 07paladox 03c01dca5 - Update www.mcpk.wiki.crt [23:53:43] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.62, 5.85, 4.94 [23:54:00] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPS3f [23:54:01] [02miraheze/ssl] 07paladox 0346a1e5d - Update www.lab612.at.crt [23:55:03] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPS3G [23:55:05] [02miraheze/ssl] 07paladox 03ad05177 - Update www.wikimicrofinanza.it.crt [23:56:03] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPS3W [23:56:05] [02miraheze/ssl] 07paladox 0399f16ee - Update www.zenbuddhism.info.crt [23:56:12] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [23:57:00] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JPS3E [23:57:02] [02miraheze/ssl] 07paladox 0349bdf98 - Update www.bluepageswiki.org.crt [23:57:38] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.37, 5.57, 4.99 [23:59:36] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.38, 4.87, 4.82