[00:00:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.94, 3.42, 3.45 [00:00:25] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.84, 2.56, 1.60 [00:00:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4de63250f8df...cd8b5bef0c72 [00:00:45] [url] Comparing 4de63250f8df...cd8b5bef0c72 · miraheze/ssl · GitHub | github.com [00:00:46] [02miraheze/ssl] 07MirahezeSSLBot 03cd8b5be - Bot: Update SSL cert for data.nonbinary.wiki [00:01:00] RECOVERY - wikileague.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'wikileague.cf' will expire on Mon 22 Aug 2022 22:21:46 GMT +0000. [00:01:25] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cd8b5bef0c72...0c3091b04ef4 [00:01:26] [url] Comparing cd8b5bef0c72...0c3091b04ef4 · miraheze/ssl · GitHub | github.com [00:01:27] [02miraheze/ssl] 07MirahezeSSLBot 030c3091b - Bot: Update SSL cert for mediawiki.ecfin.eu [00:01:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.67, 3.64 [00:01:39] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.22, 9.28, 8.66 [00:02:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.45, 1.76, 1.97 [00:02:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.76, 3.18, 2.62 [00:03:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.51, 4.27, 3.86 [00:03:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.85, 3.14, 1.68 [00:04:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.96, 4.13, 3.71 [00:04:21] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.99, 1.95, 1.59 [00:04:26] RECOVERY - cp20 Disk Space on cp20 is OK: DISK OK - free space: / 9728 MB (24% inode=96%); [00:04:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.76, 3.11, 2.67 [00:05:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.49, 8.78, 8.64 [00:05:41] PROBLEM - wiki.consid.vn - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.consid.vn' expires in 15 day(s) (Thu 09 Jun 2022 23:40:45 GMT +0000). [00:05:52] RECOVERY - cp21 Disk Space on cp21 is OK: DISK OK - free space: / 9588 MB (24% inode=96%); [00:05:55] RECOVERY - dariawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dariawiki.org' will expire on Mon 22 Aug 2022 22:44:29 GMT +0000. [00:06:20] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.80, 1.57, 1.49 [00:06:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [00:06:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406672716140747 seconds [00:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.00, 10.25, 9.88 [00:07:30] RECOVERY - wiki.ssij.se - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ssij.se' will expire on Mon 22 Aug 2022 22:25:45 GMT +0000. [00:07:51] RECOVERY - fc.songcontests.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'fc.songcontests.eu' will expire on Mon 22 Aug 2022 22:44:59 GMT +0000. [00:07:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.36, 1.68, 1.41 [00:10:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0c3091b04ef4...cb4ce2a33f43 [00:10:02] [url] Comparing 0c3091b04ef4...cb4ce2a33f43 · miraheze/ssl · GitHub | github.com [00:10:03] [02miraheze/ssl] 07MirahezeSSLBot 03cb4ce2a - Bot: Update SSL cert for wiki.consid.vn [00:10:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.98, 3.80 [00:11:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.84, 3.88 [00:13:20] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.66, 10.84, 9.35 [00:14:17] RECOVERY - wiki.grottocenter.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.grottocenter.org' will expire on Mon 22 Aug 2022 22:30:31 GMT +0000. [00:15:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.71, 9.51, 9.78 [00:15:06] RECOVERY - speleo.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'speleo.wiki' will expire on Mon 22 Aug 2022 22:45:26 GMT +0000. [00:15:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.41, 9.42, 9.01 [00:16:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.84, 3.76 [00:16:55] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.49, 4.08, 3.90 [00:17:40] RECOVERY - wiki.tulpa.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tulpa.info' will expire on Mon 22 Aug 2022 22:32:33 GMT +0000. [00:17:52] RECOVERY - www.connorjwat.net - LetsEncrypt on sslhost is OK: OK - Certificate 'www.connorjwat.net' will expire on Mon 22 Aug 2022 22:20:46 GMT +0000. [00:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.28, 1.93, 1.90 [00:22:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.58, 3.57, 3.74 [00:24:03] RECOVERY - www.dariawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dariawiki.org' will expire on Mon 22 Aug 2022 22:44:29 GMT +0000. [00:24:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 2.00, 1.97 [00:24:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.70, 3.81 [00:26:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.70, 3.74 [00:28:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.41, 3.63 [00:29:56] PROBLEM - wiki.make717.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.make717.org' expires in 15 day(s) (Fri 10 Jun 2022 00:04:23 GMT +0000). [00:30:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.16, 3.86, 3.77 [00:30:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.01, 3.99, 3.88 [00:32:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.71, 3.45, 3.64 [00:32:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.81, 3.40, 3.67 [00:35:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cb4ce2a33f43...a7474d5144f7 [00:35:15] [url] Comparing cb4ce2a33f43...a7474d5144f7 · miraheze/ssl · GitHub | github.com [00:35:16] [02miraheze/ssl] 07MirahezeSSLBot 03a7474d5 - Bot: Update SSL cert for wiki.make717.org [00:35:35] RECOVERY - wiki.consid.vn - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.consid.vn' will expire on Mon 22 Aug 2022 23:09:56 GMT +0000. [00:35:51] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.46, 10.67, 9.97 [00:36:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.95, 2.74, 3.31 [00:36:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.44, 1.50, 1.69 [00:37:45] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.82, 10.87, 10.12 [00:38:12] PROBLEM - aman.awiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'aman.awiki.org' expires in 15 day(s) (Fri 10 Jun 2022 00:09:50 GMT +0000). [00:38:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 3.27, 3.94 [00:38:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.41, 2.93, 3.40 [00:38:53] PROBLEM - zamlung.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'zamlung.tk' expires in 15 day(s) (Fri 10 Jun 2022 00:18:08 GMT +0000). [00:40:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.89, 3.66, 4.01 [00:41:34] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.45, 9.96, 9.95 [00:42:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.95, 3.58, 3.95 [00:43:37] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.57, 10.63, 9.64 [00:45:05] RECOVERY - adadevelopersacademy.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'adadevelopersacademy.wiki' will expire on Mon 22 Aug 2022 22:51:43 GMT +0000. [00:45:25] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a7474d5144f7...55a8d61043fb [00:45:25] [url] Comparing a7474d5144f7...55a8d61043fb · miraheze/ssl · GitHub | github.com [00:45:26] [02miraheze/ssl] 07MirahezeSSLBot 0355a8d61 - Bot: Update SSL cert for aman.awiki.org [00:46:14] PROBLEM - wiki.ricochetuniverse.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ricochetuniverse.com' expires in 15 day(s) (Fri 10 Jun 2022 00:24:56 GMT +0000). [00:46:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 4.10, 4.06 [00:47:29] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.57, 9.35, 9.40 [00:47:46] PROBLEM - mrlove.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mrlove.wiki' expires in 15 day(s) (Fri 10 Jun 2022 00:31:43 GMT +0000). [00:48:08] PROBLEM - www.mh142.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mh142.com' expires in 15 day(s) (Fri 10 Jun 2022 00:44:36 GMT +0000). [00:48:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.88, 3.99 [00:48:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.53, 3.86, 3.08 [00:50:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.60, 2.03, 1.77 [00:50:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/55a8d61043fb...0dcfda484b56 [00:50:35] [url] Comparing 55a8d61043fb...0dcfda484b56 · miraheze/ssl · GitHub | github.com [00:50:37] [02miraheze/ssl] 07MirahezeSSLBot 030dcfda4 - Bot: Update SSL cert for zamlung.tk [00:50:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.15, 3.63, 3.09 [00:51:44] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.15, 10.02, 8.86 [00:52:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 1.60, 3.00, 2.93 [00:52:54] RECOVERY - mediawiki.ecfin.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'mediawiki.ecfin.eu' will expire on Mon 22 Aug 2022 23:01:19 GMT +0000. [00:54:17] RECOVERY - data.nonbinary.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'data.nonbinary.wiki' will expire on Mon 22 Aug 2022 23:00:39 GMT +0000. [00:54:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.78, 4.01, 3.97 [00:54:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.27, 10.08, 9.54 [00:55:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.36, 10.73, 10.23 [00:55:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.54, 1.81, 1.44 [00:55:34] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.55, 9.66, 9.03 [00:55:42] alerting : [FIRING:1] (!sre High Job Queue Backlog yes mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [00:56:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.33, 3.74, 3.88 [00:56:32] PROBLEM - beaconspace.unrestrictedlorefare.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'beaconspace.unrestrictedlorefare.com' expires in 15 day(s) (Fri 10 Jun 2022 00:37:57 GMT +0000). [00:56:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.95, 10.15, 9.63 [00:56:53] PROBLEM - test101 Puppet on test101 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 4 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/logrotate.d/mediawiki_cron_logs] [00:57:10] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.88, 1.49, 1.37 [00:58:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0dcfda484b56...39653a6fa11f [00:58:31] [url] Comparing 0dcfda484b56...39653a6fa11f · miraheze/ssl · GitHub | github.com [00:58:32] [02miraheze/ssl] 07MirahezeSSLBot 0339653a6 - Bot: Update SSL cert for mrlove.wiki [00:59:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.78, 11.33, 10.55 [00:59:23] RECOVERY - wiki.make717.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.make717.org' will expire on Mon 22 Aug 2022 23:35:07 GMT +0000. [01:00:24] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.76, 10.99, 10.04 [01:00:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.95, 4.04, 3.96 [01:00:34] PROBLEM - mh142.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mh142.com' expires in 15 day(s) (Fri 10 Jun 2022 00:44:36 GMT +0000). [01:00:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.97, 4.02, 3.35 [01:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.90, 10.76, 10.44 [01:01:04] PROBLEM - index.pdcommunity.ir - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'index.pdcommunity.ir' expires in 15 day(s) (Fri 10 Jun 2022 00:49:23 GMT +0000). [01:02:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.85, 4.00, 3.96 [01:02:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.91, 3.60, 3.28 [01:03:57] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.76, 4.19, 3.74 [01:04:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.93, 4.05, 3.48 [01:04:41] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/39653a6fa11f...cd071f236529 [01:04:41] [url] Comparing 39653a6fa11f...cd071f236529 · miraheze/ssl · GitHub | github.com [01:04:42] [02miraheze/ssl] 07MirahezeSSLBot 03cd071f2 - Bot: Update SSL cert for wiki.ricochetuniverse.com [01:04:59] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.17, 6.90, 6.14 [01:05:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.71, 11.33, 10.69 [01:05:42] alerting : [FIRING:2] (mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [01:05:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.30, 3.53, 3.55 [01:06:11] PROBLEM - wiki.chouverse.cludisciples.cyou - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.chouverse.cludisciples.cyou' expires in 15 day(s) (Fri 10 Jun 2022 00:59:37 GMT +0000). [01:06:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cd071f236529...19bf11e6cfcd [01:06:15] [url] Comparing cd071f236529...19bf11e6cfcd · miraheze/ssl · GitHub | github.com [01:06:16] [02miraheze/ssl] 07MirahezeSSLBot 0319bf11e - Bot: Update SSL cert for mh142.com [01:06:54] RECOVERY - aman.awiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'aman.awiki.org' will expire on Mon 22 Aug 2022 23:45:19 GMT +0000. [01:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.05, 10.50, 10.48 [01:07:48] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.86, 2.95, 3.33 [01:08:12] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.39, 11.62, 10.88 [01:08:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.77, 1.98 [01:08:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.77, 3.57, 3.43 [01:08:50] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.27, 7.29, 6.44 [01:10:04] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.30, 10.24, 9.79 [01:10:25] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/19bf11e6cfcd...169571267542 [01:10:25] [url] Comparing 19bf11e6cfcd...169571267542 · miraheze/ssl · GitHub | github.com [01:10:26] [02miraheze/ssl] 07MirahezeSSLBot 031695712 - Bot: Update SSL cert for wiki.chouverse.cludisciples.cyou [01:10:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.21, 9.89, 9.29 [01:12:00] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.33, 10.98, 10.11 [01:12:41] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.33, 7.39, 6.74 [01:13:05] PROBLEM - permanentfuturelab.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'permanentfuturelab.wiki' expires in 15 day(s) (Fri 10 Jun 2022 01:08:25 GMT +0000). [01:13:56] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.97, 9.92, 9.84 [01:13:57] PROBLEM - monarchists.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'monarchists.wiki' expires in 15 day(s) (Fri 10 Jun 2022 00:54:22 GMT +0000). [01:14:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.02, 3.80, 3.73 [01:14:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.21, 3.04, 3.28 [01:14:55] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.57, 9.51, 9.25 [01:15:42] alerting : [FIRING:1, RESOLVED:1] (mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [01:16:00] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.87, 9.11, 10.15 [01:16:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/169571267542...7c08ea22c3d8 [01:16:18] [url] Comparing 169571267542...7c08ea22c3d8 · miraheze/ssl · GitHub | github.com [01:16:19] [02miraheze/ssl] 07MirahezeSSLBot 037c08ea2 - Bot: Update SSL cert for index.pdcommunity.ir [01:16:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.28, 1.94, 1.97 [01:17:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.37, 9.44, 10.04 [01:17:16] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 14600 MB (10% inode=99%); [01:18:28] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.58, 6.64, 6.65 [01:18:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.70, 3.71 [01:20:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.33, 3.95, 3.78 [01:22:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.57, 3.82, 3.46 [01:23:36] RECOVERY - test101 Puppet on test101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:24:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.90, 4.44, 3.68 [01:24:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.05, 4.41, 3.58 [01:24:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.15, 3.59, 3.42 [01:25:16] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 15564 MB (11% inode=99%); [01:26:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.62, 3.67, 2.31 [01:26:28] PROBLEM - www.permanentfuturelab.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'permanentfuturelab.wiki' expires in 15 day(s) (Fri 10 Jun 2022 01:08:25 GMT +0000). [01:26:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.16, 3.27, 3.31 [01:29:07] RECOVERY - mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Tue 23 Aug 2022 00:06:10 GMT +0000. [01:29:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c3ec35362c3c...f68898907e5d [01:29:43] [url] Comparing c3ec35362c3c...f68898907e5d · miraheze/puppet · GitHub | github.com [01:29:44] [02miraheze/puppet] 07paladox 03f688989 - matomo101: remove 30s php timeout [01:31:02] RECOVERY - index.pdcommunity.ir - LetsEncrypt on sslhost is OK: OK - Certificate 'index.pdcommunity.ir' will expire on Tue 23 Aug 2022 00:16:11 GMT +0000. [01:31:15] Was just about to do that paladox. Thanks! [01:31:18] PROBLEM - www.publictestwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'publictestwiki.com' expires in 15 day(s) (Fri 10 Jun 2022 01:25:28 GMT +0000). [01:31:21] Sorry about that last time. [01:31:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7c08ea22c3d8...d88d587daf93 [01:31:28] [url] Comparing 7c08ea22c3d8...d88d587daf93 · miraheze/ssl · GitHub | github.com [01:31:29] [02miraheze/ssl] 07MirahezeSSLBot 03d88d587 - Bot: Update SSL cert for permanentfuturelab.wiki [01:32:04] PROBLEM - archive.stellurgists.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'archive.stellurgists.wiki' expires in 15 day(s) (Fri 10 Jun 2022 01:13:34 GMT +0000). [01:32:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d88d587daf93...1d51c475db7f [01:32:51] [url] Comparing d88d587daf93...1d51c475db7f · miraheze/ssl · GitHub | github.com [01:32:52] [02miraheze/ssl] 07MirahezeSSLBot 031d51c47 - Bot: Update SSL cert for beaconspace.unrestrictedlorefare.com [01:33:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1d51c475db7f...12eea70ac0ee [01:33:09] [url] Comparing 1d51c475db7f...12eea70ac0ee · miraheze/ssl · GitHub | github.com [01:33:10] [02miraheze/ssl] 07MirahezeSSLBot 0312eea70 - Bot: Update SSL cert for archive.stellurgists.wiki [01:34:26] [02miraheze/landing] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/4deb1cd17cc4...8351044856d6 [01:34:26] [url] Comparing 4deb1cd17cc4...8351044856d6 · miraheze/landing · GitHub | github.com [01:34:27] [02miraheze/landing] 07Universal-Omega 038351044 - We are a not-for-profit, not a non-profit [01:34:42] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f68898907e5d...9dcc1aba82fe [01:34:43] [url] Comparing f68898907e5d...9dcc1aba82fe · miraheze/puppet · GitHub | github.com [01:34:44] [02miraheze/puppet] 07paladox 039dcc1ab - matomo101: increase php::fpm::fpm_workers_multiplier to 5.0 [01:35:27] miraheze/landing - Universal-Omega the build passed. [01:35:51] RECOVERY - wiki.chouverse.cludisciples.cyou - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.chouverse.cludisciples.cyou' will expire on Tue 23 Aug 2022 00:10:19 GMT +0000. [01:35:59] [02miraheze/landing] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/8351044856d6...91f79a7d8e6d [01:36:00] [url] Comparing 8351044856d6...91f79a7d8e6d · miraheze/landing · GitHub | github.com [01:36:01] [02miraheze/landing] 07Universal-Omega 0391f79a7 - We are a not-for-profit, not a non-profit [01:36:11] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.27, 1.80, 1.94 [01:36:57] miraheze/landing - Universal-Omega the build passed. [01:38:13] RECOVERY - zamlung.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'zamlung.tk' will expire on Mon 22 Aug 2022 23:50:28 GMT +0000. [01:39:59] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.90, 10.01, 9.50 [01:40:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 1.77, 1.88 [01:41:53] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.68, 9.26, 9.29 [01:42:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.92, 2.00 [01:44:27] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.23, 9.16, 8.21 [01:44:39] PROBLEM - publictestwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'publictestwiki.com' expires in 15 day(s) (Fri 10 Jun 2022 01:25:28 GMT +0000). [01:44:42] RECOVERY - wiki.ricochetuniverse.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ricochetuniverse.com' will expire on Tue 23 Aug 2022 00:04:36 GMT +0000. [01:44:48] PROBLEM - memipedia.ga - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'memipedia.ga' expires in 15 day(s) (Fri 10 Jun 2022 01:33:32 GMT +0000). [01:46:11] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.23, 1.85, 1.90 [01:46:16] RECOVERY - mrlove.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'mrlove.wiki' will expire on Mon 22 Aug 2022 23:58:24 GMT +0000. [01:46:27] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.21, 8.73, 8.17 [01:47:24] RECOVERY - www.mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Tue 23 Aug 2022 00:06:10 GMT +0000. [01:48:02] PROBLEM - electowiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'electowiki.org' expires in 15 day(s) (Fri 10 Jun 2022 01:27:48 GMT +0000). [01:48:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.26, 1.87, 1.89 [01:48:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/12eea70ac0ee...59d518934a46 [01:48:17] [url] Comparing 12eea70ac0ee...59d518934a46 · miraheze/ssl · GitHub | github.com [01:48:18] [02miraheze/ssl] 07MirahezeSSLBot 0359d5189 - Bot: Update SSL cert for publictestwiki.com [01:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.31, 1.93, 1.95 [01:48:53] PROBLEM - wiki.mineland.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mineland.eu' expires in 15 day(s) (Fri 10 Jun 2022 01:19:57 GMT +0000). [01:50:11] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.45, 1.81, 1.88 [01:50:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/59d518934a46...bc382a47c047 [01:50:30] [url] Comparing 59d518934a46...bc382a47c047 · miraheze/ssl · GitHub | github.com [01:50:30] [02miraheze/ssl] 07MirahezeSSLBot 03bc382a4 - Bot: Update SSL cert for electowiki.org [01:51:18] PROBLEM - antiguabarbudacalypso.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'antiguabarbudacalypso.com' expires in 15 day(s) (Fri 10 Jun 2022 01:40:51 GMT +0000). [01:52:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.99, 2.55, 2.12 [01:53:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [01:56:11] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.40, 1.83, 1.92 [01:56:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.58, 1.94, 1.99 [01:57:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bc382a47c047...b1efdbd4d2b8 [01:57:34] [url] Comparing bc382a47c047...b1efdbd4d2b8 · miraheze/ssl · GitHub | github.com [01:57:35] [02miraheze/ssl] 07MirahezeSSLBot 03b1efdbd - Bot: Update SSL cert for memipedia.ga [02:00:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.94, 1.98 [02:00:36] RECOVERY - www.publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Tue 23 Aug 2022 00:48:11 GMT +0000. [02:00:37] RECOVERY - archive.stellurgists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'archive.stellurgists.wiki' will expire on Tue 23 Aug 2022 00:33:03 GMT +0000. [02:02:09] PROBLEM - guia.cineastas.pt - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'guia.cineastas.pt' expires in 15 day(s) (Fri 10 Jun 2022 01:42:50 GMT +0000). [02:02:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.79, 1.31, 1.67 [02:02:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.85, 1.94 [02:05:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b1efdbd4d2b8...a0f85eec50b1 [02:05:37] [url] Comparing b1efdbd4d2b8...a0f85eec50b1 · miraheze/ssl · GitHub | github.com [02:05:37] [02miraheze/ssl] 07MirahezeSSLBot 03a0f85ee - Bot: Update SSL cert for antiguabarbudacalypso.com [02:10:11] PROBLEM - ndg.nenawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nenawiki.org' expires in 15 day(s) (Fri 10 Jun 2022 02:05:28 GMT +0000). [02:10:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.69, 3.61, 3.96 [02:11:15] RECOVERY - permanentfuturelab.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'permanentfuturelab.wiki' will expire on Tue 23 Aug 2022 00:31:22 GMT +0000. [02:12:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.87, 4.01 [02:12:31] PROBLEM - ao90.pinho.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ao90.pinho.org' expires in 15 day(s) (Fri 10 Jun 2022 01:51:11 GMT +0000). [02:13:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a0f85eec50b1...29c8d69a1b84 [02:13:23] [url] Comparing a0f85eec50b1...29c8d69a1b84 · miraheze/ssl · GitHub | github.com [02:13:24] [02miraheze/ssl] 07MirahezeSSLBot 0329c8d69 - Bot: Update SSL cert for guia.cineastas.pt [02:13:49] RECOVERY - publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Tue 23 Aug 2022 00:48:11 GMT +0000. [02:16:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.79, 1.80 [02:16:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.95, 3.29, 3.86 [02:18:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 10.64, 5.66, 4.63 [02:20:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.56, 2.26, 1.65 [02:21:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/29c8d69a1b84...241c5bbd45ce [02:21:20] [url] Comparing 29c8d69a1b84...241c5bbd45ce · miraheze/ssl · GitHub | github.com [02:21:21] [02miraheze/ssl] 07MirahezeSSLBot 03241c5bb - Bot: Update SSL cert for ao90.pinho.org [02:22:11] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.72, 1.98, 1.61 [02:22:49] PROBLEM - nenawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nenawiki.org' expires in 15 day(s) (Fri 10 Jun 2022 02:05:28 GMT +0000). [02:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [02:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.407233238220215 seconds [02:24:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.15, 1.68, 1.55 [02:24:22] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.70, 9.83, 9.34 [02:24:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.81, 1.97, 1.92 [02:24:30] RECOVERY - www.permanentfuturelab.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'permanentfuturelab.wiki' will expire on Tue 23 Aug 2022 00:31:22 GMT +0000. [02:24:32] RECOVERY - beaconspace.unrestrictedlorefare.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beaconspace.unrestrictedlorefare.com' will expire on Tue 23 Aug 2022 00:32:44 GMT +0000. [02:29:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/241c5bbd45ce...0bf631865d4f [02:29:19] [url] Comparing 241c5bbd45ce...0bf631865d4f · miraheze/ssl · GitHub | github.com [02:29:20] [02miraheze/ssl] 07MirahezeSSLBot 030bf6318 - Bot: Update SSL cert for wiki.mineland.eu [02:30:05] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.10, 9.32, 9.31 [02:30:05] PROBLEM - enc.for.uz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'enc.for.uz' expires in 15 day(s) (Fri 10 Jun 2022 01:58:53 GMT +0000). [02:30:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.15, 3.72, 4.00 [02:30:33] PROBLEM - www.allthetropes.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'allthetropes.org' expires in 15 day(s) (Fri 10 Jun 2022 02:20:17 GMT +0000). [02:30:49] RECOVERY - guia.cineastas.pt - LetsEncrypt on sslhost is OK: OK - Certificate 'guia.cineastas.pt' will expire on Tue 23 Aug 2022 01:13:17 GMT +0000. [02:31:07] PROBLEM - vitriol.spoon.army - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'vitriol.spoon.army' expires in 15 day(s) (Fri 10 Jun 2022 02:14:35 GMT +0000). [02:31:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.47, 9.74, 8.87 [02:32:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [02:34:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.21, 3.36, 3.99 [02:35:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.09, 10.18, 9.31 [02:35:45] PROBLEM - archive.a2b2.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'archive.a2b2.org' expires in 15 day(s) (Fri 10 Jun 2022 02:12:44 GMT +0000). [02:35:49] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0bf631865d4f...dec8df6891d4 [02:35:50] [url] Comparing 0bf631865d4f...dec8df6891d4 · miraheze/ssl · GitHub | github.com [02:35:51] [02miraheze/ssl] 07MirahezeSSLBot 03dec8df6 - Bot: Update SSL cert for nenawiki.org [02:36:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [02:36:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.57, 4.13, 4.19 [02:36:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.60, 10.14, 9.71 [02:38:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.56, 3.63, 3.78 [02:38:42] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.33, 9.63, 9.58 [02:40:11] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dec8df6891d4...263784ae972f [02:40:12] [url] Comparing dec8df6891d4...263784ae972f · miraheze/ssl · GitHub | github.com [02:40:13] [02miraheze/ssl] 07MirahezeSSLBot 03263784a - Bot: Update SSL cert for vitriol.spoon.army [02:40:26] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/263784ae972f...3e8cf6cea9c2 [02:40:26] [url] Comparing 263784ae972f...3e8cf6cea9c2 · miraheze/ssl · GitHub | github.com [02:40:27] [02miraheze/ssl] 07MirahezeSSLBot 033e8cf6c - Bot: Update SSL cert for archive.a2b2.org [02:40:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.02, 3.64, 3.98 [02:41:26] PROBLEM - wiki.thunis.eu - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.thunis.eu' expires in 15 day(s) (Fri 10 Jun 2022 02:33:13 GMT +0000). [02:42:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.98, 3.52, 3.72 [02:42:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.35, 1.53, 1.66 [02:42:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3e8cf6cea9c2...dbd317896b53 [02:42:51] [url] Comparing 3e8cf6cea9c2...dbd317896b53 · miraheze/ssl · GitHub | github.com [02:42:52] [02miraheze/ssl] 07MirahezeSSLBot 03dbd3178 - Bot: Update SSL cert for enc.for.uz [02:43:03] PROBLEM - franchise.franchising.org.ua - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'franchise.franchising.org.ua' expires in 15 day(s) (Fri 10 Jun 2022 02:39:12 GMT +0000). [02:43:25] RECOVERY - memipedia.ga - LetsEncrypt on sslhost is OK: OK - Certificate 'memipedia.ga' will expire on Tue 23 Aug 2022 00:57:28 GMT +0000. [02:44:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.18, 4.04, 4.06 [02:46:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.86, 1.70, 1.70 [02:46:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.70, 3.24, 3.77 [02:47:06] RECOVERY - electowiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'electowiki.org' will expire on Tue 23 Aug 2022 00:50:24 GMT +0000. [02:47:22] PROBLEM - allthetropes.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'allthetropes.org' expires in 15 day(s) (Fri 10 Jun 2022 02:20:17 GMT +0000). [02:47:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dbd317896b53...51fc7d2c5267 [02:47:30] [url] Comparing dbd317896b53...51fc7d2c5267 · miraheze/ssl · GitHub | github.com [02:47:31] [02miraheze/ssl] 07MirahezeSSLBot 0351fc7d2 - Bot: Update SSL cert for wiki.thunis.eu [02:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.28, 1.93, 1.79 [02:49:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.40, 6.56, 5.66 [02:50:53] RECOVERY - antiguabarbudacalypso.com - LetsEncrypt on sslhost is OK: OK - Certificate 'antiguabarbudacalypso.com' will expire on Tue 23 Aug 2022 01:05:30 GMT +0000. [02:51:47] PROBLEM - hololive.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'hololive.wiki' expires in 15 day(s) (Fri 10 Jun 2022 02:47:12 GMT +0000). [02:51:47] PROBLEM - history.sdtef.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'history.sdtef.org' expires in 15 day(s) (Fri 10 Jun 2022 02:26:35 GMT +0000). [02:51:51] PROBLEM - wiki.fraterniterebellion.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.fraterniterebellion.com' expires in 15 day(s) (Fri 10 Jun 2022 02:41:13 GMT +0000). [02:53:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.98, 6.47, 5.85 [02:53:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [02:54:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.95, 1.87 [02:54:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.11, 3.44, 3.62 [02:54:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/51fc7d2c5267...ca09a5c4c8a9 [02:54:45] [url] Comparing 51fc7d2c5267...ca09a5c4c8a9 · miraheze/ssl · GitHub | github.com [02:54:46] [02miraheze/ssl] 07MirahezeSSLBot 03ca09a5c - Bot: Update SSL cert for allthetropes.org [02:56:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.94, 2.98, 3.32 [02:56:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.95, 1.87 [02:56:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.18, 3.00, 3.44 [02:57:38] PROBLEM - guia.esporo.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'guia.esporo.net' expires in 15 day(s) (Fri 10 Jun 2022 02:54:41 GMT +0000). [02:58:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.84, 1.85, 1.84 [02:59:40] RECOVERY - enc.for.uz - LetsEncrypt on sslhost is OK: OK - Certificate 'enc.for.uz' will expire on Tue 23 Aug 2022 01:42:45 GMT +0000. [03:00:14] RECOVERY - vitriol.spoon.army - LetsEncrypt on sslhost is OK: OK - Certificate 'vitriol.spoon.army' will expire on Tue 23 Aug 2022 01:40:06 GMT +0000. [03:00:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.06, 1.91, 1.86 [03:00:38] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [03:00:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.12, 2.66, 3.22 [03:02:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 2.00, 1.96, 1.89 [03:02:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ca09a5c4c8a9...4971c1ea4c04 [03:02:44] [url] Comparing ca09a5c4c8a9...4971c1ea4c04 · miraheze/ssl · GitHub | github.com [03:02:45] [02miraheze/ssl] 07MirahezeSSLBot 034971c1e - Bot: Update SSL cert for hololive.wiki [03:04:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 2.01, 1.91 [03:04:32] PROBLEM - wiki.danvs.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.danvs.net' expires in 15 day(s) (Fri 10 Jun 2022 02:51:34 GMT +0000). [03:04:46] PROBLEM - thepolicyhub.org.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'thepolicyhub.org.uk' expires in 15 day(s) (Fri 10 Jun 2022 02:59:53 GMT +0000). [03:05:00] RECOVERY - archive.a2b2.org - LetsEncrypt on sslhost is OK: OK - Certificate 'archive.a2b2.org' will expire on Tue 23 Aug 2022 01:40:20 GMT +0000. [03:06:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.12, 3.58, 3.96 [03:06:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [03:06:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406351089477539 seconds [03:07:42] PROBLEM - www.fraterniterebellion.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.fraterniterebellion.com' expires in 15 day(s) (Fri 10 Jun 2022 02:41:13 GMT +0000). [03:08:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4971c1ea4c04...d7302246bbc1 [03:08:23] [url] Comparing 4971c1ea4c04...d7302246bbc1 · miraheze/ssl · GitHub | github.com [03:08:24] [02miraheze/ssl] 07MirahezeSSLBot 03d730224 - Bot: Update SSL cert for franchise.franchising.org.ua [03:08:34] RECOVERY - ndg.nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Tue 23 Aug 2022 01:35:44 GMT +0000. [03:10:20] RECOVERY - wiki.thunis.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thunis.eu' will expire on Tue 23 Aug 2022 01:47:24 GMT +0000. [03:10:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.93, 1.92 [03:10:31] RECOVERY - ao90.pinho.org - LetsEncrypt on sslhost is OK: OK - Certificate 'ao90.pinho.org' will expire on Tue 23 Aug 2022 01:21:14 GMT +0000. [03:10:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.79, 4.48, 4.22 [03:10:39] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.09, 3.85, 3.43 [03:10:43] PROBLEM - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'cyberlaw.ccdcoe.org' expires in 15 day(s) (Fri 10 Jun 2022 02:46:12 GMT +0000). [03:12:33] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.55, 3.37 [03:13:04] PROBLEM - scpvncommunitywiki.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'scpvncommunitywiki.xyz' expires in 15 day(s) (Fri 10 Jun 2022 02:55:50 GMT +0000). [03:13:05] PROBLEM - nonbinary.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'nonbinary.wiki' expires in 15 day(s) (Fri 10 Jun 2022 02:49:55 GMT +0000). [03:16:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d7302246bbc1...9bd0f48f1cf5 [03:16:15] [url] Comparing d7302246bbc1...9bd0f48f1cf5 · miraheze/ssl · GitHub | github.com [03:16:16] [02miraheze/ssl] 07MirahezeSSLBot 039bd0f48 - Bot: Update SSL cert for wiki.danvs.net [03:16:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.65, 3.08, 3.22 [03:16:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.86, 1.87 [03:16:53] RECOVERY - wiki.mineland.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mineland.eu' will expire on Tue 23 Aug 2022 01:29:10 GMT +0000. [03:22:26] RECOVERY - nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Tue 23 Aug 2022 01:35:44 GMT +0000. [03:23:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.40, 9.95, 9.62 [03:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [03:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406140089035034 seconds [03:24:03] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9bd0f48f1cf5...22047e668f9a [03:24:03] [url] Comparing 9bd0f48f1cf5...22047e668f9a · miraheze/ssl · GitHub | github.com [03:24:04] [02miraheze/ssl] 07MirahezeSSLBot 0322047e6 - Bot: Update SSL cert for guia.esporo.net [03:25:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.16, 9.68, 9.56 [03:26:24] PROBLEM - miraheze.ml - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.ml' expires in 15 day(s) (Fri 10 Jun 2022 03:13:21 GMT +0000). [03:28:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.09, 3.48, 3.26 [03:29:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/22047e668f9a...6e64be7d414e [03:29:17] [url] Comparing 22047e668f9a...6e64be7d414e · miraheze/ssl · GitHub | github.com [03:29:18] [02miraheze/ssl] 07MirahezeSSLBot 036e64be7 - Bot: Update SSL cert for history.sdtef.org [03:29:23] PROBLEM - wiki.cdntennis.ca - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.cdntennis.ca' expires in 15 day(s) (Fri 10 Jun 2022 03:09:43 GMT +0000). [03:30:26] RECOVERY - www.allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Tue 23 Aug 2022 01:54:39 GMT +0000. [03:32:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.83, 1.97 [03:32:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.71, 3.73, 3.38 [03:34:20] RECOVERY - wiki.danvs.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.danvs.net' will expire on Tue 23 Aug 2022 02:16:10 GMT +0000. [03:34:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6e64be7d414e...c004af7c479f [03:34:21] [url] Comparing 6e64be7d414e...c004af7c479f · miraheze/ssl · GitHub | github.com [03:34:22] [02miraheze/ssl] 07MirahezeSSLBot 03c004af7 - Bot: Update SSL cert for scpvncommunitywiki.xyz [03:34:28] PROBLEM - wiki.graalmilitary.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.graalmilitary.com' expires in 15 day(s) (Fri 10 Jun 2022 03:03:45 GMT +0000). [03:34:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.71, 4.01, 3.52 [03:36:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.79, 3.50 [03:36:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.87, 3.53, 3.39 [03:37:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c004af7c479f...47e0dc396581 [03:37:19] [url] Comparing c004af7c479f...47e0dc396581 · miraheze/ssl · GitHub | github.com [03:37:20] [02miraheze/ssl] 07MirahezeSSLBot 0347e0dc3 - Bot: Update SSL cert for miraheze.ml [03:38:25] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.73, 3.32, 3.35 [03:38:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.58, 3.69, 3.45 [03:40:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.24, 3.42, 3.37 [03:41:09] RECOVERY - franchise.franchising.org.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'franchise.franchising.org.ua' will expire on Tue 23 Aug 2022 02:08:17 GMT +0000. [03:42:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.38, 3.34, 3.34 [03:44:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/47e0dc396581...4afd5d3daf75 [03:44:20] [url] Comparing 47e0dc396581...4afd5d3daf75 · miraheze/ssl · GitHub | github.com [03:44:20] [02miraheze/ssl] 07MirahezeSSLBot 034afd5d3 - Bot: Update SSL cert for wiki.cdntennis.ca [03:45:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4afd5d3daf75...d248dee26679 [03:45:08] [url] Comparing 4afd5d3daf75...d248dee26679 · miraheze/ssl · GitHub | github.com [03:45:09] [02miraheze/ssl] 07MirahezeSSLBot 03d248dee - Bot: Update SSL cert for wiki.fraterniterebellion.com [03:45:22] RECOVERY - allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Tue 23 Aug 2022 01:54:39 GMT +0000. [03:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.40, 1.98, 1.91 [03:49:48] RECOVERY - hololive.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hololive.wiki' will expire on Tue 23 Aug 2022 02:02:38 GMT +0000. [03:50:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.13, 4.69, 3.80 [03:51:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d248dee26679...d3b39dbe536b [03:51:30] [url] Comparing d248dee26679...d3b39dbe536b · miraheze/ssl · GitHub | github.com [03:51:31] [02miraheze/ssl] 07MirahezeSSLBot 03d3b39db - Bot: Update SSL cert for nonbinary.wiki [03:51:36] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.56, 5.36, 4.14 [03:53:34] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [03:56:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.58, 1.99, 2.00 [03:56:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d3b39dbe536b...0d8beda1f6f2 [03:56:42] [url] Comparing d3b39dbe536b...0d8beda1f6f2 · miraheze/ssl · GitHub | github.com [03:56:43] [02miraheze/ssl] 07MirahezeSSLBot 030d8beda - Bot: Update SSL cert for thepolicyhub.org.uk [03:57:23] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.16, 3.84, 3.81 [03:58:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.98, 3.81 [03:59:15] RECOVERY - wiki.cdntennis.ca - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cdntennis.ca' will expire on Tue 23 Aug 2022 02:44:14 GMT +0000. [04:02:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [04:02:04] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:02:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0d8beda1f6f2...b1f270b5c6dc [04:02:41] [url] Comparing 0d8beda1f6f2...b1f270b5c6dc · miraheze/ssl · GitHub | github.com [04:02:42] [02miraheze/ssl] 07MirahezeSSLBot 03b1f270b - Bot: Update SSL cert for wiki.graalmilitary.com [04:03:09] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.78, 2.62, 3.29 [04:05:16] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 14599 MB (10% inode=99%); [04:06:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.45, 3.52 [04:06:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b1f270b5c6dc...6746a5dede2a [04:06:30] [url] Comparing b1f270b5c6dc...6746a5dede2a · miraheze/ssl · GitHub | github.com [04:06:31] [02miraheze/ssl] 07MirahezeSSLBot 036746a5d - Bot: Update SSL cert for cyberlaw.ccdcoe.org [04:06:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [04:07:06] RECOVERY - www.fraterniterebellion.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fraterniterebellion.com' will expire on Tue 23 Aug 2022 02:45:02 GMT +0000. [04:08:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.49, 1.47, 1.68 [04:10:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.32, 3.08, 3.37 [04:11:12] RECOVERY - scpvncommunitywiki.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'scpvncommunitywiki.xyz' will expire on Tue 23 Aug 2022 02:34:15 GMT +0000. [04:12:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.05, 3.75, 4.00 [04:16:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.73, 4.14, 4.06 [04:19:47] RECOVERY - history.sdtef.org - LetsEncrypt on sslhost is OK: OK - Certificate 'history.sdtef.org' will expire on Tue 23 Aug 2022 02:29:10 GMT +0000. [04:19:51] RECOVERY - wiki.fraterniterebellion.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fraterniterebellion.com' will expire on Tue 23 Aug 2022 02:45:02 GMT +0000. [04:20:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.58, 2.02, 1.78 [04:20:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.72, 3.86, 3.97 [04:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [04:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406173944473267 seconds [04:24:25] RECOVERY - miraheze.ml - LetsEncrypt on sslhost is OK: OK - Certificate 'miraheze.ml' will expire on Tue 23 Aug 2022 02:37:13 GMT +0000. [04:25:38] RECOVERY - guia.esporo.net - LetsEncrypt on sslhost is OK: OK - Certificate 'guia.esporo.net' will expire on Tue 23 Aug 2022 02:23:57 GMT +0000. [04:26:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.25, 1.75, 1.79 [04:26:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.94, 3.47, 3.65 [04:28:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.13, 3.09 [04:30:31] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.25, 2.76, 2.96 [04:30:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 3.31, 3.55 [04:33:24] RECOVERY - wiki.graalmilitary.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.graalmilitary.com' will expire on Tue 23 Aug 2022 03:02:35 GMT +0000. [04:34:03] RECOVERY - thepolicyhub.org.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'thepolicyhub.org.uk' will expire on Tue 23 Aug 2022 02:56:36 GMT +0000. [04:34:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.52, 3.00, 3.38 [04:36:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.69, 1.62, 1.69 [04:36:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [04:36:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406737565994263 seconds [04:40:33] RECOVERY - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is OK: OK - Certificate 'cyberlaw.ccdcoe.org' will expire on Tue 23 Aug 2022 03:06:24 GMT +0000. [04:41:05] RECOVERY - nonbinary.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'nonbinary.wiki' will expire on Tue 23 Aug 2022 02:51:23 GMT +0000. [04:43:35] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.01, 6.57, 5.99 [04:44:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.23, 3.80, 3.43 [04:45:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.84, 1.75 [04:45:31] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.14, 6.30, 5.95 [04:46:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.60, 3.82, 3.50 [04:50:59] !log RENAMED webkinzpictureguidewiki to webkinzguidewiki [04:51:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:52:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.03, 2.87, 3.21 [04:55:19] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.99, 1.95 [05:00:04] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [05:03:21] PROBLEM - prometheus101 Current Load on prometheus101 is CRITICAL: CRITICAL - load average: 2.13, 1.34, 0.76 [05:05:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.03, 3.41, 3.27 [05:05:21] RECOVERY - prometheus101 Current Load on prometheus101 is OK: OK - load average: 0.82, 1.19, 0.78 [05:05:45] alerting : [FIRING:1, RESOLVED:1] (yes mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [05:07:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.35, 3.56, 3.32 [05:10:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.72, 3.19 [05:12:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.05, 4.16, 3.41 [05:12:52] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.15, 3.66, 3.01 [05:12:54] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.95, 3.56 [05:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.25, 3.46, 3.25 [05:14:48] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.25, 3.14, 2.90 [05:16:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.04, 2.90, 3.06 [05:16:43] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.77, 3.06, 3.31 [05:17:13] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.22, 1.89, 1.86 [05:23:12] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.95, 1.92 [05:23:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [05:36:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [05:39:09] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.41, 1.86, 1.79 [05:41:08] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.77, 1.76 [05:47:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.84, 3.72, 3.21 [05:49:09] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.83, 1.77 [05:50:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.59, 3.45, 3.24 [05:51:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.97, 6.62, 5.44 [05:52:53] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.90, 3.17, 3.16 [05:53:08] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.99, 1.87 [05:53:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.63, 6.16, 5.42 [06:06:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [06:06:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.405835866928101 seconds [06:07:04] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.48, 2.40, 1.99 [06:15:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.27, 7.10, 6.19 [06:17:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.59, 7.37, 6.41 [06:21:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.92, 6.58, 6.36 [06:23:07] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [06:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [06:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406811237335205 seconds [06:25:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.50, 9.94, 8.71 [06:27:45] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.36, 9.48, 8.68 [07:06:12] CosmicAlpha, Reception123, or RhinosF1, anyone around to run `fixStuckGlobalRename.php` for https://meta.miraheze.org/wiki/Special:GlobalRenameProgress/Vardan97? First wiki is a deleted wiki...that long-standing bug :( [07:06:14] [url] Global rename progress - Miraheze Meta | meta.miraheze.org [07:07:08] dmehus: I just turned off my PC like 10 seconds before you sent that.... [07:07:13] Have to sleep now, but send me a DM when it's run...thanks in advance! :) [07:07:27] CosmicAlpha, that's okay. Can you see if RhinosF1 or Reception123 can? [07:07:51] dmehus: Well you've pinged them so they'll see it when they get on. [07:07:54] dmehus: I'm mobile and Reception123 has disappeared after causing a paging job queue backlog 2 days in a row [07:08:18] RhinosF1, what's a paging job queue, and what caused it? [07:08:25] CosmicAlpha, ack [07:08:35] RhinosF1: let's not point blame like that please. Things just happen. [07:08:51] Agreed. That seemed a tad snippy [07:08:58] dmehus: the backlog triggers an alert with !_sre in if it gets too high [07:09:01] dmehus: scroll up a bit just before your message. [07:09:11] Which it has done twice after Reception123's done imports [07:09:24] oh [07:09:43] well, I'd argue the problem isn't so much an issue with him but our paging process being too strict :) [07:09:55] It's a geniue issue [07:10:00] but maybe he should spread out his imports more, yes. Still, I agree with CosmicAlpha bove [07:10:00] That can cause user impact [07:10:03] above* [07:10:11] Including renames not running for one [07:10:22] yeah, that's true [07:10:23] and many other stuff [07:10:57] We've had 3 unbreak now issues in less than 48 hours caused by sysadmins performing maintenance and not monitoring what they are doing [07:11:07] 2 sysadmins [07:11:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 2.96, 2.68 [07:12:17] RhinosF1: things just happen. Sometimes we don't realize it, and nothing can be done to prevent it from having ever happened. We can only nicely remind others what needs done, not that tone just used here. Please. [07:13:12] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.30, 3.01, 2.73 [07:13:18] CosmicAlpha: it's extremely frustrating when we have easily preventable incidents like this [07:13:38] You wouldn't set something on fire cooking and leave your house to burn down [07:13:43] So don't do it here [07:15:48] RhinosF1, yeah, it's a valid concern...I just think you could've said it more tactfully and not pointed fingers at specific team members, but it's perhaps something that might be useful for an MWE team meeting or maybe a Phabricator task [07:16:22] dmehus: it definitely needs some sort of reminder [07:16:36] * RhinosF1 is often grumpy on mornings when preventable stuff cause issues [07:17:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.67, 3.58, 3.04 [07:17:08] Fair, but that's probably something that needs to be addressed to the whole SRE team re: more caution when performing maintenance [07:17:12] I think we need to learn to work more as a team, that assists each other, not blame each other. [07:17:18] yeah [07:20:51] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.16, 3.01, 2.96 [07:20:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.33, 9.07, 7.88 [07:22:55] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.18, 8.80, 7.93 [07:27:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.27, 1.65, 1.33 [07:29:39] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.70, 1.55, 1.33 [07:30:13] RhinosF1: someone has to do imports [07:30:32] we can't just not do them because they cause a job backlog, the issue isn't the imports it's the job queue [07:30:58] Yesterday evening I only did a rebuildall which was spaced out from the others [07:31:22] While I don't want to cast blame either, I would argue that so many imports wouldn't have had to be done if they were during my absence... [07:34:31] Reception123: runJobs [07:34:39] it's one extra command [07:35:01] just open another screen and do runJobs.php --wait --wiki=whatever [07:35:34] That's fair, but rebuildall can take forever and I didn't expect there to be an issue. Though now that I know I can plan for that happening [07:35:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.72, 1.53, 1.36 [07:35:57] Reception123: the alert went off as soon as you started it [07:36:01] You can see the rise [07:37:39] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.14, 1.40, 1.33 [07:53:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [07:56:31] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.97, 1.84, 1.60 [07:58:29] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.07, 1.88, 1.64 [08:00:26] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.72, 1.97, 1.71 [08:02:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [08:02:23] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.57, 2.19, 1.82 [08:06:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [08:07:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.70, 7.02, 5.84 [08:09:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.20, 6.26, 5.71 [08:10:12] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.79, 1.96, 1.89 [08:19:59] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.20, 1.51, 1.69 [08:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [08:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406986474990845 seconds [08:23:51] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.74, 2.04, 1.87 [08:29:42] PROBLEM - wiki.joust.ro - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.joust.ro' expires in 15 day(s) (Fri 10 Jun 2022 08:18:55 GMT +0000). [08:30:10] PROBLEM - wiki.seamly.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.seamly.net' expires in 15 day(s) (Fri 10 Jun 2022 08:26:17 GMT +0000). [08:30:39] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [08:32:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6746a5dede2a...6ff1cd4d84f8 [08:32:59] [url] Comparing 6746a5dede2a...6ff1cd4d84f8 · miraheze/ssl · GitHub | github.com [08:33:00] [02miraheze/ssl] 07MirahezeSSLBot 036ff1cd4 - Bot: Update SSL cert for wiki.joust.ro [08:33:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.10, 1.71, 1.88 [08:34:38] PROBLEM - miraheze.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.tk' expires in 15 day(s) (Fri 10 Jun 2022 08:32:23 GMT +0000). [08:35:11] PROBLEM - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.worlduniversityandschool.org' expires in 15 day(s) (Fri 10 Jun 2022 08:29:48 GMT +0000). [08:36:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [08:36:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.410606861114502 seconds [08:37:39] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.02, 1.36, 1.70 [08:40:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6ff1cd4d84f8...71d6f5187d44 [08:40:11] [url] Comparing 6ff1cd4d84f8...71d6f5187d44 · miraheze/ssl · GitHub | github.com [08:40:12] [02miraheze/ssl] 07MirahezeSSLBot 0371d6f51 - Bot: Update SSL cert for wiki.seamly.net [08:43:40] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.74, 2.38, 2.03 [08:48:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/71d6f5187d44...11cc3a87b1a1 [08:48:14] [url] Comparing 71d6f5187d44...11cc3a87b1a1 · miraheze/ssl · GitHub | github.com [08:48:15] [02miraheze/ssl] 07MirahezeSSLBot 0311cc3a8 - Bot: Update SSL cert for miraheze.tk [08:49:40] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.39, 1.77, 1.87 [08:50:42] PROBLEM - tl.awiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tl.awiki.org' expires in 15 day(s) (Fri 10 Jun 2022 08:43:58 GMT +0000). [08:52:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/11cc3a87b1a1...b9fbd0d7d377 [08:52:24] [url] Comparing 11cc3a87b1a1...b9fbd0d7d377 · miraheze/ssl · GitHub | github.com [08:52:25] [02miraheze/ssl] 07MirahezeSSLBot 03b9fbd0d - Bot: Update SSL cert for wiki.worlduniversityandschool.org [08:53:55] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b9fbd0d7d377...8464d43f25cb [08:53:56] [url] Comparing b9fbd0d7d377...8464d43f25cb · miraheze/ssl · GitHub | github.com [08:53:57] [02miraheze/ssl] 07MirahezeSSLBot 038464d43 - Bot: Update SSL cert for tl.awiki.org [08:55:40] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.69, 2.07, 1.94 [08:57:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.59, 1.96, 1.92 [08:57:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.13, 1.75, 1.38 [08:58:05] PROBLEM - wiki.insideearth.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.insideearth.info' expires in 15 day(s) (Fri 10 Jun 2022 08:37:56 GMT +0000). [08:58:55] RECOVERY - wiki.joust.ro - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.joust.ro' will expire on Tue 23 Aug 2022 07:32:52 GMT +0000. [08:59:49] RECOVERY - wiki.seamly.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.seamly.net' will expire on Tue 23 Aug 2022 07:40:05 GMT +0000. [09:01:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.03, 1.56, 1.39 [09:02:20] PROBLEM - tensegritywiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tensegritywiki.com' expires in 15 day(s) (Fri 10 Jun 2022 08:39:41 GMT +0000). [09:03:39] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.66, 1.30, 1.66 [09:04:17] RECOVERY - miraheze.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'miraheze.tk' will expire on Tue 23 Aug 2022 07:48:08 GMT +0000. [09:05:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8464d43f25cb...d573ac504d52 [09:05:40] [url] Comparing 8464d43f25cb...d573ac504d52 · miraheze/ssl · GitHub | github.com [09:05:41] [02miraheze/ssl] 07MirahezeSSLBot 03d573ac5 - Bot: Update SSL cert for wiki.insideearth.info [09:10:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d573ac504d52...d85b37678380 [09:10:23] [url] Comparing d573ac504d52...d85b37678380 · miraheze/ssl · GitHub | github.com [09:10:24] [02miraheze/ssl] 07MirahezeSSLBot 03d85b376 - Bot: Update SSL cert for tensegritywiki.com [09:16:57] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 2.96, 2.53 [09:17:20] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.45, 1.93, 1.72 [09:18:51] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.02, 2.97, 2.58 [09:19:41] PROBLEM - wiki.funkey-project.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.funkey-project.com' expires in 15 day(s) (Fri 10 Jun 2022 09:08:05 GMT +0000). [09:20:12] PROBLEM - infectowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'infectowiki.com' expires in 15 day(s) (Fri 10 Jun 2022 08:51:40 GMT +0000). [09:22:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.73, 3.03, 2.54 [09:23:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.77, 9.32, 8.11 [09:23:07] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [09:24:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.18, 2.85, 2.52 [09:25:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.73, 8.08, 7.81 [09:26:25] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d85b37678380...7c65c0286d23 [09:26:26] [url] Comparing d85b37678380...7c65c0286d23 · miraheze/ssl · GitHub | github.com [09:26:26] [02miraheze/ssl] 07MirahezeSSLBot 037c65c02 - Bot: Update SSL cert for wiki.funkey-project.com [09:27:10] RECOVERY - wiki.insideearth.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.insideearth.info' will expire on Tue 23 Aug 2022 08:05:34 GMT +0000. [09:27:41] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7c65c0286d23...b57754521e1e [09:27:41] [url] Comparing 7c65c0286d23...b57754521e1e · miraheze/ssl · GitHub | github.com [09:27:42] [02miraheze/ssl] 07MirahezeSSLBot 03b577545 - Bot: Update SSL cert for infectowiki.com [09:27:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.44, 1.75, 1.38 [09:27:57] PROBLEM - wiki.conworlds.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.conworlds.org' expires in 15 day(s) (Fri 10 Jun 2022 09:25:45 GMT +0000). [09:28:50] PROBLEM - tep.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tep.wiki' expires in 15 day(s) (Fri 10 Jun 2022 09:18:32 GMT +0000). [09:29:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.25, 1.55, 1.35 [09:30:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.50, 1.72, 1.98 [09:31:07] PROBLEM - wiki.ciptamedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ciptamedia.org' expires in 15 day(s) (Fri 10 Jun 2022 08:59:55 GMT +0000). [09:31:09] RECOVERY - tensegritywiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'tensegritywiki.com' will expire on Tue 23 Aug 2022 08:10:17 GMT +0000. [09:31:23] PROBLEM - pwiki.arkcls.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pwiki.arkcls.com' expires in 15 day(s) (Fri 10 Jun 2022 09:06:34 GMT +0000). [09:33:11] RECOVERY - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.worlduniversityandschool.org' will expire on Tue 23 Aug 2022 07:52:18 GMT +0000. [09:34:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b57754521e1e...1b96f5164e04 [09:34:22] [url] Comparing b57754521e1e...1b96f5164e04 · miraheze/ssl · GitHub | github.com [09:34:23] [02miraheze/ssl] 07MirahezeSSLBot 031b96f51 - Bot: Update SSL cert for wiki.conworlds.org [09:37:10] PROBLEM - stablestate.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'stablestate.org' expires in 15 day(s) (Fri 10 Jun 2022 09:12:10 GMT +0000). [09:39:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1b96f5164e04...bdc49432773d [09:39:16] [url] Comparing 1b96f5164e04...bdc49432773d · miraheze/ssl · GitHub | github.com [09:39:17] [02miraheze/ssl] 07MirahezeSSLBot 03bdc4943 - Bot: Update SSL cert for pwiki.arkcls.com [09:40:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bdc49432773d...7a611ba780e3 [09:40:16] [url] Comparing bdc49432773d...7a611ba780e3 · miraheze/ssl · GitHub | github.com [09:40:17] [02miraheze/ssl] 07MirahezeSSLBot 037a611ba - Bot: Update SSL cert for tep.wiki [09:40:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.73, 1.84 [09:41:16] PROBLEM - wiki.thehall.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.thehall.xyz' expires in 15 day(s) (Fri 10 Jun 2022 09:37:32 GMT +0000). [09:41:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7a611ba780e3...46e993caccbe [09:41:54] [url] Comparing 7a611ba780e3...46e993caccbe · miraheze/ssl · GitHub | github.com [09:41:55] [02miraheze/ssl] 07MirahezeSSLBot 0346e993c - Bot: Update SSL cert for stablestate.org [09:42:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.74, 1.83 [09:45:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/46e993caccbe...c1880cd22c75 [09:45:05] [url] Comparing 46e993caccbe...c1880cd22c75 · miraheze/ssl · GitHub | github.com [09:45:06] [02miraheze/ssl] 07MirahezeSSLBot 03c1880cd - Bot: Update SSL cert for wiki.ciptamedia.org [09:46:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.14, 2.61 [09:46:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.12, 3.45, 2.85 [09:46:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.93, 4.28, 3.27 [09:47:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.10, 2.02, 1.61 [09:48:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.03, 3.10, 2.66 [09:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 1.95, 1.86 [09:48:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.15, 3.41, 2.92 [09:49:06] RECOVERY - tl.awiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'tl.awiki.org' will expire on Tue 23 Aug 2022 07:53:50 GMT +0000. [09:49:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.72, 1.98, 1.65 [09:50:12] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c1880cd22c75...de9e82f4c26d [09:50:13] [url] Comparing c1880cd22c75...de9e82f4c26d · miraheze/ssl · GitHub | github.com [09:50:14] [02miraheze/ssl] 07MirahezeSSLBot 03de9e82f - Bot: Update SSL cert for wiki.thehall.xyz [09:50:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.64, 3.66, 3.25 [09:51:07] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:51:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.46, 2.32, 1.82 [09:52:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.16, 4.34, 3.50 [09:52:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.77, 3.82, 3.13 [09:53:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [09:54:52] PROBLEM - pubwiki.lab.co.pl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pubwiki.lab.co.pl' expires in 15 day(s) (Fri 10 Jun 2022 09:51:37 GMT +0000). [09:56:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.63, 4.70, 3.58 [09:56:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.88, 3.33, 3.09 [09:56:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/de9e82f4c26d...ace0a7b0323d [09:56:46] [url] Comparing de9e82f4c26d...ace0a7b0323d · miraheze/ssl · GitHub | github.com [09:56:47] [02miraheze/ssl] 07MirahezeSSLBot 03ace0a7b - Bot: Update SSL cert for pubwiki.lab.co.pl [09:57:06] PROBLEM - wiki.scvo.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.scvo.org' expires in 15 day(s) (Fri 10 Jun 2022 09:27:11 GMT +0000). [09:57:42] PROBLEM - wiki.shaazzz.ir - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.shaazzz.ir' expires in 15 day(s) (Fri 10 Jun 2022 09:28:40 GMT +0000). [09:58:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.08, 3.80, 3.39 [09:58:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.97, 3.50, 3.43 [09:58:39] RECOVERY - tep.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'tep.wiki' will expire on Tue 23 Aug 2022 08:40:10 GMT +0000. [10:00:15] RECOVERY - wiki.ciptamedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ciptamedia.org' will expire on Tue 23 Aug 2022 08:44:56 GMT +0000. [10:01:11] RECOVERY - pwiki.arkcls.com - LetsEncrypt on sslhost is OK: OK - Certificate 'pwiki.arkcls.com' will expire on Tue 23 Aug 2022 08:39:11 GMT +0000. [10:01:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ace0a7b0323d...12edf1150a96 [10:01:17] [url] Comparing ace0a7b0323d...12edf1150a96 · miraheze/ssl · GitHub | github.com [10:01:18] [02miraheze/ssl] 07MirahezeSSLBot 0312edf11 - Bot: Update SSL cert for wiki.scvo.org [10:01:19] PROBLEM - www.programming.red - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'programming.red' expires in 15 day(s) (Fri 10 Jun 2022 09:58:04 GMT +0000). [10:02:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [10:02:04] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:02:11] PROBLEM - n64brew.dev - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'n64brew.dev' expires in 15 day(s) (Fri 10 Jun 2022 09:36:41 GMT +0000). [10:02:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.39, 3.10, 3.21 [10:02:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.71, 3.10, 3.28 [10:06:19] RECOVERY - stablestate.org - LetsEncrypt on sslhost is OK: OK - Certificate 'stablestate.org' will expire on Tue 23 Aug 2022 08:41:48 GMT +0000. [10:06:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.66, 1.94, 1.99 [10:06:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [10:06:52] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/12edf1150a96...cefd8c744aae [10:06:53] [url] Comparing 12edf1150a96...cefd8c744aae · miraheze/ssl · GitHub | github.com [10:06:54] [02miraheze/ssl] 07MirahezeSSLBot 03cefd8c7 - Bot: Update SSL cert for wiki.shaazzz.ir [10:07:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.32, 1.68, 1.99 [10:10:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cefd8c744aae...ab76c1bd3010 [10:10:17] [url] Comparing cefd8c744aae...ab76c1bd3010 · miraheze/ssl · GitHub | github.com [10:10:18] [02miraheze/ssl] 07MirahezeSSLBot 03ab76c1b - Bot: Update SSL cert for n64brew.dev [10:12:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.82, 1.90 [10:13:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.59, 2.09, 2.04 [10:16:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.93, 1.95, 1.94 [10:17:23] PROBLEM - wiki.openhatch.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.openhatch.org' expires in 15 day(s) (Fri 10 Jun 2022 10:00:15 GMT +0000). [10:17:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.32, 1.70, 1.89 [10:18:09] PROBLEM - wiki.mikrodev.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mikrodev.com' expires in 15 day(s) (Fri 10 Jun 2022 09:56:20 GMT +0000). [10:18:12] RECOVERY - infectowiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'infectowiki.com' will expire on Tue 23 Aug 2022 08:27:35 GMT +0000. [10:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.69, 2.20, 2.04 [10:19:03] RECOVERY - wiki.funkey-project.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.funkey-project.com' will expire on Tue 23 Aug 2022 08:26:19 GMT +0000. [10:19:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.86, 2.49, 2.18 [10:21:22] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.46, 3.20, 2.92 [10:22:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ab76c1bd3010...906d6aad0f16 [10:22:36] [url] Comparing ab76c1bd3010...906d6aad0f16 · miraheze/ssl · GitHub | github.com [10:22:37] [02miraheze/ssl] 07MirahezeSSLBot 03906d6aa - Bot: Update SSL cert for wiki.openhatch.org [10:23:17] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.51, 2.98, 2.87 [10:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [10:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406968832015991 seconds [10:23:37] PROBLEM - programming.red - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'programming.red' expires in 15 day(s) (Fri 10 Jun 2022 09:58:04 GMT +0000). [10:25:36] PROBLEM - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pablojprieto.com.es' expires in 15 day(s) (Fri 10 Jun 2022 10:06:40 GMT +0000). [10:25:57] RECOVERY - wiki.conworlds.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.conworlds.org' will expire on Tue 23 Aug 2022 08:34:15 GMT +0000. [10:28:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/906d6aad0f16...97430f5534a4 [10:28:29] [url] Comparing 906d6aad0f16...97430f5534a4 · miraheze/ssl · GitHub | github.com [10:28:30] [02miraheze/ssl] 07MirahezeSSLBot 0397430f5 - Bot: Update SSL cert for wiki.mikrodev.com [10:30:52] RECOVERY - n64brew.dev - LetsEncrypt on sslhost is OK: OK - Certificate 'n64brew.dev' will expire on Tue 23 Aug 2022 09:10:11 GMT +0000. [10:31:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/97430f5534a4...5900aefb755c [10:31:34] [url] Comparing 97430f5534a4...5900aefb755c · miraheze/ssl · GitHub | github.com [10:31:35] [02miraheze/ssl] 07MirahezeSSLBot 035900aef - Bot: Update SSL cert for programming.red [10:31:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.12, 1.78, 2.00 [10:34:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.81, 1.97 [10:36:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [10:36:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.405076742172241 seconds [10:39:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5900aefb755c...e734a0467a60 [10:39:32] [url] Comparing 5900aefb755c...e734a0467a60 · miraheze/ssl · GitHub | github.com [10:39:33] [02miraheze/ssl] 07MirahezeSSLBot 03e734a04 - Bot: Update SSL cert for wiki.pablojprieto.com.es [10:40:57] RECOVERY - wiki.thehall.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thehall.xyz' will expire on Tue 23 Aug 2022 08:50:07 GMT +0000. [10:41:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.98, 1.36, 1.68 [10:42:37] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [10:46:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.39, 3.21, 2.83 [10:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.60, 1.96, 1.88 [10:48:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.10, 3.06, 2.82 [10:49:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.58, 1.91, 1.99 [10:51:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.64, 3.09, 2.77 [10:51:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.10, 2.01, 2.01 [10:52:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.66, 3.13 [10:52:52] RECOVERY - pubwiki.lab.co.pl - LetsEncrypt on sslhost is OK: OK - Certificate 'pubwiki.lab.co.pl' will expire on Tue 23 Aug 2022 08:56:40 GMT +0000. [10:55:10] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.48, 2.91, 2.77 [10:55:43] RECOVERY - wiki.shaazzz.ir - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.shaazzz.ir' will expire on Tue 23 Aug 2022 09:06:44 GMT +0000. [10:55:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.77, 1.69, 1.66 [10:56:16] RECOVERY - wiki.scvo.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.scvo.org' will expire on Tue 23 Aug 2022 09:01:11 GMT +0000. [10:56:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.40, 1.85, 1.91 [10:56:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.89, 3.53, 3.22 [10:57:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.86, 2.10, 1.80 [10:58:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.53, 3.25, 3.16 [10:59:57] RECOVERY - www.programming.red - LetsEncrypt on sslhost is OK: OK - Certificate 'programming.red' will expire on Tue 23 Aug 2022 09:31:28 GMT +0000. [11:00:04] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:01:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.56, 1.89, 1.80 [11:03:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.22, 2.01, 1.86 [11:05:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.00, 1.65, 1.74 [11:07:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.29, 1.94, 1.84 [11:09:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.50, 3.38, 2.91 [11:09:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.93, 1.51, 1.69 [11:10:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.59, 3.57, 3.39 [11:10:37] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:12:57] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.00, 3.11, 2.90 [11:14:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.47, 1.59, 1.70 [11:15:17] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.82, 3.77, 3.24 [11:15:23] RECOVERY - wiki.openhatch.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.openhatch.org' will expire on Tue 23 Aug 2022 09:22:30 GMT +0000. [11:16:33] PROBLEM - gluster121 Disk Space on gluster121 is WARNING: DISK WARNING - free space: / 88066 MB (10% inode=80%); [11:17:12] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.60, 3.51, 3.20 [11:17:27] RECOVERY - wiki.mikrodev.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mikrodev.com' will expire on Tue 23 Aug 2022 09:28:23 GMT +0000. [11:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.84, 2.09, 1.86 [11:19:06] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.82, 3.24, 3.14 [11:21:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.48, 1.71, 1.96 [11:22:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.22, 2.94, 3.29 [11:23:21] RECOVERY - programming.red - LetsEncrypt on sslhost is OK: OK - Certificate 'programming.red' will expire on Tue 23 Aug 2022 09:31:28 GMT +0000. [11:23:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.12, 1.84, 1.98 [11:24:48] RECOVERY - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.pablojprieto.com.es' will expire on Tue 23 Aug 2022 09:39:26 GMT +0000. [11:25:40] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.78, 1.82, 1.95 [11:27:40] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.28, 2.45, 2.17 [11:28:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.48, 3.54, 3.01 [11:28:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.90, 1.95 [11:28:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.89, 3.61, 3.42 [11:28:34] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.41, 2.88, 2.09 [11:28:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 3.58, 3.07 [11:30:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.14, 3.36, 3.01 [11:30:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.97, 3.40, 3.37 [11:30:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.23, 3.04, 2.93 [11:33:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.68, 6.66, 5.71 [11:34:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.38, 3.52, 3.42 [11:35:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.44, 6.50, 5.77 [11:36:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.34, 3.37, 3.38 [11:40:01] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.55, 1.92, 1.99 [11:40:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.92, 1.88 [11:44:10] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.47, 3.48, 3.29 [11:48:00] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.04, 3.26, 3.25 [11:51:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.18, 11.21, 9.73 [11:51:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.01, 1.33, 1.62 [11:53:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.87, 11.56, 10.05 [11:53:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [11:54:48] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.65, 3.96, 3.35 [11:56:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.78, 3.36 [11:57:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.01, 1.81, 1.75 [11:58:37] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.87, 3.39, 3.26 [11:59:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.46, 1.60, 1.67 [12:03:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.74, 9.31, 9.78 [12:04:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.86, 1.98 [12:14:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.03, 1.84, 1.90 [12:15:40] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.38, 1.45, 1.97 [12:16:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.87, 1.86, 1.90 [12:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.25, 2.09, 1.99 [12:21:04] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.85, 3.68, 3.29 [12:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [12:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4053990840911865 seconds [12:25:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.27, 1.92, 1.93 [12:26:12] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.54, 1.90, 1.66 [12:26:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.59, 1.89, 1.99 [12:27:09] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.59, 3.32, 2.94 [12:28:43] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.49, 2.75, 3.06 [12:29:04] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.23, 2.83, 2.80 [12:30:01] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.13, 1.74, 1.67 [12:31:20] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.54, 9.44, 8.59 [12:31:40] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 0.76, 1.58, 1.84 [12:33:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.76, 9.39, 8.66 [12:33:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.86, 1.96, 1.74 [12:33:58] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.60, 3.47, 3.15 [12:35:52] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 1.73, 2.95, 3.00 [12:36:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [12:37:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.73, 2.85, 2.25 [12:37:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.54, 1.93, 1.79 [12:38:12] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.58, 1.94, 1.43 [12:40:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.56, 1.35 [12:40:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 3.35, 3.01 [12:41:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.57, 2.13, 1.90 [12:42:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.59, 3.56, 3.14 [12:43:51] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.84, 10.87, 9.67 [12:43:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.48, 1.82, 1.81 [12:44:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.11, 3.29, 3.09 [12:45:27] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.47, 3.42, 3.20 [12:45:36] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [12:47:10] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.37, 3.71, 3.40 [12:47:22] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.26, 3.39, 3.22 [12:47:45] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.54, 10.05, 9.70 [12:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.51, 1.92, 1.81 [12:48:27] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.84, 10.53, 9.21 [12:49:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.63, 1.28, 1.60 [12:50:59] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.67, 3.14, 3.25 [12:51:12] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.45, 3.67, 3.38 [12:52:03] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.28, 10.11, 9.78 [12:52:16] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.78, 11.52, 9.94 [12:53:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [12:53:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.73, 9.91, 9.75 [12:54:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.89, 1.96, 1.89 [12:55:02] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.59, 3.34, 3.33 [12:56:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.83, 3.93, 3.52 [12:58:00] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.68, 9.38, 9.50 [12:58:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.62, 4.14, 3.42 [12:58:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.84, 5.23, 3.91 [13:01:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.73, 2.07, 1.83 [13:04:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.27, 3.63, 3.49 [13:04:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.14, 1.94, 1.87 [13:04:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.93, 3.58, 3.36 [13:05:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.46, 1.84, 1.80 [13:06:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.64, 2.94, 3.25 [13:06:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.86, 1.85 [13:06:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.82, 3.33, 3.30 [13:06:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [13:06:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406978130340576 seconds [13:06:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.55, 3.46, 3.71 [13:09:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.12, 1.79, 1.77 [13:10:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.84, 3.82, 3.94 [13:14:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.46, 10.20, 9.81 [13:14:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.34, 4.98, 4.34 [13:14:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.52, 3.54, 3.32 [13:15:20] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.30, 10.07, 9.08 [13:16:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 3.75, 3.47 [13:17:51] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.90, 9.72, 9.73 [13:18:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.43, 3.45, 3.38 [13:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 2.13, 1.96 [13:18:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.85, 3.89, 3.52 [13:19:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.86, 10.18, 9.37 [13:20:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.50, 3.08, 3.25 [13:20:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.47, 4.14, 3.66 [13:21:33] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 11.13, 9.08, 7.23 [13:22:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.37, 3.96, 3.66 [13:25:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.18, 7.60, 7.05 [13:26:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.30, 3.88, 3.68 [13:28:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.82, 3.47, 3.96 [13:28:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.73, 3.04, 3.38 [13:29:19] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.75, 10.98, 10.33 [13:29:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.50, 6.77, 6.80 [13:30:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.43, 3.74, 3.68 [13:33:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.41, 10.19, 9.83 [13:34:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.95, 3.61, 3.81 [13:35:03] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.16, 11.76, 10.75 [13:35:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [13:36:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.92, 1.99 [13:36:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.91, 3.77, 3.55 [13:37:21] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.33, 9.83, 9.78 [13:38:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.73, 3.62, 3.74 [13:38:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.84, 3.34, 3.42 [13:40:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.21, 4.11, 3.90 [13:40:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.75, 4.22, 3.73 [13:40:46] PROBLEM - aarthal.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'aarthal.com' expires in 15 day(s) (Fri 10 Jun 2022 13:38:13 GMT +0000). [13:41:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.84, 11.88, 11.17 [13:41:48] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.72, 10.40, 9.83 [13:42:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.98, 2.00 [13:43:44] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.83, 9.72, 9.65 [13:44:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.87, 3.55 [13:44:51] PROBLEM - www.sidem.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.sidem.wiki' expires in 15 day(s) (Fri 10 Jun 2022 13:42:02 GMT +0000). [13:45:10] PROBLEM - tot.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tot.wiki' expires in 15 day(s) (Fri 10 Jun 2022 13:18:20 GMT +0000). [13:45:10] PROBLEM - wikimas.kr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikimas.kr' expires in 15 day(s) (Fri 10 Jun 2022 13:29:51 GMT +0000). [13:45:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e734a0467a60...fbf541082cb9 [13:45:34] [url] Comparing e734a0467a60...fbf541082cb9 · miraheze/ssl · GitHub | github.com [13:45:35] [02miraheze/ssl] 07MirahezeSSLBot 03fbf5410 - Bot: Update SSL cert for aarthal.com [13:46:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.66, 4.37, 3.76 [13:47:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.97, 11.10, 10.35 [13:47:21] PROBLEM - ipv6bolivia.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ipv6bolivia.tk' expires in 15 day(s) (Fri 10 Jun 2022 13:36:09 GMT +0000). [13:48:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.95, 3.69 [13:49:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.26, 11.53, 11.17 [13:50:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 5.18, 3.94, 3.67 [13:51:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.42, 10.92, 10.99 [13:51:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/fbf541082cb9...4eee82e8a3e3 [13:51:14] [url] Comparing fbf541082cb9...4eee82e8a3e3 · miraheze/ssl · GitHub | github.com [13:51:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.16, 10.09, 10.12 [13:51:15] [02miraheze/ssl] 07MirahezeSSLBot 034eee82e - Bot: Update SSL cert for tot.wiki [13:52:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.72, 4.12, 3.82 [13:52:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.18, 3.77, 3.66 [13:54:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.98, 3.81 [13:56:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.08, 4.54, 4.04 [13:56:18] PROBLEM - wiki.misslessfighters.club - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.misslessfighters.club' expires in 15 day(s) (Fri 10 Jun 2022 13:25:09 GMT +0000). [13:56:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.08, 3.73, 3.66 [13:57:38] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.82, 10.56, 9.81 [13:58:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.18, 3.47, 3.57 [13:59:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4eee82e8a3e3...4758169c84c9 [13:59:17] [url] Comparing 4eee82e8a3e3...4758169c84c9 · miraheze/ssl · GitHub | github.com [13:59:18] [02miraheze/ssl] 07MirahezeSSLBot 034758169 - Bot: Update SSL cert for www.sidem.wiki [13:59:43] PROBLEM - mabi.myomi.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mabi.myomi.net' expires in 15 day(s) (Fri 10 Jun 2022 13:44:33 GMT +0000). [14:01:27] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.03, 9.86, 9.70 [14:01:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4758169c84c9...2e45f934bdb9 [14:01:30] [url] Comparing 4758169c84c9...2e45f934bdb9 · miraheze/ssl · GitHub | github.com [14:01:31] [02miraheze/ssl] 07MirahezeSSLBot 032e45f93 - Bot: Update SSL cert for wikimas.kr [14:02:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.89, 3.81, 3.89 [14:02:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.52, 3.86, 3.70 [14:04:03] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.33, 10.24, 10.09 [14:04:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.64, 4.03, 3.95 [14:04:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.79, 1.98 [14:05:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.79, 11.65, 11.26 [14:05:06] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [14:06:00] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.67, 9.95, 9.99 [14:06:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.07, 3.33, 3.71 [14:06:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/2e45f934bdb9...023f8debbc26 [14:06:30] [url] Comparing 2e45f934bdb9...023f8debbc26 · miraheze/ssl · GitHub | github.com [14:06:31] [02miraheze/ssl] 07MirahezeSSLBot 03023f8de - Bot: Update SSL cert for wiki.misslessfighters.club [14:06:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.93, 3.98, 3.77 [14:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.80, 11.53, 11.25 [14:07:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.15, 1.64, 1.99 [14:08:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.41, 3.28, 3.86 [14:09:07] PROBLEM - gs.sidem.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'gs.sidem.wiki' expires in 15 day(s) (Fri 10 Jun 2022 13:44:54 GMT +0000). [14:09:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/023f8debbc26...d4a1655631a1 [14:09:30] [url] Comparing 023f8debbc26...d4a1655631a1 · miraheze/ssl · GitHub | github.com [14:09:31] [02miraheze/ssl] 07MirahezeSSLBot 03d4a1655 - Bot: Update SSL cert for ipv6bolivia.tk [14:09:44] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d4a1655631a1...511e09b5c817 [14:09:44] [url] Comparing d4a1655631a1...511e09b5c817 · miraheze/ssl · GitHub | github.com [14:09:46] [02miraheze/ssl] 07MirahezeSSLBot 03511e09b - Bot: Update SSL cert for gs.sidem.wiki [14:09:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.59, 2.07, 2.11 [14:10:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.09, 3.60, 3.71 [14:10:32] RECOVERY - aarthal.com - LetsEncrypt on sslhost is OK: OK - Certificate 'aarthal.com' will expire on Tue 23 Aug 2022 12:45:25 GMT +0000. [14:10:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.07, 3.88, 3.77 [14:10:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.34, 4.18, 4.12 [14:10:57] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.12, 11.19, 10.38 [14:11:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.06, 11.09, 11.09 [14:11:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/511e09b5c817...13d025975bcc [14:11:22] [url] Comparing 511e09b5c817...13d025975bcc · miraheze/ssl · GitHub | github.com [14:11:23] [02miraheze/ssl] 07MirahezeSSLBot 0313d0259 - Bot: Update SSL cert for mabi.myomi.net [14:12:27] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.43, 9.69, 8.50 [14:12:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.75, 3.94, 3.81 [14:12:53] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.76, 11.40, 10.56 [14:13:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.46, 11.24, 11.15 [14:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.79, 3.75 [14:14:42] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.54, 11.30, 10.47 [14:16:27] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.84, 8.82, 8.43 [14:16:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.50, 3.41, 3.84 [14:17:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.22, 1.60, 1.90 [14:18:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.25, 1.95, 1.89 [14:18:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.44, 10.71, 10.44 [14:18:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.63, 4.03, 3.86 [14:18:41] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.52, 11.22, 10.64 [14:20:37] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.48, 10.85, 10.58 [14:22:30] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.64, 9.46, 10.04 [14:22:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.26, 3.93 [14:22:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.82, 2.48, 3.30 [14:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [14:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.405184268951416 seconds [14:24:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.22, 3.09, 3.36 [14:24:29] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.68, 9.51, 10.07 [14:24:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.07, 3.75, 4.01 [14:24:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 1.99, 3.51, 3.80 [14:25:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.95, 8.64, 9.88 [14:25:06] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [14:25:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.36, 1.79, 1.84 [14:26:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 2.00, 1.98 [14:26:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.64, 3.93 [14:28:12] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.27, 1.69, 1.32 [14:28:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.74, 4.26, 3.76 [14:28:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 3.99, 4.02 [14:28:57] RECOVERY - mabi.myomi.net - LetsEncrypt on sslhost is OK: OK - Certificate 'mabi.myomi.net' will expire on Tue 23 Aug 2022 13:11:15 GMT +0000. [14:29:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.45, 9.94, 10.16 [14:29:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.62, 1.88, 1.88 [14:30:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.85, 1.40, 1.26 [14:30:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.01, 3.40, 3.50 [14:30:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.67, 3.38, 3.78 [14:32:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.17, 3.38, 3.55 [14:33:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.32, 9.44, 9.92 [14:34:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.86, 3.21, 3.47 [14:35:57] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.39, 1.41, 1.66 [14:36:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.45, 3.76, 3.76 [14:36:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.99, 3.82, 3.66 [14:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.44, 10.96, 10.46 [14:38:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.92, 3.50, 3.67 [14:38:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.72, 3.72, 3.64 [14:38:43] RECOVERY - gs.sidem.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'gs.sidem.wiki' will expire on Tue 23 Aug 2022 13:09:38 GMT +0000. [14:41:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.57, 9.55, 10.00 [14:41:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.62, 1.75, 1.73 [14:42:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 3.45, 3.48 [14:42:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.52, 1.69 [14:42:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.66, 4.04, 3.77 [14:43:33] RECOVERY - www.sidem.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.sidem.wiki' will expire on Tue 23 Aug 2022 12:59:12 GMT +0000. [14:44:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.47, 3.48 [14:44:29] RECOVERY - tot.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'tot.wiki' will expire on Tue 23 Aug 2022 12:51:08 GMT +0000. [14:44:30] RECOVERY - wikimas.kr - LetsEncrypt on sslhost is OK: OK - Certificate 'wikimas.kr' will expire on Tue 23 Aug 2022 13:01:24 GMT +0000. [14:44:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.22, 4.08, 3.83 [14:45:21] RECOVERY - ipv6bolivia.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'ipv6bolivia.tk' will expire on Tue 23 Aug 2022 13:09:22 GMT +0000. [14:47:39] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.65, 6.30, 5.56 [14:48:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.09, 3.01, 3.31 [14:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.47, 2.06, 1.86 [14:49:34] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.28, 6.52, 5.75 [14:49:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.19, 1.98, 1.84 [14:50:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.58, 3.75 [14:51:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.95, 10.75, 10.11 [14:51:07] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.57, 10.24, 9.41 [14:52:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.43, 4.47, 4.04 [14:53:02] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.58, 9.85, 9.36 [14:54:18] RECOVERY - wiki.misslessfighters.club - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.misslessfighters.club' will expire on Tue 23 Aug 2022 13:06:24 GMT +0000. [14:55:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.37, 10.97, 10.35 [14:55:36] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.10, 9.74, 9.04 [14:56:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.60, 3.66, 3.81 [14:57:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.17, 11.31, 10.55 [14:57:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 10.20, 9.97, 9.21 [14:58:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.90, 1.94 [14:58:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.60, 3.75 [14:59:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.15, 11.15, 10.61 [14:59:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.09, 6.81, 6.13 [15:00:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.58, 3.58, 3.26 [15:02:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.93, 3.97, 3.90 [15:03:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.95, 9.37, 10.00 [15:03:21] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.93, 10.48, 9.64 [15:03:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.45, 7.35, 6.49 [15:04:25] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.18, 3.71, 3.36 [15:06:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.00, 3.49, 3.32 [15:06:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.25, 3.48, 3.85 [15:07:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.06, 7.42, 6.75 [15:08:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.91, 3.24, 3.25 [15:09:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.17, 9.85, 9.67 [15:10:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.09, 1.88, 1.89 [15:10:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.63, 2.73, 3.34 [15:11:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.35, 7.54, 6.93 [15:11:51] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.88, 10.21, 10.12 [15:13:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.06, 7.63, 7.06 [15:16:47] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.75, 10.48, 9.49 [15:17:34] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.62, 9.96, 10.13 [15:17:49] PROBLEM - ping6 on ns1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 120.87 ms [15:18:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 3.65, 3.51 [15:18:36] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [15:18:43] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.26, 9.56, 9.30 [15:19:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.80, 10.91, 10.25 [15:19:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.25, 3.45, 3.20 [15:20:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.72, 3.56 [15:21:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.33, 10.84, 10.46 [15:21:46] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.14, 3.07, 3.10 [15:22:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.80, 3.96, 3.65 [15:23:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [15:25:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.19, 9.95, 10.07 [15:25:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.59, 6.09, 6.62 [15:25:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.99, 3.43, 3.27 [15:26:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 3.68, 3.64 [15:27:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.06, 3.34, 3.25 [15:28:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 3.88, 3.70 [15:30:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.88, 3.49, 3.29 [15:30:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.09, 3.44, 3.56 [15:32:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.13, 3.80, 3.68 [15:34:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.10, 3.67, 3.40 [15:34:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.35, 3.77, 3.69 [15:35:08] PROBLEM - mw122 APT on mw122 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:35:29] PROBLEM - mw111 APT on mw111 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:36:23] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:36:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.79, 1.99 [15:36:24] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:36:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.08, 4.49, 3.98 [15:36:31] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:36:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.88, 3.12, 3.36 [15:36:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [15:38:25] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 4.616 second response time [15:38:25] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14656 bytes in 4.129 second response time [15:38:30] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14642 bytes in 4.215 second response time [15:39:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.22, 10.74, 10.57 [15:39:10] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.37, 4.16, 3.65 [15:39:40] PROBLEM - phab121 APT on phab121 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:39:53] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 118.19 ms [15:40:03] PROBLEM - db111 APT on db111 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (3 critical updates). [15:40:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.21, 3.74, 3.61 [15:41:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.57, 11.08, 10.72 [15:41:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.54, 3.65, 3.53 [15:41:07] PROBLEM - cloud12 APT on cloud12 is CRITICAL: APT CRITICAL: 73 packages available for upgrade (3 critical updates). [15:42:30] PROBLEM - bast121 APT on bast121 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:42:34] PROBLEM - cloud10 APT on cloud10 is CRITICAL: APT CRITICAL: 73 packages available for upgrade (3 critical updates). [15:42:36] PROBLEM - mw102 APT on mw102 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:42:51] PROBLEM - mw121 APT on mw121 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:43:01] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 4.06, 3.69 [15:43:19] PROBLEM - ldap111 APT on ldap111 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:43:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.14, 6.53, 6.20 [15:43:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.59, 1.61, 2.00 [15:44:00] PROBLEM - ping6 on ns1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 120.43 ms [15:44:53] PROBLEM - es121 APT on es121 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (3 critical updates). [15:44:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.49, 3.44, 3.51 [15:45:20] PROBLEM - mail121 APT on mail121 is CRITICAL: APT CRITICAL: 47 packages available for upgrade (3 critical updates). [15:45:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.58, 6.02, 6.05 [15:46:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 4.00, 3.73 [15:47:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.38, 9.55, 10.19 [15:47:22] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.24, 3.56, 3.38 [15:48:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.42, 3.45, 3.56 [15:48:15] PROBLEM - gluster101 APT on gluster101 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.51, 2.08, 2.02 [15:48:48] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.47, 3.18, 3.40 [15:48:50] PROBLEM - es101 APT on es101 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (3 critical updates). [15:48:58] PROBLEM - test101 APT on test101 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:49:38] PROBLEM - jobchron121 APT on jobchron121 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:49:39] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:50:35] PROBLEM - cp31 APT on cp31 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (3 critical updates). [15:50:58] PROBLEM - matomo101 APT on matomo101 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:51:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.14, 11.06, 10.62 [15:51:12] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.05, 3.61, 3.46 [15:51:32] PROBLEM - db121 APT on db121 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (3 critical updates). [15:51:46] PROBLEM - mw112 APT on mw112 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:52:13] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.72, 2.70, 3.24 [15:52:30] PROBLEM - mem101 APT on mem101 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:52:48] PROBLEM - mwtask111 APT on mwtask111 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [15:52:48] PROBLEM - cp30 APT on cp30 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (3 critical updates). [15:53:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.81, 10.95, 10.65 [15:53:06] PROBLEM - cloud11 APT on cloud11 is CRITICAL: APT CRITICAL: 73 packages available for upgrade (3 critical updates). [15:53:13] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.69, 1.73, 1.53 [15:53:42] PROBLEM - prometheus101 APT on prometheus101 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:54:31] PROBLEM - db101 APT on db101 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (3 critical updates). [15:54:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.75, 3.40, 3.84 [15:54:47] PROBLEM - es111 APT on es111 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (3 critical updates). [15:54:54] PROBLEM - puppet111 APT on puppet111 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (3 critical updates). [15:55:13] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.11, 1.52, 1.48 [15:55:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.44, 1.99, 1.92 [15:56:02] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 119.90 ms [15:56:33] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.57, 10.14, 10.03 [15:57:21] PROBLEM - cp20 APT on cp20 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (3 critical updates). [15:57:33] PROBLEM - cp21 APT on cp21 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (3 critical updates). [15:58:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.03, 3.79, 3.47 [15:58:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.87, 1.99 [15:58:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.48, 3.80, 3.88 [15:58:51] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (3 critical updates). [15:58:52] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.03, 3.63, 3.50 [15:59:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.13, 11.24, 10.85 [15:59:21] PROBLEM - graylog121 APT on graylog121 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (3 critical updates). [15:59:41] PROBLEM - gluster111 APT on gluster111 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [15:59:50] PROBLEM - gluster121 APT on gluster121 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [16:00:09] PROBLEM - ping6 on ns1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 126.25 ms [16:00:23] PROBLEM - mon111 APT on mon111 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (3 critical updates). [16:00:27] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.26, 10.13, 10.08 [16:00:47] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.14, 3.51, 3.48 [16:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.69, 10.98, 10.80 [16:02:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.94, 3.79, 3.54 [16:02:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.38, 3.68, 3.82 [16:03:18] PROBLEM - mw101 APT on mw101 is CRITICAL: APT CRITICAL: 3 packages available for upgrade (3 critical updates). [16:03:28] PROBLEM - mem121 APT on mem121 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (3 critical updates). [16:04:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.44, 3.27, 3.37 [16:04:13] PROBLEM - db112 APT on db112 is CRITICAL: APT CRITICAL: 48 packages available for upgrade (3 critical updates). [16:05:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.13, 11.59, 11.03 [16:05:40] PROBLEM - bast101 APT on bast101 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (3 critical updates). [16:06:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.53, 4.18, 3.95 [16:06:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [16:06:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406927585601807 seconds [16:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.04, 11.48, 11.07 [16:10:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.83, 3.59 [16:10:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.55, 3.86, 3.60 [16:11:20] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.54, 3.78, 3.42 [16:11:57] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.45, 10.25, 9.80 [16:12:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.76, 4.30, 3.80 [16:13:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [16:13:53] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.38, 10.20, 9.84 [16:14:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.60, 3.92, 3.70 [16:15:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.24, 3.49, 3.43 [16:16:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.76, 3.72 [16:16:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.11, 1.77, 1.79 [16:16:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.10, 3.90, 3.71 [16:19:02] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.68, 3.22, 3.35 [16:19:43] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.07, 10.62, 10.18 [16:21:39] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.14, 9.95, 10.00 [16:22:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.11, 2.90, 3.39 [16:23:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.07, 6.78, 6.06 [16:23:48] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.34, 10.28, 9.89 [16:23:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.34, 1.79, 1.99 [16:24:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.74, 1.96, 1.93 [16:24:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.45, 3.44, 3.98 [16:25:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.05, 10.93, 10.70 [16:25:29] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.38, 10.75, 9.76 [16:25:45] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.68, 11.19, 10.28 [16:25:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.62, 2.01, 2.04 [16:27:23] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.55, 10.02, 9.62 [16:27:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.53, 6.41, 6.14 [16:27:42] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.02, 10.57, 10.15 [16:29:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.31, 11.44, 11.02 [16:29:39] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 5.30, 8.97, 9.64 [16:30:16] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 115.60 ms [16:32:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.41, 4.33, 4.01 [16:33:30] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.25, 11.15, 10.33 [16:34:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.22, 3.83, 3.52 [16:34:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.54, 3.29 [16:36:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.82, 3.74, 3.86 [16:36:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.04, 3.31, 3.23 [16:37:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.11, 11.44, 11.04 [16:37:24] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.82, 10.74, 10.40 [16:38:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.87, 1.85 [16:40:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.85, 1.85 [16:41:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.33, 11.45, 11.14 [16:42:02] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.03, 10.35, 9.86 [16:42:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.83, 3.95, 3.62 [16:42:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.60, 4.14, 3.99 [16:43:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.20, 6.91, 6.34 [16:43:58] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.32, 10.19, 9.86 [16:44:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.96, 3.96 [16:45:52] PROBLEM - cp21 Disk Space on cp21 is WARNING: DISK WARNING - free space: / 4276 MB (10% inode=96%); [16:46:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 4.12, 4.02 [16:47:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.65, 7.32, 6.65 [16:47:49] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.66, 10.17, 9.94 [16:48:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.52, 2.06, 1.92 [16:49:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.18, 7.84, 6.91 [16:51:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.36, 7.51, 6.91 [16:51:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.32, 1.74, 2.00 [16:52:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.83, 4.26, 3.68 [16:53:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [16:53:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4066221714019775 seconds [16:54:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.59, 3.56, 3.97 [16:55:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.29, 9.27, 10.01 [16:55:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.58, 1.92, 2.00 [16:59:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.98, 10.91, 10.53 [16:59:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.97, 10.31, 9.95 [17:00:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.62, 3.92, 3.93 [17:01:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.04, 11.86, 10.91 [17:01:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.78, 6.73, 6.79 [17:02:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.73, 3.57, 3.77 [17:03:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.16, 10.06, 9.96 [17:04:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.28, 3.64, 3.76 [17:05:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.16, 11.16, 10.71 [17:05:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.59, 11.95, 11.20 [17:05:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.27, 7.10, 6.99 [17:06:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.05, 3.45, 3.66 [17:06:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.53, 3.22, 3.59 [17:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.81, 11.51, 10.90 [17:07:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.12, 1.68, 1.92 [17:10:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.94, 3.47, 3.63 [17:11:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.75, 11.62, 11.04 [17:11:20] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.34, 10.83, 10.20 [17:11:43] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.88, 10.43, 9.45 [17:12:36] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [17:12:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.53, 3.81, 3.73 [17:13:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.94, 11.31, 11.00 [17:13:20] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.67, 10.77, 10.27 [17:13:37] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.81, 9.55, 9.23 [17:13:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.05, 1.70, 1.83 [17:14:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.01, 2.75, 3.31 [17:14:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.77, 3.67, 3.68 [17:14:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.42, 2.96, 3.36 [17:15:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.03, 10.14, 10.09 [17:15:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.49, 1.59, 1.78 [17:16:26] PROBLEM - cp20 Disk Space on cp20 is WARNING: DISK WARNING - free space: / 4285 MB (10% inode=96%); [17:16:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.24, 3.49, 3.95 [17:17:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.15, 6.29, 6.80 [17:18:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.31, 3.61, 3.63 [17:19:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.89, 2.03, 1.90 [17:20:14] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.41, 1.83, 1.29 [17:20:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.86, 3.50, 3.60 [17:21:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.35, 1.76, 1.82 [17:22:14] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.77, 1.81, 1.35 [17:22:26] RECOVERY - cp20 Disk Space on cp20 is OK: DISK OK - free space: / 4827 MB (12% inode=96%); [17:22:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.50, 3.67, 3.85 [17:22:34] !log rm varnishlog-5xx.log* on cp* [17:22:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [17:22:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:23:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.26, 7.19, 7.03 [17:23:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [17:23:52] RECOVERY - cp21 Disk Space on cp21 is OK: DISK OK - free space: / 4601 MB (11% inode=96%); [17:24:10] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.78, 10.25, 10.15 [17:24:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.86, 3.64, 3.44 [17:24:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.42, 3.74, 3.87 [17:24:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.83, 4.08, 3.64 [17:26:06] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.13, 9.59, 9.93 [17:26:14] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.94, 1.47, 1.34 [17:26:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.93, 3.83, 3.59 [17:27:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.95, 6.09, 6.62 [17:27:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.55, 2.01, 1.89 [17:28:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.59, 4.00, 3.68 [17:29:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.32, 1.74, 1.80 [17:30:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.32, 3.79, 3.55 [17:30:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.07, 3.37, 3.44 [17:30:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.67, 3.59 [17:31:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.97, 10.75, 10.72 [17:31:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.14, 7.44, 7.05 [17:31:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.72, 2.06, 1.91 [17:32:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.51, 3.47 [17:33:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.92, 11.24, 10.91 [17:33:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.57, 6.61, 6.79 [17:34:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.42, 3.64, 3.55 [17:35:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.11, 11.13, 10.85 [17:35:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.76, 1.94, 1.89 [17:36:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.19, 3.58, 3.48 [17:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.53, 10.58, 10.70 [17:37:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.38, 11.45, 11.06 [17:38:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.05, 3.35, 3.41 [17:38:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.79, 1.99 [17:38:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.39, 3.21, 3.39 [17:40:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.18, 3.57, 3.47 [17:40:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.01, 4.33, 4.00 [17:41:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.48, 10.73, 10.92 [17:42:37] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.14, 11.17, 10.29 [17:42:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.95, 3.41, 3.45 [17:43:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.04, 1.45, 1.70 [17:44:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.67, 3.58 [17:44:33] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.72, 10.56, 10.17 [17:44:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.16, 3.67, 3.60 [17:45:05] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.86, 1.93, 1.36 [17:46:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.91, 1.94 [17:47:04] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.27, 1.64, 1.32 [17:48:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.17, 3.67, 3.65 [17:48:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.80, 3.26, 3.39 [17:49:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.08, 1.75, 1.75 [17:50:06] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [17:50:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.45, 3.47 [17:50:13] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 12.00, 10.65, 9.59 [17:50:21] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.02, 9.64, 9.96 [17:50:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.53, 3.99, 3.77 [17:51:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.28, 6.71, 6.41 [17:51:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.66, 1.76, 1.76 [17:52:07] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.77, 9.85, 9.42 [17:52:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.70, 3.56, 3.49 [17:53:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.12, 6.47, 6.37 [17:53:37] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [17:53:37] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.4063193798065186 seconds [17:54:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.64, 3.89, 3.79 [17:55:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.95, 9.49, 10.10 [17:56:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.58, 3.79, 3.62 [17:56:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.33, 1.81, 1.95 [17:57:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.56, 11.26, 10.77 [17:57:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 3.15, 2.56, 2.07 [17:57:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.99, 10.93, 9.99 [17:58:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.35, 3.95, 3.82 [17:59:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.89, 11.25, 10.81 [17:59:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.45, 10.73, 10.42 [18:00:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.45, 3.15, 3.40 [18:00:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.50, 3.70, 3.53 [18:01:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.12, 1.84, 1.89 [18:02:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.31, 3.91, 3.85 [18:02:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.43, 3.53, 3.49 [18:03:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.97, 11.00, 10.62 [18:04:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.10, 3.49, 3.52 [18:04:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.06, 3.89, 3.84 [18:04:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.34, 3.92, 3.63 [18:05:33] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.54, 9.98, 9.93 [18:05:52] PROBLEM - cp21 Disk Space on cp21 is WARNING: DISK WARNING - free space: / 4280 MB (10% inode=96%); [18:06:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.60, 1.90, 1.85 [18:06:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.66, 3.32, 3.86 [18:06:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.66, 3.81, 3.81 [18:06:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.36, 3.66, 3.57 [18:07:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.57, 9.22, 9.98 [18:08:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.84, 3.99 [18:08:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.53, 4.06, 3.91 [18:09:46] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.51, 9.93, 9.84 [18:10:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.23, 3.84, 3.99 [18:10:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.59, 3.89, 3.87 [18:10:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 4.01, 3.73 [18:11:43] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.07, 9.89, 9.86 [18:11:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.33, 1.52, 1.68 [18:12:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.06, 4.06, 3.73 [18:12:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.23, 4.62, 4.25 [18:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.86, 3.69 [18:18:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.07, 4.18, 3.86 [18:20:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.14, 3.81, 3.77 [18:20:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.91, 3.70, 3.70 [18:20:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.15, 3.77, 3.84 [18:22:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.83, 3.61, 3.66 [18:22:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.58, 4.13, 3.96 [18:23:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [18:23:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.38, 1.78, 1.67 [18:24:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.56, 3.65, 3.81 [18:25:06] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [18:25:45] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.20, 10.68, 9.99 [18:26:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.57, 4.11, 3.85 [18:26:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.01, 3.94, 3.88 [18:27:39] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.97, 9.80, 9.76 [18:27:54] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 10.32, 8.08, 6.94 [18:28:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.87, 3.80 [18:28:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.40, 3.25, 3.84 [18:28:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.03, 3.58, 3.76 [18:30:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.03, 4.12, 3.89 [18:30:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.55, 4.11, 4.08 [18:30:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.12, 3.49, 3.54 [18:30:45] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.56, 10.47, 9.81 [18:31:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.86, 9.74, 10.17 [18:31:04] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.73, 1.78, 1.31 [18:32:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.25, 3.85, 3.82 [18:32:36] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [18:32:36] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [18:32:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.80, 3.58, 3.57 [18:32:42] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.67, 10.33, 9.86 [18:32:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.36, 3.93, 3.85 [18:33:05] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.07, 1.51, 1.26 [18:34:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.21, 3.76, 3.95 [18:34:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.52, 3.70 [18:35:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.84, 11.07, 10.58 [18:36:26] PROBLEM - cp20 Disk Space on cp20 is WARNING: DISK WARNING - free space: / 4283 MB (10% inode=96%); [18:36:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.01, 4.10, 4.04 [18:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.17, 11.10, 10.66 [18:37:31] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 4.68, 7.04, 7.21 [18:38:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.00, 9.90, 9.84 [18:40:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.26, 3.80, 3.95 [18:42:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 3.99, 3.80 [18:42:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.55, 4.59, 4.22 [18:42:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.83, 3.17, 3.37 [18:42:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.72, 3.86, 3.72 [18:44:31] [02puppet] 07Universal-Omega synchronize pull request 03#2588: Reformat grafana-webhook - 13https://github.com/miraheze/puppet/pull/2588 [18:44:31] [url] Page not found · GitHub · GitHub | github.com [18:45:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.28, 9.74, 10.20 [18:45:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.35, 1.83, 1.97 [18:46:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.11, 3.52, 3.70 [18:46:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.06, 3.57, 3.49 [18:46:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.70, 3.72 [18:47:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.40, 6.34, 6.79 [18:48:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.12, 3.40, 3.44 [18:50:35] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.48, 10.49, 9.80 [18:50:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.08, 3.65, 3.53 [18:51:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.07, 11.05, 10.58 [18:51:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.85, 6.78, 6.86 [18:52:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.00, 3.35, 3.43 [18:53:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.29, 6.14, 6.61 [18:53:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [18:53:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406902551651001 seconds [18:54:24] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.00, 9.98, 9.79 [18:54:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.26, 3.17, 3.34 [18:54:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.17, 3.50, 3.58 [18:55:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.16, 11.37, 10.84 [18:56:06] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [18:56:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.33, 3.49, 3.58 [18:57:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.08, 11.79, 11.07 [18:58:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.76, 3.55, 3.46 [19:00:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.04, 3.46, 3.98 [19:00:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.26, 3.25, 3.35 [19:00:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.22, 3.60, 3.59 [19:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.60, 11.81, 11.31 [19:01:10] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [19:01:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.22, 10.61, 9.97 [19:01:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.21, 1.46, 1.65 [19:02:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.79, 3.90, 4.10 [19:02:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.51, 3.56 [19:03:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.21, 11.01, 10.22 [19:03:40] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.48, 10.69, 9.93 [19:04:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.51, 4.24, 3.90 [19:05:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.87, 2.05, 1.39 [19:05:36] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.23, 9.56, 9.60 [19:06:39] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.stardewmodding.wiki All nameservers failed to answer the query. [19:06:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.23, 4.04, 3.77 [19:07:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.66, 1.79, 1.37 [19:08:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.83, 3.63, 3.48 [19:08:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.63, 3.92, 3.76 [19:09:15] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.49, 1.63, 1.36 [19:09:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.37, 10.19, 10.15 [19:10:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.21, 3.43, 3.68 [19:10:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.87, 4.00, 3.63 [19:12:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.86, 3.91, 3.82 [19:12:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.75, 3.92, 3.65 [19:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.58, 3.81, 3.80 [19:14:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.99, 10.38, 9.97 [19:14:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.01, 3.49, 3.96 [19:16:17] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.36, 10.08, 9.92 [19:16:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.89, 3.77, 4.01 [19:17:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.60, 10.92, 10.39 [19:18:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.56, 3.01, 3.34 [19:19:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.69, 11.09, 10.51 [19:21:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.67, 10.85, 10.72 [19:22:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.06, 3.42, 3.47 [19:23:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.75, 10.73, 10.69 [19:23:33] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [19:24:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.32, 3.67, 3.60 [19:24:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.05, 4.45, 3.83 [19:25:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.24, 3.16, 1.99 [19:26:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.89, 3.44, 3.54 [19:27:55] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.51, 10.68, 10.05 [19:28:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.38, 3.75, 3.58 [19:28:47] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.89, 1.80, 1.53 [19:29:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.70, 1.86, 1.72 [19:30:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.95, 3.82, 3.63 [19:30:41] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.38, 1.65, 1.51 [19:31:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.20, 10.94, 10.61 [19:31:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.98, 1.55, 1.62 [19:32:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.86, 3.70, 3.81 [19:34:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.54, 3.87, 3.65 [19:34:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.77, 4.34, 4.02 [19:35:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.07, 10.63, 10.62 [19:35:16] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.08, 1.72, 1.72 [19:36:40] PROBLEM - www.stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.re [19:36:40] stname, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.407137393951416 seconds [19:37:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.95, 1.43, 1.62 [19:37:29] [02puppet] 07Universal-Omega synchronize pull request 03#2588: Reformat grafana-webhook - 13https://github.com/miraheze/puppet/pull/2588 [19:37:29] [url] Page not found · GitHub · GitHub | github.com [19:37:35] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.61, 9.68, 9.96 [19:37:49] [02puppet] 07Universal-Omega edited pull request 03#2588: Rewrite grafana-webhook - 13https://github.com/miraheze/puppet/pull/2588 [19:37:49] [url] Page not found · GitHub · GitHub | github.com [19:38:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.96, 3.77 [19:38:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.51, 3.56, 3.98 [19:38:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.43, 3.52, 3.79 [19:39:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.51, 8.97, 9.99 [19:40:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.04, 5.31, 4.27 [19:40:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.27, 5.04, 4.44 [19:40:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.63, 3.02, 3.35 [19:40:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.07, 4.67, 4.18 [19:42:38] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [19:43:25] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.06, 10.59, 10.20 [19:44:30] [02puppet] 07RhinosF1 reviewed pull request 03#2588 commit - 13https://github.com/miraheze/puppet/pull/2588#discussion_r882050025 [19:44:31] [url] Rewrite grafana-webhook by Universal-Omega · Pull Request #2588 · miraheze/puppet · GitHub | github.com [19:44:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.25, 3.33, 3.41 [19:45:21] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.30, 10.73, 10.29 [19:47:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.96, 9.75, 10.19 [19:47:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.03, 10.49, 10.14 [19:47:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.73, 10.14, 10.13 [19:48:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 5.38, 4.33, 3.80 [19:48:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.36, 10.34, 9.56 [19:48:59] !sre [19:49:31] RhinosF1: does this works [19:49:44] hm? [19:49:51] Reception123: access? [19:50:47] Mobile only atm, why? [19:51:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.78, 10.62, 10.49 [19:51:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.15, 11.39, 10.64 [19:51:20] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.55, 10.26, 10.26 [19:51:45] Reception123: need someone for a security issue [19:51:47] CosmicAlpha: ^ [19:51:52] i'm here [19:53:15] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.90, 11.93, 10.93 [19:53:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.53, 9.97, 10.17 [19:54:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.74, 4.00 [19:54:55] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/9834e3120d16...97152080d4e0 [19:54:55] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.57, 10.03, 9.72 [19:54:55] [url] Comparing 9834e3120d16...97152080d4e0 · miraheze/mediawiki · GitHub | github.com [19:54:56] [02miraheze/mediawiki] 07RhinosF1 039715208 - Update composer.json [19:55:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.19, 11.49, 10.85 [19:55:14] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/ab89c197187d...fa91c692a9e9 [19:55:14] [url] Comparing ab89c197187d...fa91c692a9e9 · miraheze/mediawiki · GitHub | github.com [19:55:15] [02miraheze/mediawiki] 07RhinosF1 03fa91c69 - Update composer.json [19:55:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.98, 10.94, 10.70 [19:56:29] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.45, 1.93, 1.60 [19:56:33] !log [macfan@mwtask111] starting deploy of {'world': True} to all [19:56:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:57:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.23, 11.04, 10.77 [19:57:20] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.84, 10.44, 10.31 [19:58:24] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.84, 1.89, 1.62 [19:58:52] [02mediawiki] 07dependabot[bot] edited pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [19:58:53] [url] Page not found · GitHub · GitHub | github.com [19:58:56] [02mediawiki] 07dependabot[bot] edited pull request 03#6885: Bump extensions/Renameuser from `538b76d` to `44318c3` - 13https://github.com/miraheze/mediawiki/pull/6885 [19:58:57] [url] Page not found · GitHub · GitHub | github.com [19:58:58] [02mediawiki] 07dependabot[bot] edited pull request 03#6920: Bump extensions/VisualEditor from `f9f2532` to `b6619cd` - 13https://github.com/miraheze/mediawiki/pull/6920 [19:58:58] ... [19:59:01] [02mediawiki] 07dependabot[bot] edited pull request 03#6894: Bump extensions/Cite from `ab197bb` to `c4e959b` - 13https://github.com/miraheze/mediawiki/pull/6894 [19:59:01] [url] Page not found · GitHub · GitHub | github.com [19:59:07] [02mediawiki] 07dependabot[bot] edited pull request 03#6898: Bump extensions/Math from `49e4eb7` to `8bde5da` - 13https://github.com/miraheze/mediawiki/pull/6898 [19:59:07] [url] Page not found · GitHub · GitHub | github.com [19:59:07] !log [macfan@test101] starting deploy of {'world': True} to all [19:59:08] [02mediawiki] 07dependabot[bot] edited pull request 03#6905: Bump extensions/MultimediaViewer from `0c0f6bb` to `fc9d458` - 13https://github.com/miraheze/mediawiki/pull/6905 [19:59:08] [url] Page not found · GitHub · GitHub | github.com [19:59:10] [02mediawiki] 07dependabot[bot] edited pull request 03#6914: Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` - 13https://github.com/miraheze/mediawiki/pull/6914 [19:59:10] ... [19:59:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:59:14] [02mediawiki] 07dependabot[bot] edited pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [19:59:15] ... [19:59:16] [02mediawiki] 07dependabot[bot] edited pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [19:59:18] [02mediawiki] 07dependabot[bot] edited pull request 03#6966: Bump extensions/OATHAuth from `ed0ae3f` to `6055650` - 13https://github.com/miraheze/mediawiki/pull/6966 [19:59:20] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.01, 9.76, 10.09 [19:59:57] [02mediawiki] 07dependabot[bot] edited pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [19:59:59] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/AbuseFilter-3bf3b61 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d2f73c42f4cc...8dace2afc6a5 [19:59:59] [url] Comparing d2f73c42f4cc...8dace2afc6a5 · miraheze/mediawiki · GitHub | github.com [20:00:00] [02miraheze/mediawiki] 07dependabot[bot] 038dace2a - Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` [20:00:02] [02mediawiki] 07dependabot[bot] edited pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [20:00:02] [url] Page not found · GitHub · GitHub | github.com [20:00:03] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [20:00:03] [url] Page not found · GitHub · GitHub | github.com [20:00:05] [02mediawiki] 07dependabot[bot] edited pull request 03#6920: Bump extensions/VisualEditor from `f9f2532` to `b6619cd` - 13https://github.com/miraheze/mediawiki/pull/6920 [20:00:05] ... [20:00:06] [02mediawiki] 07dependabot[bot] edited pull request 03#6914: Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` - 13https://github.com/miraheze/mediawiki/pull/6914 [20:00:07] [url] Page not found · GitHub · GitHub | github.com [20:00:08] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/VisualEditor-b6619cd [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/ed6f051f06ee...5bcbd689d885 [20:00:09] [url] Comparing ed6f051f06ee...5bcbd689d885 · miraheze/mediawiki · GitHub | github.com [20:00:09] [02miraheze/mediawiki] 07dependabot[bot] 035bcbd68 - Bump extensions/VisualEditor from `f9f2532` to `b6619cd` [20:00:11] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/TitleBlacklist-fd3440d [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/368a67590ed1...6668402dc11b [20:00:11] [url] Comparing 368a67590ed1...6668402dc11b · miraheze/mediawiki · GitHub | github.com [20:00:12] [02miraheze/mediawiki] 07dependabot[bot] 036668402 - Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` [20:00:14] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6920: Bump extensions/VisualEditor from `f9f2532` to `b6619cd` - 13https://github.com/miraheze/mediawiki/pull/6920 [20:00:14] [url] Page not found · GitHub · GitHub | github.com [20:00:15] [02mediawiki] 07dependabot[bot] edited pull request 03#6920: Bump extensions/VisualEditor from `f9f2532` to `b6619cd` - 13https://github.com/miraheze/mediawiki/pull/6920 [20:00:15] [url] Page not found · GitHub · GitHub | github.com [20:00:17] [02mediawiki] 07dependabot[bot] edited pull request 03#6894: Bump extensions/Cite from `ab197bb` to `c4e959b` - 13https://github.com/miraheze/mediawiki/pull/6894 [20:00:17] ... [20:00:18] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6914: Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` - 13https://github.com/miraheze/mediawiki/pull/6914 [20:00:18] [url] Page not found · GitHub · GitHub | github.com [20:00:18] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 4.69, 2.55, 1.87 [20:00:20] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Cite-c4e959b [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/782d107a3928...6b1835877c47 [20:00:20] [url] Comparing 782d107a3928...6b1835877c47 · miraheze/mediawiki · GitHub | github.com [20:00:21] [02miraheze/mediawiki] 07dependabot[bot] 036b18358 - Bump extensions/Cite from `ab197bb` to `c4e959b` [20:00:23] [02mediawiki] 07dependabot[bot] edited pull request 03#6914: Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` - 13https://github.com/miraheze/mediawiki/pull/6914 [20:00:23] [url] Page not found · GitHub · GitHub | github.com [20:00:24] [02mediawiki] 07dependabot[bot] edited pull request 03#6885: Bump extensions/Renameuser from `538b76d` to `44318c3` - 13https://github.com/miraheze/mediawiki/pull/6885 [20:00:25] [url] Page not found · GitHub · GitHub | github.com [20:00:26] [02mediawiki] 07dependabot[bot] edited pull request 03#6894: Bump extensions/Cite from `ab197bb` to `c4e959b` - 13https://github.com/miraheze/mediawiki/pull/6894 [20:00:26] ... [20:00:27] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6894: Bump extensions/Cite from `ab197bb` to `c4e959b` - 13https://github.com/miraheze/mediawiki/pull/6894 [20:00:27] ... [20:00:29] [02mediawiki] 07dependabot[bot] edited pull request 03#6898: Bump extensions/Math from `49e4eb7` to `8bde5da` - 13https://github.com/miraheze/mediawiki/pull/6898 [20:00:29] [url] Page not found · GitHub · GitHub | github.com [20:00:30] [02mediawiki] 07dependabot[bot] edited pull request 03#6905: Bump extensions/MultimediaViewer from `0c0f6bb` to `fc9d458` - 13https://github.com/miraheze/mediawiki/pull/6905 [20:00:31] [url] Page not found · GitHub · GitHub | github.com [20:00:32] [02mediawiki] 07dependabot[bot] edited pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [20:00:32] ... [20:00:33] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ConfirmEdit-b12fbf1 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/8ee8feb0912d...77dd148cfcd7 [20:00:34] [url] Comparing 8ee8feb0912d...77dd148cfcd7 · miraheze/mediawiki · GitHub | github.com [20:00:35] [02miraheze/mediawiki] 07dependabot[bot] 0377dd148 - Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` [20:00:36] [02mediawiki] 07dependabot[bot] edited pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [20:00:36] [url] Page not found · GitHub · GitHub | github.com [20:00:38] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [20:00:38] [url] Page not found · GitHub · GitHub | github.com [20:00:39] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Renameuser-6c7a9d9 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/1f5e489e6b8a [20:00:41] [02miraheze/mediawiki] 07dependabot[bot] 031f5e489 - Bump extensions/Renameuser from `8ed6678` to `6c7a9d9` [20:00:42] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Math-f67e18f [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c683a82b355f [20:00:44] [02miraheze/mediawiki] 07dependabot[bot] 03c683a82 - Bump extensions/Math from `5e6085d` to `f67e18f` [20:00:45] [02mediawiki] 07dependabot[bot] opened pull request 03#7218: Bump extensions/Renameuser from `8ed6678` to `6c7a9d9` - 13https://github.com/miraheze/mediawiki/pull/7218 [20:00:46] [url] Page not found · GitHub · GitHub | github.com [20:00:47] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Renameuser-6c7a9d9 - 13https://github.com/miraheze/mediawiki [20:00:47] ... [20:00:48] [02mediawiki] 07dependabot[bot] edited pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [20:00:49] ... [20:00:50] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Math-f67e18f - 13https://github.com/miraheze/mediawiki [20:00:50] [url] Page not found · GitHub · GitHub | github.com [20:00:51] [02mediawiki] 07dependabot[bot] opened pull request 03#7219: Bump extensions/Math from `5e6085d` to `f67e18f` - 13https://github.com/miraheze/mediawiki/pull/7219 [20:00:52] [url] Page not found · GitHub · GitHub | github.com [20:00:53] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-7ceb91c [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/981bdbad8024 [20:00:54] [02miraheze/mediawiki] 07dependabot[bot] 03981bdba - Bump extensions/MultimediaViewer from `82bc0e3` to `7ceb91c` [20:00:56] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-7ceb91c - 13https://github.com/miraheze/mediawiki [20:00:56] ... [20:00:57] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CategoryTree-b8e1389 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/bada3cec86cd...6cb2f506f2ca [20:00:58] [url] Comparing bada3cec86cd...6cb2f506f2ca · miraheze/mediawiki · GitHub | github.com [20:00:59] [02miraheze/mediawiki] 07dependabot[bot] 036cb2f50 - Bump extensions/CategoryTree from `d8c554c` to `b8e1389` [20:01:00] [02mediawiki] 07dependabot[bot] opened pull request 03#7220: Bump extensions/MultimediaViewer from `82bc0e3` to `7ceb91c` - 13https://github.com/miraheze/mediawiki/pull/7220 [20:01:01] [url] Page not found · GitHub · GitHub | github.com [20:01:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.64, 11.70, 11.09 [20:01:02] [02mediawiki] 07dependabot[bot] edited pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [20:01:02] [url] Page not found · GitHub · GitHub | github.com [20:01:03] [02mediawiki] 07dependabot[bot] synchronize pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [20:01:04] [url] Page not found · GitHub · GitHub | github.com [20:01:05] [02mediawiki] 07dependabot[bot] closed pull request 03#7150: Bump extensions/MultimediaViewer from `82bc0e3` to `16584a7` - 13https://github.com/miraheze/mediawiki/pull/7150 [20:01:05] ... [20:01:06] [02mediawiki] 07dependabot[bot] commented on pull request 03#7150: Bump extensions/MultimediaViewer from `82bc0e3` to `16584a7` - 13https://github.com/miraheze/mediawiki/pull/7150#issuecomment-1137793703 [20:01:06] [url] Bump extensions/MultimediaViewer from `82bc0e3` to `16584a7` by dependabot[bot] · Pull Request #7150 · miraheze/mediawiki · GitHub | github.com [20:01:08] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-16584a7 - 13https://github.com/miraheze/mediawiki [20:01:08] [url] Page not found · GitHub · GitHub | github.com [20:01:09] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-16584a7 [20:01:11] [02mediawiki] 07github-actions[bot] labeled pull request 03#7218: Bump extensions/Renameuser from `8ed6678` to `6c7a9d9` - 13https://github.com/miraheze/mediawiki/pull/7218 [20:01:11] [url] Page not found · GitHub · GitHub | github.com [20:01:12] [02mediawiki] 07github-actions[bot] labeled pull request 03#7218: Bump extensions/Renameuser from `8ed6678` to `6c7a9d9` - 13https://github.com/miraheze/mediawiki/pull/7218 [20:01:13] ... [20:01:14] [02mediawiki] 07github-actions[bot] labeled pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [20:01:14] ... [20:01:15] [02mediawiki] 07github-actions[bot] labeled pull request 03#7219: Bump extensions/Math from `5e6085d` to `f67e18f` - 13https://github.com/miraheze/mediawiki/pull/7219 [20:01:16] [url] Page not found · GitHub · GitHub | github.com [20:01:17] [02mediawiki] 07github-actions[bot] labeled pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [20:01:17] [url] Page not found · GitHub · GitHub | github.com [20:01:18] [02mediawiki] 07github-actions[bot] labeled pull request 03#7220: Bump extensions/MultimediaViewer from `82bc0e3` to `7ceb91c` - 13https://github.com/miraheze/mediawiki/pull/7220 [20:01:18] [url] Page not found · GitHub · GitHub | github.com [20:01:20] [02mediawiki] 07github-actions[bot] labeled pull request 03#7220: Bump extensions/MultimediaViewer from `82bc0e3` to `7ceb91c` - 13https://github.com/miraheze/mediawiki/pull/7220 [20:01:20] ... [20:01:21] [02mediawiki] 07github-actions[bot] labeled pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [20:02:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.56, 3.93 [20:03:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.96, 11.19, 10.97 [20:04:01] [02puppet] 07Universal-Omega synchronize pull request 03#2588: Rewrite grafana-webhook - 13https://github.com/miraheze/puppet/pull/2588 [20:04:01] [url] Page not found · GitHub · GitHub | github.com [20:04:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.74, 3.83, 3.97 [20:04:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.74, 3.95, 3.96 [20:04:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.25, 4.41, 4.13 [20:05:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.92, 11.68, 11.17 [20:05:52] !log [macfan@test101] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [20:05:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:06:02] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.97, 1.97, 1.80 [20:06:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.16, 3.97, 3.97 [20:07:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.84, 11.45, 11.15 [20:07:15] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.73, 9.27, 10.09 [20:07:31] !log [macfan@mwtask111] finished deploy of {'world': True} to all - SUCCESS in 657s [20:07:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:07:56] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 3.34, 2.49, 2.02 [20:09:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.23, 11.68, 11.26 [20:09:03] !log [macfan@test101] starting deploy of {'world': True} to all [20:09:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:37] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:11:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.53, 11.79, 11.37 [20:13:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.36, 11.97, 11.48 [20:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.77, 3.94 [20:14:25] !log [macfan@test101] finished deploy of {'world': True} to all - SUCCESS in 321s [20:14:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.47, 3.91, 3.99 [20:15:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.15, 11.29, 11.30 [20:15:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.43, 1.96, 2.00 [20:16:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 3.97, 3.99 [20:16:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.11, 3.96, 4.00 [20:17:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.04, 2.13, 2.06 [20:18:29] [02puppet] 07Universal-Omega synchronize pull request 03#2588: Rewrite grafana-webhook - 13https://github.com/miraheze/puppet/pull/2588 [20:18:29] [url] Page not found · GitHub · GitHub | github.com [20:18:39] [02puppet] 07Universal-Omega reviewed pull request 03#2588 commit - 13https://github.com/miraheze/puppet/pull/2588#discussion_r882081281 [20:18:40] [url] Rewrite grafana-webhook by Universal-Omega · Pull Request #2588 · miraheze/puppet · GitHub | github.com [20:22:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.73, 3.66, 3.89 [20:24:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.94, 3.97 [20:24:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.09, 3.74, 3.97 [20:25:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.29, 12.02, 11.46 [20:26:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.06, 3.83, 3.97 [20:27:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.67, 11.60, 11.36 [20:27:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.14, 1.80, 1.98 [20:28:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.88, 3.50, 3.83 [20:29:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.32, 1.84, 1.96 [20:31:32] [02mediawiki] 07dependabot[bot] edited pull request 03#6966: Bump extensions/OATHAuth from `ed0ae3f` to `6055650` - 13https://github.com/miraheze/mediawiki/pull/6966 [20:31:32] [url] Page not found · GitHub · GitHub | github.com [20:31:39] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/OATHAuth-bfd2bce [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ebd09fbef560 [20:31:40] [02miraheze/mediawiki] 07dependabot[bot] 03ebd09fb - Bump extensions/OATHAuth from `75fbeef` to `bfd2bce` [20:31:42] [02mediawiki] 07dependabot[bot] opened pull request 03#7221: Bump extensions/OATHAuth from `75fbeef` to `bfd2bce` - 13https://github.com/miraheze/mediawiki/pull/7221 [20:31:42] ... [20:31:43] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/OATHAuth-bfd2bce - 13https://github.com/miraheze/mediawiki [20:31:44] ... [20:31:45] [02mediawiki] 07dependabot[bot] commented on pull request 03#7147: Bump extensions/OATHAuth from `75fbeef` to `5f0e4e2` - 13https://github.com/miraheze/mediawiki/pull/7147#issuecomment-1137819562 [20:31:45] [url] Bump extensions/OATHAuth from `75fbeef` to `5f0e4e2` by dependabot[bot] · Pull Request #7147 · miraheze/mediawiki · GitHub | github.com [20:31:46] [02mediawiki] 07dependabot[bot] closed pull request 03#7147: Bump extensions/OATHAuth from `75fbeef` to `5f0e4e2` - 13https://github.com/miraheze/mediawiki/pull/7147 [20:31:47] [url] Page not found · GitHub · GitHub | github.com [20:31:48] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/OATHAuth-5f0e4e2 - 13https://github.com/miraheze/mediawiki [20:31:48] [url] Page not found · GitHub · GitHub | github.com [20:31:49] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/OATHAuth-5f0e4e2 [20:31:55] [02mediawiki] 07dependabot[bot] edited pull request 03#6894: Bump extensions/Cite from `ab197bb` to `c4e959b` - 13https://github.com/miraheze/mediawiki/pull/6894 [20:31:55] [url] Page not found · GitHub · GitHub | github.com [20:32:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [20:32:04] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [20:32:06] [02mediawiki] 07github-actions[bot] labeled pull request 03#7221: Bump extensions/OATHAuth from `75fbeef` to `bfd2bce` - 13https://github.com/miraheze/mediawiki/pull/7221 [20:32:06] [url] Page not found · GitHub · GitHub | github.com [20:32:07] [02mediawiki] 07github-actions[bot] labeled pull request 03#7221: Bump extensions/OATHAuth from `75fbeef` to `bfd2bce` - 13https://github.com/miraheze/mediawiki/pull/7221 [20:32:08] ... [20:34:59] [02mediawiki] 07dependabot[bot] edited pull request 03#6883: Bump extensions/AbuseFilter from `2c3f411` to `3bf3b61` - 13https://github.com/miraheze/mediawiki/pull/6883 [20:34:59] [url] Page not found · GitHub · GitHub | github.com [20:35:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.98, 12.40, 11.66 [20:35:04] [02mediawiki] 07dependabot[bot] edited pull request 03#6926: Bump extensions/CategoryTree from `d8c554c` to `b8e1389` - 13https://github.com/miraheze/mediawiki/pull/6926 [20:35:04] ... [20:35:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.81, 1.85, 1.94 [20:37:39] [02mediawiki] 07dependabot[bot] edited pull request 03#6914: Bump extensions/TitleBlacklist from `e94f6e3` to `fd3440d` - 13https://github.com/miraheze/mediawiki/pull/6914 [20:37:39] [url] Page not found · GitHub · GitHub | github.com [20:38:14] [02mediawiki] 07dependabot[bot] edited pull request 03#6918: Bump extensions/ConfirmEdit from `ea3ac57` to `b12fbf1` - 13https://github.com/miraheze/mediawiki/pull/6918 [20:38:14] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.33, 10.14, 9.53 [20:38:14] ... [20:39:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.26, 11.72, 11.62 [20:40:00] [02mediawiki] 07dependabot[bot] edited pull request 03#6920: Bump extensions/VisualEditor from `f9f2532` to `b6619cd` - 13https://github.com/miraheze/mediawiki/pull/6920 [20:40:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.84, 2.96, 3.34 [20:42:37] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [20:43:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.47, 1.47, 1.70 [20:47:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.25, 1.49, 1.92 [20:47:54] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.60, 9.73, 9.76 [20:49:39] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.58, 1.96, 2.08 [20:50:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.11, 3.80, 3.55 [20:52:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.06, 3.64, 3.53 [20:53:16] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.04, 1.91, 1.57 [20:53:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [20:53:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.405844688415527 seconds [20:55:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.59, 1.50 [20:56:06] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [20:56:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.84, 3.41, 3.91 [20:57:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.63, 11.18, 10.95 [20:57:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.33, 1.79, 1.96 [20:57:55] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.58, 2.03, 1.72 [20:58:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 1.96, 2.93, 3.28 [20:59:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.67, 10.77, 10.85 [20:59:55] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.53, 1.86, 1.69 [21:00:38] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [21:00:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.58, 3.17, 3.83 [21:01:55] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.88, 1.51, 1.59 [21:06:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.69, 3.33, 3.93 [21:07:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.06, 8.84, 9.95 [21:07:40] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.01, 1.26, 1.62 [21:08:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.16, 3.60, 3.60 [21:08:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.78, 4.04, 4.10 [21:08:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.72, 2.76, 3.37 [21:10:37] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:12:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.59, 3.15, 3.42 [21:12:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.25, 3.39, 3.85 [21:13:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.06, 9.92, 9.97 [21:14:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.71, 3.56, 3.83 [21:15:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.52, 9.20, 9.70 [21:16:13] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.79, 3.03, 3.30 [21:16:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.33, 3.48, 3.78 [21:20:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.59, 4.14, 3.94 [21:22:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.40, 3.93, 3.63 [21:22:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 3.74, 3.43 [21:22:51] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.05, 3.48, 3.22 [21:23:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.71, 10.41, 9.87 [21:24:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.72, 3.52, 3.52 [21:24:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.75, 3.82 [21:24:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.64, 3.43 [21:24:45] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.98, 3.72, 3.35 [21:26:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.29, 3.02, 3.33 [21:26:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.12, 3.66, 3.76 [21:27:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.14, 9.79, 9.75 [21:28:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.98, 3.29, 3.32 [21:30:04] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [21:30:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.69, 3.12, 3.51 [21:30:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.58, 3.81, 3.48 [21:32:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.96, 2.91, 3.38 [21:32:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.91, 3.91, 3.57 [21:34:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 1.85, 3.15, 3.33 [21:36:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.27, 1.36, 1.95 [21:40:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.80, 3.55 [21:42:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.30, 3.52, 3.36 [21:44:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.67, 1.83 [21:44:26] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.77, 3.43, 3.30 [21:44:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.67, 3.55 [21:44:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.44, 3.41, 3.34 [21:46:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.00, 3.71, 3.40 [21:46:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.64, 4.66, 3.89 [21:47:05] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.94, 1.56, 1.37 [21:48:54] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.58, 4.24, 3.60 [21:49:02] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.99, 1.38, 1.33 [21:50:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 5.53, 4.10, 3.61 [21:51:55] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.82, 10.37, 9.22 [21:52:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 2.70, 3.41, 3.41 [21:53:35] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - stardewmodding.wiki All nameservers failed to answer the query. [21:54:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.21, 3.62, 3.48 [21:54:51] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.28, 1.80, 1.50 [21:55:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.85, 10.35, 9.78 [21:55:03] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.66, 1.94, 1.50 [21:55:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.91, 11.53, 9.97 [21:56:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.86, 1.93 [21:56:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.75, 3.56, 3.47 [21:56:58] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.17, 1.71, 1.47 [21:58:52] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.10, 1.47, 1.41 [21:59:34] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.30, 10.19, 9.79 [21:59:47] [02mw-config] 07ugochimobi opened pull request 03#4711: add Nigerian Pidgin to wgExtraLanguageNames - 13https://github.com/miraheze/mw-config/pull/4711 [21:59:48] [url] Page not found · GitHub · GitHub | github.com [22:01:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.55, 9.83, 9.85 [22:01:19] miraheze/mw-config - ugochimobi the build passed. [22:02:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.96, 3.23, 3.36 [22:02:40] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.26, 1.71, 1.62 [22:04:37] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.42, 1.62, 1.60 [22:06:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.11, 1.82, 1.84 [22:08:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.84, 3.64, 3.49 [22:12:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.71, 3.14, 3.32 [22:14:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.11, 3.42, 3.87 [22:15:14] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.11, 3.34, 3.96 [22:17:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.27, 10.52, 10.06 [22:18:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.63, 3.55, 3.79 [22:19:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.75, 10.69, 10.17 [22:20:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.88, 3.71, 3.82 [22:20:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.21, 3.66, 3.41 [22:22:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.30, 3.50, 3.39 [22:23:36] PROBLEM - stardewmodding.wiki - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 123, in main rdns_hostname = get_reverse_dnshostname(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 108, in get_reverse_dnshostname resolved_ip_addr = str(dns_resolver.resolv [22:23:36] me, 'AAAA')[0]) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.406674861907959 seconds [22:24:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.09, 3.29, 3.32 [22:28:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.47, 4.14, 3.87 [22:28:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.66, 3.91, 3.74 [22:32:05] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.03, 10.69, 9.48 [22:32:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.41, 3.81, 3.86 [22:32:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.83, 3.76 [22:34:02] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.17, 9.65, 9.24 [22:34:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.55, 3.91, 3.79 [22:35:52] PROBLEM - cp21 Disk Space on cp21 is CRITICAL: DISK CRITICAL - free space: / 2340 MB (5% inode=96%); [22:36:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 3.95, 3.89 [22:37:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.33, 9.65, 10.07 [22:38:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.40, 3.78, 3.77 [22:40:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.12, 3.93, 3.81 [22:42:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.18, 3.64, 3.72 [22:45:36] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [22:46:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.31, 3.85, 3.96 [22:48:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.97, 4.43, 4.17 [22:48:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.04, 3.99, 3.86 [22:50:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.85, 3.82 [22:59:04] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.11, 1.81, 1.48 [23:00:18] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 10.75, 4.64, 2.54 [23:00:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.77, 1.97 [23:00:36] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/GtxbP1Xnk/mediawiki [23:01:04] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.57, 1.75, 1.50 [23:02:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.72, 3.55, 3.88 [23:02:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.10, 3.05, 3.40 [23:03:04] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.21, 1.67, 1.51 [23:07:05] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.84, 1.62 [23:08:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.43, 1.41, 1.70 [23:09:05] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.13, 1.58, 1.56 [23:10:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.51, 3.65 [23:10:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.71, 3.67, 3.96 [23:12:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.35, 3.57 [23:14:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.80, 1.38, 1.89 [23:14:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 3.36, 3.72 [23:14:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.62, 3.17, 2.90 [23:15:39] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.45, 1.83, 1.36 [23:16:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.41, 3.99, 3.74 [23:17:32] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.22, 3.75, 3.49 [23:17:39] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.12, 1.69, 1.37 [23:18:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.31, 3.39, 3.56 [23:18:18] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.71, 1.13, 1.67 [23:18:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.49, 3.43, 3.71 [23:18:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.91, 3.19, 2.98 [23:19:27] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.75, 3.31, 3.35 [23:20:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.96, 1.88, 1.73 [23:20:55] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.35, 9.45, 8.51 [23:22:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.90, 3.12, 3.40 [23:22:24] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.31, 2.28, 1.91 [23:22:37] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.09, 3.69, 3.28 [23:22:55] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.58, 8.90, 8.44 [23:23:20] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.52, 3.45 [23:24:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.94, 2.23, 1.91 [23:24:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.64, 2.00, 1.85 [23:25:15] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.90, 3.31, 3.38 [23:26:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.30, 1.81, 1.80 [23:26:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.59, 3.32, 3.23 [23:28:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.92, 1.46, 1.67 [23:28:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.76, 4.27, 3.94 [23:29:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.70, 4.03, 3.68 [23:29:08] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.86, 4.14, 3.69 [23:29:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.84, 10.30, 9.64 [23:30:54] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.85, 3.65 [23:31:03] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.40, 3.77, 3.60 [23:32:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.29, 4.23, 3.81 [23:34:24] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.58, 1.56, 1.70 [23:34:26] PROBLEM - cp20 Disk Space on cp20 is CRITICAL: DISK CRITICAL - free space: / 2334 MB (5% inode=96%); [23:34:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.38, 3.43, 3.70 [23:34:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.89, 3.63, 3.63 [23:34:54] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.30, 3.13, 3.38 [23:35:34] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.74, 11.39, 10.30 [23:36:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.67, 3.75 [23:36:37] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.37, 3.37, 3.18 [23:38:37] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 2.48, 3.05, 3.09 [23:40:25] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.36, 3.39, 3.47 [23:40:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.86, 3.63, 3.70 [23:42:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.99, 4.47, 4.00 [23:43:13] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.91, 10.50, 10.52 [23:44:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.52, 3.52 [23:45:07] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.70, 9.50, 10.15 [23:48:21] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.77, 2.05, 1.80 [23:50:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.96, 3.18, 3.38 [23:50:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.83, 2.03, 1.58 [23:51:26] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.35, 2.91, 1.96 [23:52:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.48, 1.77, 1.54 [23:54:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 2.00, 1.65 [23:56:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.15, 1.67, 1.57 [23:56:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.76, 3.50, 3.78 [23:57:21] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.76, 1.57, 1.67