[00:01:34] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.93, 9.71, 9.39 [00:03:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.00, 8.19, 8.21 [00:03:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.00, 4.50, 3.57 [00:03:32] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.90, 8.94, 9.14 [00:03:43] PROBLEM - mem121 Puppet on mem121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:04:15] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Puppet has 806 failures. Last run 3 minutes ago with 806 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[authority certificates],File[/etc/apt/apt.conf.d/50unattended-upgrades] [00:05:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.87, 4.34, 3.38 [00:06:32] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.28, 7.63, 8.38 [00:07:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.55, 9.13, 8.63 [00:07:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.38, 3.06 [00:07:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.14, 9.07, 8.97 [00:08:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.15, 9.55, 8.85 [00:09:11] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.26, 3.38, 3.10 [00:09:38] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.90, 3.74, 3.36 [00:11:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.88, 3.91, 3.77 [00:11:25] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.07, 7.90, 8.50 [00:11:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.03, 9.27, 9.09 [00:13:17] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.00, 4.31, 3.92 [00:13:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.63, 9.68, 9.26 [00:15:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.84, 3.80 [00:15:18] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.11, 9.21, 8.82 [00:15:35] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.98, 2.89, 3.13 [00:17:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.82, 9.22, 8.76 [00:17:27] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.18, 1.83, 1.37 [00:19:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.91, 8.75, 8.65 [00:19:25] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.31, 1.65, 1.36 [00:21:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.92, 7.64, 8.31 [00:23:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.07, 3.52, 3.60 [00:23:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.77, 8.34, 8.50 [00:24:55] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.83, 9.17, 8.86 [00:25:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.17, 3.44 [00:25:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.13, 9.67, 9.75 [00:27:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.03, 3.62, 3.56 [00:27:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.06, 10.62, 10.09 [00:31:43] RECOVERY - mem121 Puppet on mem121 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [00:32:21] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [00:36:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.23, 8.70, 8.30 [00:37:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.32, 3.70, 3.83 [00:38:15] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.83, 8.50, 8.27 [00:38:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.77, 9.20, 9.84 [00:39:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.57, 3.98, 3.90 [00:40:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.33, 9.63, 9.91 [00:41:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.30, 3.67, 3.79 [00:42:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.00, 9.56, 9.84 [00:44:10] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.60, 9.64, 8.83 [00:45:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.28, 8.88, 9.77 [00:46:08] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.94, 8.79, 8.60 [00:46:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.26, 9.88, 9.86 [00:47:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.24, 2.90, 3.40 [00:47:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.92, 9.38, 9.82 [00:48:07] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.73, 8.42, 8.49 [00:48:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.64, 9.29, 8.84 [00:48:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.08, 9.37, 9.69 [00:49:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.97, 8.88, 9.59 [00:50:32] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.35, 8.62, 8.66 [00:50:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.59, 9.57, 9.70 [00:54:03] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [00:54:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.33, 8.06, 8.44 [00:54:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.62, 3.32, 3.07 [00:56:40] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.26, 2.93, 2.96 [00:56:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 9.93, 9.92 [01:01:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.58, 8.82, 9.07 [01:02:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.13, 8.61, 8.40 [01:02:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.75, 10.02, 9.92 [01:03:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.74, 8.46, 8.91 [01:06:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.77, 8.99, 8.65 [01:08:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.56, 9.88, 9.95 [01:09:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:10:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.09, 7.99, 8.33 [01:11:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.51, 3.22 [01:11:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.04, 3.55, 3.17 [01:11:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.61, 8.85, 8.77 [01:13:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.23, 3.00, 3.06 [01:13:04] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.50, 3.20 [01:13:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.71, 8.92, 8.79 [01:14:59] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.05, 4.20, 3.47 [01:17:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 13.42, 10.32, 9.32 [01:18:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.96, 3.68, 3.42 [01:20:45] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.32, 3.13, 3.25 [01:20:50] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.39, 6.63, 8.23 [01:22:03] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:22:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:25:29] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.58, 8.32, 7.97 [01:25:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.19, 9.90, 9.80 [01:27:23] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.65, 8.78, 8.15 [01:27:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.02, 10.56, 10.04 [01:29:18] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.28, 8.48, 8.11 [01:29:26] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.66, 7.99, 7.92 [01:31:31] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.06, 9.32, 9.67 [01:33:33] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.80, 9.32, 8.77 [01:37:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.80, 8.54, 8.26 [01:37:25] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.83, 9.19, 8.90 [01:38:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.04, 7.79, 8.02 [01:41:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.56, 3.25 [01:41:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.37, 3.09 [01:42:12] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.02, 8.75, 8.25 [01:43:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.98, 3.15, 3.04 [01:46:48] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.52, 2.74, 3.03 [01:47:05] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.81, 9.48, 8.96 [01:47:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:48:07] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.23, 8.30, 8.28 [01:49:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.99, 8.99, 8.85 [01:49:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.42, 10.36, 9.71 [01:52:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:53:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.40, 9.28, 9.46 [01:57:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:57:56] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.81, 8.72, 8.41 [01:59:54] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.25, 9.80, 8.83 [02:00:50] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.62, 8.01, 8.49 [02:01:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:01:52] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.36, 9.51, 8.83 [02:04:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.91, 8.74, 8.22 [02:06:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:07:47] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.32, 7.83, 8.30 [02:08:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.64, 7.96, 8.05 [02:09:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:10:42] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.90, 8.54, 8.50 [02:11:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.08, 9.13, 9.04 [02:13:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.96, 9.36, 9.16 [02:14:34] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.27, 7.85, 8.23 [02:15:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.31, 9.90, 9.36 [02:17:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.04, 9.14, 9.15 [02:19:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:22:21] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.69, 8.22, 8.19 [02:24:17] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.15, 8.23, 8.20 [02:25:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.22, 9.09, 8.98 [02:29:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 6.00, 8.46, 8.84 [02:29:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:32:17] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.87, 3.77, 3.44 [02:33:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.30, 7.48, 8.33 [02:36:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.74, 3.47 [02:40:07] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.88, 3.35, 3.38 [02:49:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.97, 3.62 [02:49:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:50:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.85, 3.56, 3.22 [02:52:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.75, 3.28, 3.15 [02:57:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.25, 4.16, 3.73 [02:57:09] PROBLEM - cp20 APT on cp20 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [02:58:42] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 5.56, 2.91, 1.39 [02:59:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.09, 3.50, 3.55 [02:59:06] RECOVERY - cp20 APT on cp20 is OK: APT OK: 37 packages available for upgrade (0 critical updates). [03:02:42] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.22, 1.53, 1.18 [03:04:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:05:00] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.71, 2.09, 1.32 [03:05:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.36, 3.44 [03:07:01] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.71, 1.64, 1.25 [03:09:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.92, 3.50, 3.47 [03:14:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:17:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.85, 4.06, 3.66 [03:18:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.62, 3.42, 3.26 [03:19:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.69, 3.60, 3.52 [03:20:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.08, 3.65, 3.34 [03:21:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.71, 3.24, 3.39 [03:22:36] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.80, 3.54, 3.33 [03:24:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.78, 3.29, 3.26 [03:25:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.13, 8.93, 8.10 [03:27:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.28, 9.13, 7.91 [03:27:31] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.37, 8.99, 8.22 [03:29:22] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.59, 8.61, 7.85 [03:29:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.32, 8.50, 8.12 [03:31:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.16, 8.15, 7.78 [03:31:53] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.27, 4.02, 3.73 [03:33:51] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.67, 3.53, 3.58 [03:35:01] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [03:35:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 3.94, 3.72 [03:37:45] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.02, 3.16, 3.46 [03:41:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:41:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.02, 2.88, 3.30 [03:46:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:49:57] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.95, 8.59, 7.99 [03:50:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:51:51] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.18, 7.74, 7.75 [03:55:32] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:57:25] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.10, 8.86, 7.86 [03:59:21] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.97, 8.47, 7.81 [04:01:17] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.05, 7.84, 7.65 [04:03:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 3.59, 3.29 [04:04:47] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [04:05:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.55, 8.49, 7.97 [04:07:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.25, 3.57, 3.35 [04:07:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.16, 7.61, 7.71 [04:09:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.69, 3.22, 3.25 [04:19:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.33, 8.75, 7.79 [04:19:41] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.01, 2.52 [04:21:30] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.65, 7.92, 7.61 [04:21:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:21:40] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.91, 2.69, 2.47 [04:26:42] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 4.89, 2.19, 1.08 [04:26:47] PROBLEM - cp20 APT on cp20 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [04:28:43] RECOVERY - cp20 APT on cp20 is OK: APT OK: 37 packages available for upgrade (0 critical updates). [04:31:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 3.44, 2.95 [04:31:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:31:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.04, 3.43, 2.83 [04:32:42] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.30, 1.80, 1.39 [04:33:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.98, 3.72, 3.12 [04:33:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 4.00, 3.67, 3.00 [04:34:42] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.25, 1.25, 1.23 [04:35:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.85, 3.32, 3.04 [04:35:33] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.27, 3.28, 2.92 [04:39:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.40, 3.63, 3.12 [04:41:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.40, 3.08 [04:43:28] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.30, 3.20, 3.04 [04:44:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.98, 3.59, 3.26 [04:46:24] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.82, 4.16, 3.52 [04:46:50] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.42, 3.22 [04:48:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.70, 3.83, 3.38 [04:50:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:50:45] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.66, 3.38 [04:51:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.00, 3.62, 3.33 [04:53:23] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.05, 3.17, 3.21 [04:54:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.93, 3.71 [04:54:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.09, 3.16, 3.22 [04:55:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:56:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.35, 4.36, 3.88 [05:08:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.95, 3.64, 3.90 [05:08:17] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.60, 4.03, 3.56 [05:10:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.85, 3.71, 3.48 [05:11:22] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.57, 8.52, 7.33 [05:11:31] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.98, 8.07, 7.24 [05:12:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.55, 3.22, 3.33 [05:13:22] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.21, 7.95, 7.26 [05:13:30] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.68, 7.66, 7.18 [05:18:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.60, 3.44 [05:20:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.69, 3.23, 3.33 [05:20:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:25:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:28:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.91, 2.81, 3.39 [05:28:19] PROBLEM - mabi.myomi.net - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/lib/py [05:28:19] st-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.409630537033081 seconds [05:37:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:37:59] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [05:37:59] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:38:32] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.50, 4.28, 3.40 [05:39:00] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.07, 3.20, 2.92 [05:40:30] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.50, 3.89, 3.37 [05:40:59] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.11, 3.02, 2.88 [05:42:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:44:25] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.95, 3.06, 3.16 [05:57:35] RECOVERY - mabi.myomi.net - reverse DNS on sslhost is OK: SSL OK - mabi.myomi.net reverse DNS resolves to cp21.miraheze.org - CNAME OK [06:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:13:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:18:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:36:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.70, 9.27, 7.83 [06:37:31] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [06:38:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.18, 9.84, 8.21 [06:40:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.52, 9.70, 8.36 [06:44:31] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.46, 9.68, 8.63 [06:58:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.42, 9.94, 9.66 [07:07:18] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [07:08:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.93, 6.77, 8.15 [07:34:22] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.32, 3.76, 3.11 [07:35:53] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.49, 4.55, 3.28 [07:36:07] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.25, 2.63 [07:38:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.36, 3.16, 2.67 [07:38:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.81, 3.44, 3.15 [07:40:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.97, 3.26, 3.11 [07:47:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.35, 3.85, 3.77 [07:51:34] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.22, 3.39, 3.55 [07:53:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.45, 3.35, 3.51 [07:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:55:28] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.88, 3.10, 3.39 [07:59:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.68, 3.61 [07:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:07:10] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.13, 3.86, 3.59 [08:10:42] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [08:10:43] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:13:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.82, 3.70 [08:19:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.32, 3.09, 3.39 [08:23:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.69, 3.57 [08:25:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.80, 3.53, 3.54 [08:27:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.79, 2.96, 3.33 [08:38:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.52, 3.50, 2.87 [08:38:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.15, 2.57 [08:39:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.02, 4.11, 3.63 [08:40:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.19, 3.45, 2.94 [08:40:29] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [08:40:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.08, 2.79, 2.51 [08:41:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.58, 3.93, 3.63 [08:42:05] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 7.71, 6.95 [08:42:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.25, 3.01, 2.84 [08:44:03] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.96, 7.35, 6.91 [08:45:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.04, 3.17, 3.37 [08:51:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:56:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:57:42] [02mw-config] 07ugochimobi opened pull request 03#4537: T8874: Add the otherProjectsLink on gratispaideiawiki - 13https://github.com/miraheze/mw-config/pull/4537 [08:58:51] miraheze/mw-config - ugochimobi the build passed. [09:39:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.00, 2.42 [09:41:04] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.36, 2.98, 2.48 [09:57:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 2.97, 2.46 [09:59:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.08, 2.80, 2.47 [10:12:34] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:16:42] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.74, 2.25, 1.39 [10:17:34] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:18:42] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.03, 1.86, 1.35 [10:20:42] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.75, 1.46, 1.26 [10:22:03] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [10:36:16] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.16, 1.83, 1.11 [10:38:14] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.03, 1.51, 1.07 [10:47:24] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.33, 3.41, 2.90 [10:49:20] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.01, 3.72, 3.08 [10:51:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.45, 3.23, 2.97 [10:52:02] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.77, 3.58, 1.76 [10:52:03] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:55:03] PROBLEM - wiki.eversd.com - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/lib/p [10:55:03] ist-packages/dns/resolver.py", line 1043, in resolve timeout = self._compute_timeout(start, lifetime) File "/usr/lib/python3/dist-packages/dns/resolver.py", line 950, in _compute_timeout raise Timeout(timeout=duration)dns.exception.Timeout: The DNS operation timed out after 5.403478622436523 seconds [10:56:02] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.57, 1.97, 1.51 [10:58:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.73, 1.61, 1.43 [11:04:46] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.66, 3.24 [11:06:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.30, 3.64, 3.29 [11:08:37] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.12, 3.06, 3.12 [11:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:17:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:23:51] RECOVERY - wiki.eversd.com - reverse DNS on sslhost is OK: SSL OK - wiki.eversd.com reverse DNS resolves to cp20.miraheze.org - CNAME OK [11:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:29:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:53:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:58:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:04:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.26, 8.12, 6.96 [12:06:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.80, 7.72, 6.96 [12:07:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:12:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:17:54] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.25, 2.80 [12:19:21] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.04, 1.70, 1.24 [12:21:31] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.31, 8.67, 7.46 [12:21:49] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.95, 3.17, 2.87 [12:23:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.75, 7.97, 7.35 [12:27:00] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.4422645271 secs [12:29:10] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.32, 1.83, 1.63 [12:30:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 4.00, 3.96, 3.34 [12:31:08] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.32, 1.70, 1.61 [12:34:29] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.14, 3.60, 3.31 [12:35:00] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 5.13, 5.87, 2.87 [12:36:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.11, 3.42, 3.27 [12:37:03] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.84, 8.64, 7.79 [12:39:00] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.95, 8.25, 7.74 [12:40:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.62, 3.14, 3.19 [12:40:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:42:56] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.37, 8.72, 7.58 [12:43:00] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.62, 1.64, 1.95 [12:44:54] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.39, 8.18, 7.52 [12:45:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:47:01] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.48, 1.10, 1.65 [12:49:06] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.56, 4.22, 3.61 [12:49:31] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.35, 9.54, 8.12 [12:50:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:51:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.41, 3.48, 3.41 [12:51:31] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.79, 9.52, 8.28 [12:52:41] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.28, 1.83, 1.62 [12:53:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.71, 4.08, 3.64 [12:53:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.93, 3.50, 3.12 [12:54:39] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.69, 1.76, 1.62 [12:55:04] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.38, 3.95, 3.66 [12:55:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.38, 3.36, 3.11 [12:57:00] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.001300662756 secs [12:57:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.44, 8.23, 8.14 [12:58:35] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.66, 1.69, 1.63 [12:59:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.94, 4.13, 3.76 [13:00:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:03:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:05:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.69, 9.35, 8.61 [13:06:35] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.70, 8.50, 7.59 [13:08:31] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.28, 8.01, 7.52 [13:09:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.29, 8.23, 8.31 [13:14:18] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [13:14:19] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:15:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.87, 3.91, 3.46 [13:19:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.62, 4.18, 3.67 [13:21:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.77, 3.58 [13:23:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:26:15] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.60, 8.36, 7.50 [13:27:05] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.74, 8.47, 7.68 [13:27:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.58, 3.12, 3.33 [13:27:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:28:14] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.30, 8.16, 7.52 [13:32:55] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.18, 3.87, 3.56 [13:34:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.61, 3.50, 3.26 [13:34:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.76, 9.46, 8.41 [13:36:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.89, 3.44 [13:36:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.69, 9.41, 8.54 [13:38:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.71, 3.50, 3.35 [13:40:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.25, 3.22, 3.28 [13:44:05] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [13:44:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.27, 8.05, 7.64 [13:44:47] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.75, 4.18, 3.59 [13:44:50] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.40, 8.43, 8.45 [13:46:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.88, 3.57, 3.44 [13:48:32] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.72, 8.10, 7.79 [13:48:42] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.42, 9.84, 8.57 [13:48:47] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.14, 3.03, 3.25 [13:48:50] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.86, 8.79, 8.60 [13:50:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.74, 3.40, 3.94 [13:52:31] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.98, 8.62, 8.08 [13:52:47] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.28, 9.11, 8.38 [13:52:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.97, 9.60, 8.94 [13:54:25]