[00:04:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.90, 1.64, 1.50 [00:04:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.62, 3.90, 3.60 [00:06:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.74, 1.27, 1.38 [00:06:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.85, 1.98 [00:07:39] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.74, 1.67, 1.31 [00:08:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 4.33, 3.86 [00:09:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.10, 1.46, 1.27 [00:09:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.62, 3.42, 3.82 [00:10:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.95, 3.77 [00:12:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.67, 4.26, 3.88 [00:12:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.15, 1.94, 1.97 [00:13:45] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.53, 4.08, 3.96 [00:14:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.94, 3.87, 3.77 [00:18:13] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.17, 3.21, 1.95 [00:22:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.77, 3.70 [00:24:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.75, 3.38, 3.57 [00:27:18] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.43, 10.26, 9.42 [00:28:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.08, 3.92, 3.73 [00:29:17] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.06, 9.94, 9.41 [00:30:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.85, 3.92, 3.76 [00:30:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.78, 1.95 [00:33:15] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.76, 10.32, 9.72 [00:38:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.81, 4.00, 3.74 [00:42:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.97, 3.83 [00:44:13] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 2.41, 3.50, 3.82 [00:45:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.80, 9.49, 9.80 [00:46:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.85, 1.84 [00:48:13] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.60, 2.44, 3.30 [00:48:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.90, 3.49, 3.57 [00:50:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.07, 3.39, 3.54 [00:54:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.79, 3.66 [00:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.99, 1.95 [00:56:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.40, 3.53 [00:56:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.42, 2.07, 1.99 [00:58:04] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.46, 10.19, 9.97 [01:00:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.99, 1.98 [01:02:03] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.81, 10.06, 10.00 [01:03:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.99, 3.60, 3.91 [01:04:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.46, 3.10, 3.35 [01:04:27] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.39, 7.16, 6.55 [01:04:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.02, 3.36, 3.85 [01:08:00] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.46, 10.21, 10.02 [01:08:25] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.24, 7.32, 6.71 [01:09:59] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.43, 9.52, 9.79 [01:10:24] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.17, 7.28, 6.78 [01:11:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.76, 2.42, 3.24 [01:12:24] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.56, 6.64, 6.60 [01:14:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.20, 4.31, 3.87 [01:15:25] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 3.78, 3.61 [01:16:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.65, 3.36 [01:16:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.66, 3.47, 3.61 [01:17:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.71, 3.60 [01:18:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.18, 3.19, 3.23 [01:18:21] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.41, 7.10, 6.79 [01:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.59, 2.08, 1.92 [01:19:23] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.19, 3.14, 3.40 [01:22:20] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.31, 6.31, 6.56 [01:26:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.88, 1.95, 1.96 [01:26:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.58, 3.57, 3.53 [01:28:19] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.82, 3.45, 3.46 [01:28:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.31, 3.28, 3.42 [01:30:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.05, 2.82, 3.23 [01:32:18] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.62, 3.18, 3.36 [01:32:19] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:42:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.91, 1.85 [01:42:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.55, 3.40 [01:44:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.43, 3.72, 3.48 [01:44:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.89, 1.89, 1.85 [01:44:29] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.82, 1.60, 1.39 [01:45:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.59, 3.53, 3.29 [01:46:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.18, 3.98, 3.59 [01:46:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.99, 3.85, 3.57 [01:48:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.88, 3.91, 3.61 [01:48:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.49, 2.06, 1.92 [01:50:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.22, 3.16, 3.36 [01:50:28] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.49, 1.94, 1.58 [01:50:56] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.32, 3.10, 3.22 [01:52:10] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.84, 3.23, 3.39 [01:54:28] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.53, 1.83, 1.63 [01:56:28] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.60, 1.57 [02:01:08] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [02:04:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.86, 1.98 [02:05:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.15, 3.60, 3.38 [02:06:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.74, 3.62, 3.43 [02:10:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.60, 4.11, 3.67 [02:11:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.52, 3.35 [02:12:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.77, 3.71, 3.58 [02:13:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.55, 4.23, 3.71 [02:13:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.50, 4.08, 3.56 [02:14:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.76, 4.38, 3.85 [02:15:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.31, 3.61, 3.44 [02:16:28] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.77, 1.70, 1.48 [02:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.73, 2.02, 1.90 [02:18:28] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.10, 1.43, 1.41 [02:19:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.46, 3.88, 3.58 [02:20:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.90, 3.82 [02:21:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.46, 3.46 [02:22:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.26, 4.32, 3.98 [02:24:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.91, 1.91 [02:24:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.85, 3.84 [02:26:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.12, 1.87, 1.88 [02:28:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.79, 1.85 [02:28:52] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.68, 3.55 [02:30:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.06, 1.89, 1.88 [02:30:49] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.32, 3.43 [02:32:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.57, 3.70, 3.91 [02:32:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.86, 1.90, 1.89 [02:34:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.76, 4.05, 4.01 [02:34:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.26, 4.02, 3.83 [02:34:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.72, 3.15, 3.35 [02:38:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.09, 3.63, 3.86 [02:38:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.35, 10.34, 9.35 [02:38:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.84, 3.82 [02:44:22] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.99, 9.90, 9.54 [02:46:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.16, 2.77, 3.38 [02:48:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.49, 2.02, 1.89 [02:48:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.90, 2.96, 3.35 [02:52:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.93, 1.90 [02:58:02] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.94, 3.89, 3.57 [03:00:02] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.58, 3.29, 3.38 [03:03:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.74, 3.85, 3.31 [03:05:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.34, 3.34, 3.19 [03:06:51] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.95, 2.25, 1.57 [03:08:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.79, 1.80 [03:10:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.76, 3.76, 3.40 [03:10:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.77, 1.79 [03:11:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.67, 3.36 [03:12:46] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.77, 1.59, 1.52 [03:14:55] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.48, 3.35, 3.42 [03:15:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.97, 2.90, 3.13 [03:16:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.24, 3.16, 3.26 [03:16:54] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.92, 3.21, 3.36 [03:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.62, 2.01, 1.86 [03:23:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.90, 3.83, 3.49 [03:24:54] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.91, 3.55 [03:25:31] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.18, 2.04, 1.75 [03:25:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.76, 3.79, 3.52 [03:26:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.70, 3.50, 3.45 [03:27:29] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.25, 1.72, 1.67 [03:27:50] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.08, 4.42, 3.76 [03:29:02] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.73, 3.21 [03:29:27] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.73, 1.36, 1.54 [03:30:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.64, 1.91, 1.97 [03:30:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.54, 3.73, 3.28 [03:32:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.17, 3.91, 3.61 [03:32:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.70, 4.42, 3.58 [03:33:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 4.00, 3.79 [03:34:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.72, 3.58 [03:34:53] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.23, 3.87, 3.47 [03:36:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.20, 1.95, 1.95 [03:38:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.90, 1.93 [03:38:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.35, 4.15, 3.79 [03:40:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.20, 3.72, 3.67 [03:40:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.71, 3.30, 3.36 [03:43:44] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.19, 3.36, 3.49 [03:44:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.93, 3.87, 3.69 [03:44:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.42, 3.42 [03:45:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 3.36, 3.47 [03:46:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.83, 3.71 [03:47:43] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.53, 3.72, 3.58 [03:48:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.66, 2.07, 1.96 [03:48:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.57, 3.19, 3.34 [03:49:10] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.14, 1.70, 1.35 [03:50:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.49, 4.56, 4.02 [03:52:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.79, 3.55 [03:53:03] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.20, 1.90, 1.58 [03:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.66, 1.95, 1.96 [03:55:00] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.62, 1.45, 1.45 [03:56:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.24, 3.60, 3.53 [03:56:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.95, 3.91 [04:00:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.05, 4.76, 3.99 [04:00:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 4.11, 3.98 [04:02:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.11, 4.00, 3.80 [04:02:50] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.29, 2.92, 2.22 [04:04:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.74, 3.57, 3.81 [04:05:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.84, 3.32, 3.76 [04:06:43] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.98, 1.85, 1.93 [04:08:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.58, 1.53, 1.70 [04:10:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.70, 2.92, 3.39 [04:12:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.68, 2.52, 3.23 [04:14:30] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.35, 1.04, 1.54 [04:17:33] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.74, 3.11, 3.40 [04:18:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.13, 3.43, 3.38 [04:19:14] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.32, 2.03, 1.85 [04:20:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.96, 3.73, 3.50 [04:23:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.33, 4.18, 3.79 [04:23:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.02, 3.44, 3.40 [04:24:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.60, 3.72, 3.52 [04:26:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.61, 3.59, 3.49 [04:27:07] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.89, 1.89 [04:29:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.00, 3.97, 3.90 [04:30:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.32, 3.30, 3.39 [04:33:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.35, 3.07, 3.36 [04:34:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.67, 3.74, 3.53 [04:35:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.87, 4.03, 3.90 [04:37:34] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.46, 4.85, 3.98 [04:38:55] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.24, 1.48, 1.68 [04:41:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.58, 1.70, 1.33 [04:42:49] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.83, 1.77, 1.76 [04:45:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.05, 1.40, 1.29 [04:46:45] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.21, 1.77, 1.75 [04:48:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.04, 3.55, 3.87 [04:49:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.87, 3.46, 3.89 [04:51:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.58, 3.01, 3.89 [04:54:37] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.95, 1.91 [04:56:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.27, 2.56, 3.24 [04:57:19] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.18, 2.95, 3.50 [04:59:19] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.47, 3.21, 3.53 [05:00:56] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.36, 3.10, 3.37 [05:02:30] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.82, 1.84 [05:04:28] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.79, 1.82 [05:05:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.35, 3.32 [05:07:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.93, 2.93, 3.17 [05:10:00] PROBLEM - wiki.aoemods.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.aoemods.com' expires in 15 day(s) (Tue 19 Jul 2022 05:01:51 GMT +0000). [05:10:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.18, 2.00, 1.91 [05:11:15] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 3.32, 3.47 [05:12:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9dbd24800595...2ceac6e8b30a [05:12:41] [url] Comparing 9dbd24800595...2ceac6e8b30a · miraheze/ssl · GitHub | github.com [05:12:42] [02miraheze/ssl] 07MirahezeSSLBot 032ceac6e - Bot: Update SSL cert for wiki.aoemods.com [05:13:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.53, 3.10, 3.13 [05:14:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.59, 1.90, 1.90 [05:15:12] PROBLEM - squareenixwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'squareenixwiki.com' expires in 15 day(s) (Tue 19 Jul 2022 04:58:43 GMT +0000). [05:15:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.71, 3.02, 3.10 [05:16:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.35, 4.20, 3.62 [05:17:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.46, 3.53 [05:17:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.78, 3.60, 3.47 [05:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.71, 2.14, 1.98 [05:18:34] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.70, 1.73, 1.28 [05:19:12] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.65, 3.16, 3.40 [05:20:32] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.70, 1.74, 1.34 [05:20:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/2ceac6e8b30a...6896ac85daeb [05:20:43] [url] Comparing 2ceac6e8b30a...6896ac85daeb · miraheze/ssl · GitHub | github.com [05:20:44] [02miraheze/ssl] 07MirahezeSSLBot 036896ac8 - Bot: Update SSL cert for squareenixwiki.com [05:21:52] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.56, 3.21, 3.35 [05:22:30] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.94, 1.47, 1.29 [05:25:06] PROBLEM - mystiverse.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mystiverse.wiki' expires in 15 day(s) (Tue 19 Jul 2022 05:00:22 GMT +0000). [05:26:48] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6896ac85daeb...b3ae32bb2cf3 [05:26:48] [url] Comparing 6896ac85daeb...b3ae32bb2cf3 · miraheze/ssl · GitHub | github.com [05:26:49] [02miraheze/ssl] 07MirahezeSSLBot 03b3ae32b - Bot: Update SSL cert for mystiverse.wiki [05:30:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.90, 1.98 [05:39:00] RECOVERY - wiki.aoemods.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.aoemods.com' will expire on Sat 01 Oct 2022 04:12:34 GMT +0000. [05:46:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.54, 1.56, 1.70 [05:50:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.46, 1.97, 1.83 [05:56:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.86, 1.94, 1.90 [06:06:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.66, 1.99, 1.90 [06:06:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.20, 10.49, 9.38 [06:06:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.15, 3.45, 3.02 [06:08:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.61, 10.12, 9.37 [06:10:50] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.36, 3.44, 3.11 [06:11:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.36, 2.85 [06:12:47] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.56, 3.88, 3.31 [06:13:52] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.48, 3.23, 2.88 [06:14:36] RECOVERY - squareenixwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'squareenixwiki.com' will expire on Sat 01 Oct 2022 04:20:30 GMT +0000. [06:14:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.04, 3.15, 3.11 [06:20:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.14, 3.34, 3.01 [06:22:28] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.52, 3.28 [06:23:19] RECOVERY - mystiverse.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'mystiverse.wiki' will expire on Sat 01 Oct 2022 04:26:41 GMT +0000. [06:26:33] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.58, 3.19 [06:28:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.70, 3.27 [06:30:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.54, 3.84, 3.62 [06:30:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.72, 3.33 [06:32:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.86, 3.65 [06:34:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.56, 4.27, 3.82 [06:34:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.92, 3.95, 3.46 [06:36:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.38, 10.62, 9.89 [06:36:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.56, 3.84, 3.49 [06:38:11] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.46, 9.80, 9.68 [06:38:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.83, 3.62, 3.67 [06:42:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.72, 3.18, 3.32 [06:44:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.19, 2.93, 3.37 [06:46:45] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.03, 3.17, 3.37 [06:56:41] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.87, 4.16, 3.51 [06:58:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.47, 3.24 [07:00:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.81, 3.13, 3.14 [07:04:03] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.99, 3.70, 3.43 [07:04:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.82, 4.24, 3.57 [07:05:59] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.95, 3.38, 3.34 [07:06:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.44, 3.71, 3.45 [07:08:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.13, 3.40, 3.59 [07:12:36] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.64, 2.99, 3.37 [07:16:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.46, 2.74, 3.17 [07:16:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.10, 3.39, 3.47 [07:18:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.49, 3.16, 3.38 [07:31:22] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.04, 3.67, 3.41 [07:31:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.72, 9.52, 7.63 [07:33:19] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.59, 3.42 [07:33:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.11, 8.95, 7.66 [07:35:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.45, 3.15, 3.27 [07:39:10] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.88, 3.81, 3.55 [07:42:56] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.88, 10.21, 8.96 [07:43:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.03, 2.96, 3.27 [07:44:56] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.27, 9.98, 9.03 [08:02:13] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.27, 2.92, 1.92 [08:06:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.56, 10.24, 9.58 [08:10:45] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.88, 9.68, 9.52 [08:18:13] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.99, 3.67, 3.66 [08:21:01] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.45, 3.13 [08:22:58] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.29, 2.95, 2.98 [08:24:13] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.45, 2.64, 3.22 [08:32:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.68, 3.82, 3.29 [08:32:19] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:34:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.13, 3.14, 3.10 [08:38:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.01, 3.41, 3.19 [08:40:29] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.40, 1.86, 1.31 [08:42:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.10, 3.29, 3.18 [08:42:19] PROBLEM - atnarsia.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query atnarsia.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:42:29] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.66, 1.44, 1.22 [08:54:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.85, 3.41, 3.12 [08:55:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.70, 3.80, 3.09 [08:55:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.88, 3.54, 3.17 [08:56:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.59, 3.62, 3.22 [08:58:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.46, 3.24, 3.14 [08:59:15] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.99, 3.09, 2.97 [08:59:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.39, 3.36, 3.24 [09:01:08] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [09:10:23] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.62, 1.98 [09:12:08] RECOVERY - atnarsia.com - reverse DNS on sslhost is OK: SSL OK - atnarsia.com reverse DNS resolves to cp20.miraheze.org - NS RECORDS OK [09:14:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.96, 3.49, 3.22 [09:15:56] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.17, 1.84, 1.38 [09:16:47] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.51, 3.19, 3.15 [09:17:56] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.76, 1.43, 1.29 [09:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.81, 1.90 [09:22:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.93, 1.94 [09:25:53] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.44, 1.71, 1.43 [09:26:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.17, 1.85, 1.89 [09:27:53] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.18, 1.45, 1.36 [09:28:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.55, 1.75, 1.85 [09:36:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.55, 1.53, 1.70 [09:43:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.73, 1.74, 1.74 [09:45:15] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.48, 1.64, 1.70 [09:51:07] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.24, 2.01, 1.84 [09:52:11] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.75, 6.70, 5.74 [09:54:11] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.39, 6.37, 5.72 [09:55:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.66, 1.98, 1.88 [09:56:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.51, 3.22, 2.79 [10:00:35] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.09, 2.79, 2.72 [10:10:47] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.56, 1.60, 1.69 [10:15:40] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.95, 1.52, 1.11 [10:17:40] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.95, 1.32, 1.09 [10:19:35] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.97, 2.38, 1.97 [10:25:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.79, 3.47, 2.76 [10:26:38] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.61, 3.79, 3.01 [10:29:26] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.94, 1.97 [10:30:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.73, 3.72, 3.15 [10:32:36] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.72, 3.35 [10:32:56] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.92, 3.40, 3.09 [10:34:35] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.15, 4.03, 3.49 [10:36:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.17, 3.52, 3.36 [10:36:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.40, 3.54, 3.23 [10:38:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.07, 2.97, 3.17 [10:38:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.31, 3.17, 3.14 [10:42:48] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.97, 10.86, 8.55 [10:44:47] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.75, 9.77, 8.42 [10:47:08] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.87, 1.89 [10:53:02] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.98, 1.96 [10:59:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.20, 2.87 [11:02:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.84, 3.47, 3.04 [11:02:52] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.28, 1.45, 1.70 [11:03:02] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.87, 3.31, 3.00 [11:06:43] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.37, 3.38, 3.12 [11:08:19] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.93, 9.78, 8.30 [11:10:19] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.99, 9.09, 8.23 [11:10:40] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.97, 1.76, 1.74 [11:12:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.61, 9.82, 8.77 [11:14:36] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.36, 9.73, 8.88 [11:18:33] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.44, 2.00, 1.83 [11:18:33] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.82, 10.18, 9.25 [11:20:32] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.49, 10.07, 9.34 [11:20:48] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.06, 1.89, 1.34 [11:22:46] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.91, 1.53, 1.27 [11:24:27] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.44, 1.88, 1.87 [11:32:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.25, 1.50, 1.69 [11:34:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.98, 3.60, 3.16 [11:36:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.43, 3.31, 3.11 [11:39:18] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.79, 2.57, 1.76 [11:40:29] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.82, 2.02, 1.52 [11:42:29] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.72, 1.83, 1.50 [11:43:17] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.03, 1.71, 1.59 [11:44:29] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.06, 1.47, 1.41 [11:45:17] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.15, 1.57, 1.56 [11:50:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.46, 2.07, 1.82 [11:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.96, 1.85 [12:01:05] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.85, 6.29, 5.83 [12:03:04] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.42, 6.13, 5.82 [12:04:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.01, 1.84, 1.81 [12:06:14] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.05, 10.14, 8.83 [12:08:13] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.26, 10.13, 8.99 [12:10:12] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.37, 9.65, 8.97 [12:14:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.42, 3.16 [12:14:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.64, 1.86, 1.87 [12:14:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.43, 3.77, 3.20 [12:16:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.93, 1.90 [12:16:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.82, 4.05, 3.39 [12:18:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.50, 3.83, 3.37 [12:18:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.68, 3.56, 3.28 [12:20:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.07, 3.21, 3.21 [12:20:24] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.85, 3.35, 3.24 [12:23:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.57, 7.25, 6.38 [12:25:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.96, 6.51, 6.22 [12:26:23] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.99, 1.98, 1.99 [12:26:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.93, 10.45, 8.96 [12:28:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.13, 3.78, 3.41 [12:28:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.99, 9.79, 9.33 [12:28:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.66, 9.82, 8.92 [12:29:30] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.48, 3.84, 3.18 [12:30:11] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.06, 8.89, 9.06 [12:31:26] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.84, 3.27 [12:33:22] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 3.21, 3.09 [12:34:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.72, 3.78, 3.63 [12:36:12] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 16.40, 12.44, 10.36 [12:36:33] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:36:45] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [12:38:29] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.020 second response time [12:38:33] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 1 backends are down. mw111 [12:38:43] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 0.420 second response time [12:40:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 5.55, 10.38, 10.20 [12:42:11] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.49, 9.95, 10.04 [12:42:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.54, 1.52, 1.70 [12:42:33] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [12:46:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.00, 3.02, 3.39 [12:47:00] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.17, 3.73, 3.37 [12:50:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.36, 1.99, 1.83 [12:50:52] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 4.06, 3.58 [12:52:11] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.37, 10.39, 9.96 [12:52:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.91, 3.62, 3.48 [12:53:58] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.72, 1.55, 1.27 [12:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.91, 1.84 [12:55:58] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.63, 1.61, 1.33 [12:56:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.12, 10.74, 10.23 [12:56:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.76, 3.21, 3.34 [12:58:11] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.54, 11.62, 10.63 [13:00:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.29, 11.51, 10.72 [13:06:29] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.54, 1.87, 1.41 [13:08:29] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.65, 1.36, 1.27 [13:10:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.50, 1.59, 1.70 [13:10:37] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.10, 6.49, 5.99 [13:12:11] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.85, 10.86, 10.57 [13:12:36] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.91, 6.31, 5.98 [13:14:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.52, 10.05, 10.32 [13:17:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.37, 3.64, 3.34 [13:18:11] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.42, 9.79, 10.15 [13:19:14] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.70, 2.06, 1.83 [13:20:33] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 10.20, 8.09, 6.76 [13:21:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.95, 3.62, 3.26 [13:22:32] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.42, 7.44, 6.69 [13:22:53] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.92, 3.45, 3.21 [13:22:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.63, 3.68, 3.47 [13:24:52] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.36, 3.80, 3.37 [13:25:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.76, 3.81, 3.39 [13:26:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.84, 3.88, 3.46 [13:27:07] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.59, 1.88, 1.88 [13:27:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.47, 3.68, 3.40 [13:28:30] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.61, 6.42, 6.49 [13:28:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.11, 4.15, 3.62 [13:31:30] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.77, 3.11, 3.28 [13:32:41] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.71, 3.24, 3.39 [13:32:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.90, 3.67 [13:34:49] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.25, 3.99, 3.72 [13:36:36] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.91, 3.68, 3.54 [13:36:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.59, 3.62, 3.63 [13:37:06] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.54, 10.12, 9.83 [13:38:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.67, 3.63 [13:38:55] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.56, 1.55, 1.69 [13:39:05] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.00, 9.43, 9.61 [13:40:29] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.36, 3.30, 3.39 [13:40:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.97, 3.30, 3.49 [13:42:49] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.71, 1.73 [13:44:46] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.19, 4.56, 3.95 [13:44:47] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.62, 1.64, 1.70 [13:45:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.78, 3.60, 3.37 [13:46:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.73, 4.36, 3.88 [13:47:02] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.23, 3.07, 3.20 [13:48:42] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.21, 2.38, 1.98 [13:50:53] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 3.87, 3.50 [13:52:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.94, 3.96, 3.90 [13:52:49] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.27, 3.38, 3.38 [13:58:32] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.89, 1.96 [13:58:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.75, 3.71, 3.53 [14:00:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.41, 3.72, 3.76 [14:00:30] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.21, 1.98, 1.98 [14:02:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.61, 3.28, 3.59 [14:02:28] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.84, 1.96, 1.98 [14:02:40] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.96, 3.55, 3.87 [14:06:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.32, 3.87, 3.59 [14:06:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.39, 4.23, 4.09 [14:10:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.41, 4.19, 3.93 [14:18:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.60, 3.78, 3.89 [14:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.39, 1.96, 1.91 [14:18:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.02, 3.50, 4.00 [14:20:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.88, 3.72, 4.02 [14:24:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.82, 1.89 [14:24:33] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.96, 3.57, 3.91 [14:28:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 3.61, 3.70 [14:30:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.97, 2.97, 3.45 [14:32:30] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.55, 3.80, 3.82 [14:34:30] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 3.92, 3.87 [14:36:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.30, 3.66, 3.58 [14:36:29] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.22, 4.08, 3.94 [14:37:32] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.93, 1.76, 1.41 [14:38:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.76, 1.77 [14:38:28] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.79, 3.85 [14:39:32] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.91, 1.43, 1.33 [14:40:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 4.03, 3.93 [14:42:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.84, 1.81 [14:44:26] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.46, 3.94, 3.93 [14:46:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.83, 1.80 [14:46:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.60, 4.17, 4.01 [14:54:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.43, 3.67, 3.94 [14:56:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.35, 3.90, 4.02 [14:56:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.48, 1.96, 1.97 [15:02:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.46, 3.61, 3.94 [15:02:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.17, 3.47, 3.95 [15:07:24] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.03, 1.74, 1.47 [15:08:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.99, 3.42, 3.89 [15:09:23] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.99, 1.56, 1.44 [15:10:19] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.41, 3.85, 3.86 [15:12:18] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.76, 3.37, 3.68 [15:12:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.13, 3.95, 3.99 [15:14:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 3.81, 3.81 [15:14:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.45, 3.91, 3.81 [15:16:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.70, 3.78, 3.80 [15:16:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.53, 1.57, 1.69 [15:18:16] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.53, 3.91, 3.84 [15:20:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.87, 3.83 [15:20:23] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 2.05, 1.88 [15:24:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.76, 4.25, 3.95 [15:24:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.94, 1.89 [15:32:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.09, 1.87, 1.85 [15:33:04] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:33:14] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.75, 10.03, 9.22 [15:35:12] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.99, 10.04, 9.33 [15:36:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.63, 3.60, 3.97 [15:38:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.05, 3.83, 4.00 [15:40:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.58, 3.86, 4.00 [15:42:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.64, 1.88, 1.89 [15:44:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.81, 4.34, 4.16 [15:48:23] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.61, 2.14, 1.99 [15:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 2.00, 2.00 [16:01:53] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [16:02:38] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.46, 10.34, 9.29 [16:04:36] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.80, 9.73, 9.17 [16:06:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.95, 3.51, 3.99 [16:06:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.90, 3.53, 4.00 [16:12:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.31, 3.69, 3.87 [16:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.80, 1.77 [16:20:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.58, 3.60, 3.84 [16:20:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.46, 3.36, 4.00 [16:22:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.41, 4.31, 4.04 [16:22:08] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [16:22:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 3.51, 3.55 [16:22:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.73, 4.24, 4.23 [16:24:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.95, 3.93, 3.93 [16:26:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.59, 1.93, 1.89 [16:27:56] PROBLEM - wiki.fbpml.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.fbpml.org' expires in 15 day(s) (Tue 19 Jul 2022 16:21:56 GMT +0000). [16:28:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.41, 4.90, 4.31 [16:30:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.45, 3.64, 3.66 [16:30:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.16, 3.65, 3.98 [16:30:28] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.50, 4.60, 2.50 [16:34:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b3ae32bb2cf3...25d9c46b30c5 [16:34:02] [url] Comparing b3ae32bb2cf3...25d9c46b30c5 · miraheze/ssl · GitHub | github.com [16:34:03] [02miraheze/ssl] 07MirahezeSSLBot 0325d9c46 - Bot: Update SSL cert for wiki.fbpml.org [16:34:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.66, 3.65, 3.95 [16:34:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.49, 4.14, 4.03 [16:36:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.04, 3.93, 3.77 [16:38:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.32, 3.65, 3.68 [16:38:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.19, 3.71, 3.90 [16:38:28] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.71, 1.82, 1.98 [16:44:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.89, 3.84, 3.80 [16:44:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.20, 1.50, 1.65 [16:44:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.45, 4.39, 4.04 [16:46:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.73, 3.62 [16:46:28] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.16, 1.34, 1.66 [16:48:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.77, 3.59, 3.76 [16:48:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.41, 3.51 [16:49:11] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.13, 6.59, 5.80 [16:50:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.81, 2.14, 1.86 [16:52:08] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:52:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.67, 3.51, 3.83 [16:54:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.20, 3.62, 3.70 [16:54:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.54, 3.52 [16:54:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.50, 2.00, 1.89 [16:55:09] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 5.73, 7.07, 6.41 [16:56:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.50, 3.33, 3.59 [16:56:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.63, 3.19, 3.40 [16:56:51] RECOVERY - wiki.fbpml.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fbpml.org' will expire on Sat 01 Oct 2022 15:33:56 GMT +0000. [16:57:09] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.84, 6.64, 6.33 [16:58:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 3.92, 3.78 [16:59:12] PROBLEM - www.hoolehistoryheritagesociety.org.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.hoolehistoryheritagesociety.org.uk' expires in 15 day(s) (Tue 19 Jul 2022 16:36:45 GMT +0000). [17:00:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.65, 3.69 [17:00:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.53, 3.73 [17:02:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.51, 3.57, 3.72 [17:03:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/25d9c46b30c5...b327e0b26c85 [17:03:20] [url] Comparing 25d9c46b30c5...b327e0b26c85 · miraheze/ssl · GitHub | github.com [17:03:21] [02miraheze/ssl] 07MirahezeSSLBot 03b327e0b - Bot: Update SSL cert for www.hoolehistoryheritagesociety.org.uk [17:03:40] PROBLEM - wiki.qadrishattari.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.qadrishattari.xyz' expires in 15 day(s) (Tue 19 Jul 2022 16:32:29 GMT +0000). [17:04:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.35, 2.91, 3.37 [17:04:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.55, 1.70 [17:05:17] PROBLEM - wiki.slimelab.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.slimelab.net' expires in 15 day(s) (Tue 19 Jul 2022 16:33:58 GMT +0000). [17:06:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.31, 9.33, 8.89 [17:08:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.74, 3.72, 3.67 [17:08:46] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.55, 8.36, 8.56 [17:09:26] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b327e0b26c85...75a1746f9ed8 [17:09:27] [url] Comparing b327e0b26c85...75a1746f9ed8 · miraheze/ssl · GitHub | github.com [17:09:28] [02miraheze/ssl] 07MirahezeSSLBot 0375a1746 - Bot: Update SSL cert for wiki.qadrishattari.xyz [17:10:02] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.71, 3.66, 3.48 [17:10:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.38, 3.55, 3.52 [17:13:25] PROBLEM - you.r-fit.cc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'you.r-fit.cc' expires in 15 day(s) (Tue 19 Jul 2022 17:05:04 GMT +0000). [17:14:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.45, 3.15, 3.36 [17:14:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.71, 1.68 [17:14:36] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 36.01, 16.37, 7.19 [17:16:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.22, 1.90, 1.76 [17:16:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/75a1746f9ed8...547da0c50b05 [17:16:32] [url] Comparing 75a1746f9ed8...547da0c50b05 · miraheze/ssl · GitHub | github.com [17:16:33] [02miraheze/ssl] 07MirahezeSSLBot 03547da0c - Bot: Update SSL cert for wiki.slimelab.net [17:17:15] PROBLEM - wmworld.sktz.live - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wmworld.sktz.live' expires in 15 day(s) (Tue 19 Jul 2022 17:01:17 GMT +0000). [17:17:49] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.55, 3.47 [17:19:02] PROBLEM - gp.ct777.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'gp.ct777.cf' expires in 15 day(s) (Tue 19 Jul 2022 16:53:42 GMT +0000). [17:21:04] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.25, 3.71, 3.52 [17:21:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.70, 3.25, 3.38 [17:22:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.06, 1.62, 1.38 [17:23:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/547da0c50b05...da7a83fedd36 [17:23:03] [url] Comparing 547da0c50b05...da7a83fedd36 · miraheze/ssl · GitHub | github.com [17:23:03] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.66, 3.53 [17:23:04] [02miraheze/ssl] 07MirahezeSSLBot 03da7a83f - Bot: Update SSL cert for you.r-fit.cc [17:23:59] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.14, 1.48, 1.36 [17:24:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.54, 3.69, 3.87 [17:24:41] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 1595 bytes in 0.013 second response time [17:24:54] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.232 second response time [17:25:02] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.27, 4.14, 3.71 [17:25:12] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:25:15] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:25:49] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/da7a83fedd36...d1f8f020a608 [17:25:50] [url] Comparing da7a83fedd36...d1f8f020a608 · miraheze/ssl · GitHub | github.com [17:25:51] [02miraheze/ssl] 07MirahezeSSLBot 03d1f8f02 - Bot: Update SSL cert for gp.ct777.cf [17:26:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.44, 1.92, 1.95 [17:26:48] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:27:02] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.50, 3.74, 3.61 [17:27:54] RECOVERY - www.hoolehistoryheritagesociety.org.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'www.hoolehistoryheritagesociety.org.uk' will expire on Sat 01 Oct 2022 16:03:14 GMT +0000. [17:29:15] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:29:33] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:12] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:27] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.33, 2.66, 3.34 [17:30:34] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:36] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:38] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:41] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:30:53] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:31:00] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.29, 3.07, 3.38 [17:32:19] RECOVERY - wiki.qadrishattari.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.qadrishattari.xyz' will expire on Sat 01 Oct 2022 16:09:21 GMT +0000. [17:32:40] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.927 second response time [17:32:45] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.217 second response time [17:32:57] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.580 second response time [17:33:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d1f8f020a608...958e35e8c30d [17:33:22] [url] Comparing d1f8f020a608...958e35e8c30d · miraheze/ssl · GitHub | github.com [17:33:23] [02miraheze/ssl] 07MirahezeSSLBot 03958e35e - Bot: Update SSL cert for wmworld.sktz.live [17:33:23] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.200 second response time [17:33:40] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.386 second response time [17:34:21] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:34:23] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:34:33] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:35:06] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:35:11] RECOVERY - wiki.slimelab.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.slimelab.net' will expire on Sat 01 Oct 2022 16:16:26 GMT +0000. [17:35:59] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:36:00] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:36:09] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:36:19] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 19.24, 8.56, 3.86 [17:36:45] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:37:00] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.027 second response time [17:37:44] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:38:01] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:38:06] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.325 second response time [17:38:44] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.326 second response time [17:39:41] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:39:45] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:39:51] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:40:26] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:40:35] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.443 second response time [17:40:41] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:40:46] PROBLEM - cp20 Stunnel HTTP for mw121 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:40:49] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:40:58] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.023 second response time [17:41:01] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:41:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [17:41:07] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 0.322 second response time [17:41:08] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:41:14] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:41:25] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:41:34] PROBLEM - db121 NTP time on db121 is WARNING: NTP WARNING: Offset -0.2612053752 secs [17:41:35] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:41:43] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.017 second response time [17:41:45] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:42:02] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.307 second response time [17:42:08] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:42:13] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 0.70, 3.11, 2.87 [17:42:20] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:43:48] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.760 second response time [17:44:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.21, 1.46, 1.68 [17:44:33] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.172 second response time [17:44:52] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 8.829 second response time [17:45:33] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:45:52] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.551 second response time [17:46:01] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.325 second response time [17:46:03] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:46:22] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:46:46] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.758 second response time [17:46:49] paladox: around? [17:46:52] we've got a bit of a situation here [17:47:24] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.028 second response time [17:47:32] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 0.258 second response time [17:47:36] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.125 second response time [17:47:38] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.408 second response time [17:47:43] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.979 second response time [17:47:45] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 1.388 second response time [17:47:59] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 6.819 second response time [17:48:01] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.008 second response time [17:48:03] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.020 second response time [17:48:12] or JohnLewis [17:48:16] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.691 second response time [17:48:18] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.461 second response time [17:48:18] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 0.751 second response time [17:48:33] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.495 second response time [17:48:36] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.475 second response time [17:48:37] TL;DR we've tried re-enabling Cargo on that wiki (per a discussion with CosmicAlpha|AWAY ) now load is very high again but we can't reboot db121 as the actor migration script is on a db121 and reboot will break it [17:48:56] RECOVERY - cp20 Stunnel HTTP for mw121 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.274 second response time [17:49:09] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27067 bytes in 7.315 second response time [17:49:10] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 2.592 second response time [17:49:16] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.513 second response time [17:49:17] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.240 second response time [17:49:28] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.317 second response time [17:49:50] service stability comes first, the actor migration script will have to be stopped if its going to get into a point where we can't maintain a stable service because of fears of things breaking around it [17:50:07] PROBLEM - test101 MediaWiki Rendering on test101 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 854 bytes in 0.647 second response time [17:50:14] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset 0.000785946846 secs [17:50:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 2.00, 1.84 [17:50:37] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.018 second response time [17:50:40] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.019 second response time [17:50:47] JohnLewis: yeah definitely, though I was wondering if you had any ideas of how to avoid having to reboot [17:50:56] like how to kill all queries coming from one wiki [17:51:35] show processlist and then KILL ; [17:52:01] JohnLewis: yeah that I imagined but there's probably around 200 so is there no other way but manually? [17:52:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.85, 1.81 [17:52:23] like sort of a KILL WHERE db = x? [17:53:10] JohnLewis: agreed for sure. The issue is we must do the actor migration at some point. We can never upgrade to 1.39 otherwise, so there is an issue here for sure, but yeah if it is that or service stability, service stability comes first for sure. [17:53:19] SELECT CONCAT('KILL ',id,';') FROM information_schema.processlist WHERE db='' INTO OUTFILE '/tmp/process_list.txt'; SOURCE '/tmp/process_list.txt'; [17:53:52] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 10.09, 5.33, 3.73 [17:53:54] thanks! [17:54:14] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:54:17] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 12.86, 6.37, 3.79 [17:54:28] CosmicAlpha|AWAY: we won't we have like 6 months to do it? Also why is it so slow? Sleeping for 5 seconds also probably doesn't help - is speed consistent or does it change based on db size? [17:54:38] *won't we [17:54:40] PROBLEM - ircwiki.cf - LetsEncrypt on sslhost is CRITICAL: connect to address ircwiki.cf and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [17:54:53] JohnLewis: it could take that long to finish if even started now. [17:54:54] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:54:56] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:55:06] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:55:12] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:55:32] It's incredibly strange it's taking this long though? That would imply it'll takes years on enwiki alone nevermind all their wikis [17:55:50] The WMF have faster servers. [17:55:58] yeah it's been very slow and all other maint scripts are also affected [17:56:01] RECOVERY - test101 MediaWiki Rendering on test101 is OK: HTTP OK: HTTP/1.1 200 OK - 32552 bytes in 2.223 second response time [17:56:15] But they also have significantly more data as well [17:56:46] Something isn't right, I don't think we should be running scripts that have end dates of months - hours alone is pushing it personally [17:56:58] it is on c wikis right now, speed is not consistent, but it is slow on all wikis (a few minutes for very small wikis), and takes days for larger ones. [17:57:09] plus we have plans to do hardware work in the next month or two, so reboots will be required then [17:57:38] !log mass kill all anotheredenwiki queries on db121 [17:57:44] CosmicAlpha|AWAY: all have been killed now [17:57:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:57:56] processes back to 99 [17:58:16] there's still quite a few left but not 500 [17:58:18] CosmicAlpha|AWAY: we should disable [17:58:22] JohnLewis: That is fine, we could finish it later, my goal is to at least get it done before the upgrade, otherwise we would be blindsided by not being able to upgrade. It took the WMF a couple months to do it though I think. [17:58:27] Reception123: do it. [18:00:22] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:00:33] done [18:01:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [18:01:58] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [18:02:03] !log mass-kill anotheredenwiki queries on db121 again [18:02:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:02:26] PROBLEM - ircwiki.cf - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ircwiki.cf could not be found [18:03:33] CosmicAlpha|AWAY: load seems to be going down [18:03:38] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.86, 1.33, 0.98 [18:03:49] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:03:51] Great! [18:04:01] CosmicAlpha|AWAY: but somehow there's new queries even after I killed them? [18:04:04] and disabled in MW? [18:04:12] PROBLEM - mwtask111 NTP time on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:04:13] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:04:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.56, 1.61, 1.70 [18:04:26] Queued queries probably? [18:04:35] good thing John was around with that query killing command otherwise we would've had to reboot [18:04:49] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:04:53] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:04:54] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:05:20] !log repeat previous [18:05:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.00, 4.04, 3.17 [18:05:26] PROBLEM - mwtask111 Check Gluster Clients on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:05:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:05:37] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.00, 1.15, 0.96 [18:05:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.51, 3.72, 3.17 [18:06:26] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:06:50] PROBLEM - mwtask111 JobRunner Service on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:07:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.43, 3.55, 3.10 [18:07:45] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.49, 3.33, 3.09 [18:08:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.30, 2.31, 1.95 [18:08:23] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.48, 10.68, 7.97 [18:09:09] PROBLEM - mwtask111 Disk Space on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:10:11] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [18:10:13] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [18:10:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.69, 9.92, 8.00 [18:10:28] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [18:11:24] RECOVERY - mwtask111 Disk Space on mwtask111 is OK: DISK OK - free space: / 30790 MB (46% inode=89%); [18:11:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.55, 3.27 [18:11:47] RECOVERY - mwtask111 JobRunner Service on mwtask111 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [18:12:14] RECOVERY - you.r-fit.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'you.r-fit.cc' will expire on Sat 01 Oct 2022 16:22:56 GMT +0000. [18:13:11] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.96, 3.26, 3.13 [18:13:21] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [18:13:37] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.29, 3.31, 3.20 [18:14:21] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.00, 10.67, 8.77 [18:14:53] RECOVERY - mwtask111 NTP time on mwtask111 is OK: NTP OK: Offset -0.01198852062 secs [18:15:15] RECOVERY - wmworld.sktz.live - LetsEncrypt on sslhost is OK: OK - Certificate 'wmworld.sktz.live' will expire on Sat 01 Oct 2022 16:33:15 GMT +0000. [18:16:20] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.32, 9.84, 8.67 [18:17:05] RECOVERY - gp.ct777.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'gp.ct777.cf' will expire on Sat 01 Oct 2022 16:25:42 GMT +0000. [18:18:18] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 2.642 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [18:18:23] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 53 minutes ago with 0 failures [18:19:27] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [18:20:04] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [18:21:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.94, 3.41, 3.29 [18:23:21] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.96, 3.12, 3.19 [18:25:05] JohnLewis: enwiki too months [18:25:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.81, 3.42, 3.95 [18:26:40] s/too/took [18:26:40] RhinosF1 meant to say: JohnLewis: enwiki took months [18:27:19] geez [18:27:31] Amir1 will likely know the best out DB stuff and how to make it less a mess [18:27:31] Months for what? [18:27:41] JohnLewis: yes some of the migrations are ridiculous [18:27:41] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.17, 3.91, 4.05 [18:27:46] Expect more like it though [18:27:50] dmehus: actor migration [18:28:44] RhinosF1, oh, is this for MW 1.39 and is it related to the user table schema having changed dramatically? [18:30:18] dmehus: it's not the user table [18:30:21] But for 1.39 [18:30:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 4.40, 3.73 [18:30:38] The comment normalisation and links normalisation is still to go too [18:30:45] They will be for future versions [18:31:33] RhinosF1, ah, fair, but it's related to the way in which performing or target actors are referenced in various SQL tables in MW 1.39 then? Just not specifically the user table itself [18:31:40] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.73, 3.97 [18:32:33] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.48, 6.86, 5.77 [18:32:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.83, 3.79, 3.59 [18:33:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.09, 3.82, 3.97 [18:34:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.26, 3.87, 3.63 [18:34:32] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.70, 6.78, 5.88 [18:35:30] dmehus: yes [18:35:36] Because not all are actors are users [18:35:42] All users are actors [18:35:47] But IPs can also be actors [18:36:28] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.68, 3.43, 3.50 [18:37:01] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.89, 3.59 [18:37:38] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.41, 3.31, 3.75 [18:38:24] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.81, 3.11, 3.36 [18:40:54] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.70, 3.14, 3.34 [18:41:49] RhinosF1, yeah, hopefully this will help to finally resolve the imported user CheckUser extension bug [18:42:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.45, 3.43 [18:43:56] dmehus: which one [18:44:10] There are too many bugs to know them all [18:44:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.36, 3.33, 3.38 [18:45:35] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.88, 2.83, 3.38 [18:47:02] RhinosF1, the one where a sysop imports pages and assigns edits locally where such users exist locally, the imported edits, and thus imported user, are attributed to the importing sysop [18:47:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.64, 4.00, 3.65 [18:47:50] dmehus: i doubt it. Does that have a task [18:48:31] RhinosF1, I think so. I think I recall either creating a task, or finding a task. Or maybe someone like Reception123 made an upstream task for me. [18:49:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.47, 3.75, 3.60 [18:51:05] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.02, 10.05, 9.45 [18:51:33] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.45, 3.79, 3.60 [18:51:38] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 3.96, 3.70 [18:53:04] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.46, 9.66, 9.38 [18:53:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.95, 4.08, 3.63 [18:53:32] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.81, 3.88, 3.66 [18:53:50] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.341 second response time [18:54:03] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.363 second response time [18:54:26] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.046 second response time [18:54:42] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.033 second response time [18:54:55] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27075 bytes in 0.375 second response time [18:55:00] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.016 second response time [18:57:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.96, 3.90, 3.64 [18:57:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.26, 3.88, 3.78 [18:59:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.82, 4.62, 4.05 [19:02:13] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.03, 1.37, 3.78 [19:05:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.05, 3.56, 3.53 [19:06:13] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.41, 1.32, 3.19 [19:06:59] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.84, 10.89, 10.04 [19:07:28] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.90, 3.67, 3.58 [19:10:36] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 1.83, 2.11, 7.32 [19:10:44] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.85, 3.12, 3.38 [19:12:36] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 2.03, 2.06, 6.66 [19:12:55] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.50, 10.10, 8.56 [19:12:56] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.52, 9.98, 9.91 [19:14:29] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.72, 1.67, 1.12 [19:14:53] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.26, 9.57, 8.54 [19:16:29] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.88, 1.41, 1.09 [19:16:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.21, 3.48, 3.39 [19:17:39] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.40, 1.90, 1.27 [19:19:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.48, 1.69, 1.27 [19:21:23] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.61, 3.83, 3.65 [19:23:22] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.48, 3.53 [19:24:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.78, 3.64 [19:28:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.26, 3.86 [19:30:14] PROBLEM - archive.a2b2.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for archive.a2b2.org could not be found [19:30:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.52, 3.08, 3.35 [19:32:19] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:32:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.37, 3.90, 3.94 [19:33:01] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.54, 10.18, 9.17 [19:34:59] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.48, 9.61, 9.09 [19:36:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.37, 3.90, 3.95 [19:41:16] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.86, 3.72 [19:41:50] dmehus: I think I created the task for that. [19:42:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.45, 3.68, 3.79 [19:43:31] Ah no the one I did was different I think [19:44:27] Hey Voidwalker [19:44:31] hi [19:44:41] CosmicAlpha|AWAY: can you talk to Amir1 next week about actor migration [19:44:49] Because not very feasible [19:44:55] How's Voidwalker today [19:45:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.79, 3.95, 3.46 [19:45:38] air quality is horrible [19:45:48] RhinosF1: After Tuesday I can try. Technically away till Tuesday. Just came on today for the Cargo issue. [19:46:11] CosmicAlpha|AWAY: yeah, I'm surprised Monday isn't a WMF holiday anyway [19:46:22] it's a day for you isn't it [19:46:26] but I'm slowly adding support to Void-bot for globalblock actions [19:46:30] * RhinosF1 tired and brain broke [19:46:33] Voidwalker: cool [19:46:48] Send me like to code because I can have fun reviewing it if I get bored [19:46:56] And somehow run out of stuff [19:47:18] RhinosF1: Yep. It is for me. I am surprised it is not for the WMF though... [19:47:35] CosmicAlpha|AWAY: it might be tbh [19:47:41] I'd be surprised if not [19:47:51] But I have no idea what the days are [19:47:56] It's normally 1 Monday a month [19:50:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.96, 3.66 [19:53:33] current change is on github, but that's the easy part [19:54:39] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.71, 10.99, 9.93 [19:56:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.62, 3.66, 3.56 [19:56:48] [discord] Hi! [19:56:49] [discord] We do not have any settings for Babel like wgBabelCategoryNames and wgBabelMainCategory in LS.php, right? [19:57:22] [discord] [19:57:22] [url] Extension:Babel - MediaWiki | www.mediawiki.org [19:57:51] [02miraheze/TSPortal] 07OwenBaines pushed 037 commits to 03master [+7/-0/±25] 13https://github.com/miraheze/TSPortal/compare/62bc6944d027...9176e93a44e2 [19:57:52] [url] Comparing 62bc6944d027...9176e93a44e2 · miraheze/TSPortal · GitHub | github.com [19:57:52] [02miraheze/TSPortal] 07OwenBaines 03cb60364 - Correctly assign subject field based on modified Report structure [19:57:54] [02miraheze/TSPortal] 07OwenBaines 03f50e20d - Add 'no comments' default text for stylistic purposes [19:57:55] [02miraheze/TSPortal] 07OwenBaines 03238ef73 - Update usages of subject to user in DPA [19:57:57] [02miraheze/TSPortal] ... and 4 more commits. [19:57:58] [02miraheze/TSPortal] 07OwenBaines tagged 039176e93 as 03v4 13https://github.com/miraheze/TSPortal/commit/9176e93a44e2a5c8462c59a4aae51f1badb2350f [19:58:00] [02TSPortal] 07OwenBaines created tag 03v4 - 13https://github.com/miraheze/TSPortal [19:58:00] [url] Page not found · GitHub · GitHub | github.com [19:58:38] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.12, 10.09, 9.82 [19:59:51] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:00:48] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.04, 10.59, 9.18 [20:01:48] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.023 second response time [20:02:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.47, 3.57, 3.63 [20:02:42] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.88, 9.27, 8.87 [20:06:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.62, 3.99, 3.78 [20:06:34] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.92, 10.36, 9.86 [20:12:10] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/92921afb1405...5302fe970e75 [20:12:10] [url] Comparing 92921afb1405...5302fe970e75 · miraheze/puppet · GitHub | github.com [20:12:11] [02miraheze/puppet] 07Reception123 035302fe9 - reports: update to v4 [20:13:38] [02miraheze/puppet] 07Reception123 deleted branch 03Reception123-patch-3 [20:13:39] [02puppet] 07Reception123 deleted branch 03Reception123-patch-3 - 13https://github.com/miraheze/puppet [20:13:40] [url] Page not found · GitHub · GitHub | github.com [20:13:41] [02miraheze/puppet] 07Reception123 deleted branch 03Reception123-patch-2 [20:13:42] [02puppet] 07Reception123 deleted branch 03Reception123-patch-2 - 13https://github.com/miraheze/puppet [20:13:43] [url] Page not found · GitHub · GitHub | github.com [20:13:44] [02miraheze/puppet] 07Reception123 deleted branch 03Reception123-patch-1 [20:13:45] [02puppet] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://github.com/miraheze/puppet [20:13:46] [url] Page not found · GitHub · GitHub | github.com [20:13:47] [02miraheze/puppet] 07Reception123 created branch 03Reception123-patch-1 13https://github.com/miraheze/puppet/commit/5302fe970e75e42d061648f96438b154a9245315 [20:13:48] [02puppet] 07Reception123 created branch 03Reception123-patch-1 - 13https://github.com/miraheze/puppet [20:13:49] [url] Page not found · GitHub · GitHub | github.com [20:14:31] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.71, 9.80, 9.82 [20:14:44] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5302fe970e75...5a6546d58c4c [20:14:45] [url] Comparing 5302fe970e75...5a6546d58c4c · miraheze/puppet · GitHub | github.com [20:14:46] [02miraheze/puppet] 07Reception123 035a6546d - add discord webhook variable for TSPortal [20:15:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.34, 3.33, 3.89 [20:15:17] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03Reception123-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5a6546d58c4c...026f598cd7fb [20:15:18] [url] Comparing 5a6546d58c4c...026f598cd7fb · miraheze/puppet · GitHub | github.com [20:15:19] [02miraheze/puppet] 07Reception123 03026f598 - Update init.pp [20:16:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.40, 3.58, 3.85 [20:16:27] [02puppet] 07Reception123 opened pull request 03#2690: add discord webhook variable for TSPortal - 13https://github.com/miraheze/puppet/pull/2690 [20:16:28] ... [20:18:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 3.91, 3.91 [20:20:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.62, 3.80 [20:20:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.11, 10.72, 10.19 [20:20:58] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.52, 7.03, 6.53 [20:21:03] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.20, 4.20, 4.06 [20:21:23] [02puppet] 07Reception123 closed pull request 03#2690: add discord webhook variable for TSPortal - 13https://github.com/miraheze/puppet/pull/2690 [20:21:23] [url] Page not found · GitHub · GitHub | github.com [20:21:24] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/5302fe970e75...8285868af7fe [20:21:25] [url] Comparing 5302fe970e75...8285868af7fe · miraheze/puppet · GitHub | github.com [20:21:26] [02miraheze/puppet] 07Reception123 038285868 - add discord webhook variable for TSPortal (#2690) [20:21:27] [02miraheze/puppet] 07Reception123 deleted branch 03Reception123-patch-1 [20:21:29] [02puppet] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://github.com/miraheze/puppet [20:21:29] [url] Page not found · GitHub · GitHub | github.com [20:22:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.73, 3.94, 3.89 [20:22:27] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.04, 10.28, 10.09 [20:23:13] !log added discord webhook for tsportal to private hieradata [20:23:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:26] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.37, 10.92, 10.34 [20:26:55] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.03, 6.67, 6.56 [20:28:25] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.00, 10.61, 10.30 [20:30:19] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [20:31:51] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.20, 10.45, 9.46 [20:32:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.66, 10.09, 10.15 [20:33:49] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.61, 9.51, 9.24 [20:34:18] CosmicAlpha|AWAY: ah, yeah, quite possible (thanks in that case :) ) [20:34:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.16, 3.51, 3.85 [20:40:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.58, 3.98 [20:46:17] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.20, 10.32, 10.11 [20:48:16] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.77, 10.07, 10.03 [20:48:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.75, 2.72, 3.22 [20:48:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.06, 3.53, 3.91 [20:50:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.66, 3.84, 3.86 [20:52:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.86, 4.44, 4.16 [20:54:13] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.40, 10.10, 10.01 [20:54:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.01, 3.74, 3.93 [20:56:13] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.68, 9.77, 9.91 [20:56:33] [02miraheze/TSPortal] 07OwenBaines pushed 031 commit to 03master [+0/-0/±8] 13https://github.com/miraheze/TSPortal/compare/9176e93a44e2...6b6eb1049b08 [20:56:33] [url] Comparing 9176e93a44e2...6b6eb1049b08 · miraheze/TSPortal · GitHub | github.com [20:56:34] [02miraheze/TSPortal] 07OwenBaines 036b6eb10 - [Release] Version 5: Add support for HTTP web proxies [20:56:36] [02miraheze/TSPortal] 07OwenBaines tagged 036b6eb10 as 03v5 13https://github.com/miraheze/TSPortal/commit/6b6eb1049b0835c161b05fca33b61aebdc2a00f0 [20:56:37] [02TSPortal] 07OwenBaines created tag 03v5 - 13https://github.com/miraheze/TSPortal [20:56:37] [url] Page not found · GitHub · GitHub | github.com [20:56:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.75, 3.50, 3.77 [20:58:35] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.73, 1.57, 1.21 [20:59:12] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8285868af7fe...578073afff07 [20:59:13] [url] Comparing 8285868af7fe...578073afff07 · miraheze/puppet · GitHub | github.com [20:59:14] [02miraheze/puppet] 07Reception123 03578073a - reports: update to v5 [21:00:32] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.18, 1.39, 1.19 [21:03:34] PROBLEM - reports121 Puppet on reports121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 39 seconds ago with 1 failures. Failed resources (up to 3 shown): Exec[git_checkout_TSPortal] [21:04:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.81, 3.23, 3.05 [21:04:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.57, 3.63, 3.68 [21:06:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.64, 3.15, 3.05 [21:06:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.84, 3.62, 3.67 [21:07:32] RECOVERY - reports121 Puppet on reports121 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [21:07:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/578073afff07...c01955e39852 [21:07:50] [url] Comparing 578073afff07...c01955e39852 · miraheze/puppet · GitHub | github.com [21:07:51] [02miraheze/puppet] 07paladox 03c01955e - reports: set WEB_PROXY to bast.miraheze.org [21:08:22] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.47, 1.73, 1.37 [21:08:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.60, 3.92, 3.76 [21:10:19] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.72, 1.41, 1.30 [21:10:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.19, 3.35, 3.58 [21:14:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.77, 3.47, 3.54 [21:14:46] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.61, 2.92, 3.33 [21:24:42] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.90, 4.27, 3.72 [21:26:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.58, 2.49, 1.96 [21:27:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.86, 3.94, 3.51 [21:28:40] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.90, 3.69 [21:30:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.68, 1.99 [21:30:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.12, 3.80, 3.94 [21:30:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.85, 3.69 [21:30:46] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.18, 1.95, 1.86 [21:31:02] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 3.65, 3.45 [21:32:19] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:32:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.27, 4.60, 4.20 [21:36:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.02, 1.69, 1.87 [21:36:40] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.08, 1.40, 1.64 [21:36:57] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.48, 10.39, 9.54 [21:38:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.66, 1.84 [21:38:38] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.43, 3.93, 3.89 [21:40:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.20, 4.07, 3.95 [21:40:55] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.52, 10.10, 9.63 [21:48:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.50, 1.95, 1.88 [21:48:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.82, 3.98, 3.91 [21:48:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.21, 3.36, 3.73 [21:51:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.14, 1.83, 1.73 [21:52:28] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.80, 4.15, 3.99 [21:52:33] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.48, 3.76, 3.80 [21:53:21] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.31, 1.65, 1.68 [21:56:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.32, 1.85, 1.94 [21:58:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.32, 3.88, 3.96 [21:58:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.75, 3.86, 3.89 [22:01:08] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [22:02:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.23, 3.24, 3.99 [22:04:29] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.58, 3.81, 3.75 [22:06:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.25, 2.75, 3.30 [22:06:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.23, 3.60, 3.68 [22:08:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.53, 1.70 [22:10:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.36, 3.45 [22:10:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.53, 3.81 [22:12:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.82, 3.20, 3.38 [22:12:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.87, 3.42, 3.75 [22:12:27] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 3.86, 3.78 [22:13:57] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.41, 2.07, 1.82 [22:14:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.69, 1.72 [22:14:26] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.56, 3.67 [22:15:55] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.11, 1.85, 1.77 [22:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.58, 2.01, 1.83 [22:18:25] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.62, 3.78, 3.70 [22:18:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.74, 4.34, 3.98 [22:21:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.06, 4.51, 3.79 [22:25:46] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.29, 1.54, 1.67 [22:28:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.40, 3.73, 3.76 [22:30:19] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [22:30:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.50, 3.45, 3.74 [22:30:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.60, 3.64, 3.99 [22:32:20] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.38, 4.30, 4.05 [22:32:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.66, 1.94, 1.97 [22:32:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.66, 3.88, 4.05 [22:34:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.61, 3.92 [22:34:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.51, 4.22, 3.92 [22:36:19] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.70, 3.79, 3.90 [22:36:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.63, 3.74 [22:39:30] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.24, 2.23, 1.85 [22:40:18] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.12, 3.68, 3.80 [22:42:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.82, 3.56, 3.73 [22:43:26] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.65, 1.87, 1.79 [22:44:16] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.66, 3.74 [22:44:19] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 14.18, 11.15, 8.74 [22:44:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.06, 1.88, 1.88 [22:45:24] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.02, 1.78, 1.76 [22:46:19] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.70, 9.80, 8.54 [22:46:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 3.60, 3.60 [22:48:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.30, 3.48, 3.55 [22:50:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.58, 3.73, 3.62 [22:50:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.82, 3.68 [22:52:14] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.36, 3.88, 3.83 [22:52:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.68, 3.97, 3.75 [22:52:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.90, 3.84, 3.71 [22:53:17] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.33, 1.91, 1.96 [22:54:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.74, 4.10, 3.79 [22:56:12] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.56, 4.12, 3.91 [22:56:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.94, 1.99 [22:58:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.74, 3.89, 3.85 [22:58:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.90, 3.72, 3.72 [22:59:58] RECOVERY - archive.a2b2.org - reverse DNS on sslhost is OK: SSL OK - archive.a2b2.org reverse DNS resolves to cp21.miraheze.org - CNAME OK [23:00:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.76, 3.79 [23:00:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.44, 2.91, 3.32 [23:01:10] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.64, 1.19, 1.61 [23:02:10] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.42, 3.10, 3.54 [23:02:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.28, 3.88, 3.76 [23:02:59] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 7.16, 4.68, 3.19 [23:04:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.67, 3.55, 3.66 [23:06:09] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.27, 2.85, 3.32 [23:10:59] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 3.63, 5.11, 4.14 [23:12:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.09, 3.68, 3.59 [23:14:21] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.24, 3.14 [23:16:21] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.04, 2.87, 3.02 [23:16:59] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 4.08, 5.04, 4.47 [23:18:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.67, 3.54, 3.39 [23:18:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.44, 1.95, 1.84 [23:18:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.58, 3.87, 3.73 [23:20:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.85, 2.99, 3.21 [23:20:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.33, 3.97, 3.78 [23:23:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.49, 3.88, 3.38 [23:24:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.20, 4.38, 3.72 [23:25:09] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.72, 3.83, 3.43 [23:26:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 2.00, 1.96 [23:27:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.66, 4.38, 3.66 [23:32:19] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:36:50] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.87, 3.98, 3.85 [23:40:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.84, 4.03, 3.88 [23:44:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.58, 3.87, 3.86 [23:46:06] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.81, 3.96 [23:46:11] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.09, 6.77, 5.95 [23:48:22] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.68, 2.06, 1.87 [23:50:11] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.17, 7.17, 6.28 [23:50:20] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.61, 2.10, 1.66 [23:52:11] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.31, 6.99, 6.34 [23:52:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.44, 1.88, 1.63 [23:52:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.85, 4.19, 3.84 [23:54:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.63, 4.05, 3.98 [23:54:11] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.18, 6.38, 6.19 [23:56:05] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.99, 4.00, 3.97 [23:56:14] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.65, 1.68, 1.60 [23:56:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.52, 1.87, 1.91 [23:58:06] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.36, 4.34, 4.10