[00:00:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.79, 1.82 [00:02:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.77, 1.81 [00:02:52] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.29, 1.71, 1.49 [00:03:09] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.89, 4.27, 3.91 [00:04:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.13, 4.72, 4.05 [00:04:52] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.83, 1.71, 1.52 [00:06:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.65, 1.36, 1.41 [00:08:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.53, 1.68 [00:09:04] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.95, 3.98 [00:11:03] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.57, 4.82, 4.28 [00:18:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.84, 2.07, 1.81 [00:24:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.90, 1.87 [00:24:52] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.70, 3.57, 3.91 [00:28:49] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.33, 3.89, 3.92 [00:29:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:30:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.45, 4.00 [00:32:46] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.65, 3.80 [00:34:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.60, 1.58, 1.70 [00:38:41] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.70, 2.74, 3.38 [00:43:10] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.83, 1.76 [00:43:46] miraheze/ImportDump - Universal-Omega the build passed. [00:44:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:45:08] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.72, 1.73 [00:49:05] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.47, 2.00, 1.83 [00:50:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.35, 2.98, 3.34 [00:55:00] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.91, 1.87 [01:00:20] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.59, 3.60, 3.33 [01:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:00:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.69, 3.42, 3.40 [01:02:19] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.28, 3.07, 3.16 [01:02:30] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.34, 10.24, 9.04 [01:02:52] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 3.16, 3.31 [01:04:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.83, 10.11, 9.15 [01:04:52] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.47, 1.53, 1.69 [01:10:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:11:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.60, 3.43 [01:11:53] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.79, 2.11, 1.56 [01:13:37] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.83, 3.53 [01:13:53] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.04, 1.72, 1.48 [01:17:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.36, 3.50, 3.51 [01:18:50] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.68, 7.19, 6.25 [01:19:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.26, 3.07, 3.34 [01:19:37] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.67, 2.02, 1.78 [01:19:53] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.94, 1.59, 1.55 [01:23:34] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.82, 1.77 [01:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:25:53] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.98, 1.86, 1.68 [01:27:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.93, 3.06, 3.78 [01:27:53] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.84, 1.44, 1.54 [01:28:18] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.73, 10.19, 9.45 [01:29:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.16, 3.88, 3.99 [01:29:29] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.45, 1.58, 1.68 [01:29:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:30:16] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.31, 9.51, 9.29 [01:32:23] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.18, 6.77, 6.58 [01:33:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.81, 3.92, 3.98 [01:36:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:37:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.29, 3.60, 3.78 [01:39:10] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.90, 1.76, 1.37 [01:40:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.37, 3.24 [01:41:09] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.28, 1.50, 1.32 [01:41:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.54, 10.95, 9.84 [01:42:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 3.07, 3.14 [01:42:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.50, 10.41, 9.78 [01:43:25] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.42, 10.82, 9.91 [01:43:27] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.43, 10.29, 9.84 [01:44:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.43, 9.75, 9.62 [01:45:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.87, 1.67, 1.59 [01:45:22] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.11, 9.96, 9.72 [01:45:22] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.00, 10.17, 9.87 [01:46:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.70, 3.43 [01:47:57] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.34, 7.05, 6.55 [01:49:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.44, 3.88 [01:49:13] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.56, 1.98, 1.72 [01:49:53] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.72, 1.55, 1.38 [01:49:54] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.30, 6.69, 6.50 [01:50:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.85, 3.19, 3.03 [01:51:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.28, 9.31, 8.62 [01:51:53] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.02, 1.71, 1.46 [01:52:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.15, 2.99, 3.25 [01:53:10] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.95, 1.78 [01:53:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.92, 9.19, 8.66 [01:53:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.50, 1.41 [01:54:31] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.29, 2.71, 2.87 [01:58:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.38, 10.85, 10.12 [01:59:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.59, 10.66, 9.46 [02:00:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.34, 10.61, 9.98 [02:01:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.30, 10.02, 9.37 [02:03:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.85, 3.95, 3.75 [02:04:37] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.46, 9.75, 9.88 [02:04:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.09, 11.11, 10.36 [02:04:59] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.63, 1.69 [02:05:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.34, 3.57, 3.63 [02:06:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.78, 10.46, 10.24 [02:07:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.92, 3.75 [02:07:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.68, 10.67, 9.86 [02:08:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.29, 9.95, 10.07 [02:09:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.97, 9.39, 9.49 [02:11:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.27, 3.21, 3.53 [02:11:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.38, 10.52, 10.21 [02:12:34] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.66, 10.96, 10.04 [02:12:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.74, 10.62, 10.29 [02:13:21] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.35, 10.90, 10.37 [02:14:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.05, 9.52, 9.92 [02:15:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.08, 2.62, 3.22 [02:17:11] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.74, 10.83, 10.45 [02:18:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.69, 9.98, 9.96 [02:18:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.27, 10.76, 10.36 [02:20:44] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.56, 2.00, 1.77 [02:21:02] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.04, 11.39, 10.71 [02:22:52] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.19, 6.33, 5.84 [02:24:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 16.52, 13.21, 11.28 [02:24:41] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.82, 1.99, 1.83 [02:25:41] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.18, 3.93, 3.55 [02:26:44] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.52, 6.52, 6.03 [02:26:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 17.26, 12.89, 11.27 [02:28:43] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.11, 11.80, 11.35 [02:30:39] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.01, 7.03, 6.40 [02:32:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.01, 11.65, 11.37 [02:32:34] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.03, 12.08, 11.47 [02:34:29] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.09, 11.99, 11.54 [02:35:06] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.70, 2.03, 1.61 [02:35:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.78, 3.70 [02:36:27] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.38, 7.27, 6.62 [02:37:06] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.81, 1.56, 1.49 [02:38:24] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.17, 7.12, 6.67 [02:38:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.06, 11.54, 11.96 [02:40:27] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.42, 1.56, 1.68 [02:42:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.33, 10.79, 10.92 [02:44:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.08, 11.45, 11.28 [02:44:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.63, 10.28, 10.72 [02:46:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.55, 6.50, 6.66 [02:46:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:47:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.40, 2.87, 3.35 [02:48:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.86, 11.24, 11.23 [02:48:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.72, 10.34, 10.86 [02:50:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.31, 1.90, 1.75 [02:50:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.01, 11.96, 11.50 [02:50:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.80, 10.69, 10.92 [02:51:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.54, 3.30, 3.43 [02:51:20] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.64, 10.77, 9.31 [02:52:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.60, 11.44, 11.18 [02:53:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.49, 2.93, 3.28 [02:54:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.83, 1.77 [02:54:26] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.83, 11.26, 11.36 [02:54:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.44, 8.92, 9.92 [02:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:55:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.75, 10.57, 9.62 [02:55:52] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.37, 9.51, 8.74 [02:57:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.01, 10.06, 9.56 [02:57:47] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.74, 8.74, 8.54 [02:58:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.56, 1.64, 1.70 [02:58:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.01, 11.87, 11.50 [02:59:48] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.09, 3.62, 3.51 [03:01:42] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.16, 3.09, 3.32 [03:06:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.13, 11.90, 11.88 [03:06:51] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.22, 6.96, 6.49 [03:06:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.94, 11.33, 11.91 [03:08:48] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.60, 6.51, 6.37 [03:08:52] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.73, 1.47 [03:08:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.29, 11.76, 11.97 [03:10:14] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.49, 4.38, 3.68 [03:10:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.22, 1.44, 1.39 [03:10:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.39, 11.11, 11.71 [03:12:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.96, 3.72, 3.52 [03:14:02] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.81, 3.57 [03:15:55] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.06, 3.64, 3.55 [03:16:03] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.39, 3.55, 2.21 [03:17:49] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.11, 3.04, 3.33 [03:19:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.59, 2.04, 1.75 [03:19:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:20:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.47, 3.36, 3.01 [03:20:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.89, 8.94, 10.18 [03:22:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.20, 3.36, 3.05 [03:23:08] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.95, 1.79 [03:24:02] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.88, 1.76, 1.92 [03:24:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.03, 9.39, 10.17 [03:24:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.78, 9.93, 10.29 [03:26:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:26:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.50, 3.98, 3.36 [03:28:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.81, 3.37 [03:28:02] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.69, 1.28, 1.69 [03:28:52] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.15, 3.10, 2.04 [03:28:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.53, 11.56, 10.81 [03:30:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.76, 3.39 [03:30:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.17, 10.83, 10.62 [03:31:17] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.65, 11.13, 10.63 [03:33:13] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.97, 11.24, 10.76 [03:34:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.57, 3.41 [03:34:52] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.10, 1.78, 1.77 [03:34:58] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.59, 1.69 [03:35:08] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.48, 9.41, 10.14 [03:36:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:36:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.79, 2.95, 3.20 [03:38:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.05, 1.48, 1.65 [03:38:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.71, 9.65, 10.11 [03:46:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.48, 3.82, 3.40 [03:48:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.13, 2.90 [03:48:47] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.59, 3.35, 3.28 [03:50:05] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.07, 2.81, 2.82 [03:50:41] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 2.20, 1.90 [03:54:38] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.42, 1.87, 1.84 [04:04:29] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.66, 1.60, 1.70 [04:04:32] PROBLEM - puzzles.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - puzzles.wiki All nameservers failed to answer the query. [04:08:24] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.75, 1.69, 1.73 [04:11:06] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.55, 3.56, 3.14 [04:13:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.57, 3.20 [04:14:58] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.25, 3.38, 3.17 [04:18:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.32, 1.84, 1.76 [04:20:52] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.27, 1.80, 1.40 [04:22:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.84, 1.96, 1.84 [04:22:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.30, 1.61, 1.39 [04:29:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:31:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.68, 3.52, 2.90 [04:31:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.67, 3.60, 3.04 [04:32:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.30, 1.53, 1.70 [04:33:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.65, 3.72, 3.07 [04:33:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.57, 3.10 [04:34:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:34:05] RECOVERY - puzzles.wiki - reverse DNS on sslhost is OK: SSL OK - puzzles.wiki reverse DNS resolves to cp21.miraheze.org - NS RECORDS OK [04:35:22] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.87, 3.37, 3.01 [04:35:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.68, 3.85, 3.24 [04:41:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:41:17] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.75, 3.44 [04:44:08] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.93, 6.48, 5.86 [04:46:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.95, 6.13, 5.80 [04:48:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.35, 1.90, 1.74 [04:49:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.81, 4.89, 4.00 [04:52:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.92, 1.81 [04:53:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.96, 3.83 [04:56:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:57:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.63, 4.71, 4.13 [04:58:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.46, 1.57, 1.68 [05:03:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.75, 3.95 [05:05:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.66, 4.60, 4.24 [05:09:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.60, 3.90 [05:11:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.81, 3.93 [05:13:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.42, 3.77 [05:15:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.59, 4.40, 4.09 [05:17:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.84, 3.89, 3.95 [05:19:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.79, 4.37, 4.10 [05:20:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 1.93, 1.71 [05:22:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.57, 1.84, 1.71 [05:26:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.39, 1.64, 1.66 [05:30:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.50, 3.24, 2.89 [05:32:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.71, 1.69 [05:32:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.26, 3.26, 2.94 [05:34:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.48, 1.64, 1.67 [05:39:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.39, 3.25, 3.93 [05:41:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.52, 4.56, 4.33 [05:45:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.46, 3.36, 3.89 [05:50:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.62, 2.09, 1.80 [05:53:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.21, 2.51, 3.29 [05:54:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.93, 1.81 [06:02:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.70, 9.23, 7.99 [06:04:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.54, 1.60, 1.70 [06:04:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.79, 8.60, 7.91 [06:08:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.88, 2.19, 1.91 [06:32:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.97, 10.57, 8.93 [06:32:58] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.40, 7.12, 6.47 [06:34:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.87, 10.99, 9.28 [06:34:44] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.24, 3.54, 2.93 [06:34:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.45, 3.62, 2.86 [06:34:54] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.41, 7.93, 6.84 [06:35:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.06, 3.72, 2.83 [06:36:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:37:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.31, 3.61, 2.91 [06:38:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.60, 9.56, 9.12 [06:38:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.13, 3.71, 3.13 [06:38:46] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 4.71, 6.88, 6.71 [06:39:23] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.15, 3.27, 2.87 [06:39:50] Reception123: mind merging https://github.com/miraheze/puppet/pull/2526? [06:39:51] [url] ImageMagic policy: add stealth to shared-secret and resource limits by Universal-Omega · Pull Request #2526 · miraheze/puppet · GitHub | github.com [06:40:32] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.93, 3.11, 2.98 [06:40:42] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.08, 3.09, 2.91 [06:40:42] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.26, 6.05, 6.43 [06:41:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:42:52] [02puppet] 07Universal-Omega edited pull request 03#2526: ImageMagick policy: add stealth to shared-secret and limit resource - 13https://github.com/miraheze/puppet/pull/2526 [06:42:52] [url] Page not found · GitHub · GitHub | github.com [06:43:34] [02puppet] 07Universal-Omega synchronize pull request 03#2526: ImageMagick policy: add stealth to shared-secret and limit resource - 13https://github.com/miraheze/puppet/pull/2526 [06:43:34] [url] Page not found · GitHub · GitHub | github.com [06:43:54] [02puppet] 07Universal-Omega synchronize pull request 03#2526: ImageMagick policy: add stealth to shared-secret and limit resource - 13https://github.com/miraheze/puppet/pull/2526 [06:43:54] ... [06:44:13] [02puppet] 07Universal-Omega edited pull request 03#2526: ImageMagick policy: add stealth to shared-secret and limit resources - 13https://github.com/miraheze/puppet/pull/2526 [06:44:13] [url] Page not found · GitHub · GitHub | github.com [06:50:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 16.93, 12.29, 9.36 [06:52:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.01, 10.65, 9.13 [06:54:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 5.92, 9.12, 8.76 [06:54:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:02:29] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.70, 10.41, 9.32 [07:06:25] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.69, 9.98, 9.42 [07:14:28] [02puppet] 07Reception123 closed pull request 03#2526: ImageMagick policy: add stealth to shared-secret and limit resources - 13https://github.com/miraheze/puppet/pull/2526 [07:14:28] [url] Page not found · GitHub · GitHub | github.com [07:14:29] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cf2f78a76e87...43ced668dcf2 [07:14:29] ^ CosmicAlpha [07:14:30] [url] Comparing cf2f78a76e87...43ced668dcf2 · miraheze/puppet · GitHub | github.com [07:14:31] [02miraheze/puppet] 07Universal-Omega 0343ced66 - ImageMagick policy: add stealth to shared-secret and limit resources (#2526) [07:24:04] Thanks Reception123! [07:24:28] no problem [07:27:59] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.85, 9.85, 9.05 [07:29:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.65, 8.65, 8.71 [08:18:54] PROBLEM - ping6 on ns1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 147.58 ms [08:26:35] PROBLEM - mon111 APT on mon111 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [08:28:36] PROBLEM - mon111 APT on mon111 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (4 critical updates). [08:29:00] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 102.77 ms [08:32:42] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [08:32:42] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:48:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:53:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:02:13] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [09:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:13:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:35:24] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.35, 2.66 [09:37:20] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.47, 2.91, 2.58 [09:47:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:52:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:58:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.53, 1.68, 1.99 [10:02:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:07:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:10:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.39, 1.43, 1.68 [10:20:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.40, 1.98, 1.80 [10:22:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.79, 1.75 [10:32:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.38, 1.60, 1.68 [10:49:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.96, 1.78 [10:51:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:52:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.56, 3.12, 2.72 [10:53:08] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.91, 1.82 [10:54:41] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.93, 2.81, 2.66 [10:56:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:02:10] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [11:08:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:08:54] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.84, 1.77 [11:10:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.79, 1.76 [11:13:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:18:46] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.03, 1.82, 1.76 [11:22:42] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.55, 1.80, 1.78 [11:26:39] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.59, 1.70 [11:31:41] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [11:31:41] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:32:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.05, 9.77, 8.49 [11:34:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.94, 9.80, 8.66 [11:49:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 1.88, 1.70 [11:53:14] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.68, 1.86, 1.74 [11:55:12] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.29, 1.68, 1.69 [11:57:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:01:11] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [12:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:02:32] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.27, 10.29, 9.57 [12:02:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.39, 3.11, 2.69 [12:04:27] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.08, 9.49, 9.36 [12:04:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.62, 3.08, 2.72 [12:06:26] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.90, 3.02, 2.74 [12:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:15:20] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.66, 10.79, 8.96 [12:17:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.07, 9.70, 8.79 [12:18:48] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.48, 1.99, 1.74 [12:22:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.23, 9.76, 8.95 [12:22:45] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.99, 1.82 [12:24:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.69, 9.56, 8.97 [12:29:39] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.45, 10.86, 10.16 [12:31:37] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.10, 10.12, 9.99 [12:32:40] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.66, 9.88, 9.49 [12:33:56] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.97, 10.66, 9.84 [12:35:31] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.19, 11.09, 10.41 [12:35:52] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.83, 10.06, 9.70 [12:37:29] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.96, 10.89, 10.41 [12:40:30] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.46, 1.56, 1.68 [12:42:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.81, 10.13, 9.91 [12:42:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.30, 10.57, 9.73 [12:44:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.35, 9.55, 9.44 [12:45:21] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.41, 9.80, 10.19 [12:47:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [12:48:21] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.43, 1.91, 1.77 [12:51:13] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.75, 11.25, 10.64 [12:53:11] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.32, 11.22, 10.71 [12:56:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.91, 1.88 [13:02:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:03:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.23, 9.46, 10.18 [13:06:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.30, 1.50, 1.69 [13:08:08] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.12, 6.99, 6.12 [13:08:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.02, 9.42, 8.83 [13:09:13] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.42, 10.00, 9.59 [13:10:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.44, 6.72, 6.12 [13:11:08] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.01, 10.43, 9.78 [13:12:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.22, 9.79, 9.15 [13:13:04] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.70, 10.12, 9.74 [13:15:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.87, 3.36 [13:17:19] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.71, 3.37 [13:20:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.46, 1.93, 1.74 [13:20:33] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.81, 1.55, 1.08 [13:21:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.84, 4.91, 3.88 [13:21:23] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.77, 3.39, 2.97 [13:21:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.76, 4.05, 3.39 [13:22:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:22:33] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.88, 1.29, 1.04 [13:23:03] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.28, 6.93, 6.29 [13:23:17] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.24, 3.35, 3.00 [13:24:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.81, 1.75 [13:24:59] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.09, 7.53, 6.57 [13:26:56] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.58, 7.12, 6.54 [13:27:05] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.36, 3.51, 3.14 [13:28:52] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.33, 7.46, 6.72 [13:28:54] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.32, 3.65, 3.78 [13:29:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.65, 2.91, 2.97 [13:29:26] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.73, 10.89, 10.20 [13:29:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.34, 3.49, 3.52 [13:30:19] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.39, 1.59, 1.67 [13:30:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.55, 9.98, 9.40 [13:30:48] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.47, 7.46, 6.81 [13:32:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.20, 10.97, 9.82 [13:33:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:34:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.94, 11.01, 9.99 [13:34:41] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.77, 6.69, 6.65 [13:35:12] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 7.94, 10.58, 10.45 [13:35:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.51, 2.56, 3.12 [13:36:38] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.12, 2.61, 3.28 [13:37:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.50, 10.38, 10.20 [13:37:35] PROBLEM - mw102 Disk Space on mw102 is WARNING: DISK WARNING - free space: / 2428 MB (10% inode=83%); [13:39:22] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.46, 10.08, 10.11 [13:44:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.89, 9.84, 10.03 [13:46:19] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.31, 3.69, 3.45 [13:48:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.35, 3.52, 3.42 [13:49:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.33, 11.31, 10.52 [13:50:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.58, 1.94, 1.67 [13:51:08] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.89, 11.06, 10.53 [13:53:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [13:54:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.66, 3.22, 3.35 [13:54:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.72, 1.96, 1.76 [13:54:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.83, 9.74, 10.15 [13:57:02] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.42, 11.90, 10.92 [13:58:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.36, 3.41 [13:58:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.79, 10.37, 10.35 [13:58:51] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.10, 1.90, 1.33 [13:59:00] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.51, 11.74, 10.99 [14:00:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.09, 3.26, 3.36 [14:00:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.02, 10.72, 10.46 [14:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:00:51] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.81, 1.47, 1.24 [14:02:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.47, 1.66, 1.70 [14:02:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.39, 11.13, 10.66 [14:06:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.58, 1.67, 1.71 [14:08:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.24, 1.55, 1.66 [14:08:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.42, 9.80, 10.18 [14:08:51] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.22, 1.76, 1.49 [14:08:55] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.51, 10.28, 9.86 [14:10:51] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.82, 1.43, 1.40 [14:12:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.27, 11.32, 10.68 [14:12:50] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.91, 11.73, 10.49 [14:14:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.93, 1.75, 1.70 [14:14:47] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.84, 10.62, 10.24 [14:15:38] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.47, 4.24, 3.60 [14:16:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.69, 1.70, 1.69 [14:16:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.22, 10.68, 10.65 [14:16:45] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.08, 9.66, 9.92 [14:16:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.69, 9.52, 10.15 [14:17:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.49, 3.11, 3.00 [14:17:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.76, 3.62, 3.45 [14:19:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.83, 2.80, 2.88 [14:20:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.72, 2.31, 1.95 [14:20:40] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.89, 11.09, 10.41 [14:21:24] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.13, 9.68, 8.76 [14:22:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.05, 10.64, 10.42 [14:23:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.00, 9.39, 8.77 [14:24:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.97, 1.91 [14:24:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 10.19, 9.75, 10.16 [14:24:34] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.82, 10.33, 10.30 [14:24:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.60, 9.77, 10.12 [14:25:18] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.88, 2.93, 3.24 [14:26:32] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.48, 9.29, 9.92 [14:28:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.01, 10.88, 10.43 [14:30:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:30:33] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.27, 2.01, 1.42 [14:30:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.27, 10.53, 10.35 [14:31:04] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.16, 3.48, 3.23 [14:32:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.42, 10.31, 10.12 [14:32:33] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.91, 1.58, 1.33 [14:33:02] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 3.62, 3.30 [14:33:33] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 86, in check_records cname = str(dns_resolver.resolve(hostname, 'CNAME')[0]) File "/usr/li [14:33:33] 3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:34:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.68, 11.49, 10.76 [14:35:01] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.47, 3.13, 3.15 [14:35:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:35:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.93, 10.24, 9.26 [14:36:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.64, 8.96, 9.66 [14:36:27] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.38, 10.23, 10.03 [14:36:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.07, 11.33, 10.80 [14:37:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.75, 9.90, 9.24 [14:38:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.10, 9.34, 9.72 [14:38:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.06, 3.44, 3.41 [14:40:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:40:52] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.34, 3.84, 3.54 [14:42:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.37, 9.27, 10.02 [14:44:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.41, 1.57, 1.70 [14:44:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.60, 9.61, 9.49 [14:44:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.05, 3.97, 3.50 [14:45:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [14:46:25] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.15, 9.27, 9.40 [14:47:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.60, 3.25 [14:48:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.29, 3.74, 3.52 [14:49:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.00, 2.96, 3.06 [14:50:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.24, 1.98, 1.84 [14:50:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.70, 10.15, 9.75 [14:50:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.77, 3.78 [14:50:46] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.39, 3.17, 3.33 [14:50:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.56, 10.20, 10.03 [14:52:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.67, 1.90, 1.83 [14:56:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.46, 10.95, 10.19 [14:57:06] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.03, 10.11, 9.87 [14:59:03] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.76, 10.99, 10.21 [15:00:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.50, 1.60, 1.70 [15:00:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.76, 11.22, 10.48 [15:00:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.86, 3.29, 3.19 [15:00:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.25, 10.07, 10.16 [15:02:06] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.59, 3.55 [15:02:15] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.75, 1.75, 1.27 [15:02:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.19, 12.23, 10.93 [15:02:58] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 8.97, 10.44, 10.22 [15:04:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.86, 3.50, 3.51 [15:04:15] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.65, 1.68, 1.30 [15:04:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.42, 11.79, 10.92 [15:04:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.94, 3.22, 3.20 [15:05:00] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.21, 6.51, 5.68 [15:06:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.54, 4.23, 3.75 [15:06:56] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.16, 6.13, 5.63 [15:06:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.06, 10.61, 10.36 [15:08:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.47, 3.83, 3.67 [15:08:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.86, 3.51 [15:12:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.49, 3.33, 3.38 [15:12:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.09, 9.21, 9.92 [15:14:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.82, 3.71 [15:14:42] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.30, 9.79, 10.12 [15:14:43] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.51, 7.07, 6.18 [15:16:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.76, 11.36, 11.03 [15:16:39] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.08, 6.69, 6.15 [15:20:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.63, 3.61, 3.72 [15:20:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.93, 1.73 [15:20:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.14, 10.57, 10.87 [15:20:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.35, 11.05, 10.37 [15:22:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.41, 3.96, 3.83 [15:22:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.86, 1.73 [15:24:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.80, 3.66, 3.74 [15:24:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.44, 1.69, 1.68 [15:24:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.60, 11.34, 10.70 [15:27:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.39, 5.04, 3.73 [15:28:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.93, 5.55, 4.45 [15:28:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 8.64, 5.42, 4.05 [15:29:24] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.99, 4.08, 2.54 [15:30:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.84, 11.16, 10.86 [15:32:35] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [15:34:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 14.14, 11.69, 10.92 [15:35:07] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.04, 10.92, 10.12 [15:36:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.71, 11.44, 11.22 [15:36:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.21, 11.33, 10.88 [15:37:04] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.88, 10.05, 9.90 [15:39:07] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.69, 1.66, 1.98 [15:40:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.23, 11.49, 11.03 [15:42:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.35, 11.81, 11.42 [15:42:53] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.39, 11.11, 10.40 [15:42:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.61, 11.58, 11.13 [15:43:00] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.49, 1.08, 1.67 [15:46:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.74, 1.71, 1.64 [15:48:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.32, 1.99, 1.76 [15:50:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.83, 10.25, 9.37 [15:50:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.84, 11.84, 11.75 [15:52:16] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.79, 10.03, 9.40 [15:54:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.65, 1.98, 1.87 [15:54:37] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.38, 9.62, 10.14 [15:58:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.26, 2.03, 1.91 [15:58:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.27, 11.43, 11.46 [15:58:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.01, 11.87, 11.29 [16:00:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.95, 1.90 [16:00:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.31, 11.64, 11.28 [16:01:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.56, 3.42, 3.94 [16:01:27] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 13.66, 11.29, 10.63 [16:02:05] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [16:02:05] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:03:25] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.35, 11.26, 10.70 [16:06:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.65, 11.12, 11.53 [16:09:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.94, 4.13, 4.01 [16:13:11] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.77, 11.22, 10.80 [16:14:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.16, 11.60, 11.48 [16:15:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.77, 3.89 [16:16:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.11, 1.76, 1.76 [16:16:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.06, 11.07, 11.31 [16:17:06] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.99, 11.09, 10.89 [16:19:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.99, 4.37, 4.06 [16:19:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.18, 2.13, 1.46 [16:20:14] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.93, 10.67, 9.83 [16:21:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.29, 3.56, 3.81 [16:21:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.71, 1.90, 1.46 [16:23:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.85, 1.55, 1.38 [16:24:05] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.64, 10.01, 9.78 [16:24:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.27, 3.71, 3.97 [16:24:55] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.08, 10.74, 10.70 [16:25:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 3.63, 3.75 [16:26:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.53, 1.97, 1.96 [16:26:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.14, 11.63, 11.38 [16:26:52] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.95, 10.90, 10.77 [16:27:00] !log [rhinos@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/populateContentTables.php --wiki=lakehubwiki (END - exit=0) [16:27:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:28:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.97, 11.51, 11.38 [16:29:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.48, 3.68, 3.75 [16:30:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.10, 4.76, 4.29 [16:31:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.95, 4.67, 4.09 [16:31:36] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [16:32:02] !log INSERT INTO content_models (model_id, model_name) VALUES ( 2, 'css'); INSERT INTO content_models (model_id, model_name) VALUES ( 3, 'javascript'); INSERT INTO content_models (model_id, model_name) VALUES ( 4, 'Scribunto'); INSERT INTO content_models (model_id, model_name) VALUES ( 5, 'json'); (lakehubwiki) [16:32:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.25, 12.12, 11.60 [16:34:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.50, 11.16, 10.80 [16:36:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.56, 10.40, 10.56 [16:37:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.18, 3.64, 3.89 [16:38:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.87, 11.60, 11.57 [16:39:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.34, 2.07, 1.66 [16:40:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.47, 3.48, 3.95 [16:41:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.10, 1.72, 1.58 [16:42:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.35, 1.49, 1.68 [16:42:31] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.98, 11.56, 10.99 [16:42:36] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:42:36] [url] Page not found · GitHub · GitHub | github.com [16:43:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.08, 1.51, 1.51 [16:44:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.72, 3.90, 4.00 [16:44:28] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.42, 11.04, 10.87 [16:45:14] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:45:14] [url] Page not found · GitHub · GitHub | github.com [16:45:45] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:45:45] ... [16:46:15] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:46:15] ... [16:46:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.34, 9.74, 10.15 [16:47:57] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:47:57] [url] Page not found · GitHub · GitHub | github.com [16:48:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.05, 3.66, 3.89 [16:49:52] [02puppet] 07Universal-Omega synchronize pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [16:49:53] ... [16:50:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.90, 1.79 [16:50:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.51, 3.75, 3.88 [16:51:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.53, 3.96, 3.85 [16:52:14] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.03, 6.81, 6.06 [16:52:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.97, 1.83 [16:52:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.21, 3.63, 3.83 [16:53:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.18, 3.73, 3.78 [16:54:10] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.47, 6.24, 5.94 [16:54:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.29, 10.70, 10.42 [16:56:16] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.32, 1.59, 1.70 [16:56:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.12, 3.91, 3.87 [17:00:08] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.37, 7.40, 6.52 [17:02:08] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.38, 7.04, 6.50 [17:02:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.73, 12.09, 11.74 [17:02:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.21, 9.71, 10.15 [17:03:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.54, 4.07, 3.91 [17:04:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.36, 11.96, 11.76 [17:04:27] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.76, 9.67, 10.20 [17:05:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.98, 3.89, 3.85 [17:06:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.48, 6.26, 6.33 [17:06:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.34, 10.66, 10.48 [17:07:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.14, 4.07, 3.93 [17:08:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.03, 9.88, 10.20 [17:10:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.82, 11.99, 11.75 [17:11:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.33, 3.67, 3.84 [17:12:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.48, 3.40, 3.91 [17:12:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.10, 3.30, 3.83 [17:12:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.29, 11.37, 11.54 [17:12:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.86, 10.75, 10.50 [17:14:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.45, 3.56, 3.89 [17:14:09] [02dns] 07MacFan4000 opened pull request 03#270: add datawiki.online zone - 13https://github.com/miraheze/dns/pull/270 [17:14:10] [url] Page not found · GitHub · GitHub | github.com [17:14:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.48, 9.62, 10.12 [17:16:17] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.61, 2.47, 3.37 [17:17:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.62, 9.76, 9.15 [17:18:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.91, 10.25, 10.25 [17:20:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.69, 2.00, 1.68 [17:20:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.45, 3.57 [17:20:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.46, 11.17, 10.60 [17:20:59] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.29, 7.26, 6.64 [17:21:20] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.05, 8.71, 8.91 [17:22:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.65, 3.20, 3.47 [17:22:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.16, 12.01, 11.63 [17:22:40] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+1/-0/±0] 13https://github.com/miraheze/ssl/commit/0b006fde8eec [17:22:42] [02miraheze/ssl] 07MacFan4000 030b006fd - add wiki.veloren.net cert per T9173 [17:22:43] [02ssl] 07MacFan4000 created branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [17:22:44] [url] Page not found · GitHub · GitHub | github.com [17:22:55] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.22, 6.81, 6.54 [17:24:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.77, 2.00, 1.77 [17:24:17] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.53, 3.09, 3.40 [17:24:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.69, 11.81, 11.60 [17:24:52] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.14, 6.57, 6.48 [17:24:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.80, 11.22, 10.77 [17:25:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.10, 2.98, 3.39 [17:26:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.76, 3.68, 3.96 [17:26:33] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03MacFan4000-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0b006fde8eec...81f2d294d357 [17:26:33] [url] Comparing 0b006fde8eec...81f2d294d357 · miraheze/ssl · GitHub | github.com [17:26:34] [02miraheze/ssl] 07MacFan4000 0381f2d29 - Update certs.yaml [17:27:03] [02ssl] 07MacFan4000 opened pull request 03#511: add wiki.veloren.net cert per T9173 - 13https://github.com/miraheze/ssl/pull/511 [17:27:03] [url] Page not found · GitHub · GitHub | github.com [17:28:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.32, 4.11, 4.07 [17:28:10] [02ssl] 07MacFan4000 closed pull request 03#511: add wiki.veloren.net cert per T9173 - 13https://github.com/miraheze/ssl/pull/511 [17:28:11] [url] Page not found · GitHub · GitHub | github.com [17:28:12] [02miraheze/ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 [17:28:13] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/8abc6ef06674...9de772a17c1d [17:28:14] [url] Comparing 8abc6ef06674...9de772a17c1d · miraheze/ssl · GitHub | github.com [17:28:15] [02miraheze/ssl] 07MacFan4000 039de772a - add wiki.veloren.net cert per T9173 (#511) [17:28:16] [02ssl] 07MacFan4000 deleted branch 03MacFan4000-patch-1 - 13https://github.com/miraheze/ssl [17:28:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.89, 1.77 [17:28:17] ... [17:29:52] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.75, 10.51, 10.23 [17:30:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.88, 1.90, 1.79 [17:30:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.34, 3.95, 3.67 [17:30:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.51, 11.84, 11.68 [17:31:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.02, 3.45, 3.48 [17:32:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.69, 3.94 [17:32:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.98, 3.73, 3.63 [17:32:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.75, 1.79, 1.51 [17:33:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.96, 3.91, 3.63 [17:34:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.35, 1.68, 1.51 [17:35:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.47, 3.51 [17:36:17] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.37, 3.01, 3.36 [17:36:31] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.72, 7.82, 6.86 [17:36:58] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.04, 11.15, 10.74 [17:38:27] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.10, 7.12, 6.73 [17:38:58] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.18, 10.16, 10.44 [17:39:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 10.85, 4.87, 3.93 [17:40:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.12, 4.18, 4.07 [17:40:23] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.37, 6.31, 6.48 [17:40:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.43, 11.71, 11.84 [17:42:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.95, 4.00 [17:42:58] [02puppet] 07Universal-Omega edited pull request 03#2446: T8848: add MediaWiki Icinga docs entries - 13https://github.com/miraheze/puppet/pull/2446 [17:42:58] [url] Page not found · GitHub · GitHub | github.com [17:43:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.68, 3.52, 3.58 [17:44:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 4.11, 4.05 [17:45:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.26, 3.07, 3.40 [17:46:15] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.62, 7.73, 7.04 [17:46:58] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.46, 9.69, 10.14 [17:48:16] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.68, 2.15, 1.93 [17:50:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.30, 3.89, 4.00 [17:50:25] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.60, 11.75, 11.63 [17:50:48] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.76, 1.53, 1.38 [17:52:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.81, 4.09, 4.05 [17:52:48] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.50, 1.55, 1.40 [17:53:41] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.70, 3.48 [17:53:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.45, 10.39, 10.29 [17:54:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.31, 6.18, 6.65 [17:54:11] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.23, 1.76, 1.39 [17:54:25] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.12, 10.94, 11.38 [17:55:10] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.98, 4.36, 3.72 [17:55:23] !log macfan@mw102:/$ sudo -u www-data rm -rf /tmp/* [17:55:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:55:35] RECOVERY - mw102 Disk Space on mw102 is OK: DISK OK - free space: / 12518 MB (56% inode=83%); [17:56:11] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.77, 1.39, 1.30 [18:00:45] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.67, 9.45, 10.06 [18:00:54] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.14, 10.97, 11.10 [18:01:28] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.88, 1.96 [18:02:52] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.52, 10.51, 10.93 [18:03:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.32, 9.81, 10.17 [18:05:21] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.31, 1.96, 1.96 [18:06:59] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.31, 11.21, 10.62 [18:08:54] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.99, 10.74, 10.50 [18:10:30] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 1.90, 2.17, 1.37 [18:12:28] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.50, 1.57, 1.24 [18:13:39] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:14:42] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.83, 9.31, 10.13 [18:15:38] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14640 bytes in 0.311 second response time [18:17:09] RECOVERY - mw121 Puppet on mw121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:18:24] PROBLEM - matomo101 APT on matomo101 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [18:18:31] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.36, 9.53, 10.05 [18:20:22] PROBLEM - matomo101 APT on matomo101 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (4 critical updates). [18:21:08] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.13, 10.00, 9.27 [18:22:22] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 12.00, 10.88, 10.50 [18:23:03] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 11.75, 10.75, 9.64 [18:26:53] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.31, 11.00, 9.94 [18:28:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.02, 11.17, 10.70 [18:28:28] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.36, 11.12, 10.47 [18:28:48] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.02, 10.67, 9.95 [18:30:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.36, 11.20, 10.77 [18:30:43] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.83, 9.92, 9.75 [18:32:17] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.61, 10.45, 9.86 [18:32:24] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.88, 11.89, 10.93 [18:33:51] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.42, 11.77, 11.08 [18:34:07] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.70, 2.83, 1.85 [18:34:11] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.11, 9.23, 9.48 [18:35:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.14, 11.56, 11.09 [18:36:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.38, 3.31, 3.81 [18:36:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.21, 3.34, 3.97 [18:38:04] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.01, 1.96, 1.73 [18:38:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.78, 4.04, 4.02 [18:38:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.03, 3.59, 3.98 [18:40:02] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.19, 1.70, 1.66 [18:40:44] [02puppet] 07Universal-Omega opened pull request 03#2527: MediaWiki: add hourly cron to delete tmp files (T9175) - 13https://github.com/miraheze/puppet/pull/2527 [18:40:45] [url] Page not found · GitHub · GitHub | github.com [18:41:32] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:42:20] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.12, 6.50, 6.21 [18:42:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.70, 3.93 [18:42:40] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.66, 3.93 [18:43:33] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22334 bytes in 4.937 second response time [18:43:37] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.96, 11.57, 11.13 [18:43:58] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 1.58, 2.01, 1.80 [18:45:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [18:45:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.15, 10.48, 10.80 [18:45:57] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.48, 1.51, 1.65 [18:46:13] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.06, 11.74, 10.93 [18:48:06] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.49, 6.49, 6.31 [18:50:09] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.80, 11.14, 10.91 [18:53:17] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:53:33] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:53:34] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:53:34] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:54:06] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.32, 9.03, 10.14 [18:54:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.59, 2.88, 3.37 [18:55:13] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 0.019 second response time [18:55:30] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14634 bytes in 0.022 second response time [18:55:32] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 1.707 second response time [18:55:34] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14626 bytes in 1.161 second response time [18:56:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 3.57, 3.58 [18:58:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.42, 3.89, 3.68 [18:58:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.62, 3.49, 3.54 [19:00:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.15, 3.27, 3.49 [19:00:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.72, 2.87, 3.31 [19:01:37] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.72, 11.32, 10.90 [19:01:58] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.56, 10.09, 10.15 [19:02:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.13, 2.99, 3.36 [19:03:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.78, 10.29, 10.57 [19:04:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.64, 3.56, 3.45 [19:05:55] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.42, 9.42, 9.91 [19:06:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.19, 3.86, 3.60 [19:10:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.92, 3.84, 3.65 [19:12:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.96, 4.21, 3.80 [19:14:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.36, 3.91, 3.74 [19:14:49] [02puppet] 07Universal-Omega synchronize pull request 03#2527: MediaWiki: add hourly cron to delete tmp files (T9175) - 13https://github.com/miraheze/puppet/pull/2527 [19:14:49] [url] Page not found · GitHub · GitHub | github.com [19:15:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:16:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.39, 4.39, 3.93 [19:16:49] [02puppet] 07paladox closed pull request 03#2527: MediaWiki: add hourly cron to delete tmp files (T9175) - 13https://github.com/miraheze/puppet/pull/2527 [19:16:49] [url] Page not found · GitHub · GitHub | github.com [19:16:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/43ced668dcf2...7d537b8e7441 [19:16:51] [url] Comparing 43ced668dcf2...7d537b8e7441 · miraheze/puppet · GitHub | github.com [19:16:52] [02miraheze/puppet] 07Universal-Omega 037d537b8 - MediaWiki: add hourly cron to delete tmp files (T9175) (#2527) [19:17:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.13, 10.22, 9.81 [19:18:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.99, 3.85 [19:18:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.63, 3.75, 3.81 [19:19:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.65, 9.03, 9.43 [19:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [19:24:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.67, 2.83, 3.40 [19:25:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.76, 3.32, 3.94 [19:29:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.03, 3.55, 3.83 [19:32:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.88, 3.48, 3.41 [19:32:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.63, 4.26, 3.91 [19:34:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.92, 3.83 [19:35:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.40, 12.15, 10.53 [19:36:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.60, 3.17, 3.31 [19:37:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.16, 11.03, 10.31 [19:39:37] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.12, 11.26, 10.74 [19:39:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.79, 12.05, 10.76 [19:40:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.66, 3.72, 3.51 [19:40:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.12, 3.74, 3.75 [19:41:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.79, 11.09, 10.57 [19:42:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.09, 3.38, 3.61 [19:43:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.93, 10.68, 10.74 [19:47:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.23, 10.97, 10.53 [19:48:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.41, 3.01, 3.29 [19:49:37] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.49, 10.89, 10.65 [19:49:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.52, 3.74, 3.95 [19:49:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.72, 11.08, 10.62 [19:51:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.74, 10.62, 10.58 [19:52:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.52, 3.58, 3.52 [19:53:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.45, 4.34, 4.12 [19:53:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.17, 9.48, 10.08 [19:55:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.91, 4.00 [19:56:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.95, 3.41, 3.48 [19:57:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.10, 4.32, 4.13 [19:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:00:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.78, 3.19, 3.38 [20:00:45] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/46d237afd0f6 [20:00:47] [02miraheze/mw-config] 07Universal-Omega 0346d237a - Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki [20:00:48] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [20:00:49] [url] Page not found · GitHub · GitHub | github.com [20:00:50] [02mw-config] 07Universal-Omega opened pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:00:50] ... [20:01:22] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/46d237afd0f6...113b40df81d9 [20:01:23] [url] Comparing 46d237afd0f6...113b40df81d9 · miraheze/mw-config · GitHub | github.com [20:01:24] [02miraheze/mw-config] 07github-actions 03113b40d - CI: lint code to MediaWiki standards [20:01:25] [02mw-config] 07github-actions[bot] synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:01:25] ... [20:02:02] miraheze/mw-config - Universal-Omega the build passed. [20:04:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.58, 4.18, 3.76 [20:05:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.89, 3.98, 3.55 [20:06:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.97, 3.58, 3.58 [20:07:05] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.31, 3.32, 3.36 [20:07:37] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.42, 9.58, 10.10 [20:08:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.36, 3.76, 3.64 [20:09:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.93, 3.57, 3.92 [20:10:13] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.05, 6.47, 5.63 [20:10:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.68, 3.21, 3.45 [20:11:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.15, 3.49, 3.83 [20:12:08] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.75, 6.47, 5.72 [20:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [20:12:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.49, 3.66, 3.58 [20:13:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.17, 3.39, 3.75 [20:14:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.82, 2.94, 3.32 [20:15:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.66, 1.97, 1.47 [20:16:59] [02mw-config] 07Universal-Omega synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:16:59] [url] Page not found · GitHub · GitHub | github.com [20:17:00] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/113b40df81d9...47c1c5aa9057 [20:17:01] [url] Comparing 113b40df81d9...47c1c5aa9057 · miraheze/mw-config · GitHub | github.com [20:17:02] [02miraheze/mw-config] 07Universal-Omega 0347c1c5a - Use InitializeArticleMaybeRedirect hook [20:17:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.95, 4.30, 3.98 [20:18:18] miraheze/mw-config - Universal-Omega the build passed. [20:18:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.25, 3.71, 3.38 [20:18:38] [02mw-config] 07Universal-Omega synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:18:39] [url] Page not found · GitHub · GitHub | github.com [20:18:40] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/47c1c5aa9057...5dc073a9291c [20:18:41] [url] Comparing 47c1c5aa9057...5dc073a9291c · miraheze/mw-config · GitHub | github.com [20:18:41] [02miraheze/mw-config] 07Universal-Omega 035dc073a - Update LocalWiki.php [20:19:10] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.36, 1.33 [20:19:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.63, 3.64, 3.79 [20:19:49] miraheze/mw-config - Universal-Omega the build passed. [20:20:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.48, 3.34 [20:21:08] [02mw-config] 07Universal-Omega synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:21:08] [url] Page not found · GitHub · GitHub | github.com [20:21:09] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5dc073a9291c...59dbd336a8cf [20:21:10] [url] Comparing 5dc073a9291c...59dbd336a8cf · miraheze/mw-config · GitHub | github.com [20:21:11] [02miraheze/mw-config] 07Universal-Omega 0359dbd33 - Update LocalWiki.php [20:21:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.87, 10.17, 9.70 [20:22:25] miraheze/mw-config - Universal-Omega the build passed. [20:22:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.26, 3.33, 3.29 [20:25:29] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.42, 3.43 [20:25:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.36, 3.94, 3.86 [20:26:53] [02mw-config] 07Universal-Omega synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [20:26:53] [url] Page not found · GitHub · GitHub | github.com [20:26:54] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/59dbd336a8cf...2659017c99da [20:26:55] [url] Comparing 59dbd336a8cf...2659017c99da · miraheze/mw-config · GitHub | github.com [20:26:56] [02miraheze/mw-config] 07Universal-Omega 032659017 - Update LocalWiki.php [20:27:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.96, 3.18, 3.33 [20:27:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.52, 3.42, 3.68 [20:27:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.73, 8.99, 9.43 [20:27:59] miraheze/mw-config - Universal-Omega the build passed. [20:31:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.42, 10.83, 10.03 [20:33:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.34, 9.47, 9.63 [20:33:58] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.43, 1.87, 1.17 [20:35:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.91, 2.82, 3.36 [20:35:58] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.45, 1.36, 1.07 [20:37:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.57, 10.29, 9.94 [20:39:30] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.70, 10.29, 9.33 [20:41:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.64, 10.87, 9.68 [20:41:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.36, 9.43, 9.75 [20:43:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.29, 10.18, 9.56 [20:43:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 3.68, 3.49 [20:47:52] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.67, 3.90, 3.47 [20:48:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.53, 3.68, 3.20 [20:49:50] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.86, 3.40, 3.33 [20:50:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.93, 3.34, 3.13 [20:51:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.42, 10.79, 10.03 [20:55:37] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.67, 10.04, 9.93 [20:55:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.61, 4.15, 3.59 [20:58:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.18, 4.74, 3.80 [20:59:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.04, 10.29, 10.07 [20:59:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.28, 10.39, 9.99 [21:01:32] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.21, 3.96, 3.75 [21:01:37] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.17, 10.14, 10.03 [21:01:46] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 13.91, 11.29, 10.34 [21:02:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.54, 3.96, 3.73 [21:03:47] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.37, 10.86, 10.30 [21:05:27] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.19, 3.06, 3.39 [21:07:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:07:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.19, 3.19, 3.83 [21:07:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.16, 9.46, 9.86 [21:08:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.21, 3.11, 3.37 [21:11:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 3.49, 3.80 [21:13:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.93, 3.46, 3.74 [21:17:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.69, 3.74, 3.78 [21:21:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.23, 3.84, 3.83 [21:23:42] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [21:31:39] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.39, 3.83, 3.75 [21:32:33] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.76, 10.15, 9.37 [21:33:39] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.54, 3.43, 3.60 [21:34:32] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.79, 9.43, 9.19 [21:38:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:48:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [21:49:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.33, 3.00, 3.31 [21:51:42] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:02:07] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.50, 9.45, 8.78 [22:04:06] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.77, 8.91, 8.67 [22:06:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.50, 3.30 [22:08:09] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.85, 3.21, 3.21 [22:11:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:12:45] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.49, 1.55, 1.99 [22:16:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:16:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.90, 3.47, 3.28 [22:18:45] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.51, 1.89, 1.97 [22:20:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.58, 4.12, 3.58 [22:20:59] [02miraheze/CreateWiki] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/stylelint-config-wikimedia-0.13.0 [+0/-0/±2] 13https://github.com/miraheze/CreateWiki/commit/f2240cac72c2 [22:21:01] [02miraheze/CreateWiki] 07dependabot[bot] 03f2240ca - Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 [22:21:02] [02CreateWiki] 07dependabot[bot] opened pull request 03#319: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/CreateWiki/pull/319 [22:21:03] [url] Page not found · GitHub · GitHub | github.com [22:21:04] [02CreateWiki] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/stylelint-config-wikimedia-0.13.0 - 13https://github.com/miraheze/CreateWiki [22:21:04] [url] Page not found · GitHub · GitHub | github.com [22:21:05] [02CreateWiki] 07dependabot[bot] labeled pull request 03#319: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/CreateWiki/pull/319 [22:21:05] ... [22:21:07] [02CreateWiki] 07dependabot[bot] labeled pull request 03#319: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/CreateWiki/pull/319 [22:21:07] [url] Page not found · GitHub · GitHub | github.com [22:21:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:22:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.49, 3.60, 3.46 [22:22:45] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.97, 2.00 [22:23:55] [02miraheze/IncidentReporting] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/stylelint-config-wikimedia-0.13.0 [+0/-0/±2] 13https://github.com/miraheze/IncidentReporting/commit/73cbb046b53b [22:23:57] [02miraheze/IncidentReporting] 07dependabot[bot] 0373cbb04 - Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 [22:23:58] [02IncidentReporting] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/stylelint-config-wikimedia-0.13.0 - 13https://github.com/miraheze/IncidentReporting [22:23:58] [url] Page not found · GitHub · GitHub | github.com [22:24:00] [02IncidentReporting] 07dependabot[bot] opened pull request 03#41: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/IncidentReporting/pull/41 [22:24:00] ... [22:24:01] [02IncidentReporting] 07dependabot[bot] labeled pull request 03#41: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/IncidentReporting/pull/41 [22:24:01] ... [22:24:03] [02IncidentReporting] 07dependabot[bot] labeled pull request 03#41: Bump stylelint-config-wikimedia from 0.12.2 to 0.13.0 - 13https://github.com/miraheze/IncidentReporting/pull/41 [22:24:03] [url] Page not found · GitHub · GitHub | github.com [22:24:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.63, 3.34, 3.38 [22:26:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [22:28:18] miraheze/IncidentReporting - dependabot[bot] the build passed. [22:28:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.95, 3.72, 3.51 [22:28:40] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.15, 3.46, 3.07 [22:30:16] miraheze/CreateWiki - dependabot[bot] the build passed. [22:30:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 3.46, 3.11 [22:32:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.85, 3.18, 3.33 [22:32:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.16, 3.25, 3.07 [22:36:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.32, 4.50, 3.59 [22:38:45] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.43, 1.51, 1.67 [22:40:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.63, 3.52 [22:43:10] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [22:44:00] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.91, 2.97, 3.29 [22:46:39] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.39, 3.68, 3.61 [22:48:39] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.76, 3.00, 3.38 [22:49:33] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.51, 2.03, 1.81 [22:49:58] [02puppet] 07Universal-Omega opened pull request 03#2528: MediaWiki: include svg_* in the clean-tmp-files cron - 13https://github.com/miraheze/puppet/pull/2528 [22:49:58] [url] Page not found · GitHub · GitHub | github.com [22:53:27] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 1.94, 1.84 [22:56:18] !log [universalomega@test101] starting deploy of {'config': True} to all [22:56:19] !log [universalomega@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [22:56:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:57:20] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.06, 1.86, 1.82 [22:59:16] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.88, 1.94, 1.86 [23:00:02] !log [@test101] starting deploy of {'l10nupdate': True} to all [23:00:03] !log [@mwtask111] starting deploy of {'l10nupdate': True} to all [23:00:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:03:09] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.74, 2.30, 2.01 [23:05:06] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.47, 1.96, 1.92 [23:05:16] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [23:05:16] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:05:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:07:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 1.99, 2.11, 1.98 [23:09:10] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 second ago with 0 failures [23:12:10] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.10, 3.90, 3.20 [23:12:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:12:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.94, 3.77, 2.89 [23:12:43] !log [@mwtask111] finished deploy of {'l10nupdate': True} to all - SUCCESS in 760s [23:12:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:13:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.86, 4.58, 3.62 [23:14:08] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.68, 3.45, 3.12 [23:14:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.19, 3.18, 2.79 [23:16:05] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.76, 3.23, 3.08 [23:17:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 4.00, 3.62 [23:17:31] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:20:17] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.11, 9.52, 8.96 [23:21:09] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.33, 3.04, 3.30 [23:21:46] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.38, 10.25, 9.01 [23:23:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:24:55] !log [@test101] starting deploy of {'config': True} to all [23:24:57] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 2s [23:25:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:25:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.44, 9.94, 9.32 [23:27:49] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.11, 6.78, 5.45 [23:27:56] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.52, 10.09, 9.54 [23:29:49] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.36, 6.84, 5.64 [23:31:49] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.11, 6.56, 5.68 [23:32:25] !log [@test101] finished deploy of {'l10nupdate': True} to all - SUCCESS in 1942s [23:32:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:32:45] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.67, 3.89, 3.50 [23:38:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:38:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.55, 3.62, 3.17 [23:40:55] [02mw-config] 07Universal-Omega synchronize pull request 03#4662: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [23:40:55] [url] Page not found · GitHub · GitHub | github.com [23:40:56] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2659017c99da...d17814e22870 [23:40:57] [url] Comparing 2659017c99da...d17814e22870 · miraheze/mw-config · GitHub | github.com [23:40:58] [02miraheze/mw-config] 07Universal-Omega 03d17814e - Use BeforeInitialize hook [23:42:06] miraheze/mw-config - Universal-Omega the build passed. [23:42:13] [02mw-config] 07Universal-Omega edited pull request 03#4662: T9101: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [23:42:13] ... [23:42:23] [02mw-config] 07Universal-Omega closed pull request 03#4662: T9101: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki - 13https://github.com/miraheze/mw-config/pull/4662 [23:42:23] [url] Page not found · GitHub · GitHub | github.com [23:42:24] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/fc6dcef91771...4546c6400bbe [23:42:25] [url] Comparing fc6dcef91771...4546c6400bbe · miraheze/mw-config · GitHub | github.com [23:42:26] [02miraheze/mw-config] 07Universal-Omega 034546c64 - T9101: Redirect USER_WIKI_TALK to USER_TALK for polandballruwiki (#4662) [23:42:27] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/mw-config [23:42:28] [url] Page not found · GitHub · GitHub | github.com [23:42:29] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [23:42:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.87, 3.20, 3.15 [23:42:48] !log [universalomega@test101] starting deploy of {'pull': 'config', 'config': True} to all [23:42:52] !log [universalomega@test101] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 4s [23:42:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:42:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:43:25] miraheze/mw-config - Universal-Omega the build passed. [23:43:54] !log [universalomega@test101] starting deploy of {'pull': 'config', 'config': True, 'ignoretime': True} to all [23:43:56] !log [universalomega@test101] finished deploy of {'pull': 'config', 'config': True, 'ignoretime': True} to all - SUCCESS in 1s [23:44:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:44:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:48] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=criticalrolewiki --update (END - exit=0) [23:47:49] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.86, 6.64, 6.02 [23:47:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:49] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.85, 6.25, 5.95 [23:50:01] alerting : [FIRING:1] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [23:50:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.46, 3.50, 3.78 [23:52:43] !log [@mwtask111] starting deploy of {'config': True} to all [23:53:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:01] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 17s [23:53:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:55:01] ok : [RESOLVED] (PHP-FPM Worker Usage High mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki