[00:01:39] RECOVERY - matomo101 Disk Space on matomo101 is OK: DISK OK - free space: / 1758 MB (18% inode=80%); [00:03:39] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:03:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.69, 4.29, 3.36 [00:04:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.13, 4.83, 3.80 [00:04:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.40, 4.46, 3.42 [00:07:47] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 8.98, 8.37 [00:09:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.84, 8.91, 8.95 [00:09:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.69, 9.46, 9.24 [00:11:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.39, 9.17, 9.05 [00:11:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.58, 9.47, 9.27 [00:11:44] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [00:13:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.30, 9.09, 8.89 [00:13:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.89, 9.75, 9.28 [00:15:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.76, 9.39, 9.03 [00:15:27] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.82, 9.41, 8.79 [00:17:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.07, 9.67, 9.43 [00:17:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.45, 9.82, 9.43 [00:19:16] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.32, 3.42, 3.72 [00:20:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.54, 3.88 [00:21:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.36, 10.07, 9.61 [00:23:07] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.36, 3.62, 3.72 [00:23:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.78, 9.59, 9.50 [00:25:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 4.01, 8.22, 8.93 [00:25:07] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.07, 3.20, 3.56 [00:25:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.09, 7.31, 8.27 [00:25:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 5.60, 8.82, 9.45 [00:25:30] SRE, persistent 502s on `metawiki` [00:25:44] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:52] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:54] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:02] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.58, 2.87, 3.83 [00:26:16] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:19] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:24] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:27] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:28] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:29] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:34] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:39] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:45] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:51] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:54] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:56] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:56] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:56] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:26:57] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:27:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.83, 7.23, 8.46 [00:27:04] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:27:24] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 5.59, 6.76, 8.31 [00:27:44] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 5.162 second response time [00:27:49] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 1.817 second response time [00:27:57] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 5.361 second response time [00:28:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.90, 2.72, 3.33 [00:28:05] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 5.505 second response time [00:28:13] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.316 second response time [00:28:16] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.564 second response time [00:28:21] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.733 second response time [00:28:24] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.470 second response time [00:28:26] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 0.339 second response time [00:28:29] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 1.119 second response time [00:28:34] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 0.888 second response time [00:28:39] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14548 bytes in 0.574 second response time [00:28:45] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 0.570 second response time [00:28:51] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 14548 bytes in 1.393 second response time [00:28:53] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 14548 bytes in 3.231 second response time [00:28:54] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 0.943 second response time [00:28:55] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14556 bytes in 2.881 second response time [00:28:58] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 22336 bytes in 2.283 second response time [00:29:00] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 14548 bytes in 4.498 second response time [00:29:02] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14562 bytes in 0.744 second response time [00:29:07] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.19, 3.37, 3.51 [00:29:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.63, 8.65, 8.54 [00:31:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.50, 8.03, 8.52 [00:31:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.86, 3.07, 3.38 [00:31:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.57, 8.28, 8.43 [00:31:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.50, 9.52, 9.44 [00:31:39] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [00:32:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.18, 3.42 [00:32:19] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.29, 8.19, 7.24 [00:33:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.21, 7.67, 8.32 [00:33:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.99, 9.62, 9.52 [00:34:18] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.81, 8.00, 7.28 [00:35:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.09, 9.04, 8.72 [00:36:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.59, 3.96, 3.68 [00:37:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.87, 8.33, 8.43 [00:37:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.43, 9.90, 9.63 [00:38:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.36, 3.40, 3.60 [00:39:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.95, 8.13, 8.34 [00:39:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.06, 9.18, 9.40 [00:39:43] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [00:39:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.13, 4.02, 3.66 [00:40:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.81, 3.54, 3.63 [00:41:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.28, 9.23, 8.84 [00:41:49] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.76, 3.57, 3.54 [00:41:49] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.46, 8.80, 8.37 [00:42:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.22, 3.77, 3.72 [00:42:41] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.66, 8.73, 7.61 [00:43:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.20, 8.75, 8.59 [00:43:44] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.23, 9.12, 8.56 [00:44:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 4.09, 3.85 [00:44:15] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.04, 8.14, 7.51 [00:45:38] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.70, 8.34, 8.34 [00:46:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.84, 3.61, 3.70 [00:47:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.26, 8.41, 8.49 [00:47:35] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.17, 3.16, 3.36 [00:48:31] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.42, 8.28, 7.77 [00:49:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.16, 9.78, 9.45 [00:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.16, 9.82, 9.51 [00:50:14] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.06, 8.43, 7.88 [00:51:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.98, 9.78, 9.53 [00:52:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 3.46, 3.57 [00:52:14] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.35, 3.26, 3.40 [00:53:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.63, 9.30, 8.87 [00:54:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.93, 3.07, 3.41 [00:56:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.41, 2.83, 3.28 [00:58:13] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.35, 8.76, 8.24 [00:59:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.61, 9.72, 9.54 [00:59:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.99, 8.91, 8.60 [01:01:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.45, 9.44, 9.48 [01:01:24] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.83, 8.23, 8.40 [01:03:04] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.18, 3.72, 3.50 [01:03:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.69, 9.71, 9.46 [01:04:12] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 10.14, 8.81, 8.37 [01:05:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.43, 9.74, 9.57 [01:05:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.33, 8.81, 9.17 [01:06:12] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.66, 8.87, 8.47 [01:06:57] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.87, 3.78, 3.56 [01:07:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.19, 9.01, 9.32 [01:08:11] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.67, 8.48, 8.38 [01:09:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.49, 9.32, 9.27 [01:11:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.96, 9.69, 9.43 [01:12:13] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.75, 8.55, 8.36 [01:14:07] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.86, 9.61, 8.78 [01:14:09] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.68, 8.91, 8.53 [01:14:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.20, 4.71, 3.91 [01:15:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.92, 7.91, 8.50 [01:15:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.21, 9.29, 9.30 [01:15:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.34, 9.86, 9.54 [01:16:25] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.71, 3.33, 3.19 [01:16:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.24, 4.14, 3.63 [01:17:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.72, 9.45, 9.36 [01:17:22] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.98, 1.52, 1.11 [01:17:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.36, 9.44, 9.42 [01:17:55] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.83, 9.24, 8.81 [01:18:20] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.12, 3.75, 3.37 [01:18:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.96, 4.00, 3.81 [01:18:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.33, 3.94, 3.63 [01:19:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.09, 9.11, 8.91 [01:19:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.75, 1.19, 1.04 [01:19:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.39, 10.52, 9.84 [01:20:16] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.12, 3.11, 3.17 [01:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.92, 9.81, 9.66 [01:21:44] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.96, 9.65, 9.03 [01:22:28] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.19, 3.04, 3.33 [01:23:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.14, 9.24, 8.96 [01:23:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.10, 10.31, 9.86 [01:24:07] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.21, 8.26, 8.45 [01:25:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.31, 8.56, 8.75 [01:25:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.96, 9.86, 9.75 [01:25:32] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.75, 9.84, 9.25 [01:27:26] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.32, 10.22, 9.45 [01:28:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.21, 2.99, 3.37 [01:28:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:29:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.57, 9.12, 8.89 [01:31:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.55, 8.41, 8.65 [01:33:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.77, 9.46, 9.15 [01:33:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.93, 9.56, 9.42 [01:33:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.17, 9.63, 9.67 [01:33:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:37:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.78, 9.99, 9.61 [01:39:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.40, 8.07, 8.45 [01:41:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.20, 9.43, 9.41 [01:41:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.89, 9.90, 9.91 [01:42:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.27, 3.65, 3.22 [01:42:01] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.82, 1.40, 1.22 [01:43:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.92, 8.73, 8.67 [01:43:02] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.11, 8.36, 8.28 [01:44:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.77, 3.33 [01:45:01] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.25, 8.45, 8.33 [01:45:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.52, 9.89, 9.85 [01:45:59] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.59, 1.49, 1.29 [01:46:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.86, 3.51 [01:47:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.21, 7.83, 8.32 [01:47:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.56, 10.19, 9.96 [01:47:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.25, 9.90, 9.85 [01:48:14] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.47, 3.29, 3.34 [01:50:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.83, 2.95, 3.15 [01:51:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.11, 9.79, 9.87 [01:53:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.54, 8.33, 8.36 [01:53:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.73, 9.49, 9.84 [01:55:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.12, 9.18, 8.68 [01:57:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.14, 9.78, 9.81 [01:59:07] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.22, 3.60, 3.08 [01:59:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.34, 9.87, 9.24 [01:59:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.61, 9.72, 9.78 [02:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.03, 9.26, 8.96 [02:01:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.43, 3.14, 2.97 [02:01:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.59, 9.23, 9.08 [02:01:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.12, 9.87, 9.83 [02:03:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.18, 9.89, 9.22 [02:03:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.23, 9.48, 9.18 [02:03:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.84, 9.81, 9.81 [02:03:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.11, 10.25, 9.94 [02:05:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.12, 9.97, 9.87 [02:07:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.49, 9.61, 9.30 [02:07:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.01, 4.03, 3.55 [02:07:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.75, 9.71, 9.86 [02:09:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.91, 9.28, 9.28 [02:09:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.63, 3.32 [02:09:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.33, 3.96, 3.60 [02:11:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.84, 9.88, 9.98 [02:13:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.46, 3.26, 3.25 [02:13:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.25, 9.41, 9.63 [02:14:58] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.07, 3.00, 3.29 [02:17:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.22, 9.68, 9.41 [02:17:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.22, 9.76, 9.83 [02:17:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.06, 9.38, 9.59 [02:19:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.43, 9.52, 9.25 [02:19:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.21, 9.03, 9.19 [02:19:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.46, 9.63, 9.78 [02:19:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.96, 10.36, 9.92 [02:21:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.72, 9.03, 9.09 [02:21:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.23, 9.74, 9.80 [02:21:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.50, 9.61, 9.72 [02:21:45] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.88, 1.68, 1.23 [02:23:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.31, 9.47, 9.24 [02:23:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.30, 9.39, 9.65 [02:23:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.87, 1.39, 1.18 [02:25:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.92, 9.37, 9.24 [02:25:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.15, 10.53, 9.73 [02:25:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.23, 9.77, 9.77 [02:25:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.03, 10.52, 10.04 [02:26:51] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.56, 8.16, 7.95 [02:27:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.53, 9.86, 9.45 [02:27:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.24, 9.36, 9.63 [02:28:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.33, 7.80, 7.85 [02:29:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.63, 9.39, 9.30 [02:29:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.31, 9.41, 9.72 [02:29:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.82, 3.56, 3.09 [02:33:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.05, 9.30, 9.25 [02:33:22] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.00, 3.04, 3.01 [02:35:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.67, 9.33, 9.27 [02:35:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.62, 9.11, 9.62 [02:35:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.85, 9.07, 9.28 [02:37:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.60, 3.38 [02:37:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.42, 8.47, 9.05 [02:43:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 5.67, 6.95, 8.14 [02:43:24] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.99, 7.72, 8.41 [02:43:39] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:45:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.44, 7.49, 8.42 [02:47:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.86, 3.58 [02:48:59] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.75, 3.37, 3.43 [02:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.09, 8.77, 8.71 [02:49:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:50:55] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.54, 3.47 [02:51:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.22, 8.63, 8.67 [02:52:51] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.07, 3.32, 3.40 [02:53:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.14, 9.80, 9.10 [02:55:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 8.38, 8.25 [02:57:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.70, 9.78, 9.27 [02:59:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.11, 9.60, 8.71 [02:59:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.24, 9.66, 9.27 [03:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.23, 8.98, 8.59 [03:01:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.03, 9.48, 9.26 [03:03:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.18, 8.28, 8.37 [03:03:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.24, 9.02, 9.00 [03:04:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.06, 3.52, 3.42 [03:05:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.38, 8.65, 8.89 [03:05:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.53, 9.39, 9.23 [03:06:30] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.65, 2.87, 3.20 [03:07:14] PROBLEM - www.marinebiodiversitymatrix.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'marinebiodiversitymatrix.org' expires in 15 day(s) (Tue 29 Mar 2022 02:53:19 GMT +0000). [03:07:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.75, 7.64, 8.49 [03:07:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.58, 9.17, 9.18 [03:08:17] PROBLEM - wikien.wildterra2.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikien.wildterra2.com' expires in 15 day(s) (Tue 29 Mar 2022 03:03:18 GMT +0000). [03:08:49] PROBLEM - wiki.xysspon.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.xysspon.com' expires in 15 day(s) (Tue 29 Mar 2022 02:58:20 GMT +0000). [03:09:39] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:09:44] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.25, 8.70, 8.40 [03:10:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1ee85d01a5ae...ddf96f93655d [03:10:18] [02miraheze/ssl] 07MirahezeSSLBot 03ddf96f9 - Bot: Update SSL cert for wikien.wildterra2.com [03:11:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ddf96f93655d...55d03befd151 [03:11:12] [02miraheze/ssl] 07MirahezeSSLBot 0355d03be - Bot: Update SSL cert for wiki.xysspon.com [03:11:31] PROBLEM - wiki.thesimswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 15 day(s) (Tue 29 Mar 2022 02:51:09 GMT +0000). [03:11:38] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.41, 9.28, 8.64 [03:12:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:13:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.74, 9.23, 9.08 [03:13:32] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.96, 9.59, 8.83 [03:13:41] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.79, 8.43, 8.33 [03:14:17] PROBLEM - www.thesimswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.thesimswiki.com' expires in 15 day(s) (Tue 29 Mar 2022 02:51:09 GMT +0000). [03:14:17] PROBLEM - wiki.zymonic.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.zymonic.com' expires in 15 day(s) (Tue 29 Mar 2022 02:59:27 GMT +0000). [03:15:36] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.93, 8.17, 8.24 [03:15:57] PROBLEM - journeytheword.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.journeytheword.wiki' expires in 15 day(s) (Tue 29 Mar 2022 02:57:20 GMT +0000). [03:17:04] PROBLEM - marinebiodiversitymatrix.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'marinebiodiversitymatrix.org' expires in 15 day(s) (Tue 29 Mar 2022 02:53:19 GMT +0000). [03:17:15] PROBLEM - wiki.warframestat.us - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.warframestat.us' expires in 15 day(s) (Tue 29 Mar 2022 03:08:37 GMT +0000). [03:19:27] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.02, 8.70, 8.43 [03:21:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/55d03befd151...c3a84543583b [03:21:09] [02miraheze/ssl] 07MirahezeSSLBot 03c3a8454 - Bot: Update SSL cert for www.thesimswiki.com [03:21:22] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.68, 8.27, 8.31 [03:21:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.05, 9.04, 8.81 [03:21:57] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.72, 9.37, 8.71 [03:23:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.80, 2.78, 1.74 [03:23:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.52, 9.24, 8.92 [03:23:53] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.69, 9.34, 8.77 [03:27:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.70, 1.61, 1.49 [03:27:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 11.20, 9.84, 9.21 [03:27:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c3a84543583b...9b051898fce2 [03:27:35] [02miraheze/ssl] 07MirahezeSSLBot 039b05189 - Bot: Update SSL cert for wiki.warframestat.us [03:28:14] PROBLEM - www.journeytheword.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.journeytheword.wiki' expires in 15 day(s) (Tue 29 Mar 2022 02:57:20 GMT +0000). [03:29:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.77, 9.61, 9.21 [03:30:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9b051898fce2...408962e142ce [03:30:23] [02miraheze/ssl] 07MirahezeSSLBot 03408962e - Bot: Update SSL cert for marinebiodiversitymatrix.org [03:31:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.64, 9.72, 9.79 [03:31:30] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 0.95, 2.06, 1.52 [03:31:34] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 8.94, 8.42, 7.81 [03:31:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.34, 7.47, 8.22 [03:33:04] PROBLEM - opendatascot.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'opendatascot.org' expires in 15 day(s) (Tue 29 Mar 2022 03:25:39 GMT +0000). [03:33:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.98, 9.92, 9.84 [03:33:30] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.49, 1.53, 1.38 [03:33:34] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.02, 7.76, 7.63 [03:35:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.70, 9.75, 9.36 [03:35:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.13, 9.01, 9.50 [03:35:28] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 11.54, 9.35, 8.78 [03:35:52] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.40, 9.77, 8.88 [03:36:39] PROBLEM - wiki.nowchess.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.nowchess.org' expires in 15 day(s) (Tue 29 Mar 2022 03:29:04 GMT +0000). [03:37:24] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.27, 9.15, 8.77 [03:37:30] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/408962e142ce...78a0858cb839 [03:37:32] [02miraheze/ssl] 07MirahezeSSLBot 0378a0858 - Bot: Update SSL cert for wiki.zymonic.com [03:38:07] RECOVERY - wiki.xysspon.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.xysspon.com' will expire on Sat 11 Jun 2022 02:11:05 GMT +0000. [03:38:14] RECOVERY - wikien.wildterra2.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wikien.wildterra2.com' will expire on Sat 11 Jun 2022 02:10:08 GMT +0000. [03:39:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.05, 9.57, 9.37 [03:39:37] PROBLEM - with.cpt-ra.bid - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'with.cpt-ra.bid' expires in 15 day(s) (Tue 29 Mar 2022 03:20:18 GMT +0000). [03:40:48] PROBLEM - wikiru.wildterra2.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikiru.wildterra2.com' expires in 15 day(s) (Tue 29 Mar 2022 03:15:30 GMT +0000). [03:41:03] PROBLEM - wiki.seredrau.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.seredrau.xyz' expires in 15 day(s) (Tue 29 Mar 2022 03:28:24 GMT +0000). [03:42:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:43:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/78a0858cb839...ac5cd08bc353 [03:43:03] [02miraheze/ssl] 07MirahezeSSLBot 03ac5cd08 - Bot: Update SSL cert for wiki.nowchess.org [03:43:31] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.99, 7.90, 8.35 [03:43:36] PROBLEM - www.opendatascot.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'opendatascot.org' expires in 15 day(s) (Tue 29 Mar 2022 03:25:39 GMT +0000). [03:45:24] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.64, 9.82, 9.50 [03:46:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:49:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.13, 8.99, 9.23 [03:49:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.22, 10.09, 9.32 [03:51:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.82, 9.16, 8.89 [03:52:03] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ac5cd08bc353...a89fabaf8206 [03:52:05] [02miraheze/ssl] 07MirahezeSSLBot 03a89faba - Bot: Update SSL cert for wiki.seredrau.xyz [03:52:16] PROBLEM - wiki.mobilityengineer.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mobilityengineer.com' expires in 15 day(s) (Tue 29 Mar 2022 03:48:00 GMT +0000). [03:53:07] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.88, 9.06, 8.70 [03:53:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.13, 8.62, 8.74 [03:53:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.83, 9.72, 9.34 [03:53:40] PROBLEM - wiki.villagecollaborative.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.villagecollaborative.net' expires in 15 day(s) (Tue 29 Mar 2022 03:22:06 GMT +0000). [03:55:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.14, 7.89, 8.44 [03:55:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.94, 10.82, 9.78 [03:58:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.02, 2.73 [03:59:01] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.95, 9.88, 9.08 [03:59:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.02, 9.60, 9.57 [04:00:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.75, 2.81, 2.69 [04:01:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 5.57, 8.35, 8.64 [04:01:24] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.15, 7.38, 8.30 [04:01:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:05:18] RECOVERY - wiki.nowchess.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.nowchess.org' will expire on Sat 11 Jun 2022 02:42:56 GMT +0000. [04:05:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.76, 2.57, 1.64 [04:05:43] RECOVERY - www.marinebiodiversitymatrix.org - LetsEncrypt on sslhost is OK: OK - Certificate 'marinebiodiversitymatrix.org' will expire on Sat 11 Jun 2022 02:30:15 GMT +0000. [04:06:26] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a89fabaf8206...c801cac3a018 [04:06:27] [02miraheze/ssl] 07MirahezeSSLBot 03c801cac - Bot: Update SSL cert for wiki.mobilityengineer.com [04:06:38] PROBLEM - wiki.starship.digital - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.starship.digital' expires in 15 day(s) (Tue 29 Mar 2022 03:37:03 GMT +0000). [04:07:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.94, 1.94, 1.52 [04:07:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.86, 8.61, 8.50 [04:08:38] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:09:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.02, 7.97, 8.40 [04:09:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.71, 1.58, 1.44 [04:09:34] RECOVERY - wiki.thesimswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.thesimswiki.com' will expire on Sat 11 Jun 2022 02:21:01 GMT +0000. [04:11:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.41, 7.76, 8.49 [04:12:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c801cac3a018...1ca16e3ba7bc [04:12:16] [02miraheze/ssl] 07MirahezeSSLBot 031ca16e3 - Bot: Update SSL cert for wikiru.wildterra2.com [04:12:17] RECOVERY - www.thesimswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.thesimswiki.com' will expire on Sat 11 Jun 2022 02:21:01 GMT +0000. [04:12:17] RECOVERY - wiki.zymonic.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.zymonic.com' will expire on Sat 11 Jun 2022 02:37:25 GMT +0000. [04:13:40] PROBLEM - wiki.minecraftathome.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.minecraftathome.com' expires in 15 day(s) (Tue 29 Mar 2022 03:55:17 GMT +0000). [04:15:04] RECOVERY - marinebiodiversitymatrix.org - LetsEncrypt on sslhost is OK: OK - Certificate 'marinebiodiversitymatrix.org' will expire on Sat 11 Jun 2022 02:30:15 GMT +0000. [04:15:15] RECOVERY - wiki.warframestat.us - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.warframestat.us' will expire on Sat 11 Jun 2022 02:27:27 GMT +0000. [04:17:20] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.48, 9.05, 8.73 [04:18:38] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:19:20] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.37, 8.26, 8.48 [04:19:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1ca16e3ba7bc...2a6aaa584634 [04:19:33] [02miraheze/ssl] 07MirahezeSSLBot 032a6aaa5 - Bot: Update SSL cert for wiki.starship.digital [04:19:43] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.63, 1.36 [04:20:15] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.09, 8.50, 8.43 [04:20:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/2a6aaa584634...91838d506a27 [04:20:31] [02miraheze/ssl] 07MirahezeSSLBot 0391838d5 - Bot: Update SSL cert for wiki.minecraftathome.com [04:21:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.10, 8.38, 8.39 [04:21:42] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.47, 1.52, 1.35 [04:22:04] PROBLEM - wiki.rebirthofthenight.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.rebirthofthenight.com' expires in 15 day(s) (Tue 29 Mar 2022 03:59:29 GMT +0000). [04:22:39] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:23:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.72, 8.82, 8.66 [04:24:07] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.15, 9.58, 8.85 [04:25:01] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.11, 8.20, 8.31 [04:25:46] PROBLEM - wiki.whentheycry.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.whentheycry.org' expires in 15 day(s) (Tue 29 Mar 2022 03:56:46 GMT +0000). [04:26:02] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.79, 8.85, 8.67 [04:27:58] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.54, 9.36, 8.87 [04:28:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/91838d506a27...bf1fd44a46d8 [04:28:21] [02miraheze/ssl] 07MirahezeSSLBot 03bf1fd44 - Bot: Update SSL cert for wiki.villagecollaborative.net [04:29:08]