[00:00:06] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [00:00:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.73, 1.93, 1.90 [00:00:21] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1301597059 secs [00:01:39] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.11, 1.93, 1.74 [00:01:42] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 31.08, 11.86, 5.01 [00:01:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.46, 5.53, 5.46 [00:02:05] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [00:02:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.33, 2.55, 2.11 [00:03:34] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.98, 1.79 [00:05:28] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.03, 1.97, 1.80 [00:05:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.55, 5.13, 5.23 [00:07:23] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.97, 1.82 [00:07:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 4.12, 4.70, 5.06 [00:09:29] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 0.63, 3.45, 3.73 [00:10:16] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.09717941284 secs [00:11:12] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.05, 1.93, 1.82 [00:11:27] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.52, 2.46, 3.33 [00:11:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 5.45, 6.19, 5.60 [00:13:07] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.89, 1.58, 1.71 [00:15:01] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.31, 1.46, 1.64 [00:15:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.17, 5.24, 5.38 [00:17:40] RECOVERY - db131 NTP time on db131 is OK: NTP OK: Offset -0.01256972551 secs [00:17:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.44, 3.89, 4.86 [00:18:36] RECOVERY - mon141 NTP time on mon141 is OK: NTP OK: Offset -0.02611479163 secs [00:20:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.69, 1.98, 1.77 [00:21:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 5.87, 6.70, 5.90 [00:22:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.42, 1.68, 1.68 [00:23:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.15, 5.13, 5.43 [00:25:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.59, 3.86, 4.92 [00:32:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.03, 1.82, 1.71 [00:34:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.09, 1.69, 1.68 [00:37:09] PROBLEM - bast121 NTP time on bast121 is WARNING: NTP WARNING: Offset 0.3005403876 secs [00:38:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.75, 2.05, 1.82 [00:41:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.82, 5.63, 5.05 [00:45:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.88, 4.82, 4.84 [00:48:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.46, 1.87, 1.93 [00:50:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.86, 2.21, 2.04 [00:52:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.64, 1.98, 1.97 [00:53:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.27, 5.87, 5.26 [00:55:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.16, 4.93, 4.99 [00:59:56] PROBLEM - test131 NTP time on test131 is WARNING: NTP WARNING: Offset 0.2904301286 secs [01:04:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.57, 2.67, 2.13 [01:17:52] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.85, 2.94, 3.82 [01:19:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 3.54, 3.92 [01:23:51] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.67, 3.62, 3.90 [01:24:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.16, 1.54, 1.96 [01:25:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.58, 4.00, 3.98 [01:27:09] RECOVERY - bast121 NTP time on bast121 is OK: NTP OK: Offset -0.03509998322 secs [01:27:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.48, 3.57, 3.82 [01:29:27] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.19, 2.89, 1.53 [01:29:50] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 4.01, 3.96 [01:29:51] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [01:31:27] PROBLEM - cp23 Current Load on cp23 is WARNING: WARNING - load average: 3.42, 3.39, 1.90 [01:31:49] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [01:32:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.48, 1.81, 1.85 [01:33:27] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.63, 2.33, 1.69 [01:34:51] RECOVERY - test131 NTP time on test131 is OK: NTP OK: Offset -0.006271928549 secs [01:36:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.75, 1.87, 1.86 [01:37:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.81, 3.67, 3.94 [01:39:48] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.13, 4.83, 4.29 [01:42:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.40, 2.06, 1.95 [01:44:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.75, 1.83, 1.87 [01:45:22] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.62, 5.19, 4.64 [01:46:16] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset -0.1158616543 secs [01:47:18] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 4.70, 4.66, 4.49 [01:48:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.62, 1.46, 1.92 [01:50:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.26, 1.83, 2.00 [01:53:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.74, 3.60, 3.99 [01:54:16] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.09517261386 secs [01:56:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.18, 1.80, 1.96 [01:56:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.87, 1.38, 1.61 [01:58:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.70, 2.01, 2.03 [02:01:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.68, 3.57, 3.70 [02:05:39] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.62, 1.76, 1.70 [02:07:33] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.54, 1.65, 1.66 [02:07:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.29, 3.50, 3.71 [02:11:42] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.75, 3.95, 3.78 [02:12:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.72, 1.80, 1.98 [02:13:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.39, 3.78, 3.75 [02:14:15] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.11, 2.02, 1.79 [02:15:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.51, 4.09, 3.87 [02:16:10] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.91, 1.92, 1.77 [02:17:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.77, 3.57, 3.70 [02:18:04] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.91, 2.18, 1.88 [02:18:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.87, 2.12, 2.03 [02:19:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.18, 3.61, 3.69 [02:21:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.87, 3.73, 3.72 [02:22:39] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.77, 5.27, 4.77 [02:23:48] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.41, 1.88, 1.87 [02:24:34] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.34, 4.59, 4.57 [02:29:32] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.94, 2.08, 1.94 [02:32:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.66, 1.59, 1.92 [02:33:21] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.88, 1.94, 1.91 [02:36:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.49, 2.05, 2.02 [02:38:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.73, 1.89, 1.96 [02:41:00] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.87, 5.62, 4.92 [02:42:56] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.93, 4.94, 4.75 [02:43:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.87, 4.64, 4.14 [02:44:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.07, 2.02, 1.95 [02:46:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.97, 1.93, 1.93 [02:46:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.59, 1.31, 1.62 [02:47:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.97, 3.66, 3.86 [02:50:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.94, 2.16, 1.89 [02:51:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.45, 4.24, 3.94 [02:52:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.69, 2.16, 2.01 [02:52:36] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.52, 6.17, 5.07 [02:52:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.33, 1.77, 1.77 [02:54:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.65, 2.40, 1.98 [02:56:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.30, 1.95, 1.98 [02:58:23] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.75, 5.96, 5.41 [03:02:15] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.98, 6.01, 5.47 [03:04:10] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.06, 5.51, 5.37 [03:08:02] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.56, 4.20, 4.87 [03:08:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.34, 1.45, 1.69 [03:09:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.13, 3.21, 3.85 [03:11:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.66, 3.83, 3.99 [03:13:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.68, 3.90 [03:17:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.03, 1.71, 1.67 [03:19:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 9.17, 5.20, 4.36 [03:22:39] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.21, 6.97, 5.72 [03:24:34] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.68, 5.68, 5.40 [03:24:46] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.07, 1.96, 1.88 [03:26:30] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.42, 4.68, 5.07 [03:26:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.63, 1.73, 1.95 [03:28:35] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.90, 1.45, 1.70 [03:31:27] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.89, 3.27, 3.96 [03:32:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.18, 1.36, 1.70 [03:33:26] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.72, 3.43, 3.91 [03:35:26] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.96, 2.86, 3.64 [03:36:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.87, 2.01, 1.87 [03:38:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.00, 1.67, 1.77 [03:42:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.37, 1.56, 1.70 [03:43:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.11, 3.00, 3.40 [03:49:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.33, 4.67, 3.94 [03:50:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.92, 2.23, 1.93 [03:51:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.50, 3.42, 3.57 [03:52:40] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1030460596 secs [03:53:22] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.32, 2.77, 3.31 [03:56:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.65, 1.68, 1.85 [04:00:40] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08962884545 secs [04:02:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.16, 1.45, 1.69 [04:03:06] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.98, 1.56, 1.38 [04:05:01] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.02, 1.70, 1.45 [04:06:56] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.73, 1.72, 1.49 [04:08:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.04, 1.46, 1.42 [04:14:17] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.72, 3.32, 2.90 [04:16:17] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.51, 2.54, 2.65 [04:29:14] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [04:31:12] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [04:32:13] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.95, 4.15, 3.28 [04:34:13] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.74, 3.78, 3.27 [04:36:13] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.09, 4.10, 3.43 [04:38:12] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.56, 3.71, 3.36 [04:40:12] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.46, 3.31, 3.26 [04:56:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.33, 1.74, 1.48 [04:58:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.04, 1.47, 1.42 [05:01:53] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.66, 7.43, 4.87 [05:02:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.67, 5.10, 3.70 [05:02:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.05, 1.73, 1.58 [05:04:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.66, 1.35, 1.46 [05:05:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.67, 5.55, 4.76 [05:07:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.35, 4.33, 4.39 [05:09:50] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.77, 1.63, 1.35 [05:10:05] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.23, 3.87, 3.80 [05:14:04] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.49, 2.48, 3.23 [05:14:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.43, 2.84, 2.02 [05:18:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.12, 1.71, 1.72 [05:20:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.14, 1.56, 1.67 [05:21:45] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.62, 1.70, 1.57 [05:23:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.26, 1.68, 1.37 [05:24:55] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.73, 1.65, 1.39 [05:26:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.00, 1.43, 1.34 [05:27:42] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.67, 2.07, 1.75 [05:31:41] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.69, 1.95, 1.79 [05:43:36] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.05, 1.78, 1.76 [05:45:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.90, 1.83, 1.78 [05:55:27] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.35, 1.82, 1.54 [05:57:21] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.86, 1.57, 1.48 [05:57:31] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.69, 2.09, 1.89 [06:17:54] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [06:18:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:14:23] PROBLEM - christipedia.nl - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.403785705566406 seconds [07:17:01] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query roblox-wiki.tk. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:30:40] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.135053426 secs [07:38:40] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09763965011 secs [07:43:43] RECOVERY - christipedia.nl - reverse DNS on sslhost is OK: SSL OK - christipedia.nl reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK [07:46:31] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK [07:54:58] PROBLEM - eng.archiopedia.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns083.d.register.com.', 'dns179.a.register.com.', 'dns032.b.register.com.', 'dns025.c.register.com.'], 'CNAME': None} [08:24:24] RECOVERY - eng.archiopedia.org - reverse DNS on sslhost is OK: SSL OK - eng.archiopedia.org reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:26:16] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1101607382 secs [08:34:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.87, 1.88, 2.00 [08:40:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.06, 1.97, 2.00 [08:42:16] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08526596427 secs [08:44:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.69, 1.92, 1.98 [08:56:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.22, 1.85, 1.85 [09:00:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.61, 1.95, 1.92 [09:08:53] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [09:08:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:20:35] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.46, 1.61, 1.70 [09:24:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.67, 1.66, 1.71 [09:26:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.76, 1.98, 1.82 [09:32:36] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.59, 2.00, 1.93 [09:46:40] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1594155133 secs [09:48:34] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=atwmcwiki --username-prefix=mh:atwmc --report 1 /home/reception/reception/delbackups2/atwmcwiki.xml (START) [09:48:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:56:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.26, 1.92, 1.86 [09:59:01] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=atwmcwiki --username-prefix=mh:atwmc --report 1 /home/reception/reception/delbackups2/atwmcwiki.xml (END - exit=0) [09:59:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:02:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.61, 1.91, 1.91 [10:08:40] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09730890393 secs [10:20:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.09, 1.86, 1.80 [10:34:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.75, 1.93, 1.97 [10:48:40] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.1134861112 secs [10:50:40] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.09751713276 secs [10:56:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.10, 1.74, 1.76 [11:00:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.75, 1.98, 1.87 [11:16:02] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - roblox-wiki.tk All nameservers failed to answer the query. [11:16:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.16, 1.96, 1.88 [11:18:36] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.83, 1.88, 1.86 [11:21:28] PROBLEM - ping6 on cp31 is WARNING: PING WARNING - Packet loss = 0%, RTA = 107.78 ms [11:21:47] PROBLEM - ping6 on cp30 is WARNING: PING WARNING - Packet loss = 0%, RTA = 113.35 ms [11:23:35] PROBLEM - ping6 on cp31 is CRITICAL: PING CRITICAL - Packet loss = 100% [11:23:56] PROBLEM - ping6 on cp30 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 82.76 ms [11:25:35] PROBLEM - ping6 on cp31 is WARNING: PING WARNING - Packet loss = 0%, RTA = 107.49 ms [11:25:59] PROBLEM - ping6 on cp30 is WARNING: PING WARNING - Packet loss = 0%, RTA = 112.98 ms [11:26:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.14, 1.79, 1.80 [11:32:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.69, 1.94, 1.92 [11:35:42] PROBLEM - ping6 on cp31 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 672.71 ms [11:36:18] PROBLEM - ping6 on cp30 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 678.85 ms [11:39:13] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 149.56.141.75/cpweb [11:39:18] PROBLEM - cp31 NTP time on cp31 is WARNING: NTP WARNING: Offset 0.2979944944 secs [11:39:26] PROBLEM - cp30 HTTPS on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:40:33] PROBLEM - cp30 NTP time on cp30 is WARNING: NTP WARNING: Offset 0.2978548706 secs [11:40:35] PROBLEM - cp31 Stunnel HTTP for test131 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:40:37] PROBLEM - cp31 Stunnel HTTP for mwtask141 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:40:55] PROBLEM - cp31 Stunnel HTTP for mw142 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:40:56] PROBLEM - cp30 Stunnel HTTP for mw131 on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:41:03] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:41:14] PROBLEM - cp30 Stunnel HTTP for reports121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:41:20] PROBLEM - cp30 Stunnel HTTP for phab121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:41:34] PROBLEM - cp30 Stunnel HTTP for mail121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:41:37] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:41:39] PROBLEM - cp30 Stunnel HTTP for matomo131 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:41:47] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:42:21] PROBLEM - cp31 Stunnel HTTP for reports121 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:42:25] PROBLEM - cp30 Stunnel HTTP for puppet141 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:42:48] PROBLEM - cp31 Stunnel HTTP for phab121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:42:50] PROBLEM - cp31 Stunnel HTTP for matomo131 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:42:54] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:42:55] RECOVERY - cp31 Stunnel HTTP for mwtask141 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 356 bytes in 5.312 second response time [11:43:02] RECOVERY - cp30 Stunnel HTTP for mw131 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 5.311 second response time [11:43:06] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 2 backends are down. mw131 mw132 [11:43:16] PROBLEM - cp31 HTTPS on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:43:20] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [11:43:31] RECOVERY - cp30 Stunnel HTTP for reports121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 10996 bytes in 9.298 second response time [11:43:50] RECOVERY - cp30 HTTPS on cp30 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3360 bytes in 8.549 second response time [11:43:56] RECOVERY - cp30 Stunnel HTTP for matomo131 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 100159 bytes in 8.306 second response time [11:44:14] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 2 backends are down. mw122 mw131 [11:44:39] PROBLEM - cp31 Stunnel HTTP for mw132 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:44:54] RECOVERY - cp31 Stunnel HTTP for phab121 on cp31 is OK: HTTP OK: Status line output matched "500" - 2855 bytes in 2.710 second response time [11:45:06] RECOVERY - cp30 Stunnel HTTP for puppet141 on cp30 is OK: HTTP OK: Status line output matched "403" - 289 bytes in 9.151 second response time [11:45:21] PROBLEM - cp31 Stunnel HTTP for mon141 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:45:26] RECOVERY - cp31 HTTPS on cp31 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3362 bytes in 8.000 second response time [11:45:32] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp22.miraheze.org - NS RECORDS OK [11:45:34] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 3.894 second response time [11:45:49] RECOVERY - cp30 Stunnel HTTP for phab121 on cp30 is OK: HTTP OK: Status line output matched "500" - 2855 bytes in 5.636 second response time [11:45:57] PROBLEM - cp31 Stunnel HTTP for mail121 on cp31 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:46:04] PROBLEM - cp30 NTP time on cp30 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:46:25] RECOVERY - cp30 Stunnel HTTP for mail121 on cp30 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 427 bytes in 2.650 second response time [11:46:43] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb [11:47:02] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 7.909 second response time [11:47:28] RECOVERY - cp31 Stunnel HTTP for matomo131 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 100159 bytes in 7.945 second response time [11:48:48] RECOVERY - cp31 Stunnel HTTP for mw142 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.247 second response time [11:48:51] RECOVERY - cp31 Stunnel HTTP for mail121 on cp31 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 427 bytes in 0.279 second response time [11:49:06] PROBLEM - cp30 NTP time on cp30 is WARNING: NTP WARNING: Offset 0.2978551388 secs [11:49:08] RECOVERY - ping6 on cp31 is OK: PING OK - Packet loss = 0%, RTA = 78.06 ms [11:49:15] RECOVERY - cp31 Stunnel HTTP for mw132 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.240 second response time [11:49:22] RECOVERY - cp31 Stunnel HTTP for reports121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 10996 bytes in 0.305 second response time [11:49:43] RECOVERY - ping6 on cp30 is OK: PING OK - Packet loss = 0%, RTA = 83.00 ms [11:50:19] RECOVERY - cp31 Stunnel HTTP for test131 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 14740 bytes in 0.387 second response time [11:50:24] RECOVERY - cp31 Stunnel HTTP for mon141 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 36275 bytes in 0.339 second response time [11:50:49] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [11:52:37] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [11:53:02] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [11:53:09] RECOVERY - cp31 NTP time on cp31 is OK: NTP OK: Offset 0.0008575022221 secs [11:53:14] RECOVERY - cp30 NTP time on cp30 is OK: NTP OK: Offset 0.000561863184 secs [11:56:28] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36229 bytes in 2.090 second response time [11:57:24] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.62, 2.05, 1.91 [11:58:27] RECOVERY - cp20 APT on cp20 is OK: APT OK: 38 packages available for upgrade (0 critical updates). [12:00:22] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [12:00:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:07:21] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.85, 2.00, 1.98 [12:09:20] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.15, 2.10, 2.02 [12:14:30] PROBLEM - cp22 APT on cp22 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [12:15:18] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.80, 1.97, 1.99 [12:16:30] RECOVERY - cp22 APT on cp22 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:27:13] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.63, 2.03, 1.95 [12:32:03] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [12:32:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:10] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.65, 1.98, 1.99 [12:38:07] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.90, 4.69, 3.03 [12:40:03] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.17, 4.50, 3.14 [12:41:58] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.67, 3.44, 2.91 [12:51:04] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.41, 1.49, 1.68 [12:59:00] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.45, 2.20, 1.93 [13:02:58] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.66, 1.93, 1.89 [13:16:53] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.66, 1.60, 1.70 [13:28:47] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.76, 2.09, 1.86 [13:30:30] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.10, 4.16, 3.21 [13:30:46] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.79, 1.94, 1.83 [13:32:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.44, 3.47, 3.06 [13:42:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.41, 5.22, 3.93 [13:44:22] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 11.32, 7.39, 4.78 [13:44:41] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.61, 1.63, 1.70 [13:48:13] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.57, 5.65, 4.66 [13:50:09] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.21, 6.25, 5.01 [13:50:38] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.86, 1.76, 1.73 [13:52:37] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.15, 1.93, 1.80 [13:54:00] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.40, 5.63, 5.01 [13:57:22] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 1.45 ms [13:57:30] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [13:57:36] PROBLEM - cloud11 ferm_active on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:57:37] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.58 ms [13:57:41] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:57:41] PROBLEM - cloud11 conntrack_table_size on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:57:41] PROBLEM - cloud11 Disk Space on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:57:41] PROBLEM - cloud11 APT on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:57:51] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.14, 3.99, 4.50 [13:57:56] PROBLEM - cloud11 PowerDNS Recursor on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:58:46] PROBLEM - cloud11 NTP time on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [13:59:16] PROBLEM - cloud11 Current Load on cloud11 is CRITICAL: connect to address 2a10:6740::6:200 port 5666: Connection refusedconnect to host 2a10:6740::6:200 port 5666: Connection refused [14:01:44] expected [14:04:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.70, 1.97, 1.98 [14:05:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 14.12, 8.19, 6.07 [14:13:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 1.92, 5.01, 5.59 [14:15:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.29, 3.84, 5.10 [14:18:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.35, 3.10, 3.93 [14:20:20] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.40, 3.71, 4.08 [14:22:20] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.82, 3.32, 3.89 [14:26:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.55, 1.96, 1.84 [14:26:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.78, 1.37, 1.13 [14:28:19] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.57, 2.38, 3.30 [14:28:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.18, 1.62, 1.24 [14:30:35] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.68, 1.96, 1.89 [14:32:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.96, 3.18, 3.45 [14:32:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.21, 1.65, 1.36 [14:38:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.96, 3.70 [14:41:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.79, 5.73, 4.70 [14:42:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.54, 3.85, 3.78 [14:44:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 4.12, 3.89 [14:46:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.99, 3.94, 3.84 [14:47:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.60, 6.86, 5.41 [14:48:14] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.84, 4.17, 3.96 [14:48:35] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.35, 1.54, 1.67 [14:52:19] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.98, 1.45, 1.18 [14:55:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.43, 5.96, 5.86 [14:56:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.02, 1.29, 1.18 [14:57:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.49, 6.83, 6.20 [14:58:35] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.60, 1.99, 1.77 [15:00:36] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.68, 1.86, 1.75 [15:01:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.80, 5.80, 5.93 [15:03:50] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.48, 6.26, 6.04 [15:04:35] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.43, 1.66, 1.69 [15:07:50] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.14, 5.33, 5.70 [15:11:44] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/6d40ab00def6...358a1a756e24 [15:11:44] [url] Comparing 6d40ab00def6...358a1a756e24 · miraheze/dns · GitHub | github.com [15:11:45] [02miraheze/dns] 07Reception123 03358a1a7 - add pornwiki.org zone [15:11:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.23, 3.91, 5.07 [15:20:07] PROBLEM - mon141 NTP time on mon141 is CRITICAL: NTP CRITICAL: Offset 0.7884532511 secs [15:23:34] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.85, 1.77, 1.70 [15:24:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.48, 3.01, 3.97 [15:26:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.31, 3.81, 4.14 [15:27:26] PROBLEM - db131 NTP time on db131 is CRITICAL: NTP CRITICAL: Offset 0.7859133482 secs [15:27:32] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.79, 2.13, 1.85 [15:27:51] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.64, 6.98, 5.42 [15:31:51] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.25, 5.35, 5.12 [15:33:25] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [15:33:27] [02puppet] 07paladox closed pull request 03#2852: Remove varnish role from cp20/21 - 13https://github.com/miraheze/puppet/pull/2852 [15:33:28] [url] Page not found · GitHub · GitHub | github.com [15:33:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8c43cd4d89ac...0f43f4315301 [15:33:29] [url] Comparing 8c43cd4d89ac...0f43f4315301 · miraheze/puppet · GitHub | github.com [15:33:30] [02miraheze/puppet] 07paladox 030f43f43 - Remove varnish role from cp20/21 (#2852) [15:33:31] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/puppet [15:33:32] [url] Page not found · GitHub · GitHub | github.com [15:33:51] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.62, 5.03, 5.05 [15:36:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.93, 3.19, 3.97 [15:37:29] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 5.70, 3.10, 1.85 [15:37:34] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [15:41:11] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.34, 3.06, 2.21 [15:41:18] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:41:21] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.77, 2.00, 1.99 [15:43:17] RECOVERY - db131 NTP time on db131 is OK: NTP OK: Offset -0.005951821804 secs [15:45:11] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.12, 1.93, 1.96 [15:46:47] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.02, 2.36, 3.20 [15:47:08] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.56, 1.79, 1.90 [15:51:24] RECOVERY - mon141 NTP time on mon141 is OK: NTP OK: Offset -0.007964968681 secs [15:52:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.22, 3.30 [15:54:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.78, 2.69, 3.09 [15:56:40] [02mw-config] 07Universal-Omega commented on pull request 03#4812: globally enable IPInfo per T9503 - 13https://github.com/miraheze/mw-config/pull/4812#issuecomment-1252564025 [15:56:41] [url] globally enable IPInfo per T9503 by Reception123 · Pull Request #4812 · miraheze/mw-config · GitHub | github.com [15:56:41] [02mw-config] 07Universal-Omega closed pull request 03#4812: globally enable IPInfo per T9503 - 13https://github.com/miraheze/mw-config/pull/4812 [15:56:42] [url] Page not found · GitHub · GitHub | github.com [15:56:44] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Reception123-patch-2 [15:56:45] [02mw-config] 07Universal-Omega deleted branch 03Reception123-patch-2 - 13https://github.com/miraheze/mw-config [15:56:46] ... [15:57:07] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.70, 2.06, 1.93 [16:03:07] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.91, 1.98, 1.95 [16:08:45] [02miraheze/dns] 07Universal-Omega pushed 031 commit to 03cloud11-remove-ipv4 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/2f842e88c9ab [16:08:46] [02miraheze/dns] 07Universal-Omega 032f842e8 - Remove IPv4 address from cloud11 [16:08:48] [02dns] 07Universal-Omega created branch 03cloud11-remove-ipv4 - 13https://github.com/miraheze/dns [16:08:48] [url] Page not found · GitHub · GitHub | github.com [16:08:49] [02dns] 07Universal-Omega opened pull request 03#354: Remove IPv4 address from cloud11 - 13https://github.com/miraheze/dns/pull/354 [16:08:50] [url] Page not found · GitHub · GitHub | github.com [16:11:24] [02dns] 07paladox closed pull request 03#354: Remove IPv4 address from cloud11 - 13https://github.com/miraheze/dns/pull/354 [16:11:25] [url] Page not found · GitHub · GitHub | github.com [16:11:26] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/358a1a756e24...c593080673ee [16:11:27] [url] Comparing 358a1a756e24...c593080673ee · miraheze/dns · GitHub | github.com [16:11:27] [02miraheze/dns] 07Universal-Omega 03c593080 - Remove IPv4 address from cloud11 (#354) [16:11:29] [02miraheze/dns] 07paladox deleted branch 03cloud11-remove-ipv4 [16:11:30] [02dns] 07paladox deleted branch 03cloud11-remove-ipv4 - 13https://github.com/miraheze/dns [16:11:31] ... [16:16:06] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1219576001 secs [16:17:07] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.26, 1.47, 1.67 [16:21:30] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.26, 3.98, 3.56 [16:27:19] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.09793078899 secs [16:27:21] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.36, 1.89, 1.75 [16:27:23] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.3676689863 secs [16:32:13] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.24, 6.69, 5.01 [16:32:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.97, 3.84 [16:33:17] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.47, 1.85, 1.83 [16:34:43] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.39, 4.83, 4.17 [16:35:00] !log cloud11: reset; delete all Logical Drives; create new Logical Drive (2 SSDs with RAID 1+0); and install Proxmox VE 7.2 from ISO [16:35:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:02] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.40, 5.23, 4.80 [16:36:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.17, 4.00, 3.93 [16:38:00] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.43, 5.73, 5.02 [16:38:37] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.79, 5.35, 4.47 [16:39:02] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset -0.006269663572 secs [16:40:00] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.57, 4.50, 4.66 [16:41:13] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.51, 1.58, 1.70 [16:44:00] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.11, 5.49, 5.16 [16:46:00] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.54, 4.66, 4.88 [16:46:35] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0f43f4315301...12380d59bd55 [16:46:36] [url] Comparing 0f43f4315301...12380d59bd55 · miraheze/puppet · GitHub | github.com [16:46:37] [02miraheze/puppet] 07Universal-Omega 0312380d5 - graylog: upgrade to 4.3.7 [16:52:27] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 4 failures. Last run 1 minute ago with 4 failures. Failed resources (up to 3 shown): Package[logrotate],Package[nano],Package[openssl],Package[postfix] [16:53:32] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.90, 5.17, 4.89 [16:55:31] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.48, 6.27, 5.31 [16:57:29] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.75, 5.97, 5.35 [16:58:55] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.60, 2.13, 1.84 [16:59:28] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.71, 5.98, 5.41 [17:00:41] !log renamed modernpvpwiki to mcpvpwiki [17:00:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:00:49] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.63, 1.94, 1.80 [17:01:27] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.85, 4.99, 5.13 [17:02:26] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [17:03:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.15, 6.18, 5.51 [17:05:04] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:06:13] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:06:34] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.10, 1.44, 1.63 [17:08:13] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.284 second response time [17:09:09] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.251 second response time [17:10:19] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw121 [17:12:14] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [17:13:20] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.54, 5.03, 6.00 [17:18:03] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.11, 1.91, 1.76 [17:18:40] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [17:18:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:19:16] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.35, 3.18, 4.85 [17:21:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.83, 2.55, 3.81 [17:25:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.17, 1.85, 3.23 [17:29:51] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [17:30:53] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 8.89, 4.09, 2.26 [17:31:12] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.42, 5.97, 4.99 [17:31:46] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:31:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.10, 3.85, 3.64 [17:33:10] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.95, 5.96, 5.13 [17:33:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.79, 3.66 [17:35:49] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.47, 1.87, 1.98 [17:35:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.07, 4.48, 3.91 [17:36:53] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.64, 1.71, 1.77 [17:37:08] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 5.45, 6.31, 5.45 [17:38:53] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.91, 1.45, 1.67 [17:39:07] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.39, 5.35, 5.19 [17:41:05] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.75, 4.49, 4.90 [17:41:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.37, 4.00, 3.91 [17:45:07] PROBLEM - cp22 APT on cp22 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [17:45:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.16, 3.91, 3.86 [17:46:27] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.13, 3.36, 1.86 [17:47:03] RECOVERY - cp22 APT on cp22 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [17:47:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.99, 3.61, 3.76 [17:48:25] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1024897397 secs [17:48:26] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.12, 2.50, 1.72 [17:48:59] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.90, 6.67, 5.66 [17:49:49] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.48, 1.50, 1.70 [17:51:36] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 9.78, 7.44, 4.99 [17:51:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.51, 5.01, 4.23 [17:52:53] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.89, 1.94, 1.60 [17:56:53] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.09, 1.63, 1.56 [17:56:54] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.87, 5.90, 5.85 [17:58:53] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.26, 5.84, 5.79 [17:59:49] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.26, 1.98, 1.80 [18:00:52] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.36, 5.27, 5.60 [18:00:54] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [18:01:36] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.52, 7.48, 6.55 [18:01:49] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.61, 1.79, 1.75 [18:02:48] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:03:37] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 9.28, 7.87, 6.78 [18:04:25] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.0996825397 secs [18:05:36] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.42, 7.19, 6.65 [18:05:49] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.08, 1.83, 1.77 [18:05:51] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.12, 1.71, 1.62 [18:06:48] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.71, 5.29, 5.44 [18:07:37] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 9.77, 8.38, 7.15 [18:09:36] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.38, 6.81, 6.72 [18:09:50] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.58, 1.70, 1.63 [18:10:47] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.57, 5.36, 5.53 [18:11:36] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.98, 5.92, 6.41 [18:14:47] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.10, 4.12, 4.99 [18:17:47] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.01, 1.91, 1.74 [18:17:49] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 9.53, 7.15, 5.10 [18:19:46] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.75, 1.80, 1.72 [18:19:49] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.55, 5.91, 4.88 [18:20:47] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.54, 8.29, 6.44 [18:21:41] !log reboot graylog121 [18:21:45] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.97, 1.48, 1.61 [18:22:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:22:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.82, 1.76, 1.56 [18:23:19] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 5.19, 3.08, 1.28 [18:24:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.21, 2.05, 1.69 [18:25:19] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.80, 2.44, 1.26 [18:25:43] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.92, 1.91, 1.76 [18:26:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.99, 1.65, 1.59 [18:27:42] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.97, 1.60, 1.66 [18:29:56] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:29:57] PROBLEM - cp22 Stunnel HTTP for mw121 on cp22 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:29:57] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:31:57] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 30911 bytes in 2.108 second response time [18:31:58] RECOVERY - cp22 Stunnel HTTP for mw121 on cp22 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 1.412 second response time [18:31:59] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 2.299 second response time [18:33:59] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.29, 1.98, 1.73 [18:35:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.68, 1.78, 1.68 [18:36:47] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 1.46, 4.12, 5.89 [18:37:52] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.57, 1.70, 1.66 [18:38:40] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [18:38:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:39:37] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.02, 7.16, 6.65 [18:40:47] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.35, 2.37, 4.75 [18:41:36] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 8.52, 7.79, 6.93 [18:43:36] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.84, 6.31, 6.50 [18:43:42] PROBLEM - cloud11 SSH on cloud11 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:43:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.51, 2.33, 3.89 [18:44:19] PROBLEM - ping6 on cloud11 is CRITICAL: PING CRITICAL - Packet loss = 100% [18:44:26] PROBLEM - Host cloud11 is DOWN: PING CRITICAL - Packet loss = 100% [18:46:26] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.77 ms [18:47:35] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [18:48:25] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.29 ms [18:49:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.53, 3.11, 3.64 [18:51:30] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.84, 2.06, 1.78 [18:52:07] PROBLEM - cloud11 Puppet on cloud11 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 6 minutes ago with 0 failures [18:52:18] PROBLEM - cloud14 Puppet on cloud14 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 31 minutes ago with 0 failures [18:52:20] PROBLEM - cloud12 Puppet on cloud12 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 12 minutes ago with 0 failures [18:52:26] PROBLEM - cloud13 Puppet on cloud13 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 32 minutes ago with 0 failures [18:52:28] PROBLEM - cloud10 Puppet on cloud10 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 13 minutes ago with 0 failures [18:53:26] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.48, 1.85, 1.73 [18:53:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.10, 3.25, 3.66 [18:55:23] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.54, 1.36, 1.56 [18:55:36] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.89, 7.36, 6.95 [18:55:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.14, 2.56, 3.36 [18:57:36] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 11.93, 9.31, 7.71 [18:59:36] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 3.98, 7.59, 7.30 [19:01:36] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 1.78, 5.49, 6.56 [19:01:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.04, 3.53, 3.65 [19:03:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.28, 2.76, 3.35 [19:08:26] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.56, 2.03, 1.60 [19:10:25] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.12, 1.63, 1.50 [19:12:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/75c22701a82c...3812117574f8 [19:12:59] [url] Comparing 75c22701a82c...3812117574f8 · miraheze/mw-config · GitHub | github.com [19:13:00] [02miraheze/mw-config] 07Universal-Omega 033812117 - Add swiftproxy to disallowed subdomains [19:13:55] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.61, 2.52, 1.99 [19:14:11] miraheze/mw-config - Universal-Omega the build passed. [19:14:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.76, 3.78, 3.54 [19:15:30] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/c8f6c0a7cde3 [19:15:32] [02miraheze/dns] 07paladox 03c8f6c0a - Add swiftproxy111 to dns [19:15:33] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [19:15:34] [url] Page not found · GitHub · GitHub | github.com [19:15:35] [02dns] 07paladox opened pull request 03#355: Add swiftproxy111 to dns - 13https://github.com/miraheze/dns/pull/355 [19:15:36] [url] Page not found · GitHub · GitHub | github.com [19:15:40] [02dns] 07paladox closed pull request 03#355: Add swiftproxy111 to dns - 13https://github.com/miraheze/dns/pull/355 [19:15:41] [url] Page not found · GitHub · GitHub | github.com [19:15:42] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [19:15:43] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/c593080673ee...edfeb87bdce2 [19:15:44] [url] Comparing c593080673ee...edfeb87bdce2 · miraheze/dns · GitHub | github.com [19:15:45] [02miraheze/dns] 07paladox 03edfeb87 - Add swiftproxy111 to dns (#355) [19:15:46] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [19:15:47] [url] Page not found · GitHub · GitHub | github.com [19:16:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.26, 3.62, 3.51 [19:17:34] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1029099822 secs [19:17:53] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.26, 1.87, 1.85 [19:18:21] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.04, 1.97, 1.71 [19:18:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.22, 3.09, 3.32 [19:19:44] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 1.95, 1.88 [19:20:20] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.72, 1.87, 1.70 [19:22:19] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.00, 1.59, 1.62 [19:25:34] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.0872195065 secs [19:30:20] !log [@test131] starting deploy of {'config': True} to all [19:30:21] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [19:30:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:31:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.59, 1.40, 1.85 [19:31:32] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.16, 5.09, 3.51 [19:35:30] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 4.13, 4.85, 3.78 [19:39:09] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.93, 1.29, 1.63 [19:39:48] !log [@mwtask141] starting deploy of {'config': True} to all [19:39:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:55] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [19:39:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:09] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.95, 1.83, 1.58 [19:45:11] PROBLEM - wiki.mineland.eu - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.mineland.eu could not be found [19:46:58] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [19:47:19] PROBLEM - ping6 on cloud11 is CRITICAL: PING CRITICAL - Packet loss = 100% [19:47:42] PROBLEM - cloud11 SSH on cloud11 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:47:49] PROBLEM - Host cloud11 is DOWN: PING CRITICAL - Packet loss = 100% [19:49:08] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.25, 2.01, 1.70 [19:49:45] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tcharchivewiki --no-updates --username-prefix=wikia:thecorphub-archive /mnt/mediawiki-static/metawiki/ImportDump/tcharchivewiki-20220918020344.xml (START) [19:49:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:51:07] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.16, 1.69, 1.62 [19:51:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.29, 4.40, 3.45 [19:53:53] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:54:26] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:54:43] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.72, 1.42, 1.42 [19:56:15] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 17.60, 8.69, 5.45 [19:57:07] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 1.77 ms [19:57:49] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 0.75 ms [19:57:56] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [19:58:19] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tcharchivewiki --no-updates --username-prefix=wikia:thecorphub-archive /mnt/mediawiki-static/metawiki/ImportDump/tcharchivewiki-20220918020344.xml (END - exit=0) [19:58:20] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tcharchivewiki (START) [19:58:20] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:58:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:58:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:58:28] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [19:58:36] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.98, 1.27, 1.35 [19:59:31] PROBLEM - cloud11 SMART on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [19:59:36] PROBLEM - cloud11 Current Load on cloud11 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [20:01:03] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.07, 2.23, 1.81 [20:01:47] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tcharchivewiki (END - exit=0) [20:01:48] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=tcharchivewiki --active --update (END - exit=0) [20:01:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:01:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:02:11] PROBLEM - ping6 on cloud11 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:02:11] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.31, 5.59, 5.01 [20:02:31] PROBLEM - Host cloud11 is DOWN: PING CRITICAL - Packet loss = 100% [20:03:49] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fusangpediawiki --no-updates --username-prefix=w /mnt/mediawiki-static/metawiki/ImportDump/fusangpediawiki-20220918163138.xml (START) [20:03:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:04:02] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fusangpediawiki --no-updates --username-prefix=w /mnt/mediawiki-static/metawiki/ImportDump/fusangpediawiki-20220918163138.xml (END - exit=0) [20:04:03] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fusangpediawiki (START) [20:04:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:04:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:04:57] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fusangpediawiki (END - exit=0) [20:04:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=fusangpediawiki --active --update (END - exit=0) [20:05:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:05:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:05:49] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 0.82, 1.20, 1.98 [20:06:09] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.51, 4.32, 4.67 [20:07:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.77, 3.75, 4.00 [20:09:54] !log upgrade grafana on mon141 [20:10:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:11:00] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.81, 1.56, 1.74 [20:11:49] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 0.93, 1.03, 1.65 [20:12:59] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.87, 1.38, 1.65 [20:13:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.08, 3.71, 3.80 [20:15:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.01, 3.37, 3.67 [20:16:44] RECOVERY - Host cloud11 is UP: PING OK - Packet loss = 0%, RTA = 0.64 ms [20:16:52] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [20:16:56] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.21 ms [20:17:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.87, 3.82 [20:18:31] RECOVERY - cloud11 Current Load on cloud11 is OK: OK - load average: 0.98, 0.37, 0.14 [20:18:46] RECOVERY - cloud11 SMART on cloud11 is OK: OK: [cciss,0] - Device is clean --- [cciss,1] - Device is clean --- [cciss,2] - Device is clean --- [cciss,3] - Device is clean --- [cciss,4] - Device is clean --- [cciss,5] - Device is clean --- [cciss,6] - Device is clean| [20:19:09] !log cloud11: create 5 new Logical Drives (with RAID0) for HDDs [20:19:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.96, 3.87 [20:25:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.08, 2.82, 3.38 [20:30:17] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.82, 2.02, 1.59 [20:30:55] PROBLEM - cloud11 SSH on cloud11 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [20:31:22] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3812117574f8...d7b9bcef8054 [20:31:22] [url] Comparing 3812117574f8...d7b9bcef8054 · miraheze/mw-config · GitHub | github.com [20:31:23] [02miraheze/mw-config] 07Universal-Omega 03d7b9bce - Update LocalSettings.php [20:31:25] PROBLEM - ping6 on cloud11 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:31:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.02, 3.21, 3.41 [20:32:13] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.54, 1.95, 1.62 [20:32:41] miraheze/mw-config - Universal-Omega the build passed. [20:32:50] RECOVERY - cloud11 SSH on cloud11 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [20:33:27] RECOVERY - ping6 on cloud11 is OK: PING OK - Packet loss = 0%, RTA = 1.15 ms [20:33:51] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.17, 1.72, 1.54 [20:34:49] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.78, 5.51, 4.42 [20:35:50] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.98, 1.51, 1.49 [20:35:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.97, 4.29, 3.73 [20:38:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.47, 1.65, 1.59 [20:40:20] !log [@mwtask141] starting deploy of {'config': True} to all [20:40:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:40:27] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [20:40:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:41:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.82, 3.98, 4.00 [20:42:47] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.42, 5.22, 5.16 [20:43:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.24, 3.99, 3.99 [20:44:47] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 4.80, 4.98, 5.06 [20:47:13] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - roblox-wiki.tk All nameservers failed to answer the query. [20:48:44] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.33, 2.04, 1.68 [20:48:47] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 5.51, 6.50, 5.74 [20:49:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.65, 3.80, 3.98 [20:50:47] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.30, 4.87, 5.22 [20:51:40] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.88, 1.93, 1.65 [20:52:43] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.28, 1.96, 1.76 [20:52:47] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.03, 3.69, 4.76 [20:53:37] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.34, 2.16, 1.77 [20:53:56] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.03, 2.41, 3.37 [20:54:42] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.56, 1.48, 1.61 [20:55:33] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.48, 1.87, 1.70 [20:57:30] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.29, 1.66, 1.64 [20:57:37] !log [@test131] starting deploy of {'config': True} to all [20:57:38] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [20:57:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:59:51] PROBLEM - cp23 APT on cp23 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [21:01:46] RECOVERY - cp23 APT on cp23 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [21:01:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.74, 4.35, 3.91 [21:03:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.83, 3.95, 3.82 [21:05:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.08, 3.96, 3.84 [21:07:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.50, 3.43, 3.66 [21:09:34] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.85, 1.58, 1.58 [21:09:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 4.13, 3.92 [21:11:07] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.77, 2.17, 1.80 [21:11:34] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.38, 1.79, 1.65 [21:11:39] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.10, 5.40, 5.13 [21:11:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.37, 3.35, 3.65 [21:13:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.69, 1.95, 1.76 [21:13:38] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.64, 4.59, 4.87 [21:15:00] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.03, 1.69, 1.70 [21:15:32] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.90, 1.87, 1.72 [21:16:32] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK [21:17:31] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.10, 1.59, 1.63 [21:21:32] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.26, 6.02, 5.19 [21:23:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.69, 4.36, 4.01 [21:25:23] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/deleteBatch.php --wiki=monstergirlencyclopediawiki --r=Requested at [[phab:T9754 [21:25:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:30:25] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.76, 1.73, 1.66 [21:34:23] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.58, 1.66, 1.64 [21:39:53] RECOVERY - wiki.mineland.eu - reverse DNS on sslhost is OK: SSL OK - wiki.mineland.eu reverse DNS resolves to cp23.miraheze.org - CNAME OK [21:45:17] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 1.64, 4.20, 5.80 [21:47:16] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.03, 6.20, 6.31 [21:47:47] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - roblox-wiki.tk All nameservers failed to answer the query. [21:49:15] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.87, 5.38, 5.99 [21:56:13] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.83, 1.65, 1.57 [21:57:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.64, 3.07, 3.72 [21:58:12] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.98, 1.42, 1.50 [22:00:53] PROBLEM - db131 NTP time on db131 is WARNING: NTP WARNING: Offset 0.3988835216 secs [22:01:07] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.57, 3.92, 4.96 [22:03:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.59, 4.58, 4.04 [22:06:49] RECOVERY - db131 NTP time on db131 is OK: NTP OK: Offset 0.0135948658 secs [22:07:27] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.00, 5.77, 5.59 [22:09:02] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.92, 3.84, 3.85 [22:09:21] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 12.16, 7.34, 6.12 [22:10:58] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/b0fffda4f261 [22:10:59] [02miraheze/dns] 07paladox 03b0fffda - Add swiftac111 to dns [22:11:01] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:11:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.03, 4.07, 3.91 [22:11:02] [url] Page not found · GitHub · GitHub | github.com [22:11:02] [02dns] 07paladox opened pull request 03#356: Add swiftac111 to dns - 13https://github.com/miraheze/dns/pull/356 [22:11:03] ... [22:11:08] [02dns] 07paladox closed pull request 03#356: Add swiftac111 to dns - 13https://github.com/miraheze/dns/pull/356 [22:11:08] ... [22:11:09] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [22:11:11] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:11:11] [url] Page not found · GitHub · GitHub | github.com [22:11:12] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/edfeb87bdce2...c2c9b522960d [22:11:13] [url] Comparing edfeb87bdce2...c2c9b522960d · miraheze/dns · GitHub | github.com [22:11:14] [02miraheze/dns] 07paladox 03c2c9b52 - Add swiftac111 to dns (#356) [22:11:40] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/d7b9bcef8054...a84e83fcd30c [22:11:41] [url] Comparing d7b9bcef8054...a84e83fcd30c · miraheze/mw-config · GitHub | github.com [22:11:42] [02miraheze/mw-config] 07Universal-Omega 03a84e83f - add swiftac to disallowed subdomains [22:12:51] miraheze/mw-config - Universal-Omega the build passed. [22:13:01] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.71, 3.72, 3.79 [22:13:50] PROBLEM - cp22 APT on cp22 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [22:15:00] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.83, 5.08, 4.30 [22:15:01] PROBLEM - db131 NTP time on db131 is CRITICAL: NTP CRITICAL: Offset 0.7272014618 secs [22:15:48] RECOVERY - cp22 APT on cp22 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [22:17:06] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query roblox-wiki.tk. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:19:17] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.45, 1.71, 1.61 [22:21:12] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.85, 1.47, 1.54 [22:23:30] PROBLEM - swiftproxy111 NTP time on swiftproxy111 is CRITICAL: NTP CRITICAL: Offset 0.721036762 secs [22:25:32] RECOVERY - swiftproxy111 NTP time on swiftproxy111 is OK: NTP OK: Offset 0.01828527451 secs [22:26:59] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.23, 1.72, 1.63 [22:27:35] !log [@test131] starting deploy of {'config': True} to all [22:27:36] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [22:27:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:29:27] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.29, 1.51, 1.56 [22:29:39] RECOVERY - db131 NTP time on db131 is OK: NTP OK: Offset 0.01958674192 secs [22:30:11] [02miraheze/dns] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/dns/commit/d8e96de69130 [22:30:12] [02miraheze/dns] 07paladox 03d8e96de - Add swiftobject11[1-5] to dns [22:30:14] [02dns] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:30:14] [url] Page not found · GitHub · GitHub | github.com [22:30:15] [02dns] 07paladox opened pull request 03#357: Add swiftobject11[1-5] to dns - 13https://github.com/miraheze/dns/pull/357 [22:30:16] [url] Page not found · GitHub · GitHub | github.com [22:31:45] [02dns] 07paladox closed pull request 03#357: Add swiftobject11[1-5] to dns - 13https://github.com/miraheze/dns/pull/357 [22:31:45] [url] Page not found · GitHub · GitHub | github.com [22:31:46] [02dns] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/dns [22:31:47] [url] Page not found · GitHub · GitHub | github.com [22:31:48] [02miraheze/dns] 07paladox deleted branch 03paladox-patch-1 [22:31:49] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/c2c9b522960d...67d19a6a545e [22:31:50] [url] Comparing c2c9b522960d...67d19a6a545e · miraheze/dns · GitHub | github.com [22:31:51] [02miraheze/dns] 07paladox 0367d19a6 - Add swiftobject11[1-5] to dns (#357) [22:33:59] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a84e83fcd30c...c9c65f465e8b [22:34:00] [url] Comparing a84e83fcd30c...c9c65f465e8b · miraheze/mw-config · GitHub | github.com [22:34:01] [02miraheze/mw-config] 07Universal-Omega 03c9c65f4 - add swiftobject to disallowed subdomains [22:35:08] miraheze/mw-config - Universal-Omega the build passed. [22:36:52] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.35, 2.13, 1.71 [22:38:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.22, 1.84, 1.66 [22:40:07] !log [@mwtask141] starting deploy of {'config': True} to all [22:40:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:40:13] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [22:40:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:40:24] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.37, 1.88, 1.67 [22:40:45] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.47, 1.97, 1.72 [22:43:35] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/usr/local/bin/puppet-enabled] [22:44:22] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.75, 1.83, 1.70 [22:44:55] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.13, 3.98, 5.75 [22:46:37] PROBLEM - cp22 APT on cp22 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [22:46:59] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK [22:47:20] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 5.38, 3.19, 1.70 [22:48:21] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 7.78, 3.23, 2.19 [22:48:32] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.01, 1.69, 1.73 [22:48:42] RECOVERY - cp22 APT on cp22 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [22:48:47] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.19, 4.17, 5.36 [22:49:44] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.50, 3.30, 3.91 [22:50:29] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.46, 1.62, 1.70 [22:50:43] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.96, 4.20, 5.25 [22:51:12] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 0.93, 2.73, 1.93 [22:51:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.92, 4.01, 4.08 [22:52:39] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.92, 5.40, 5.58 [22:54:35] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.04, 4.67, 5.30 [22:55:36] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.71, 3.60, 3.96 [22:56:31] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.46, 3.82, 4.90 [22:56:58] !log [@test131] starting deploy of {'config': True} to all [22:56:59] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [22:57:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:57:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:17] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.12, 1.69, 1.94 [23:00:26] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.22, 4.79, 5.05 [23:01:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.43, 1.73, 1.69 [23:02:16] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.46, 1.99, 2.02 [23:02:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 12.11, 7.65, 6.06 [23:03:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.62, 1.63, 1.65 [23:06:15] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.83, 1.87, 1.96 [23:06:26] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.22, 5.84, 5.67 [23:08:14] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.44, 2.11, 2.04 [23:08:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.71, 6.50, 5.90 [23:09:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.34, 4.19, 3.84 [23:11:01] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [23:14:27] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.05, 4.93, 5.52 [23:15:34] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.91, 3.87, 3.87 [23:16:12] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.56, 1.84, 1.94 [23:16:40] PROBLEM - mon141 NTP time on mon141 is CRITICAL: NTP CRITICAL: Offset 0.6858832836 secs [23:18:11] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.12, 1.89, 1.94 [23:18:26] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.31, 4.00, 4.99 [23:18:43] RECOVERY - mon141 NTP time on mon141 is OK: NTP OK: Offset 0.01627540588 secs [23:20:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/12380d59bd55...4544b83852d2 [23:20:17] [url] Comparing 12380d59bd55...4544b83852d2 · miraheze/puppet · GitHub | github.com [23:20:18] [02miraheze/puppet] 07paladox 034544b83 - Revert "Reduce PHP-FPM childs to 16 on mw121 and mw122 (#2844)" [23:23:51] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.85, 1.83, 1.66 [23:25:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.74, 4.50, 3.98 [23:26:08] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.32, 1.82, 1.97 [23:29:51] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.82, 1.43, 1.56 [23:30:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.11, 5.39, 4.94 [23:34:26] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.62, 4.97, 4.91 [23:36:05] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.23, 1.84, 1.85 [23:38:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.37, 6.25, 5.43 [23:39:43] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.14, 1.84, 1.67 [23:40:03] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.96, 1.94, 1.89 [23:41:40] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.54, 1.62, 1.60 [23:42:26] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.71, 5.69, 5.53 [23:44:02] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.02, 2.04, 1.95 [23:44:20] PROBLEM - cp22 APT on cp22 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [23:46:01] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.04, 1.65, 1.82 [23:46:14] PROBLEM - cp22 Current Load on cp22 is CRITICAL: CRITICAL - load average: 4.52, 3.72, 1.88 [23:46:22] RECOVERY - cp22 APT on cp22 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [23:48:14] RECOVERY - cp22 Current Load on cp22 is OK: OK - load average: 1.05, 2.64, 1.71 [23:48:26] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 13.70, 7.56, 6.17 [23:50:00] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.47, 2.09, 1.95 [23:51:27] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.45, 2.15, 1.76 [23:54:08] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.93, 1.44, 1.71 [23:54:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.34, 1.78, 1.69 [23:54:57] PROBLEM - swiftobject114 PowerDNS Recursor on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:01] PROBLEM - swiftobject114 Current Load on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:01] PROBLEM - swiftobject115 PowerDNS Recursor on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:07] PROBLEM - swiftobject114 NTP time on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:09] PROBLEM - swiftobject114 Puppet on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:20] PROBLEM - swiftobject115 APT on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:27] PROBLEM - swiftobject115 conntrack_table_size on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:29] PROBLEM - swiftobject114 conntrack_table_size on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:34] PROBLEM - swiftobject115 NTP time on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:34] PROBLEM - swiftobject114 APT on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:39] PROBLEM - swiftobject115 Puppet on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:39] PROBLEM - swiftobject115 ferm_active on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:42] PROBLEM - swiftobject115 Disk Space on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:55:45] PROBLEM - swiftobject114 Disk Space on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:45] PROBLEM - swiftobject114 ferm_active on swiftobject114 is CRITICAL: connect to address 2a10:6740::6:206 port 5666: Connection refusedconnect to host 2a10:6740::6:206 port 5666: Connection refused [23:55:53] PROBLEM - swiftobject115 Current Load on swiftobject115 is CRITICAL: connect to address 2a10:6740::6:207 port 5666: Connection refusedconnect to host 2a10:6740::6:207 port 5666: Connection refused [23:56:03] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.54, 1.43, 1.67 [23:58:07] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.54, 1.69, 1.68 [23:58:18] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.96, 4.79, 5.85 [23:59:52] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.51, 1.60, 1.71