[00:00:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.83, 3.41, 3.42 [00:00:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.52, 3.89, 3.93 [00:02:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.20, 3.93, 3.93 [00:02:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.04, 3.61, 3.49 [00:03:52] PROBLEM - discordian.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for discordian.wiki could not be found [00:04:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.71, 3.91, 3.94 [00:06:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.15, 3.69, 3.84 [00:06:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.66, 3.85, 3.64 [00:08:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.36, 3.92, 3.91 [00:08:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.29, 4.07, 3.99 [00:10:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.95, 4.00, 3.95 [00:10:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.12, 3.98, 3.75 [00:10:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.79, 3.93, 3.95 [00:12:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.14, 3.65, 3.66 [00:12:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.09, 3.90, 3.93 [00:14:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.03, 4.47, 4.14 [00:16:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.20, 3.98, 4.00 [00:20:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.10, 3.92, 3.95 [00:20:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.32, 3.95, 3.78 [00:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.96, 3.93, 3.95 [00:22:20] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.57, 3.84, 3.76 [00:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 3.98, 4.04, 3.99 [00:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.63, 3.93, 3.96 [00:26:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.07, 3.82, 3.76 [00:28:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.01, 3.52, 3.65 [00:32:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.70, 3.91, 3.77 [00:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.92, 3.66, 3.93 [00:34:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.80, 3.75, 3.72 [00:38:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.52, 4.23, 4.10 [00:40:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.04, 3.67, 3.75 [00:42:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.60, 3.67, 3.74 [00:42:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.16, 3.93, 3.78 [00:44:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.58, 4.13, 3.91 [00:44:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.90, 3.88, 3.98 [00:46:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.56, 3.75, 3.74 [00:46:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.28, 4.27, 4.13 [00:48:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.87, 3.92, 3.87 [00:48:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.03, 3.88, 3.79 [00:50:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.83, 4.14, 3.95 [00:56:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.24, 4.00, 3.94 [01:02:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.49, 3.77, 3.97 [01:04:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.05, 3.77, 3.81 [01:04:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.82, 3.49, 3.99 [01:06:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.22, 3.75, 3.89 [01:08:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.69, 3.71, 3.86 [01:10:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.77, 3.90, 3.89 [01:12:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.02, 3.54, 3.73 [01:14:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.61, 3.96, 3.89 [01:14:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.63, 4.04, 4.01 [01:20:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.81, 3.90, 3.89 [01:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.17, 3.78, 3.87 [01:24:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.44, 2.74, 1.95 [01:25:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [01:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.02, 3.88, 3.86 [01:28:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.85, 2.69, 2.14 [01:30:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.93, 4.22, 3.98 [01:34:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 0.76, 2.27, 3.92 [01:34:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.19, 3.89, 3.91 [01:35:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 1.96, 3.10, 3.99 [01:38:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 0.47, 1.27, 3.13 [01:38:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.15, 3.60, 3.98 [01:40:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.06, 2.67, 3.37 [01:41:49] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.50, 2.39, 3.36 [01:46:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.39, 3.27, 3.79 [01:50:39] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.71, 2.77, 3.37 [01:54:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.26, 2.67, 3.29 [02:02:28] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.43, 2.70, 2.27 [02:03:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 4.30, 3.27, 2.88 [02:04:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.97, 3.44, 2.59 [02:08:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.42, 3.11, 2.86 [02:10:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.99, 3.00, 2.85 [02:12:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.94, 3.10, 2.77 [02:14:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.51, 3.46, 3.08 [02:14:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.68, 3.62, 3.00 [02:16:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.88, 3.57, 3.01 [02:16:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.06, 3.62, 3.18 [02:18:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.98, 3.69, 3.12 [02:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 7.29, 4.66, 3.58 [02:26:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.08, 3.64, 3.88 [02:28:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.91, 3.77, 3.68 [02:30:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.14, 3.78, 3.68 [02:32:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 5.65, 4.57, 4.15 [02:32:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.78, 3.85, 3.70 [02:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 4.00, 3.80, 3.69 [02:34:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.37, 3.83, 3.77 [02:34:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.22, 4.04, 3.79 [02:34:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.26, 3.96, 3.76 [02:36:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.19, 3.48, 3.82 [02:36:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.37, 4.08, 3.88 [02:36:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.72, 3.85, 3.75 [02:36:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.66, 3.88, 3.76 [02:38:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.79, 3.80, 3.88 [02:40:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.41, 4.16, 3.90 [02:42:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.45, 3.50, 3.77 [02:42:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.37, 3.78, 3.69 [02:42:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.29, 3.87, 3.83 [02:44:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.18, 3.80, 3.84 [02:44:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.58, 3.78, 3.71 [02:44:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.87, 4.52, 4.09 [02:52:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.47, 3.87, 3.98 [02:54:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.76, 3.90, 3.99 [02:54:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.21, 3.89, 3.72 [02:56:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.57, 4.25, 4.10 [02:58:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.31, 3.99, 3.99 [02:58:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.80, 3.77, 3.71 [03:02:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.66, 3.82, 3.72 [03:04:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.91, 3.77, 3.71 [03:06:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.03, 3.65, 3.90 [03:08:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.21, 3.83, 3.93 [03:16:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.86, 3.20, 3.90 [03:18:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.42, 3.60, 3.96 [03:20:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.63, 3.70, 3.57 [03:22:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.33, 3.77, 3.98 [03:22:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.15, 3.42, 3.48 [03:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.72, 3.83, 3.98 [03:26:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.78, 3.76, 3.88 [03:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.11, 4.35, 4.15 [03:28:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.59, 3.68, 3.53 [03:30:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.22, 3.81, 3.97 [03:30:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 2.88, 3.35, 3.43 [03:30:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.47, 3.81, 4.00 [03:32:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.19, 3.66, 3.53 [03:36:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.20, 3.78, 3.87 [03:36:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.71, 3.77, 3.60 [03:38:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.68, 3.83, 3.88 [03:38:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.79, 4.03, 3.71 [03:42:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.40, 4.01, 3.92 [03:42:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.24, 3.65, 3.62 [03:44:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.75, 4.27, 3.84 [03:44:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.00, 4.66, 4.13 [03:50:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.77, 3.94, 3.81 [03:56:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.83, 3.51, 3.91 [04:02:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.71, 3.65, 4.00 [04:04:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.21, 3.77, 3.69 [04:04:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.27, 3.81, 4.01 [04:06:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.54, 3.60, 3.63 [04:06:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.87, 3.81, 3.98 [04:10:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.00, 4.13, 4.04 [04:12:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 1.97, 2.96, 3.85 [04:12:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.12, 3.52, 3.55 [04:12:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.43, 3.80, 3.93 [04:14:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.01, 3.68, 3.69 [04:14:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.34, 3.42, 3.50 [04:16:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.76, 3.70, 3.70 [04:16:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.30, 4.47, 4.12 [04:20:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.59, 3.62, 3.70 [04:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.23, 3.94, 3.77 [04:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.85, 3.88, 3.77 [04:24:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.46, 3.98, 3.64 [04:26:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.23, 3.58, 3.53 [04:28:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.27, 3.80, 3.99 [04:30:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.01, 4.05, 3.87 [04:30:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.55, 3.12, 3.36 [04:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.36, 3.84, 3.81 [04:32:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.19, 3.78, 3.93 [04:34:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.42, 4.25, 3.96 [04:36:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.58, 3.98, 3.66 [04:40:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.72, 3.91, 3.91 [04:42:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.33, 4.21, 4.03 [04:48:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.83, 3.83, 3.93 [04:48:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.76, 3.08, 3.36 [04:48:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.31, 3.82, 3.99 [05:00:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.85, 3.96, 3.86 [05:00:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.92, 4.12, 3.97 [05:02:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.69, 3.83, 3.82 [05:02:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.80, 4.00, 3.95 [05:04:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 1.70, 3.04, 3.86 [05:04:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.10, 3.91, 3.85 [05:04:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.08, 3.95, 3.93 [05:06:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.38, 3.76, 3.80 [05:06:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.46, 3.41, 3.32 [05:08:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.28, 3.62, 3.80 [05:10:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 3.26, 3.40, 3.35 [05:12:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 1.58, 2.21, 3.18 [05:14:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.61, 4.06, 3.89 [05:15:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.57, 3.70, 4.00 [05:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.08, 3.74, 3.81 [05:18:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.60, 3.70, 3.49 [05:19:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 4.98, 4.19, 4.10 [05:20:50] PROBLEM - wiki.sketch.town - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.sketch.town could not be found [05:21:09] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.65, 3.87, 3.51 [05:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.69, 4.09, 3.84 [05:23:04] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.34, 3.67, 3.48 [05:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.68, 3.90, 3.79 [05:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 3.80, 4.04, 3.87 [05:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.04, 3.75, 3.79 [05:29:08] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.71, 3.07, 1.58 [05:30:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 3.94, 4.02, 3.89 [05:31:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 2.28, 3.37, 3.96 [05:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.45, 3.81, 3.83 [05:32:39] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 3.05, 3.22, 3.38 [05:32:40] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.19, 3.15, 3.39 [05:33:08] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 1.91, 2.95, 1.92 [05:35:49] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 1.06, 2.17, 3.33 [05:36:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 0.85, 2.55, 3.90 [05:36:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.36, 3.40, 3.45 [05:38:00] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 16.03, 12.17, 7.74 [05:40:00] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 6.36, 9.68, 7.36 [05:40:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 1.80, 1.95, 3.32 [05:40:14] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.71, 4.07, 2.55 [05:44:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.29, 3.52, 3.55 [05:44:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.19, 3.60, 3.49 [05:48:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.86, 3.81, 3.66 [05:48:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.52, 3.84, 3.63 [05:50:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.08, 3.83, 3.68 [05:52:03] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.02, 3.25, 3.35 [05:52:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.40, 4.01, 3.75 [05:54:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.38, 3.70, 3.67 [05:54:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.81, 3.88, 3.73 [06:04:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.12, 3.90, 3.81 [06:05:20] PROBLEM - cp33 Varnish Backends on cp33 is CRITICAL: 2 backends are down. mw121 mw131 [06:05:55] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 1 backends are down. mw142 [06:06:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.81, 3.62, 3.41 [06:06:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.56, 3.79, 3.78 [06:07:20] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 14 backends are healthy [06:07:55] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 14 backends are healthy [06:08:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.47, 3.92, 3.65 [06:10:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.96, 3.24, 3.30 [06:12:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.69, 3.41, 2.79 [06:16:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.38, 3.88, 3.79 [06:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.02, 3.76, 3.73 [06:18:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.80, 3.80, 3.75 [06:20:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.35, 3.07, 2.95 [06:20:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.37, 3.94, 3.83 [06:22:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.80, 3.85, 3.81 [06:24:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.93, 4.09, 3.87 [06:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.62, 3.89, 3.81 [06:26:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.59, 3.90, 3.82 [06:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.36, 3.68, 3.74 [06:29:08] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 2.80, 4.44, 2.57 [06:30:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.03, 3.86, 3.82 [06:31:08] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 0.72, 3.08, 2.30 [06:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.10, 3.61, 3.74 [06:32:50] !log reception@mwtask141:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/dumpBackup.php --full --include-files --wiki linexwiki > /home/reception/linexwiki03122022.xml [06:32:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:34:37] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.54, 3.65, 3.32 [06:34:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.07, 3.71, 3.75 [06:36:32] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.78, 3.66, 3.36 [06:36:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.36, 3.63, 3.72 [06:38:27] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.97, 3.37, 3.29 [06:40:59] !log [reception@mwtask141] test [06:41:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:42:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.09, 3.83, 3.74 [06:42:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.09, 3.53, 3.61 [06:43:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1247607768 secs [06:44:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.53, 3.49, 3.58 [06:45:48] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=pokeclickerwiki --no-updates --username-prefix=mh:pokeclicker /home/reception/ImportDump/pokeclickerwiki-20221203061528.xml (START) [06:45:49] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=pokeclickerwiki --no-updates --username-prefix=mh:pokeclicker /home/reception/ImportDump/pokeclickerwiki-20221203061528.xml (END - exit=256) [06:45:50] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pokeclickerwiki (START) [06:45:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pokeclickerwiki (END - exit=0) [06:45:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=pokeclickerwiki --active --update (END - exit=0) [06:45:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:06] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=memoniwiki --no-updates --username-prefix=mh:memoni /home/reception/ImportDump/memoniwiki-20221203055738.xml (START) [06:46:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:07] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=memoniwiki --no-updates --username-prefix=mh:memoni /home/reception/ImportDump/memoniwiki-20221203055738.xml (END - exit=256) [06:46:08] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=memoniwiki (START) [06:46:09] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=memoniwiki (END - exit=0) [06:46:10] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=memoniwiki --active --update (END - exit=0) [06:46:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.81, 3.99, 3.76 [06:46:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:46:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:48:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 2.90, 3.53, 3.62 [06:50:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.80, 3.91, 3.86 [06:56:39] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.01, 3.18, 3.40 [06:58:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.94, 3.67, 2.83 [07:00:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.36, 3.85, 2.99 [07:02:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.44, 3.39, 2.93 [07:02:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.03, 3.73, 3.74 [07:02:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.48, 3.33, 3.19 [07:04:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.74, 3.89, 3.42 [07:04:35] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 4.79, 3.95, 3.07 [07:06:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.94, 3.58, 3.15 [07:06:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.18, 3.73, 3.76 [07:06:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.07, 3.58, 3.36 [07:06:30] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.51, 3.36, 2.95 [07:06:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.61, 3.73, 3.56 [07:08:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.90, 3.33, 3.29 [07:08:46] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fohsarchivewiki --no-updates --username-prefix=mh:fohsarchive /home/reception/wiki-fohsarchivemirahezeorg_w/fohsarchivemirahezeorg_w-20220626-history.xml (START) [07:08:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:10:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.91, 3.86, 3.62 [07:12:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.38, 3.22, 3.16 [07:12:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.53, 3.79, 3.63 [07:14:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.17, 3.70, 3.68 [07:14:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.72, 3.75, 3.48 [07:15:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.02365258336 secs [07:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.49, 4.40, 3.93 [07:18:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.54, 3.39, 3.58 [07:18:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.19, 3.66, 3.47 [07:22:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.32, 3.62, 3.50 [07:22:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.58, 3.92, 3.87 [07:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.19, 3.94, 3.76 [07:28:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.60, 3.75, 3.56 [07:28:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.97, 3.77, 2.48 [07:30:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.32, 3.90, 3.86 [07:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.72, 3.97, 3.85 [07:32:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.01, 3.55, 3.54 [07:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.75, 3.85, 3.84 [07:32:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.16, 2.78, 2.43 [07:34:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 6.36, 4.66, 4.10 [07:36:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.16, 3.77, 3.62 [07:37:42] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=fohsarchivewiki --no-updates --username-prefix=mh:fohsarchive /home/reception/wiki-fohsarchivemirahezeorg_w/fohsarchivemirahezeorg_w-20220626-history.xml (END - exit=0) [07:37:43] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fohsarchivewiki (START) [07:37:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:38:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:38:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.44, 3.68, 3.61 [07:40:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.53, 3.92, 3.93 [07:40:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.18, 3.95, 3.72 [07:42:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.36, 3.65, 3.63 [07:42:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.34, 4.15, 2.59 [07:43:20] PROBLEM - vedopedia.witches-empire.com - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.405451774597168 seconds [07:44:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.27, 3.65, 3.72 [07:44:45] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.36, 3.76, 2.65 [07:46:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.62, 3.46, 2.79 [07:46:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.73, 3.86, 3.81 [07:46:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.25, 2.91, 2.47 [07:47:53] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=fohsarchivewiki (END - exit=0) [07:47:54] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=fohsarchivewii --active --update (END - exit=65280) [07:47:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:48:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:48:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.65, 3.52, 2.90 [07:50:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.86, 3.30, 2.89 [07:50:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.92, 3.11, 3.37 [07:58:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.32, 3.49, 3.14 [08:00:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.36, 3.02, 3.01 [08:06:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.98, 3.22, 3.40 [08:08:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.52, 2.96, 2.11 [08:10:18] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.55, 4.78, 2.91 [08:12:22] RECOVERY - vedopedia.witches-empire.com - reverse DNS on sslhost is OK: SSL OK - vedopedia.witches-empire.com reverse DNS resolves to cp23.miraheze.org - CNAME OK [08:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.27, 3.85, 3.63 [08:18:11] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.74, 3.28, 3.00 [08:18:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.02, 3.54, 3.55 [08:18:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.12, 3.66, 3.48 [08:20:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.83, 3.84, 3.64 [08:20:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.27, 3.53, 3.45 [08:24:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.58, 3.74, 3.65 [08:26:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.14, 3.92, 3.72 [08:28:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.49, 3.76, 3.69 [08:29:08] PROBLEM - cp23 Current Load on cp23 is CRITICAL: CRITICAL - load average: 4.67, 2.41, 1.40 [08:30:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.14, 3.94, 3.77 [08:30:41] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 3.14, 3.30, 3.39 [08:31:08] RECOVERY - cp23 Current Load on cp23 is OK: OK - load average: 2.79, 2.82, 1.69 [08:32:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.57, 3.70, 3.70 [08:35:09] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.01, 3.42, 2.68 [08:36:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 4.55, 3.41, 2.28 [08:37:04] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.44, 3.34, 2.74 [08:38:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.97, 4.07, 3.79 [08:38:58] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.93, 3.32, 2.81 [08:40:26] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.83, 3.60, 3.45 [08:40:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.42, 3.76, 3.71 [08:42:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 2.58, 3.54, 2.80 [08:42:23] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.20, 3.72, 3.51 [08:44:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.47, 3.15, 2.74 [08:44:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.89, 3.75, 3.55 [09:03:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset 0.1177396476 secs [09:18:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.66, 4.03, 3.71 [09:19:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset 0.08187460899 secs [09:19:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.89, 3.01, 2.39 [09:20:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.64, 3.12, 2.16 [09:20:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.57, 3.79, 3.65 [09:20:58] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=projectdiablo2wiki --no-updates --username-prefix=mh:projectdiablo2 /home/reception/wiki-wikiprojectdiablo2com_w/wikiprojectdiablo2com_w-20220810-history.xml (START) [09:21:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:21:10] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=projectdiablo2wiki --no-updates --username-prefix=mh:projectdiablo2 /home/reception/wiki-wikiprojectdiablo2com_w/wikiprojectdiablo2com_w-20220810-history.xml (END - exit=256) [09:21:11] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=projectdiablo2wiki (START) [09:21:13] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=projectdiablo2wiki (END - exit=0) [09:21:14] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=projectdiablo2wiki --active --update (END - exit=0) [09:21:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:21:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:21:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:21:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:21:49] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.41, 2.74, 2.36 [09:22:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 2.73, 2.96, 2.22 [09:22:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.44, 3.97, 3.76 [09:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.07, 3.78, 3.67 [09:24:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.88, 3.97, 3.79 [09:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.58, 3.70, 3.65 [09:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 6.25, 4.30, 3.84 [09:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.70, 3.85, 3.72 [09:34:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.41, 3.63, 3.66 [09:36:40] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.37, 3.79, 3.71 [09:37:46] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.77, 3.19, 1.86 [09:38:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.62, 3.97, 3.86 [09:38:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.84, 3.68, 3.66 [09:39:47] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.97, 4.04, 2.34 [09:40:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.63, 4.23, 3.97 [09:45:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.39, 3.29, 2.66 [09:46:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.73, 3.77, 3.62 [09:48:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.35, 3.84, 3.93 [09:48:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.37, 3.71, 3.63 [09:54:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.45, 3.81, 3.68 [09:54:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.38, 3.28, 1.79 [09:56:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.69, 3.80, 3.70 [09:56:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.80, 2.79, 1.80 [10:10:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.11, 3.63, 3.65 [10:12:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.18, 3.47, 3.60 [10:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.59, 3.90, 3.73 [10:18:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.55, 3.72, 3.68 [10:18:56] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=memoniwiki --no-updates --username-prefix=mh:memoni /home/reception/ImportDump/memoniwiki-20221203055738.xml (START) [10:18:58] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=pokeclickerwiki --no-updates --username-prefix=mh:pokeclicker /home/reception/ImportDump/pokeclickerwiki-20221203061528.xml (START) [10:19:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:05] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=memoniwiki --no-updates --username-prefix=mh:memoni /home/reception/ImportDump/memoniwiki-20221203055738.xml (END - exit=0) [10:19:06] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=memoniwiki (START) [10:19:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:07] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=memoniwiki (END - exit=0) [10:19:08] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=memoniwiki --active --update (END - exit=0) [10:19:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:13] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gaiawiki --no-updates --username-prefix=mh:gaia /home/reception/ImportDump/gaiawiki-20221203021410.xml (START) [10:19:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:32] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=twistedwonderlandwiki --no-updates --username-prefix=mh:twistedwonderland /home/reception/ImportDump/twistedwonderlandwiki-20221203001027.xml (START) [10:19:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=gaiawiki --no-updates --username-prefix=mh:gaia /home/reception/ImportDump/gaiawiki-20221203021410.xml (END - exit=0) [10:19:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=gaiawiki (START) [10:19:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:19:59] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=wawaforthewinwiki --no-updates --username-prefix=wikia:wawaforthewin /home/reception/ImportDump/wawaforthewinwiki-20221203000748.xml (START) [10:20:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:20:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:20:30] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=gaiawiki (END - exit=0) [10:20:31] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=gaiawiki --active --update (END - exit=0) [10:20:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:20:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:21:02] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=waysofdarknesswiki --no-updates --username-prefix=mh:waysofdarkness /home/reception/ImportDump/waysofdarknesswiki-20221203075102.xml (START) [10:21:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:24:32] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=wawaforthewinwiki --no-updates --username-prefix=wikia:wawaforthewin /home/reception/ImportDump/wawaforthewinwiki-20221203000748.xml (END - exit=0) [10:24:33] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=wawaforthewinwiki (START) [10:24:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:24:39] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.54, 2.61, 3.23 [10:24:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:26:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=wawaforthewinwiki (END - exit=0) [10:26:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=wawaforthewinwiki --active --update (END - exit=0) [10:27:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:27:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.45, 3.81, 3.66 [10:30:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.52, 3.70, 3.64 [10:40:41] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.86, 3.62, 1.96 [10:46:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 1.60, 2.75, 3.23 [10:49:02] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=waysofdarknesswiki --no-updates --username-prefix=mh:waysofdarkness /home/reception/ImportDump/waysofdarknesswiki-20221203075102.xml (END - exit=0) [10:49:03] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=waysofdarknesswiki (START) [10:49:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:49:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:52:34] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.47, 3.36, 3.58 [10:54:32] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.55, 2.46, 3.22 [10:58:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.65, 3.10, 2.59 [11:02:40] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.80, 3.03, 2.69 [11:04:49] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=pokeclickerwiki --no-updates --username-prefix=mh:pokeclicker /home/reception/ImportDump/pokeclickerwiki-20221203061528.xml (END - exit=0) [11:04:50] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pokeclickerwiki (START) [11:04:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:05:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:14:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=waysofdarknesswiki (END - exit=0) [11:14:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=waysofdarknesswiki --active --update (END - exit=0) [11:14:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:15:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:15:28] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.39, 3.71, 3.16 [11:17:25] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.47, 3.66, 3.22 [11:17:46] PROBLEM - cp22 NTP time on cp22 is WARNING: NTP WARNING: Offset -0.1009958088 secs [11:19:23] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.87, 3.04, 3.04 [11:19:46] RECOVERY - cp22 NTP time on cp22 is OK: NTP OK: Offset -0.07336002588 secs [11:20:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.61, 3.10, 2.51 [11:24:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.12, 3.56, 2.84 [11:26:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.93, 3.74, 3.00 [11:28:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.02, 3.65, 3.05 [11:30:05] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.61, 3.61, 3.11 [11:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.07, 4.04, 3.32 [11:34:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.26, 3.67, 3.27 [11:39:06] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pokeclickerwiki (END - exit=0) [11:39:07] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=pokeclickerwiki --active --update (END - exit=0) [11:39:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:39:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:40:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.66, 3.87, 3.44 [11:42:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.90, 3.86, 3.48 [12:06:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.27, 2.99, 3.31 [12:37:34] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=twistedwonderlandwiki --no-updates --username-prefix=mh:twistedwonderland /home/reception/ImportDump/twistedwonderlandwiki-20221203001027.xml (END - exit=0) [12:37:35] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=twistedwonderlandwiki (START) [12:37:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:37:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:53:33] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=kwiki --no-updates --username-prefix=mh:k /home/reception/ImportDump/kwiki-20221203103255.xml (START) [12:53:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:53:44] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=elsaversewiki --no-updates --username-prefix=mh:elsaverse /home/reception/ImportDump/elsaversewiki-20221203113640.xml (START) [12:53:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:54:59] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=elsaversewiki --no-updates --username-prefix=mh:elsaverse /home/reception/ImportDump/elsaversewiki-20221203113640.xml (END - exit=0) [12:55:00] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=elsaversewiki (START) [12:55:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:33] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=kwiki --no-updates --username-prefix=mh:k /home/reception/ImportDump/kwiki-20221203103255.xml (END - exit=0) [12:55:34] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=kwiki (START) [12:55:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:10] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=elsaversewiki (END - exit=0) [12:56:11] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=elsaversewiki --active --update (END - exit=0) [12:56:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:50] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=kwiki (END - exit=0) [12:56:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=kwiki --active --update (END - exit=0) [12:56:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:04:42] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=mockelectionswiki --no-updates --username-prefix=mh:mockelections /home/reception/mockelectionsmirahezeorg_w/mockelectionsmirahezeorg_w-20220626-history.xml (START) [13:04:43] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=mockelectionswiki --no-updates --username-prefix=mh:mockelections /home/reception/mockelectionsmirahezeorg_w/mockelectionsmirahezeorg_w-20220626-history.xml (END - exit=256) [13:04:44] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mockelectionswiki (START) [13:04:45] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mockelectionswiki (END - exit=0) [13:04:46] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=mockelectionswiki --active --update (END - exit=0) [13:04:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:04:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:04:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:04:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:05:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:05:09] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=mockelectionswiki --no-updates --username-prefix=mh:mockelections /home/reception/wiki-mockelectionsmirahezeorg_w/mockelectionsmirahezeorg_w-20220626-history.xml (START) [13:05:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:34:03] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=twistedwonderlandwiki (END - exit=0) [13:34:04] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=twistedwonderlandwiki --active --update (END - exit=0) [13:34:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:34:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:50:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [13:55:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.42, 3.16, 1.75 [13:56:58] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.89, 2.55, 1.68 [14:12:42] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/366dd09f9b1d [14:12:43] [02miraheze/mw-config] 07paladox 03366dd09 - Set redis logging channel to info as a experiment [14:12:44] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [14:12:45] [url] Page not found · GitHub · GitHub | github.com [14:12:47] [02mw-config] 07paladox opened pull request 03#5001: Set redis logging channel to info as a experiment - 13https://github.com/miraheze/mw-config/pull/5001 [14:12:48] [url] Page not found · GitHub · GitHub | github.com [14:12:50] [02mw-config] 07paladox closed pull request 03#5001: Set redis logging channel to info as a experiment - 13https://github.com/miraheze/mw-config/pull/5001 [14:12:51] [url] Page not found · GitHub · GitHub | github.com [14:12:53] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [14:12:54] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/02a627c32ad5...94f2015c0fbe [14:12:55] [url] Comparing 02a627c32ad5...94f2015c0fbe · miraheze/mw-config · GitHub | github.com [14:12:57] [02miraheze/mw-config] 07paladox 0394f2015 - Set redis logging channel to info as a experiment (#5001) [14:12:58] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/mw-config [14:12:59] [url] Page not found · GitHub · GitHub | github.com [14:13:07] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [14:13:08] !log [paladox@mwtask141] DEPLOY ABORTED: Canary check failed for publictestwiki.com@localhost [14:13:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:13:43] miraheze/mw-config - paladox the build has errored. [14:13:49] miraheze/mw-config - paladox the build has errored. [14:13:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/94f2015c0fbe...2ee6e6ab9243 [14:13:53] [url] Comparing 94f2015c0fbe...2ee6e6ab9243 · miraheze/mw-config · GitHub | github.com [14:13:54] [02miraheze/mw-config] 07paladox 032ee6e6a - Fix [14:14:00] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [14:14:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:09] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [14:14:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:52] miraheze/mw-config - paladox the build passed. [14:21:25] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=mockelectionswiki --no-updates --username-prefix=mh:mockelections /home/reception/wiki-mockelectionsmirahezeorg_w/mockelectionsmirahezeorg_w-20220626-history.xml (END - exit=0) [14:21:26] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mockelectionswiki (START) [14:21:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:21:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:58] !log [@test131] starting deploy of {'config': True} to all [14:26:59] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [14:27:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:27:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:14] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (START) [14:57:15] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (END - exit=256) [14:57:16] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (START) [14:57:17] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (END - exit=0) [14:57:18] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=onlinemusicsongcontestwiki --active --update (END - exit=0) [14:57:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:05:35] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.21, 9.50, 7.54 [15:07:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.57, 8.72, 7.51 [15:08:12] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=secosawikiwiki --missing (START) [15:08:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:08:24] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=secosawikiwiki --missing (END - exit=2) [15:08:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:09:47] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=secosawikiwiki --missing (START) [15:09:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:12:04] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=secosawikiwiki --missing (END - exit=0) [15:12:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:15:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [15:16:47] PROBLEM - jobchron121 Puppet on jobchron121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [15:16:52] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=silverionewscenterwiki --missing (START) [15:16:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:17:06] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=silverionewscenterwiki --missing (END - exit=0) [15:17:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:17:41] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=alethustriawiki --missing (START) [15:17:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:18:44] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=alethustriawiki --missing (END - exit=0) [15:18:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:23:08] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [15:24:46] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/f4a6054ea56d [15:24:47] [02miraheze/puppet] 07paladox 03f4a6054 - redis: disable lua-time-limit [15:24:49] [02puppet] 07paladox created branch 03paladox-patch-9 - 13https://github.com/miraheze/puppet [15:24:49] [url] Page not found · GitHub · GitHub | github.com [15:24:51] [02puppet] 07paladox opened pull request 03#3043: redis: disable lua-time-limit - 13https://github.com/miraheze/puppet/pull/3043 [15:24:51] ... [15:25:44] [02puppet] 07paladox edited pull request 03#3043: redis: disable lua-time-limit - 13https://github.com/miraheze/puppet/pull/3043 [15:25:44] ... [15:26:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.01, 3.37, 1.93 [15:27:08] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 7 = Critical] [15:28:16] [02puppet] 07paladox commented on pull request 03#3043: redis: disable lua-time-limit - 13https://github.com/miraheze/puppet/pull/3043#issuecomment-1336182982 [15:28:17] [url] redis: disable lua-time-limit by paladox · Pull Request #3043 · miraheze/puppet · GitHub | github.com [15:28:21] [02puppet] 07paladox closed pull request 03#3043: redis: disable lua-time-limit - 13https://github.com/miraheze/puppet/pull/3043 [15:28:21] [url] Page not found · GitHub · GitHub | github.com [15:28:31] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-9 [15:28:32] [02puppet] 07paladox deleted branch 03paladox-patch-9 - 13https://github.com/miraheze/puppet [15:28:32] [url] Page not found · GitHub · GitHub | github.com [15:30:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.53, 2.95, 2.15 [15:43:15] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mockelectionswiki (END - exit=0) [15:43:16] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=mockelectionswiki --active --update (END - exit=0) [15:43:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:43:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:50:12] PROBLEM - wiki.ultravanilla.world - LetsEncrypt on sslhost is CRITICAL: connect to address wiki.ultravanilla.world and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [15:59:09] PROBLEM - wiki.ultravanilla.world - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ultravanilla.world could not be found [16:09:47] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=silveriowikiwiki --missing (START) [16:09:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:11:18] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=silveriowikiwiki --missing (END - exit=0) [16:11:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:31] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (START) [16:20:32] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (END - exit=256) [16:20:33] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (START) [16:20:34] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (END - exit=0) [16:20:35] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=onlinemusicsongcontestwiki --active --update (END - exit=0) [16:20:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:21:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:23] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/root/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (START) [16:22:24] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/root/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (END - exit=256) [16:22:25] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (START) [16:22:26] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (END - exit=0) [16:22:27] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=onlinemusicsongcontestwiki --active --update (END - exit=0) [16:22:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:25] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/root/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (START) [16:24:29] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/root/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (END - exit=256) [16:24:30] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (START) [16:24:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:40] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (END - exit=0) [16:24:41] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=onlinemusicsongcontestwiki --active --update (END - exit=0) [16:24:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:25:20] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (START) [16:25:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:29:57] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=onlinemusicsongcontestwiki --no-updates --username-prefix=mh:onlinemusicsongcontest /home/reception/ImportDump/onlinemusicsongcontestwiki-20221203144734.xml (END - exit=0) [16:29:58] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (START) [16:30:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:30:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:31:40] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 7.76, 4.44, 2.79 [16:32:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.38, 3.58, 2.85 [16:32:38] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=onlinemusicsongcontestwiki (END - exit=0) [16:32:39] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=onlinemusicsongcontestwiki --active --update (END - exit=0) [16:32:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.88, 3.11, 2.76 [16:36:35] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 5.50, 3.70, 2.56 [16:42:20] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 2.53, 3.98, 3.14 [16:42:47] RECOVERY - jobchron121 Puppet on jobchron121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [16:44:15] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.51, 4.05, 3.25 [16:45:02] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.78, 3.68, 3.80 [16:46:10] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 2.20, 3.37, 3.10 [16:48:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 5.11, 3.51, 3.63 [16:50:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.83, 3.24, 2.65 [16:50:50] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 5.40, 3.81, 3.11 [16:52:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 5.02, 3.79, 2.92 [16:52:40] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.27, 4.31, 3.49 [16:52:48] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.27, 3.52, 3.08 [16:54:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.78, 3.62, 2.95 [16:54:37] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.89, 3.68, 3.35 [16:56:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 1.90, 2.95, 2.78 [16:56:35] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.37, 3.19, 3.20 [16:56:42] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.26, 3.35, 3.12 [16:56:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.98, 3.46, 3.75 [17:00:27] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.76, 3.95, 3.50 [17:00:55] RECOVERY - gluster112 Current Load on gluster112 is OK: OK - load average: 2.43, 2.74, 3.38 [17:02:25] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.67, 3.40, 3.35 [17:02:28] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 6.26, 4.61, 3.55 [17:07:45] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.23, 4.08, 3.73 [17:28:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.80, 2.89, 3.89 [17:30:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.67, 3.73, 4.08 [17:32:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 2.27, 3.06, 3.78 [17:37:01] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 3.37, 3.21, 3.93 [17:38:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.41, 3.32, 3.67 [17:39:01] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.17, 3.36, 3.88 [17:40:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.79, 3.02, 3.54 [17:42:55] RECOVERY - gluster112 Current Load on gluster112 is OK: OK - load average: 1.77, 2.72, 3.38 [17:44:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.11, 3.70, 2.97 [17:45:01] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 2.05, 3.28, 3.75 [17:46:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.49, 3.19, 2.87 [17:51:01] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.03, 3.45, 3.68 [17:53:01] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 1.65, 2.78, 3.41 [17:57:01] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.60, 3.51, 3.53 [17:59:01] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 3.20, 3.36, 3.47 [18:01:01] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.99, 3.92, 3.65 [18:05:01] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 2.16, 3.43, 3.55 [18:05:45] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 6.04, 4.50, 3.62 [18:06:01] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 15.94, 11.69, 7.53 [18:07:01] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 1.88, 2.92, 3.35 [18:07:07] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=waysofdarknesswiki --missing --user Miraheze_maintenance_script (START) [18:07:08] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=waysofdarknesswiki --missing --user Miraheze_maintenance_script (END - exit=256) [18:07:37] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=waysofdarknesswiki --missing (START) [18:08:00] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 6.18, 9.45, 7.21 [18:08:03] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=onlinemusicsongcontestwiki --missing (START) [18:08:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:36] PROBLEM - mw141 Current Load on mw141 is CRITICAL: CRITICAL - load average: 16.94, 13.05, 8.55 [18:09:03] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=onlinemusicsongcontestwiki --missing (END - exit=256) [18:09:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:09:12] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=onlinemusicsongcontestwiki --missing (START) [18:09:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:09:25] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=onlinemusicsongcontestwiki --missing (END - exit=256) [18:09:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:09] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 13.40, 9.31, 6.20 [18:10:35] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 5.32, 9.94, 7.96 [18:13:01] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 6.13, 5.76, 4.45 [18:14:09] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 10.32, 11.40, 7.89 [18:16:09] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 4.36, 8.81, 7.37 [18:16:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.52, 3.42, 2.84 [18:18:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.42, 3.05, 2.78 [18:21:02] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.56, 3.00, 3.93 [18:21:08] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/2e88153c2898 [18:21:11] [02miraheze/puppet] 07Universal-Omega 032e88153 - redis: use 1GB instead of 1000MB for maxmemory [18:21:14] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [18:21:15] [url] Page not found · GitHub · GitHub | github.com [18:21:15] [02puppet] 07Universal-Omega opened pull request 03#3044: redis: use 1GB instead of 1000MB for maxmemory - 13https://github.com/miraheze/puppet/pull/3044 [18:21:16] [url] Page not found · GitHub · GitHub | github.com [18:23:36] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=waysofdarknesswiki --missing (END - exit=0) [18:23:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:26:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.26, 3.33, 3.76 [18:27:35] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.52, 3.61, 2.19 [18:28:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.72, 2.68, 3.46 [18:32:17] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.15, 10.10, 7.99 [18:32:31] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 8191 bytes in 2.436 second response time [18:32:32] PROBLEM - mw141 Current Load on mw141 is CRITICAL: CRITICAL - load average: 14.38, 11.25, 7.98 [18:33:36] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.23, 3.39, 2.76 [18:34:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.77, 10.46, 8.38 [18:34:18] PROBLEM - mw121 nutcracker process on mw121 is CRITICAL: PROCS CRITICAL: 0 processes with UID = 116 (nutcracker), command name 'nutcracker' [18:34:31] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 5.74, 9.00, 7.55 [18:34:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 11.70, 6.18, 4.53 [18:36:09] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 20.22, 11.80, 7.83 [18:36:15] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.81, 12.02, 9.16 [18:36:22] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.97, 3.93, 2.58 [18:37:43] !log [universalomega@puppet141] sudo -u www-data rm /srv/mediawiki/cache/*opentaxawiki* on mw*, test131, and mwtask141 [18:37:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:37:51] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM cw_wikis WHERE wiki_dbname='opentaxawiki'; [18:37:57] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM mw_settings WHERE s_dbname='opentaxawiki'; [18:38:03] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM mw_namespaces WHERE ns_dbname='opentaxawiki'; [18:38:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:38:09] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM mw_permissions WHERE perm_dbname='opentaxawiki'; [18:38:14] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.47, 10.02, 8.80 [18:38:15] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM localnames WHERE ln_wiki='opentaxawiki'; [18:38:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:38:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:38:23] !log [universalomega@db101] MariaDB [mhglobal]> DELETE FROM localuser WHERE lu_wiki='opentaxawiki'; [18:38:31] !log [universalomega@db101] MariaDB [metawiki]> DELETE FROM echo_unread_wikis WHERE euw_wiki='opentaxawiki'; [18:38:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:40:57] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.88, 3.69, 2.50 [18:42:17] RECOVERY - mw121 nutcracker process on mw121 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [18:42:28] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.283 second response time [18:42:55] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.17, 3.25, 2.50 [18:44:22] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.23, 3.12, 2.94 [18:48:11] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.65, 3.33, 2.90 [18:50:08] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.49, 3.00, 2.83 [18:50:09] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 3.56, 10.20, 10.66 [18:50:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [18:54:09] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 4.44, 7.70, 9.60 [18:56:55] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.32, 3.43, 3.12 [18:58:52] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.79, 3.28, 3.10 [19:05:11] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=xomniwiki --missing (START) [19:05:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:10:09] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 12.87, 11.21, 9.09 [19:13:38] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=xomniwiki --missing (END - exit=0) [19:13:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:09] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 4.58, 9.80, 9.44 [19:16:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.31, 3.47, 3.02 [19:16:54] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.78, 4.04, 3.18 [19:18:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.94, 3.18, 2.41 [19:20:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.79, 3.33, 2.59 [19:21:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 5.70, 4.80, 3.07 [19:24:02] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 3.65, 3.74, 2.43 [19:24:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.28, 4.02, 3.06 [19:27:39] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 18.35, 14.25, 9.14 [19:28:02] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 1.92, 2.92, 2.42 [19:28:04] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 14.94, 12.91, 8.45 [19:29:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 2.85, 3.71, 3.30 [19:29:51] PROBLEM - cp33 Varnish Backends on cp33 is CRITICAL: 2 backends are down. mw131 mw132 [19:30:04] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 7.75, 10.61, 8.13 [19:30:09] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 16.57, 14.05, 10.49 [19:30:14] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:31:37] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 6.64, 10.19, 8.64 [19:31:49] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 4.03, 3.83, 3.39 [19:32:04] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 5.88, 8.81, 7.76 [19:32:05] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 9.76, 15.08, 11.05 [19:32:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.83, 3.97, 3.45 [19:32:39] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.50, 3.97, 3.93 [19:33:04] PROBLEM - cloud11 Current Load on cloud11 is CRITICAL: CRITICAL - load average: 48.48, 41.76, 28.46 [19:33:41] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 1 backends are down. mw121 [19:33:49] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.17, 3.15, 3.19 [19:34:03] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 1.44, 10.27, 9.76 [19:34:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.40, 3.80, 3.95 [19:35:04] PROBLEM - cloud11 Current Load on cloud11 is WARNING: WARNING - load average: 32.21, 41.10, 30.06 [19:35:42] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 14 backends are healthy [19:35:52] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 14 backends are healthy [19:36:02] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 3.03, 7.74, 8.88 [19:36:19] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 1.62, 2.94, 3.17 [19:36:30] PROBLEM - mw141 Current Load on mw141 is CRITICAL: CRITICAL - load average: 17.66, 12.69, 8.51 [19:37:04] RECOVERY - cloud11 Current Load on cloud11 is OK: OK - load average: 13.72, 30.99, 27.68 [19:37:18] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:37:35] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 21.03, 15.28, 10.40 [19:37:35] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 9.64, 12.16, 10.03 [19:38:00] PROBLEM - mw142 Current Load on mw142 is CRITICAL: CRITICAL - load average: 22.61, 16.05, 10.08 [19:38:39] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.305 second response time [19:39:15] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.224 second response time [19:39:28] PROBLEM - cp23 Varnish Backends on cp23 is CRITICAL: 1 backends are down. mw142 [19:39:35] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.26, 11.77, 9.72 [19:39:53] PROBLEM - cp33 Varnish Backends on cp33 is CRITICAL: 1 backends are down. mw141 [19:40:00] PROBLEM - mw142 Current Load on mw142 is WARNING: WARNING - load average: 5.53, 11.71, 9.21 [19:40:09] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 4.11, 9.90, 11.87 [19:40:24] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 1.60, 2.39, 3.28 [19:40:39] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 1.86, 2.59, 3.33 [19:41:28] RECOVERY - cp23 Varnish Backends on cp23 is OK: All 14 backends are healthy [19:41:35] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 4.87, 9.41, 9.10 [19:41:35] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 8.97, 11.59, 10.28 [19:41:53] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 14 backends are healthy [19:42:00] RECOVERY - mw142 Current Load on mw142 is OK: OK - load average: 3.98, 9.21, 8.60 [19:42:04] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 10.89, 12.81, 9.80 [19:42:35] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.04, 13.83, 11.13 [19:43:35] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 6.48, 9.65, 9.72 [19:43:54] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1271417439 secs [19:44:09] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 2.82, 6.00, 9.81 [19:44:29] RECOVERY - mw141 Current Load on mw141 is OK: OK - load average: 4.95, 10.03, 9.58 [19:44:34] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.74, 11.28, 10.52 [19:46:32] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.38, 9.76, 10.06 [19:48:05] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 4.52, 11.70, 10.89 [19:50:04] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 4.45, 9.42, 10.16 [19:50:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [19:52:06] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.74, 4.00, 3.48 [19:54:04] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.77, 3.58, 3.40 [19:55:54] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08155843616 secs [19:56:04] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.81, 3.19, 3.27 [19:56:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.70, 4.68, 2.59 [19:57:21] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=xyrinfewiki --no-updates --username-prefix=mh:xyrinfe /home/reception/ImportDump/xyrinfewiki-20221203162906.xml (START) [19:57:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:58:52] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.29, 3.69, 2.49 [19:59:53] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/4c005726f8bf...0588d50bd82f [19:59:54] [url] Comparing 4c005726f8bf...0588d50bd82f · miraheze/mediawiki · GitHub | github.com [19:59:55] [02miraheze/mediawiki] 07Universal-Omega 030588d50 - Switch SecureLinkFixer to master [20:00:20] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=xyrinfewiki --no-updates --username-prefix=mh:xyrinfe /home/reception/ImportDump/xyrinfewiki-20221203162906.xml (END - exit=0) [20:00:21] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=xyrinfewiki (START) [20:00:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:52] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.79, 2.69, 2.27 [20:00:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.21, 2.72, 3.68 [20:01:08] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±2] 13https://github.com/miraheze/mediawiki/compare/0588d50bd82f...37ff2ad19645 [20:01:09] [url] Comparing 0588d50bd82f...37ff2ad19645 · miraheze/mediawiki · GitHub | github.com [20:01:11] [02miraheze/mediawiki] 07Universal-Omega 0337ff2ad - Revert "Switch SecureLinkFixer to master" [20:03:09] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=xyrinfewiki (END - exit=0) [20:03:10] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=xyrinfewiki --active --update (END - exit=0) [20:03:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:03:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:08:41] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/baea9cd22429...1353567126cb [20:08:42] [url] Comparing baea9cd22429...1353567126cb · miraheze/ErrorPages · GitHub | github.com [20:08:43] [02miraheze/ErrorPages] 07Universal-Omega 031353567 - Fix 'Undefined variable: wgDBname' [20:08:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.39, 3.53, 3.71 [20:08:59] !log [universalomega@mwtask141] starting deploy of {'pull': 'errorpages', 'errorpages': True} to all [20:09:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:09:07] !log [universalomega@mwtask141] finished deploy of {'pull': 'errorpages', 'errorpages': True} to all - SUCCESS in 8s [20:09:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:09:30] miraheze/ErrorPages - Universal-Omega the build passed. [20:09:47] !log [@mwtask141] starting deploy of {'config': True} to all [20:09:52] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [20:09:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7abb7a690856...39dfb3409d54 [20:10:21] [url] Comparing 7abb7a690856...39dfb3409d54 · miraheze/ssl · GitHub | github.com [20:10:22] [02miraheze/ssl] 07MirahezeSSLBot 0339dfb34 - Bot: Add SSL cert for chuckepedia.com [20:10:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 3.78, 3.47, 3.66 [20:11:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ssl/compare/39dfb3409d54...800f40659d32 [20:11:42] [url] Comparing 39dfb3409d54...800f40659d32 · miraheze/ssl · GitHub | github.com [20:11:45] [02miraheze/ssl] 07MirahezeSSLBot 03800f406 - Bot: Add SSL cert for njtelimination.ml [20:12:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/800f40659d32...c94cc6c99114 [20:12:47] [url] Comparing 800f40659d32...c94cc6c99114 · miraheze/ssl · GitHub | github.com [20:12:48] [02miraheze/ssl] 07MirahezeSSLBot 03c94cc6c - Bot: Add SSL cert for redkrayolawiki.org [20:12:52] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=desmoswiki (START) [20:12:53] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=desmoswiki (END - exit=0) [20:12:53] miraheze/ssl - MirahezeSSLBot the build has errored. [20:12:55] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 6.65, 4.65, 4.06 [20:13:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:13:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:13:50] miraheze/ssl - MirahezeSSLBot the build has errored. [20:14:04] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/1353567126cb...e8bc00d4f942 [20:14:05] [url] Comparing 1353567126cb...e8bc00d4f942 · miraheze/ErrorPages · GitHub | github.com [20:14:07] [02miraheze/ErrorPages] 07Universal-Omega 03e8bc00d - db141Wiki: use exit() [20:14:34] !log [universalomega@mwtask141] starting deploy of {'pull': 'errorpages', 'errorpages': True} to all [20:14:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:48] !log [universalomega@mwtask141] finished deploy of {'pull': 'errorpages', 'errorpages': True} to all - SUCCESS in 14s [20:14:50] miraheze/ErrorPages - Universal-Omega the build passed. [20:14:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:15:13] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c94cc6c99114...cb7870dfba2c [20:15:14] [url] Comparing c94cc6c99114...cb7870dfba2c · miraheze/ssl · GitHub | github.com [20:15:15] [02miraheze/ssl] 07Reception123 03cb7870d - not pointed; rm [20:16:18] miraheze/ssl - Reception123 the build has errored. [20:17:25] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=0997798779wiki --missing (START) [20:17:30] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=0997798779wiki --missing (END - exit=256) [20:17:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:31] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=101dalmatianswiki --missing (START) [20:17:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:36] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=101dalmatianswiki --missing (END - exit=0) [20:17:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1134819wiki --missing (START) [20:17:38] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1134819wiki --missing (END - exit=0) [20:17:39] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=131parkhurstwiki --missing (START) [20:17:42] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=131parkhurstwiki --missing (END - exit=0) [20:17:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:43] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=15srwiki --missing (START) [20:17:48] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=15srwiki --missing (END - exit=2) [20:17:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:49] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1910sfashionwiki --missing (START) [20:17:50] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1910sfashionwiki --missing (END - exit=2) [20:17:51] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1x1wiki --missing (START) [20:17:52] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1x1wiki --missing (END - exit=0) [20:17:53] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021pediawiki --missing (START) [20:17:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021pediawiki --missing (END - exit=2) [20:17:55] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021wiki --missing (START) [20:17:56] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021wiki --missing (END - exit=2) [20:17:57] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2084wiki --missing (START) [20:17:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2084wiki --missing (END - exit=2) [20:17:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:59] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=20tomidnightwiki --missing (START) [20:18:00] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=20tomidnightwiki --missing (END - exit=2) [20:18:01] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2100bedfordwiki --missing (START) [20:18:02] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2100bedfordwiki --missing (END - exit=2) [20:18:03] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23vwiki --missing (START) [20:18:04] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23vwiki --missing (END - exit=0) [20:18:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:17] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cb7870dfba2c...56055dc0c10b [20:18:18] [url] Comparing cb7870dfba2c...56055dc0c10b · miraheze/ssl · GitHub | github.com [20:18:19] [02miraheze/ssl] 07Reception123 0356055dc - rm duplicate for njtelimination.ml, already generated before [20:18:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:18:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:19:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:55] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 2.02, 3.47, 3.89 [20:24:09] PROBLEM - chuckepedia.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - chuckepedia.com All nameservers failed to answer the query. [20:26:09] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=0997798779wiki --missing (START) [20:26:15] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=0997798779wiki --missing (END - exit=0) [20:26:16] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=101dalmatianswiki --missing (START) [20:26:17] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1134819wiki --missing (START) [20:26:18] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1134819wiki --missing (END - exit=0) [20:26:19] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=131parkhurstwiki --missing (START) [20:26:20] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=15srwiki --missing (START) [20:26:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:21] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1910sfashionwiki --missing (START) [20:26:22] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1x1wiki --missing (START) [20:26:23] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1x1wiki --missing (END - exit=0) [20:26:24] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021pediawiki --missing (START) [20:26:25] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021wiki --missing (START) [20:26:26] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2084wiki --missing (START) [20:26:27] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=20tomidnightwiki --missing (START) [20:26:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2100bedfordwiki --missing (START) [20:26:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23vwiki --missing (START) [20:26:30] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23vwiki --missing (END - exit=0) [20:26:31] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23wiki --missing (START) [20:26:32] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2b2rwiki --missing (START) [20:26:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2b2rwiki --missing (END - exit=0) [20:26:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:34] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2b2twiki --missing (START) [20:26:34] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.27, 3.63, 3.78 [20:26:35] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2beta2netwiki --missing (START) [20:26:36] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2beta2netwiki --missing (END - exit=0) [20:26:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2builders2toolswiki --missing (START) [20:26:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:38] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2cats2fatwiki --missing (START) [20:26:39] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2jai2022wiki --missing (START) [20:26:40] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2jai2022wiki --missing (END - exit=0) [20:26:41] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2s2dwiki --missing (START) [20:26:42] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2s2dwiki --missing (END - exit=0) [20:26:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:26:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:28] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 1.53, 2.79, 3.45 [20:28:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=101dalmatianswiki --missing (END - exit=2) [20:28:55] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=131parkhurstwiki --missing (END - exit=2) [20:28:56] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=15srwiki --missing (END - exit=2) [20:28:57] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1910sfashionwiki --missing (END - exit=2) [20:28:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=1x1wiki --missing (END - exit=2) [20:28:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:59] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021pediawiki --missing (END - exit=2) [20:29:00] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2021wiki --missing (END - exit=2) [20:29:01] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2084wiki --missing (END - exit=2) [20:29:02] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=20tomidnightwiki --missing (END - exit=2) [20:29:03] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2100bedfordwiki --missing (END - exit=2) [20:29:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:05] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=23wiki --missing (END - exit=2) [20:29:05] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2b2rwiki --missing (END - exit=2) [20:29:06] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2b2twiki --missing (END - exit=2) [20:29:07] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2builders2toolswiki --missing (END - exit=2) [20:29:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2cats2fatwiki --missing (END - exit=2) [20:29:09] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2ndgeneralwcwarwiki --missing (END - exit=2) [20:29:10] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2shanti2022wiki --missing (END - exit=2) [20:29:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:12] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=2speccers2toolswiki --missing (END - exit=2) [20:29:12] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=30xxwiki --missing (END - exit=2) [20:29:13] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=360homebrewwiki --missing (END - exit=2) [20:29:14] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=379thwiki --missing (END - exit=2) [20:29:15] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=3drobotwiki --missing (END - exit=2) [20:29:16] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=400yearsoldwiki --missing (END - exit=2) [20:29:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:17] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=402611wiki --missing (END - exit=2) [20:29:18] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=4dmwiki --missing (END - exit=2) [20:29:19] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=4tranwiki --missing (END - exit=2) [20:29:20] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=5000dcwiki --missing (END - exit=2) [20:29:21] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=50beardsofgreywiki --missing (END - exit=2) [20:29:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:23] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=50nincraftwiki --missing (END - exit=2) [20:29:24] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=50peoplecraftwiki --missing (END - exit=2) [20:29:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:29:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:10] PROBLEM - cp33 Puppet on cp33 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:30:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:31] PROBLEM - test131 Puppet on test131 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:30:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:34] PROBLEM - cp23 Puppet on cp23 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:31:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:57] !log start rebuildImages.php on all wikis with modified foreachwikiindblist and mwscript to log each output if exit code is not zero. [20:31:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:33:35] PROBLEM - mw142 Puppet on mw142 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:34:09] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 5.29, 3.86, 3.70 [20:35:03] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=a320wiki --missing (END - exit=256) [20:35:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:36:06] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 3.32, 3.59, 3.63 [20:38:06] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 4.12, 3.69, 3.65 [20:39:20] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:39:57] PROBLEM - mw122 Puppet on mw122 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:40:08] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:40:18] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:40:39] !log [@mwtask141] starting deploy of {'config': True} to all [20:40:50] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [20:40:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:40:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:41:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=airturkeyfictionalwiki --missing (END - exit=256) [20:41:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:41:53] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=akasatrpgruleswiki --missing (END - exit=256) [20:42:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:44:06] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 2.63, 3.62, 3.73 [20:44:10] PROBLEM - mw131 Puppet on mw131 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:44:11] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:45:17] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:45:50] PROBLEM - mw121 Puppet on mw121 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 3 minutes ago with 2 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/chuckepedia.com.crt],File[/etc/ssl/private/chuckepedia.com.key] [20:46:06] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 5.07, 3.87, 3.78 [20:48:06] PROBLEM - gluster112 Current Load on gluster112 is WARNING: WARNING - load average: 2.74, 3.42, 3.63 [20:49:57] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=altversewiki --missing (END - exit=256) [20:49:59] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=alvinandchipmunkscompilationwiki --missing (END - exit=256) [20:50:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:50:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:50:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ameniawiki --missing (END - exit=256) [20:51:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:51:29] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=amniawiki --missing (END - exit=256) [20:51:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:54:06] PROBLEM - gluster112 Current Load on gluster112 is CRITICAL: CRITICAL - load average: 11.26, 5.35, 4.21 [20:55:45] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=animeshoeswiki --missing (END - exit=256) [20:55:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:56:30] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.96, 3.62, 2.45 [20:58:01] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=anthropomorphicstatespediawiki --missing (END - exit=256) [20:58:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:58:30] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.66, 2.86, 2.31 [20:59:13] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=apeirologywiki --missing (END - exit=256) [20:59:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:00:33] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=apuntesinformaticawiki --missing (END - exit=256) [21:00:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:06:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.48, 3.35, 2.09 [21:10:43] !log [@mwtask141] starting deploy of {'config': True} to all [21:10:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:10:56] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 13s [21:11:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:14:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.11, 3.02, 2.68 [21:14:41] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5796 bytes in 0.009 second response time [21:15:51] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:16:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.70, 3.62, 2.39 [21:16:37] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.368 second response time [21:17:00] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=bearboneswiki --missing (END - exit=256) [21:17:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:17:08] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=beataimwiki --missing (END - exit=256) [21:17:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:17:38] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=belmont3216wiki --missing (END - exit=256) [21:17:48] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.235 second response time [21:17:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:18:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.33, 2.73, 2.21 [21:18:37] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=betaboundwiki --missing (END - exit=256) [21:18:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:20:51] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=bilderbibelnwiki --missing (END - exit=256) [21:20:55] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=bilgallwikiwiki --missing (END - exit=256) [21:20:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:21:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:21:57] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=biuwiki --missing (END - exit=256) [21:22:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:22:44] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=blizzardwiki --missing (END - exit=256) [21:22:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:34:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/a62f240759a6 [21:34:15] [02miraheze/puppet] 07paladox 03a62f240 - swift: add rsyslog rules [21:34:18] [02puppet] 07paladox created branch 03paladox-patch-9 - 13https://github.com/miraheze/puppet [21:34:19] [url] Page not found · GitHub · GitHub | github.com [21:34:19] [02puppet] 07paladox opened pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:34:20] [url] Page not found · GitHub · GitHub | github.com [21:34:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=carhleswiki --missing (END - exit=256) [21:35:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:35:22] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.58, 3.23, 2.90 [21:35:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+1/-0/±0] 13https://github.com/miraheze/puppet/compare/a62f240759a6...292e6a876b45 [21:35:54] [url] Comparing a62f240759a6...292e6a876b45 · miraheze/puppet · GitHub | github.com [21:35:55] [02miraheze/puppet] 07paladox 03292e6a8 - Create swift.rsyslog.conf [21:35:58] [02puppet] 07paladox synchronize pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:35:59] [url] Page not found · GitHub · GitHub | github.com [21:37:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+1/-0/±0] 13https://github.com/miraheze/puppet/compare/292e6a876b45...0e71cbe02cb0 [21:37:17] [url] Comparing 292e6a876b45...0e71cbe02cb0 · miraheze/puppet · GitHub | github.com [21:37:18] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 3.14, 3.16, 2.91 [21:37:19] [02miraheze/puppet] 07paladox 030e71cbe - Create swift.logrotate.conf [21:37:22] [02puppet] 07paladox synchronize pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:37:23] [url] Page not found · GitHub · GitHub | github.com [21:37:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-9 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0e71cbe02cb0...fadca4ad0bab [21:37:44] [url] Comparing 0e71cbe02cb0...fadca4ad0bab · miraheze/puppet · GitHub | github.com [21:37:47] [02miraheze/puppet] 07paladox 03fadca4a - Update init.pp [21:37:49] [02puppet] 07paladox synchronize pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:37:49] [url] Page not found · GitHub · GitHub | github.com [21:38:02] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ccrpivwiki --missing (END - exit=256) [21:38:11] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ccwikiwiki --missing (END - exit=256) [21:38:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:38:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:39:12] [02puppet] 07paladox edited pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:39:13] ... [21:39:42] !log [@mwtask141] starting deploy of {'config': True} to all [21:39:58] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 15s [21:40:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:40:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:40:07] [02puppet] 07paladox closed pull request 03#3045: swift: add rsyslog rules - 13https://github.com/miraheze/puppet/pull/3045 [21:40:08] [url] Page not found · GitHub · GitHub | github.com [21:40:08] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-9 [21:40:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+2/-0/±1] 13https://github.com/miraheze/puppet/compare/773a8ace22b1...f48295d4a87c [21:40:10] [url] Comparing 773a8ace22b1...f48295d4a87c · miraheze/puppet · GitHub | github.com [21:40:11] [02miraheze/puppet] 07paladox 03f48295d - swift: add rsyslog rules (#3045) [21:40:12] [02puppet] 07paladox deleted branch 03paladox-patch-9 - 13https://github.com/miraheze/puppet [21:40:13] [url] Page not found · GitHub · GitHub | github.com [21:40:20] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=ceskygeocachingwiki --missing (END - exit=256) [21:40:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:41:14] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.16, 3.61, 3.18 [21:43:11] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.75, 3.22, 3.09 [21:44:14] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5796 bytes in 0.014 second response time [21:45:14] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.68, 3.13, 2.77 [21:46:10] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.282 second response time [21:46:49] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:46:51] PROBLEM - mw142 MediaWiki Rendering on mw142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:47:10] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.65, 3.02, 2.78 [21:48:21] PROBLEM - cp33 Varnish Backends on cp33 is CRITICAL: 1 backends are down. mw121 [21:48:39] PROBLEM - cp23 Varnish Backends on cp23 is CRITICAL: 1 backends are down. mw121 [21:48:40] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: 1 backends are down. mw121 [21:48:45] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.387 second response time [21:48:50] RECOVERY - mw142 MediaWiki Rendering on mw142 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.215 second response time [21:50:21] RECOVERY - cp33 Varnish Backends on cp33 is OK: All 14 backends are healthy [21:50:39] RECOVERY - cp23 Varnish Backends on cp23 is OK: All 14 backends are healthy [21:50:40] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 14 backends are healthy [21:50:42] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 24.27, 12.21, 7.12 [21:50:51] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.80, 3.29, 2.71 [21:50:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=cldswiki --missing (END - exit=256) [21:51:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:51:28] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=clinuxwiki --missing (END - exit=256) [21:51:32] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=cllwiki --missing (END - exit=256) [21:51:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:51:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:52:04] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=cloudwheatwiki --missing (END - exit=256) [21:52:06] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=cloudwikiwiki --missing (END - exit=256) [21:52:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:52:08] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.93, 3.62, 2.74 [21:52:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:52:51] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.06, 2.88, 2.63 [21:52:54] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=cnrailwikiwiki --missing (END - exit=256) [21:52:57] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.14, 3.48, 3.27 [21:52:58] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.51, 4.23, 3.41 [21:53:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:53:18] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=codegeasswiki --missing (END - exit=256) [21:53:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:53:28] [02mw-config] 07ugochimobi reviewed pull request 03#4998 commit - 13https://github.com/miraheze/mw-config/pull/4998#discussion_r1038876213 [21:53:29] [url] disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki by ugochimobi · Pull Request #4998 · miraheze/mw-config · GitHub | github.com [21:53:50] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=colgannuwhowiki --missing (END - exit=256) [21:53:55] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=collabvmwiki --missing (END - exit=256) [21:53:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:53:58] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=collectif21wiki --missing (END - exit=256) [21:53:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:54:08] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.39, 4.52, 3.15 [21:54:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:54:25] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=colourpediawiki --missing (END - exit=256) [21:54:30] !log [universalomega@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildImages.php --wiki=combinatorialnumbertheorywiki --missing (END - exit=256) [21:54:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:54:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:54:48] [02mw-config] 07Universal-Omega reviewed pull request 03#4998 commit - 13https://github.com/miraheze/mw-config/pull/4998#discussion_r1038876350 [21:54:49] [url] disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki by ugochimobi · Pull Request #4998 · miraheze/mw-config · GitHub | github.com [21:56:51] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.18, 3.57, 3.00 [21:57:42] [02mw-config] 07ugochimobi synchronize pull request 03#4998: disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4998 [21:57:43] [url] Page not found · GitHub · GitHub | github.com [21:58:08] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.59, 3.68, 3.13 [21:58:12] [02mw-config] 07ugochimobi reviewed pull request 03#4998 commit - 13https://github.com/miraheze/mw-config/pull/4998#discussion_r1038876696 [21:58:13] [url] disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki by ugochimobi · Pull Request #4998 · miraheze/mw-config · GitHub | github.com [21:58:48] miraheze/mw-config - ugochimobi the build passed. [21:58:51] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 5.39, 4.01, 3.21 [22:00:32] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 5.17, 11.16, 9.64 [22:01:53] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/4615d20517bd [22:01:55] [02miraheze/mw-config] 07Universal-Omega 034615d20 - Re-enable Swift site notice for migration [22:01:58] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://github.com/miraheze/mw-config [22:01:59] [url] Page not found · GitHub · GitHub | github.com [22:02:01] [02mw-config] 07Universal-Omega opened pull request 03#5002: Re-enable Swift site notice for migration - 13https://github.com/miraheze/mw-config/pull/5002 [22:02:02] [url] Page not found · GitHub · GitHub | github.com [22:02:08] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.33, 3.12, 3.05 [22:02:30] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 3.96, 9.09, 9.08 [22:03:15] miraheze/mw-config - Universal-Omega the build passed. [22:04:11] [02mw-config] 07Universal-Omega closed pull request 03#4998: disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki - 13https://github.com/miraheze/mw-config/pull/4998 [22:04:11] [url] Page not found · GitHub · GitHub | github.com [22:04:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2ee6e6ab9243...2489085e05d6 [22:04:13] [url] Comparing 2ee6e6ab9243...2489085e05d6 · miraheze/mw-config · GitHub | github.com [22:04:14] [02miraheze/mw-config] 07ugochimobi 032489085 - disable localClientDatabases, enable allowDataAccessInUserLanguage on gratisdatawiki (#4998) [22:04:31] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.29, 3.78, 3.96 [22:04:51] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.94, 3.28, 3.34 [22:04:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.17, 3.43, 3.21 [22:05:25] miraheze/mw-config - Universal-Omega the build passed. [22:06:51] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 1.81, 2.90, 3.05 [22:09:38] !log [@mwtask141] starting deploy of {'config': True} to all [22:09:54] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 16s [22:09:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:10:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:10:12] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.08, 1.79, 1.46 [22:12:09] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.88, 1.85, 1.53 [22:14:06] RECOVERY - phab121 Current Load on phab121 is OK: OK - load average: 1.19, 1.60, 1.47 [22:14:09] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.84, 2.87, 3.38 [22:16:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.71, 2.78 [22:17:01] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/c7914d8595a7 [22:17:03] [02miraheze/mw-config] 07Universal-Omega 03c7914d8 - Sitenotice: use localised message for learn more button [22:17:05] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://github.com/miraheze/mw-config [22:17:05] [url] Page not found · GitHub · GitHub | github.com [22:17:06] [02mw-config] 07Universal-Omega opened pull request 03#5003: Sitenotice: use localised message for learn more button - 13https://github.com/miraheze/mw-config/pull/5003 [22:17:06] ... [22:18:06] miraheze/mw-config - Universal-Omega the build passed. [22:18:38] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.35, 2.76, 2.53 [22:20:41] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 [22:20:42] [02mw-config] 07Universal-Omega closed pull request 03#5003: Sitenotice: use localised message for learn more button - 13https://github.com/miraheze/mw-config/pull/5003 [22:20:43] [url] Page not found · GitHub · GitHub | github.com [22:20:45] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2489085e05d6...378d759c4a27 [22:20:46] [url] Comparing 2489085e05d6...378d759c4a27 · miraheze/mw-config · GitHub | github.com [22:20:47] [02miraheze/mw-config] 07Universal-Omega 03378d759 - Sitenotice: use localised message for learn more button (#5003) [22:20:50] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-3 - 13https://github.com/miraheze/mw-config [22:20:51] ... [22:21:29] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/4615d20517bd...f002b8a08f8e [22:21:29] [url] Comparing 4615d20517bd...f002b8a08f8e · miraheze/mw-config · GitHub | github.com [22:21:30] [02miraheze/mw-config] 07Universal-Omega 03f002b8a - Re-enable Swift site notice for migration [22:21:33] [02mw-config] 07Universal-Omega synchronize pull request 03#5002: Re-enable Swift site notice for migration - 13https://github.com/miraheze/mw-config/pull/5002 [22:21:34] [url] Page not found · GitHub · GitHub | github.com [22:21:39] miraheze/mw-config - Universal-Omega the build passed. [22:22:42] miraheze/mw-config - Universal-Omega the build passed. [22:23:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.13, 3.24, 3.20 [22:25:56] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 2.46, 3.06, 3.15 [22:26:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.46, 3.81, 2.40 [22:27:09] !log [@test131] starting deploy of {'config': True} to all [22:27:10] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [22:27:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:30] !log add 1g of swap to swiftproxy111/131 [22:27:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.61, 3.37, 2.41 [22:31:10] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:32:23] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.68, 6.07, 3.68 [22:33:07] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.226 second response time [22:40:05] !log [@mwtask141] starting deploy of {'config': True} to all [22:40:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:40:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.20, 3.49, 3.40 [22:40:22] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 17s [22:40:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:42:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.83, 3.12, 3.26 [22:43:54] !log /usr/local/bin/fileLockScript.sh /tmp/rotten_links_file_lock "/usr/bin/nice -n 15 /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/extensions/RottenLinks/maintenance/updateExternalLinks.php" [22:43:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 6.18, 4.45, 3.45 [22:43:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:44:06] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03fix-updateExternalLinks [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/507497a5b8dc [22:44:08] [02miraheze/puppet] 07Universal-Omega 03507497a - mediawiki: fix `updateExternalLinks.php` cron [22:44:11] [02puppet] 07Universal-Omega created branch 03fix-updateExternalLinks - 13https://github.com/miraheze/puppet [22:44:11] [url] Page not found · GitHub · GitHub | github.com [22:44:12] [02puppet] 07Universal-Omega opened pull request 03#3046: mediawiki: fix `updateExternalLinks.php` cron - 13https://github.com/miraheze/puppet/pull/3046 [22:44:12] [url] Page not found · GitHub · GitHub | github.com [22:44:50] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.97, 3.66, 3.06 [22:46:10] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 12.09, 13.70, 8.75 [22:46:15] [02puppet] 07Universal-Omega edited pull request 03#3046: mediawiki: fix `updateExternalLinks.php` cron - 13https://github.com/miraheze/puppet/pull/3046 [22:46:16] [url] Page not found · GitHub · GitHub | github.com [22:46:32] [02puppet] 07Universal-Omega closed pull request 03#3046: mediawiki: fix `updateExternalLinks.php` cron - 13https://github.com/miraheze/puppet/pull/3046 [22:46:32] ... [22:46:33] [02miraheze/puppet] 07Universal-Omega deleted branch 03fix-updateExternalLinks [22:46:36] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f48295d4a87c...3b515cc87d79 [22:46:36] [url] Comparing f48295d4a87c...3b515cc87d79 · miraheze/puppet · GitHub | github.com [22:46:37] [02miraheze/puppet] 07Universal-Omega 033b515cc - mediawiki: fix `updateExternalLinks.php` cron (#3046) [22:46:38] [02puppet] 07Universal-Omega deleted branch 03fix-updateExternalLinks - 13https://github.com/miraheze/puppet [22:46:38] [url] Page not found · GitHub · GitHub | github.com [22:47:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.59, 3.90, 3.45 [22:51:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 5.95, 4.52, 3.78 [22:52:32] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/82f7cd301aee...6b1e0dd863c0 [22:52:33] [url] Comparing 82f7cd301aee...6b1e0dd863c0 · miraheze/ManageWiki · GitHub | github.com [22:52:34] [02miraheze/ManageWiki] 07Universal-Omega 036b1e0dd - Fix Descriptor with no class for contentmodel-namespace [22:52:51] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.78, 3.37, 3.19 [22:53:32] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/37ff2ad19645...abf8bea9d338 [22:53:33] [url] Comparing 37ff2ad19645...abf8bea9d338 · miraheze/mediawiki · GitHub | github.com [22:53:35] [02miraheze/mediawiki] 07Universal-Omega 03abf8bea - Update ManageWiki [22:53:57] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/5a1b4855b86b...94c0c765d66f [22:53:58] [url] Comparing 5a1b4855b86b...94c0c765d66f · miraheze/mediawiki · GitHub | github.com [22:53:59] !log [universalomega@mwtask141] starting deploy of {'world': True} to all [22:53:59] [02miraheze/mediawiki] 07Universal-Omega 0394c0c76 - Update ManageWiki [22:54:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:04] miraheze/ManageWiki - Universal-Omega the build passed. [22:56:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.73, 3.60, 3.32 [22:59:03] !log [universalomega@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 304s [22:59:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.35, 3.96, 3.51 [23:02:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.96, 4.25, 3.16 [23:02:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.51, 3.74, 3.48 [23:03:22] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.84, 3.48, 2.98 [23:04:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.40, 3.24, 2.92 [23:04:52] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 4.10, 4.02, 3.62 [23:05:19] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.31, 3.78, 3.15 [23:06:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.91, 3.95, 3.65 [23:07:17] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.51, 3.28, 3.04 [23:08:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 6.33, 3.72, 2.64 [23:10:10] !log [@mwtask141] starting deploy of {'config': True} to all [23:10:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:10:20] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.69, 3.96, 2.89 [23:10:24] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 13s [23:10:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:12:10] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 3.63, 7.20, 11.87 [23:14:20] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.35, 2.90, 2.72 [23:15:27] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.23, 1.67, 1.40 [23:15:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 2.47, 3.30, 3.82 [23:16:10] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 3.75, 5.36, 10.04 [23:17:24] RECOVERY - phab121 Current Load on phab121 is OK: OK - load average: 0.97, 1.45, 1.35 [23:18:51] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 2.13, 2.98, 3.36 [23:21:56] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 1.87, 2.47, 3.27 [23:26:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.47, 3.29, 3.30 [23:28:00] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 13.70, 6.71, 3.49 [23:30:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.14, 4.45, 3.75 [23:31:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.38, 3.95, 3.05 [23:32:49] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.04, 4.05, 3.66 [23:32:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 3.45, 3.99, 3.66 [23:33:44] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 0.87, 2.89, 2.76 [23:34:04] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.25, 3.35, 2.91 [23:36:04] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.34, 3.07, 2.87 [23:36:39] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 3.18, 3.67, 3.59 [23:38:11] [02mw-config] 07Universal-Omega closed pull request 03#5002: Re-enable Swift site notice for migration - 13https://github.com/miraheze/mw-config/pull/5002 [23:38:11] [url] Page not found · GitHub · GitHub | github.com [23:38:13] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [23:38:14] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://github.com/miraheze/mw-config [23:38:14] [url] Page not found · GitHub · GitHub | github.com [23:38:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/378d759c4a27...8c65e9cdb313 [23:38:16] [url] Comparing 378d759c4a27...8c65e9cdb313 · miraheze/mw-config · GitHub | github.com [23:38:19] [02miraheze/mw-config] 07Universal-Omega 038c65e9c - Re-enable Swift site notice for migration (#5002) [23:38:45] !log [universalomega@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:38:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:38:56] !log [universalomega@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [23:39:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:39:18] miraheze/mw-config - Universal-Omega the build passed. [23:40:41] !log [@mwtask141] starting deploy of {'config': True} to all [23:40:51] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [23:40:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:41:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:42:04] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 4.68, 3.69, 3.13 [23:42:26] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 4.80, 4.27, 3.82 [23:42:51] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 6.31, 4.41, 3.86 [23:44:10] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 11.14, 10.31, 8.11 [23:46:10] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 14.02, 11.66, 8.86 [23:47:13] [02puppet] 07Universal-Omega closed pull request 03#3044: redis: use 1GB instead of 1000MB for maxmemory - 13https://github.com/miraheze/puppet/pull/3044 [23:47:14] [url] Page not found · GitHub · GitHub | github.com [23:47:16] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3b515cc87d79...ad63e899dc2d [23:47:17] [url] Comparing 3b515cc87d79...ad63e899dc2d · miraheze/puppet · GitHub | github.com [23:47:17] [02miraheze/puppet] 07Universal-Omega 03ad63e89 - redis: use 1GB instead of 1000MB for maxmemory (#3044) [23:47:17] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 3.29, 3.82, 3.07 [23:47:18] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [23:47:19] [url] Page not found · GitHub · GitHub | github.com [23:47:20] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [23:49:15] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 5.72, 4.34, 3.34 [23:53:28] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.92, 4.77, 3.51 [23:54:04] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 3.91, 3.85, 3.57 [23:56:36] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.70, 3.23, 2.28 [23:57:04] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 2.88, 3.79, 3.53 [23:57:15] !log [@test131] starting deploy of {'config': True} to all [23:57:16] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [23:57:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:57:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:58:04] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 2.19, 3.25, 3.40 [23:59:01] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 2.55, 3.31, 3.38