[00:01:40] [02mw-config] 07Universal-Omega commented on pull request 03#5380: Add $wgRSSUserAgent (T11425) - 13https://github.com/miraheze/mw-config/pull/5380#issuecomment-1876129926 [00:02:14] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.45, 7.83, 7.44 [00:04:21] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 17.38, 11.49, 6.25 [00:05:42] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.01, 7.59, 7.35 [00:06:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.63, 3.46, 2.82 [00:07:41] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.16, 6.95, 7.13 [00:08:13] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.40, 3.79, 3.01 [00:10:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.34, 3.23, 2.90 [00:12:21] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.13, 6.45, 6.30 [00:13:37] PROBLEM - mail121 Current Load on mail121 is CRITICAL: LOAD CRITICAL - total load average: 3.44, 1.89, 1.03 [00:13:59] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.45, 7.88, 7.51 [00:15:56] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.68, 7.97, 7.60 [00:16:13] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.53, 3.46, 3.12 [00:16:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [00:16:49] CosmicAlpha: it was unintentional [00:17:34] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.76, 7.68, 7.39 [00:18:12] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.48, 4.04, 3.36 [00:19:33] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.12, 7.10, 7.21 [00:19:36] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.53, 1.64, 1.30 [00:20:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.52, 3.50, 3.25 [00:23:31] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.73, 7.40, 7.26 [00:23:45] Alright thanks just making sure [00:25:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.91, 7.37, 7.28 [00:25:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.57, 7.39, 7.28 [00:27:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.52, 7.13, 7.21 [00:28:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.96, 3.26, 3.26 [00:35:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.03, 7.63, 7.32 [00:36:58] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.88, 3.95, 3.51 [00:37:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.62, 7.23, 7.21 [00:38:56] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.76, 3.94, 3.57 [00:43:29] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.52, 5.99, 6.66 [00:44:44] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 3.01, 3.30, 3.36 [00:45:39] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [00:46:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [00:49:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.26, 6.28, 6.71 [00:52:40] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.18, 4.20, 3.64 [00:54:35] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.01, 3.69, 3.52 [00:55:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.93, 7.14, 6.86 [00:55:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.14, 7.03, 6.83 [00:56:29] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.21, 3.09, 3.32 [01:00:21] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.72, 3.36, 3.37 [01:01:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.53, 7.42, 7.03 [01:01:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.36, 7.45, 7.15 [01:04:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.90, 3.15, 3.29 [01:08:12] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.51, 3.77, 3.50 [01:11:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.80, 7.40, 7.12 [01:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.39, 7.34, 7.50 [01:14:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.13, 3.95, 3.74 [01:17:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 5.78, 7.47, 7.36 [01:22:14] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.15, 3.67, 3.66 [01:25:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.68, 6.32, 6.78 [01:26:13] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.39, 3.67, 3.67 [01:28:13] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.92, 3.83, 3.71 [01:32:13] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.45, 3.51, 3.63 [01:33:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.24, 7.78, 7.43 [01:33:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.86, 6.78, 6.83 [01:35:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.38, 7.53, 7.37 [01:35:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.65, 6.69, 6.79 [01:36:14] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.67, 4.06, 3.78 [01:42:13] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.87, 3.74, 3.76 [01:43:07] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 11.64, 11.16, 6.75 [01:43:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.85, 7.61, 7.06 [01:44:17] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.09, 4.14, 3.89 [01:45:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.68, 7.37, 7.02 [01:46:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.71, 3.66, 3.75 [01:50:53] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 3.21, 6.94, 6.61 [01:52:50] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 17.40, 11.32, 8.25 [01:53:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.55, 6.55, 6.77 [01:56:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.79, 2.36, 3.14 [01:57:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.14, 7.69, 7.30 [01:59:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.41, 7.36, 7.21 [02:04:22] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 8131 MB (6% inode=99%); [02:06:22] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7604 MB (5% inode=99%); [02:10:22] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 8143 MB (6% inode=99%); [02:11:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.56, 7.43, 7.21 [02:12:37] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.47, 7.33, 6.83 [02:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.45, 7.26, 7.17 [02:18:28] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 5.14, 7.21, 7.04 [02:20:21] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 1.71, 3.85, 7.26 [02:22:10] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [02:22:21] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.17, 3.67, 6.77 [02:22:22] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7959 MB (5% inode=99%); [02:27:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.85, 7.69, 7.32 [02:29:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.22, 7.41, 7.26 [02:32:09] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.13, 7.42, 7.14 [02:34:07] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.04, 6.77, 6.93 [02:35:37] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [02:41:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.39, 7.68, 7.46 [02:43:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.93, 7.27, 7.33 [02:43:54] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.38, 7.43, 7.18 [02:47:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.16, 7.67, 7.37 [02:55:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.08, 6.08, 6.71 [03:00:36] RECOVERY - db131 Backups SQL on db131 is OK: FILE_AGE OK: /var/log/sql-backup.log is 35 seconds old and 0 bytes [03:03:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.69, 7.81, 7.39 [03:03:42] PROBLEM - mon141 Backups Grafana on mon141 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864219 seconds old and 785 bytes [03:07:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.46, 7.87, 7.51 [03:07:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.22, 7.11, 6.86 [03:08:18] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:09:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.66, 8.15, 7.66 [03:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.06, 7.81, 7.65 [03:15:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.20, 7.83, 7.67 [03:17:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.60, 7.64, 7.61 [03:17:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.65, 6.26, 6.71 [03:23:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 10.12, 7.43, 6.94 [03:25:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.98, 7.74, 7.14 [03:27:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.95, 8.08, 7.72 [03:29:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.94, 7.84, 7.66 [03:31:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.11, 8.09, 7.79 [03:31:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.22, 7.57, 7.21 [03:33:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 5.26, 6.79, 6.98 [03:35:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.54, 6.35, 6.79 [03:37:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.11, 7.35, 7.66 [03:43:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.99, 7.21, 6.99 [03:47:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 3.89, 5.75, 6.47 [03:57:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.84, 7.75, 7.24 [03:57:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.41, 6.86, 6.59 [03:59:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.09, 7.29, 6.78 [04:03:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.40, 7.15, 7.14 [04:03:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.16, 6.77, 6.75 [04:05:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.63, 7.60, 7.30 [04:09:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.30, 7.17, 6.91 [04:11:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.09, 7.29, 7.36 [04:17:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.79, 7.45, 7.27 [04:19:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.11, 7.23, 7.21 [04:19:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.92, 6.28, 6.71 [04:23:31] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.30, 7.77, 7.39 [04:23:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.02, 7.34, 7.05 [04:25:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.62, 7.50, 7.15 [04:27:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.32, 7.95, 7.58 [04:29:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.25, 7.46, 7.16 [04:31:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.26, 7.47, 7.21 [04:33:19] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.43, 6.68, 6.10 [04:33:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.69, 7.63, 7.45 [04:35:19] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 5.10, 6.15, 5.98 [04:37:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.80, 7.97, 7.64 [04:39:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.21, 7.42, 7.20 [04:41:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.31, 6.95, 7.05 [04:45:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.18, 8.13, 7.49 [04:47:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.06, 7.48, 7.33 [04:50:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [04:53:19] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 6.86, 6.64, 6.26 [04:53:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.96, 6.23, 6.77 [04:55:19] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 5.51, 6.27, 6.18 [04:57:36] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 2.00, 1.54, 1.03 [04:59:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.82, 6.77, 6.84 [05:01:31] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.45, 7.64, 7.42 [05:01:36] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 1.02, 1.47, 1.13 [05:09:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 4.33, 6.86, 7.35 [05:11:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.71, 6.18, 6.75 [05:15:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.59, 7.70, 7.48 [05:17:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.92, 7.74, 7.52 [05:17:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.83, 6.86, 6.83 [05:21:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.10, 7.66, 7.13 [05:23:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.78, 7.66, 7.19 [05:25:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.11, 7.35, 7.35 [05:25:53] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 7620MiB (10% inode=89%); [05:29:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.20, 7.41, 7.40 [05:29:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.06, 6.19, 6.69 [05:33:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.86, 7.60, 7.43 [05:33:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.31, 6.95, 6.86 [05:35:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.65, 6.78, 6.81 [05:41:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.38, 7.79, 7.71 [05:45:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [05:47:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.13, 7.59, 7.62 [05:47:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.32, 7.33, 7.07 [05:49:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.06, 7.36, 7.53 [05:49:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.54, 7.22, 7.04 [05:51:36] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 1.80, 1.33, 0.84 [05:51:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.06, 7.35, 7.10 [05:53:36] PROBLEM - mail121 Current Load on mail121 is CRITICAL: LOAD CRITICAL - total load average: 2.86, 1.90, 1.11 [05:53:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.28, 7.26, 7.10 [05:57:36] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.67, 1.37, 1.08 [05:57:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.35, 6.36, 6.78 [06:01:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.46, 7.10, 7.16 [06:07:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.58, 7.88, 7.53 [06:16:40] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [06:18:33] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.42, 6.86, 6.59 [06:20:30] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.43, 6.65, 6.54 [06:21:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.02, 8.02, 7.68 [06:21:33] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [06:23:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.30, 7.41, 7.49 [06:25:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.79, 7.78, 7.61 [06:27:17] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.33, 7.02, 6.76 [06:27:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.55, 7.15, 7.38 [06:29:14] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.01, 6.59, 6.63 [06:31:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.07, 7.33, 7.37 [06:33:05] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.38, 7.05, 6.80 [06:38:59] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 12.07, 9.18, 7.67 [06:40:31] PROBLEM - swiftobject112 Current Load on swiftobject112 is WARNING: WARNING - load average: 7.01, 6.55, 5.96 [06:42:27] PROBLEM - swiftobject112 Current Load on swiftobject112 is CRITICAL: CRITICAL - load average: 8.06, 6.92, 6.16 [06:44:22] RECOVERY - swiftobject112 Current Load on swiftobject112 is OK: OK - load average: 6.79, 6.74, 6.18 [06:46:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [06:46:48] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.13, 7.82, 7.64 [06:47:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.19, 7.52, 7.91 [06:53:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.13, 7.44, 7.69 [06:55:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.82, 6.70, 7.38 [06:59:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.60, 7.43, 7.50 [07:00:49] PROBLEM - kodiak.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'kodiak.wiki' expires in 15 day(s) (Sat 20 Jan 2024 06:43:17 GMT +0000). [07:01:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1cb83944a961...2d30c4185f33 [07:01:10] [02miraheze/ssl] 07MirahezeSSLBot 032d30c41 - Bot: Update SSL cert for kodiak.wiki [07:01:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.10, 6.95, 7.32 [07:02:26] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.58, 7.69, 7.40 [07:03:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.12, 7.67, 7.56 [07:06:21] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.46, 7.54, 7.42 [07:07:41] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures [07:07:53] PROBLEM - mwtask141 Disk Space on mwtask141 is CRITICAL: DISK CRITICAL - free space: / 4238MiB (5% inode=89%); [07:09:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.20, 7.88, 7.74 [07:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.14, 7.83, 7.75 [07:19:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.63, 7.15, 7.53 [07:20:02] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.73, 7.50, 7.33 [07:22:00] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.15, 6.84, 7.10 [07:25:54] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.85, 5.99, 6.70 [07:29:38] RECOVERY - kodiak.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'kodiak.wiki' will expire on Wed 03 Apr 2024 06:00:59 GMT +0000. [07:31:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.12, 7.20, 7.15 [07:31:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.97, 7.23, 7.04 [07:39:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.27, 7.43, 7.42 [07:41:53] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 5486MiB (7% inode=89%); [07:46:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:49:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.11, 5.98, 6.61 [07:51:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.18, 7.06, 7.05 [07:53:53] PROBLEM - mwtask141 Disk Space on mwtask141 is CRITICAL: DISK CRITICAL - free space: / 3795MiB (5% inode=89%); [07:55:50] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: LOAD WARNING - total load average: 10.69, 10.20, 9.54 [07:55:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.16, 7.10, 6.85 [07:57:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.25, 7.43, 7.25 [07:57:47] RECOVERY - mwtask141 Current Load on mwtask141 is OK: LOAD OK - total load average: 9.47, 9.95, 9.53 [07:57:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.27, 7.06, 6.87 [07:59:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.41, 7.79, 7.41 [07:59:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.73, 6.61, 6.73 [07:59:53] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 5511MiB (7% inode=89%); [08:01:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.30, 7.59, 7.38 [08:05:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.41, 7.56, 7.38 [08:09:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.77, 7.41, 7.38 [08:10:25] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 6.43, 4.26, 2.79 [08:12:20] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.38, 3.65, 2.73 [08:12:37] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.68, 6.86, 6.76 [08:14:14] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.50, 3.23, 2.68 [08:14:36] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.67, 7.47, 6.98 [08:16:33] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.37, 7.56, 7.09 [08:16:40] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:20:27] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.58, 6.65, 6.78 [08:21:54] PROBLEM - mwtask141 Disk Space on mwtask141 is CRITICAL: DISK CRITICAL - free space: / 4230MiB (5% inode=89%); [08:22:36] PROBLEM - tep.wiki - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [08:29:53] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 6089MiB (8% inode=89%); [08:37:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.95, 7.54, 7.15 [08:38:17] PROBLEM - barengreza.my.id - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for barengreza.my.id could not be found [08:39:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.79, 7.23, 7.09 [08:40:12] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.35, 3.36, 2.81 [08:43:27] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=infrawikiwiki --no-updates --username-prefix=infrawiki /home/reception/ImportDump/infrawikiwiki-20240104051120.xml (START) [08:43:28] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=mspfswiki --no-updates --username-prefix=wikia:mspfs /home/reception/ImportDump/mspfswiki-20240104060322.xml (START) [08:43:29] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=shaperaversewiki --no-updates --username-prefix=wikia:shaperaverse /home/reception/ImportDump/shaperaversewiki-20240103163309.xml (START) [08:43:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:43:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:43:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:44:01] PROBLEM - os141 PowerDNS Recursor on os141 is CRITICAL: CRITICAL - Plugin timed out while executing system call [08:44:04] !log [@mwtask141] starting deploy of {'config': True} to all [08:44:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:45:24] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 80s [08:45:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:45:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.78, 7.46, 6.83 [08:46:01] RECOVERY - os141 PowerDNS Recursor on os141 is OK: DNS OK: 0.417 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [08:46:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.19, 3.56, 3.14 [08:47:24] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=infrawikiwiki --no-updates --username-prefix=infrawiki /home/reception/ImportDump/infrawikiwiki-20240104051120.xml (END - exit=0) [08:47:25] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=infrawikiwiki (START) [08:47:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:47:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:48:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.08, 3.03, 2.99 [08:48:51] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=infrawikiwiki (END - exit=0) [08:48:53] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=infrawikiwiki --active --update (END - exit=0) [08:48:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:48:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:49:53] PROBLEM - mwtask141 Disk Space on mwtask141 is CRITICAL: DISK CRITICAL - free space: / 4015MiB (5% inode=89%); [08:51:53] PROBLEM - mwtask141 Disk Space on mwtask141 is WARNING: DISK WARNING - free space: / 6153MiB (8% inode=89%); [08:52:01] RECOVERY - tep.wiki - reverse DNS on sslhost is OK: SSL OK - tep.wiki reverse DNS resolves to cp25.miraheze.org - CNAME OK [08:53:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.12, 7.46, 7.28 [08:53:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.08, 6.54, 6.69 [08:55:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.03, 7.55, 7.36 [08:58:30] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=mspfswiki --no-updates --username-prefix=wikia:mspfs /home/reception/ImportDump/mspfswiki-20240104060322.xml (END - exit=0) [08:58:31] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mspfswiki (START) [08:58:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:58:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:59:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.29, 7.05, 6.86 [09:01:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.65, 8.32, 7.71 [09:01:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.57, 7.70, 7.13 [09:03:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.03, 7.77, 7.25 [09:04:12] PROBLEM - db142 Backups SQL on db142 is CRITICAL: FILE_AGE CRITICAL: /var/log/sql-backup.log is 1209707 seconds old and 82142 bytes [09:05:05] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=shaperaversewiki --no-updates --username-prefix=wikia:shaperaverse /home/reception/ImportDump/shaperaversewiki-20240103163309.xml (END - exit=0) [09:05:06] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shaperaversewiki (START) [09:05:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:05:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:05:20] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=mspfswiki (END - exit=0) [09:05:21] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=mspfswiki --active --update (END - exit=0) [09:05:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.15, 7.52, 7.51 [09:05:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:05:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:06:24] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 1.96, 1.52, 0.94 [09:08:08] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shaperaversewiki (END - exit=0) [09:08:10] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=shaperaversewiki --active --update (END - exit=0) [09:08:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:08:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:08:18] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.52, 1.14, 0.87 [09:09:52] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.42, 6.33, 6.74 [09:11:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.59, 7.73, 7.53 [09:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.44, 7.57, 7.50 [09:13:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.44, 6.72, 6.83 [09:15:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.09, 7.71, 7.18 [09:17:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.81, 7.64, 7.22 [09:27:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.34, 7.52, 7.32 [09:27:53] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.65, 7.52, 7.24 [09:29:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.72, 7.62, 7.38 [09:31:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.02, 7.60, 7.36 [09:33:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.07, 7.75, 7.44 [09:35:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 7.92, 8.04, 7.64 [09:35:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.59, 7.39, 7.35 [09:37:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.11, 7.72, 7.60 [09:41:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.35, 7.42, 7.44 [09:41:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.98, 7.18, 7.13 [09:43:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.13, 7.25, 7.38 [09:43:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.73, 7.35, 7.21 [09:44:12] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.85, 2.81, 1.80 [09:45:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.28, 8.23, 7.56 [09:46:12] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.50, 2.33, 1.75 [09:47:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.44, 7.82, 7.48 [09:55:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.82, 5.82, 6.63 [09:59:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.19, 6.69, 6.82 [10:05:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.29, 7.53, 7.24 [10:07:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 5.76, 6.87, 7.04 [10:09:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 5.45, 6.11, 6.55 [10:11:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.21, 7.32, 7.16 [10:13:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.31, 7.28, 7.16 [10:13:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 9.61, 7.19, 6.84 [10:17:30] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 9.10, 7.93, 7.43 [10:19:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.91, 7.54, 7.16 [10:21:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 6.52, 7.61, 7.45 [10:21:53] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [10:23:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.43, 6.17, 6.72 [10:29:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 6.57, 6.84, 6.84 [10:31:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.03, 7.44, 7.07 [10:33:28] PROBLEM - swiftobject111 Current Load on swiftobject111 is CRITICAL: CRITICAL - load average: 8.57, 7.46, 7.29 [10:35:52] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.74, 7.66, 7.26 [10:41:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is CRITICAL: CRITICAL - load average: 8.34, 7.21, 7.13 [10:47:29] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.01, 8.00, 7.85 [10:47:51] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.39, 7.91, 7.53 [10:53:51] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 3.55, 5.54, 6.66 [10:55:29] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 5.07, 5.71, 6.79 [10:55:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 4.27, 6.18, 7.94 [11:03:51] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 4.55, 5.23, 6.78 [11:03:58] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.45, 5.86, 7.83 [11:08:00] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:10:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.29, 6.28, 7.94 [11:13:58] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.85, 5.78, 6.79 [11:20:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.93, 6.86, 7.33 [11:22:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.66, 6.67, 7.20 [11:28:41] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.83, 5.59, 6.49 [11:51:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.05, 7.35, 6.77 [11:55:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.58, 6.65, 6.64 [12:02:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 7.03, 6.79 [12:04:09] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.45, 6.77, 6.73 [12:08:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.17, 7.11, 6.87 [12:10:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.97, 6.80, 6.80 [12:12:42] [02miraheze/ImportDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/76c9985d5326...d22d7928554b [12:12:44] [02miraheze/ImportDump] 07translatewiki 03d22d792 - Localisation updates from https://translatewiki.net. [12:12:45] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/landing/compare/495dc4d8e4ed...c3d9e17057a4 [12:12:46] [02miraheze/landing] 07translatewiki 03c3d9e17 - Localisation updates from https://translatewiki.net. [12:12:49] [02miraheze/RemovePII] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/f3ae0448b868...9ad977cec332 [12:12:51] [02miraheze/RemovePII] 07translatewiki 039ad977c - Localisation updates from https://translatewiki.net. [12:12:54] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/4e59c5563c68...0834515daa46 [12:12:55] [02miraheze/MirahezeMagic] 07translatewiki 030834515 - Localisation updates from https://translatewiki.net. [12:12:57] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/f4c42a308497...20345a40d25b [12:12:58] [02miraheze/WikiDiscover] 07translatewiki 0320345a4 - Localisation updates from https://translatewiki.net. [12:14:19] miraheze/landing - translatewiki the build passed. [12:17:52] miraheze/WikiDiscover - translatewiki the build passed. [12:21:39] miraheze/MirahezeMagic - translatewiki the build passed. [12:21:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 16.79, 9.29, 7.56 [12:22:09] miraheze/ImportDump - translatewiki the build passed. [12:22:38] miraheze/RemovePII - translatewiki the build passed. [12:27:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 7.87, 7.46 [12:33:33] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.87, 6.05, 6.79 [12:46:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [12:46:54] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.hsdfz.org.cn All nameservers failed to answer the query. [12:51:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.30, 7.39, 6.86 [12:53:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.71, 7.28, 6.87 [12:55:05] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.03, 6.40, 6.59 [13:06:45] PROBLEM - swiftobject111 Current Load on swiftobject111 is WARNING: WARNING - load average: 7.02, 6.42, 5.54 [13:08:44] RECOVERY - swiftobject111 Current Load on swiftobject111 is OK: OK - load average: 3.73, 5.43, 5.29 [13:16:39] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [13:16:56] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.hsdfz.org.cn could not be found [13:36:19] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.36, 6.82, 6.58 [13:38:16] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.65, 6.31, 6.42 [13:41:50] PROBLEM - wiki.qadrishattari.xyz - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query qadrishattari.xyz. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:44:09] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.98, 7.11, 6.64 [13:48:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.95, 7.66, 6.95 [13:53:52] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 7.85, 6.87, 5.97 [13:55:52] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.43, 6.41, 5.92 [14:03:59] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.01, 6.58, 6.30 [14:05:58] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.89, 6.22, 6.19 [14:11:01] RECOVERY - wiki.qadrishattari.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.qadrishattari.xyz reverse DNS resolves to cp25.miraheze.org - CNAME OK [14:11:32] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.41, 6.20, 6.63 [14:14:00] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.49, 6.64, 6.33 [14:17:58] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.07, 6.68, 6.46 [14:21:25]