[00:01:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.75, 2.11, 2.02 [00:03:17] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 1.82, 1.69 [00:03:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.88, 10.93, 10.22 [00:05:16] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.56, 1.74, 1.68 [00:05:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.33, 9.68, 9.85 [00:07:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.33, 1.63, 1.65 [00:11:13] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.14, 1.93, 1.77 [00:11:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.59, 11.79, 10.52 [00:13:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.01, 1.57, 1.66 [00:13:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.06, 11.06, 10.42 [00:15:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.71, 12.04, 10.82 [00:16:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.56, 5.51, 4.57 [00:17:10] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 4.76, 4.03, 2.68 [00:17:46] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.23, 12.61, 10.39 [00:18:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.89, 5.00, 4.50 [00:19:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.94, 3.20 [00:21:47] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 4.19, 3.38 [00:25:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.94, 3.45 [00:26:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.24, 5.50, 4.91 [00:28:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.28, 4.57, 4.63 [00:29:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.23, 3.24, 3.29 [00:31:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.18, 11.72, 11.94 [00:32:21] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.45, 7.74, 5.91 [00:35:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.56, 3.42, 3.32 [00:37:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.69, 11.46, 11.83 [00:37:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.17, 1.46, 1.95 [00:39:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.69, 3.09, 3.22 [00:40:44] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [00:41:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.96, 11.96, 11.67 [00:41:32] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [00:43:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.29, 11.04, 11.37 [00:43:32] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [00:44:44] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:45:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.05, 12.65, 11.95 [00:47:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.77, 11.77, 11.71 [00:47:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.13, 1.79, 1.85 [00:49:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.52, 11.80, 11.70 [00:51:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.86, 11.03, 11.43 [00:51:42] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/f6e04c15450e [00:51:44] [02miraheze/puppet] 07Universal-Omega 03f6e04c1 - elasticsearch: only use the prometheus exporter on master [00:51:45] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [00:51:45] [url] Page not found · GitHub · GitHub | github.com [00:51:47] [02puppet] 07Universal-Omega opened pull request 03#2811: elasticsearch: only use the prometheus exporter on master - 13https://github.com/miraheze/puppet/pull/2811 [00:51:47] [url] Page not found · GitHub · GitHub | github.com [00:53:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.96, 12.23, 11.83 [00:53:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.08, 11.44, 11.06 [00:55:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.69, 11.30, 11.54 [00:59:03] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.41, 4.10, 3.52 [00:59:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.36, 12.44, 11.91 [01:00:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.65, 3.54, 3.38 [01:01:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.68, 11.31, 11.56 [01:01:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.18, 11.27, 11.48 [01:02:53] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.18, 3.13, 3.25 [01:07:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.62, 8.05, 9.96 [01:13:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.16, 10.19, 10.29 [01:15:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.22, 9.33, 9.96 [01:16:23] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - roblox-wiki.tk All nameservers failed to answer the query. [01:21:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.49, 9.22, 10.15 [01:25:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.93, 9.89, 10.12 [01:25:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.59, 3.33, 3.10 [01:26:05] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.98, 10.59, 10.00 [01:27:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.01, 10.92, 10.47 [01:29:01] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.26, 2.16, 1.70 [01:31:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.34, 3.21, 3.08 [01:32:38] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.77, 1.55, 1.94 [01:33:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.64, 1.94, 1.72 [01:33:14] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [01:33:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.06, 2.70, 2.90 [01:35:13] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [01:35:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.44, 1.60, 1.98 [01:37:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.44, 1.65, 1.65 [01:38:35] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.29, 1.70, 1.86 [01:40:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.81, 4.23, 5.87 [01:40:34] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.80, 1.73, 1.85 [01:43:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.09, 1.64, 1.82 [01:44:21] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.09, 4.32, 5.47 [01:44:32] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.63, 1.94, 1.87 [01:45:46] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp30.miraheze.org - NS RECORDS OK [01:46:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.79, 3.75, 5.13 [01:47:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.61, 1.77, 1.85 [01:48:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.75, 3.13, 4.74 [01:48:31] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.13, 1.71, 1.81 [01:49:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.24, 10.81, 11.95 [01:51:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.05, 2.10, 1.96 [01:53:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.82, 1.69, 1.83 [01:56:27] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.39, 1.51, 1.69 [01:57:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.84, 2.20, 1.99 [01:59:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.94, 7.90, 10.01 [01:59:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.71, 10.36, 11.78 [02:00:25] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.64, 2.15, 1.90 [02:03:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.51, 11.38, 11.80 [02:05:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.78, 11.20, 11.67 [02:07:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.68, 11.98, 11.87 [02:09:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.72, 10.27, 10.37 [02:09:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.77, 11.41, 11.67 [02:11:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.04, 12.09, 11.92 [02:13:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.66, 9.46, 10.07 [02:13:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.57, 10.64, 11.43 [02:21:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.57, 8.49, 10.12 [02:23:32] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.93, 8.43, 5.81 [02:24:36] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03revert-4716-revert-4713-patch-82 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/bfc2008c68f7 [02:24:36] [url] Page not found · GitHub · GitHub | github.com [02:24:38] [02miraheze/mw-config] 07Universal-Omega 03bfc2008 - Revert "Revert "add gratisdata.miraheze.org as wgCrossSiteAJAXdomains on gratispaideiawiki" (#4716)" [02:24:39] [02mw-config] 07Universal-Omega created branch 03revert-4716-revert-4713-patch-82 - 13https://github.com/miraheze/mw-config [02:24:39] ... [02:24:41] [02mw-config] 07Universal-Omega opened pull request 03#4889: Revert "Revert "add gratisdata.miraheze.org as wgCrossSiteAJAXdomains on gratispaideiawiki"" - 13https://github.com/miraheze/mw-config/pull/4889 [02:24:41] ... [02:25:03] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.10, 3.44, 3.09 [02:25:26] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 15.92, 10.99, 10.06 [02:25:41] miraheze/mw-config - Universal-Omega the build passed. [02:26:40] [02mw-config] 07Universal-Omega closed pull request 03#4889: Revert "Revert "add gratisdata.miraheze.org as wgCrossSiteAJAXdomains on gratispaideiawiki"" - 13https://github.com/miraheze/mw-config/pull/4889 [02:26:41] [url] Page not found · GitHub · GitHub | github.com [02:26:42] [02miraheze/mw-config] 07Universal-Omega deleted branch 03revert-4716-revert-4713-patch-82 [02:26:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/7c31c3e77eb5...ffe8704f6e29 [02:26:44] [url] Comparing 7c31c3e77eb5...ffe8704f6e29 · miraheze/mw-config · GitHub | github.com [02:26:45] [02miraheze/mw-config] 07Universal-Omega 03ffe8704 - Revert "Revert "add gratisdata.miraheze.org as wgCrossSiteAJAXdomains on gratispaideiawiki"" (#4889) [02:26:46] [02mw-config] 07Universal-Omega deleted branch 03revert-4716-revert-4713-patch-82 - 13https://github.com/miraheze/mw-config [02:26:47] [url] Page not found · GitHub · GitHub | github.com [02:26:52] [02mw-config] 07Universal-Omega closed pull request 03#4722: Revert "Revert "add gratisdata.miraheze.org as wgCrossSiteAJAXdomains on gratispaideiawiki"" - 13https://github.com/miraheze/mw-config/pull/4722 [02:26:53] ... [02:26:58] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.68, 3.22, 3.05 [02:27:25] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.12, 5.95, 5.41 [02:27:29] !log [@test131] starting deploy of {'config': True} to all [02:27:30] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [02:27:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:27:44] miraheze/mw-config - Universal-Omega the build passed. [02:27:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:29:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.88, 10.73, 10.14 [02:30:28] [02puppet] 07Universal-Omega closed pull request 03#2811: elasticsearch: only use the prometheus exporter on master - 13https://github.com/miraheze/puppet/pull/2811 [02:30:29] [url] Page not found · GitHub · GitHub | github.com [02:30:30] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [02:30:30] [url] Page not found · GitHub · GitHub | github.com [02:30:31] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [02:30:33] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/954b7c87c92c...350de476455e [02:30:33] [url] Comparing 954b7c87c92c...350de476455e · miraheze/puppet · GitHub | github.com [02:30:34] [02miraheze/puppet] 07Universal-Omega 03350de47 - elasticsearch: only use the prometheus exporter on master (#2811) [02:31:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.37, 10.10, 10.00 [02:33:15] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.24, 6.93, 5.80 [02:33:40] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/3769a3e8f98f [02:33:40] [url] Page not found · GitHub · GitHub | github.com [02:33:42] [02miraheze/puppet] 07Universal-Omega 033769a3e - role::elasticsearch: replace hiera with lookup [02:33:43] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [02:33:43] ... [02:33:45] [02puppet] 07Universal-Omega opened pull request 03#2812: role::elasticsearch: replace hiera with lookup - 13https://github.com/miraheze/puppet/pull/2812 [02:33:45] [url] Page not found · GitHub · GitHub | github.com [02:36:50] [02puppet] 07Universal-Omega closed pull request 03#2812: role::elasticsearch: replace hiera with lookup - 13https://github.com/miraheze/puppet/pull/2812 [02:36:51] [url] Page not found · GitHub · GitHub | github.com [02:36:52] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [02:36:52] ... [02:36:53] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [02:36:55] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/350de476455e...eedd652f02e6 [02:36:55] [url] Comparing 350de476455e...eedd652f02e6 · miraheze/puppet · GitHub | github.com [02:36:56] [02miraheze/puppet] 07Universal-Omega 03eedd652 - role::elasticsearch: replace hiera with lookup (#2812) [02:37:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.25, 3.46, 3.26 [02:37:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.93, 1.66, 1.97 [02:39:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.29, 3.07, 3.14 [02:39:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.84, 1.87, 2.01 [02:40:02] !log [@mwtask141] starting deploy of {'config': True} to all [02:41:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:41:09] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:41:12] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 69s [02:41:17] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:41:25] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:42:05] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03prometheus-es-exporter-ipv6 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/66d8426a5bb8 [02:42:06] [02miraheze/puppet] 07Universal-Omega 0366d8426 - prometheus-es-exporter: use IPV6 [02:42:08] [02puppet] 07Universal-Omega created branch 03prometheus-es-exporter-ipv6 - 13https://github.com/miraheze/puppet [02:42:08] [url] Page not found · GitHub · GitHub | github.com [02:42:09] [02puppet] 07Universal-Omega opened pull request 03#2813: prometheus-es-exporter: use IPV6 - 13https://github.com/miraheze/puppet/pull/2813 [02:42:10] [url] Page not found · GitHub · GitHub | github.com [02:42:54] [02puppet] 07Universal-Omega closed pull request 03#2813: prometheus-es-exporter: use IPV6 - 13https://github.com/miraheze/puppet/pull/2813 [02:42:55] ... [02:42:56] [02miraheze/puppet] 07Universal-Omega deleted branch 03prometheus-es-exporter-ipv6 [02:42:57] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/eedd652f02e6...82373ccd26ad [02:42:58] [url] Comparing eedd652f02e6...82373ccd26ad · miraheze/puppet · GitHub | github.com [02:42:59] [02miraheze/puppet] 07Universal-Omega 0382373cc - prometheus-es-exporter: use IPV6 (#2813) [02:43:00] [02puppet] 07Universal-Omega deleted branch 03prometheus-es-exporter-ipv6 - 13https://github.com/miraheze/puppet [02:43:01] [url] Page not found · GitHub · GitHub | github.com [02:43:10] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 1.723 second response time [02:43:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:43:17] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 4.699 second response time [02:43:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.17, 5.13, 3.90 [02:43:40] PROBLEM - cp31 Stunnel HTTP for mw141 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:44] PROBLEM - cp30 Stunnel HTTP for mw141 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:45:44] RECOVERY - cp30 Stunnel HTTP for mw141 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 4.988 second response time [02:46:17] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/98f4084db195 [02:46:19] [02miraheze/puppet] 07Universal-Omega 0398f4084 - prometheus-es-exporter: use elasticsearch master only [02:46:20] [02puppet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [02:46:21] [url] Page not found · GitHub · GitHub | github.com [02:46:22] [02puppet] 07Universal-Omega opened pull request 03#2814: prometheus-es-exporter: use elasticsearch master only - 13https://github.com/miraheze/puppet/pull/2814 [02:46:22] [url] Page not found · GitHub · GitHub | github.com [02:47:19] [02puppet] 07Universal-Omega closed pull request 03#2814: prometheus-es-exporter: use elasticsearch master only - 13https://github.com/miraheze/puppet/pull/2814 [02:47:19] ... [02:47:20] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/82373ccd26ad...786b0131f2e3 [02:47:21] [url] Comparing 82373ccd26ad...786b0131f2e3 · miraheze/puppet · GitHub | github.com [02:47:22] [02miraheze/puppet] 07Universal-Omega 03786b013 - prometheus-es-exporter: use elasticsearch master only (#2814) [02:47:23] [02puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/puppet [02:47:24] [url] Page not found · GitHub · GitHub | github.com [02:47:25] [02miraheze/puppet] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [02:47:35] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.924 second response time [02:51:26] RECOVERY - es131 Puppet on es131 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [02:51:49] RECOVERY - cp31 Stunnel HTTP for mw141 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.251 second response time [02:53:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.90, 10.73, 9.86 [02:56:06] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.28, 10.04, 9.39 [02:57:48] !log upgrade grafana on mon141 [02:58:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:59:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.92, 1.73, 1.47 [02:59:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.57, 12.37, 10.77 [02:59:54] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.65, 3.91, 3.15 [02:59:58] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.11, 12.95, 10.72 [03:01:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.48, 1.62, 1.46 [03:01:53] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.07, 11.75, 10.56 [03:03:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.51, 11.34, 10.81 [03:03:50] PROBLEM - gluster122 Current Load on gluster122 is WARNING: WARNING - load average: 2.27, 3.49, 3.19 [03:05:23] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:05:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.08, 12.33, 11.23 [03:05:45] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.45, 11.73, 10.69 [03:05:48] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 2.84, 3.34, 3.17 [03:07:24] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 1.434 second response time [03:07:40] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.42, 10.62, 10.42 [03:09:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.39, 11.02, 11.05 [03:09:36] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.75, 9.86, 10.16 [03:10:19] [02miraheze/puppet] 07Universal-Omega pushed 038 commits to 03puppet-lint [+0/-0/±12] 13https://github.com/miraheze/puppet/compare/3b16f487f56c...32fa88f91aeb [03:10:19] [url] Comparing 3b16f487f56c...32fa88f91aeb · miraheze/puppet · GitHub | github.com [03:10:20] [02miraheze/puppet] 07Universal-Omega 0332fa88f - Merge branch 'master' into puppet-lint [03:10:22] [02puppet] 07Universal-Omega synchronize pull request 03#2809: CI: lint puppet code to standards - 13https://github.com/miraheze/puppet/pull/2809 [03:10:22] [url] Page not found · GitHub · GitHub | github.com [03:11:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.74, 11.34, 11.14 [03:13:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.86, 1.68, 1.54 [03:13:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.90, 10.95, 11.04 [03:17:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.81, 11.62, 11.22 [03:18:35] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [03:18:51] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.19, 3.37, 2.89 [03:19:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.39, 1.61, 1.56 [03:19:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.94, 10.61, 10.91 [03:20:34] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [03:20:47] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.99, 3.18, 2.88 [03:21:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.88, 11.55, 11.22 [03:23:10] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.21, 10.31, 10.23 [03:23:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.39, 10.89, 11.03 [03:24:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 1.94, 2.85, 5.56 [03:25:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.73, 9.19, 9.84 [03:25:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.44, 1.57, 1.96 [03:28:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.46, 2.42, 4.77 [03:31:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.75, 9.15, 10.02 [03:31:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.48, 3.38, 3.95 [03:36:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.65, 1.69, 1.99 [03:37:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.01, 2.57, 3.36 [03:37:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.25, 1.42, 1.67 [03:38:23] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.19, 1.92, 2.04 [03:40:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.05, 1.59, 1.90 [03:41:38] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.56, 3.82, 3.70 [03:41:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.66, 2.10, 1.93 [03:43:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.42, 3.56 [03:44:23] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.30, 2.12, 2.01 [03:45:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.97, 2.92, 3.36 [03:45:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.21, 1.88, 1.90 [03:47:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.18, 1.82, 1.86 [03:50:21] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.66, 5.91, 4.72 [03:53:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.00, 1.79, 1.89 [03:54:18] [Grafana] FIRING: Notification test [03:54:51] !log update mon111 to mon141 for Icinga webhook in Grafana [03:54:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:55:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.92, 2.13, 2.02 [03:56:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.62, 1.95, 2.00 [03:57:21] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [03:58:23] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.66, 2.08, 2.04 [03:59:20] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:02:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.73, 5.78, 5.54 [04:02:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.39, 1.86, 1.96 [04:03:18] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [04:05:05] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.55, 10.14, 9.29 [04:05:17] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:07:01] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.94, 9.45, 9.15 [04:07:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.46, 1.83, 1.98 [04:08:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.58, 4.06, 4.92 [04:08:23] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.15, 1.39, 1.69 [04:09:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.87, 2.24, 2.11 [04:12:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.56, 1.60, 1.71 [04:13:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.00, 1.72, 1.94 [04:14:23] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.18, 1.39, 1.62 [04:17:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.39, 2.27, 2.08 [04:21:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.42, 3.11, 2.94 [04:23:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.76, 2.91, 2.89 [04:25:18] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.70, 5.70, 4.98 [04:25:19] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.41, 2.33, 2.00 [04:27:14] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.28, 4.72, 4.70 [04:27:18] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.94, 1.88, 1.87 [04:31:16] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.38, 1.96, 1.89 [04:33:07] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [04:35:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.63, 1.85, 1.86 [04:37:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.11, 1.34, 1.97 [04:41:12] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.12, 1.91, 1.86 [04:43:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.49, 0.93, 1.60 [04:45:10] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.86, 1.94, 1.88 [04:49:01] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [04:49:20] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=2b2twiki --no-updates --username-prefix=mh:2builders2tools /mnt/mediawiki-static/metawiki/ImportDump/2b2twiki-20220908033427.xml (START) [04:49:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:51:08] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.48, 2.47, 2.07 [04:52:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.65, 2.82, 3.87 [04:55:21] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=2b2twiki --no-updates --username-prefix=mh:2builders2tools /mnt/mediawiki-static/metawiki/ImportDump/2b2twiki-20220908033427.xml (END - exit=0) [04:55:22] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=2b2twiki (START) [04:55:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:55:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:57:01] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.60, 1.91, 1.56 [05:00:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.88, 2.55, 3.36 [05:01:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.72, 1.90, 1.65 [05:01:56] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [05:02:46] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.83, 1.68, 1.60 [05:04:17] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:04:41] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.26, 1.53, 1.55 [05:05:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.38, 1.62, 1.59 [05:05:54] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:06:14] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36268 bytes in 0.052 second response time [05:09:00] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.93, 1.46, 1.93 [05:09:10] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.71, 3.43, 3.45 [05:11:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.89, 2.96, 3.28 [05:16:57] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.02, 1.28, 1.64 [05:20:59] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.31, 1.96, 1.76 [05:22:53] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.63, 2.07, 1.84 [05:22:54] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.94, 2.23, 1.87 [05:22:57] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=2b2twiki (END - exit=0) [05:22:59] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=2b2twiki --active --update (END - exit=0) [05:23:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:23:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:25:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 3.35, 2.47 [05:25:50] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.15, 6.28, 4.47 [05:26:56] !log [reception@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/moveBatch.php --wiki=2b2twiki --r=Import mistake [05:27:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:27:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.78, 3.73, 2.74 [05:28:45] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [05:28:51] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.59, 2.00, 1.93 [05:29:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.09, 3.35, 2.71 [05:30:34] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.45, 2.00, 1.95 [05:30:50] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.49, 2.23, 2.02 [05:36:43] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:42:03] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.36, 1.54, 1.70 [05:42:40] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [05:47:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.65, 1.78, 1.76 [05:48:21] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.60, 7.94, 5.19 [05:49:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.46, 1.92, 1.81 [05:50:37] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:51:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.03, 1.58, 1.70 [05:54:36] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [05:56:21] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.67, 5.56, 5.38 [05:56:35] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:57:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.28, 1.97, 1.81 [05:58:21] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.10, 4.73, 5.09 [06:01:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.47, 1.90, 1.83 [06:02:36] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.82, 1.66, 1.95 [06:05:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.38, 1.52, 1.68 [06:09:01] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.47, 1.94, 1.55 [06:13:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.51, 2.92, 2.16 [06:14:07] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:14:31] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.89, 1.29, 1.63 [06:15:26] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [06:16:28] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 7.261 second response time [06:27:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.36, 1.71, 1.94 [06:35:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.86, 1.32, 1.65 [06:36:40] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.20, 2.64, 3.95 [06:38:28] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:40:27] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36230 bytes in 0.037 second response time [06:40:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.71, 3.45, 3.90 [06:41:17] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [06:44:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.86, 3.43, 3.80 [06:52:12] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.64, 2.12, 1.68 [06:54:11] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.86, 1.64, 1.56 [06:54:22] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.21, 2.47, 3.21 [06:57:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.05, 8.78, 6.83 [06:59:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.58, 9.68, 7.42 [07:00:21] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.92, 1.58, 1.44 [07:01:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.05, 8.56, 7.28 [07:02:15] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.59, 1.48, 1.41 [07:11:05] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [07:14:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.04, 1.60, 1.45 [07:16:00] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.36, 1.49, 1.43 [07:19:34] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.26, 2.45, 1.83 [07:21:57] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.87, 2.04, 1.67 [07:23:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.04, 1.29, 1.92 [07:23:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.22, 1.82, 1.65 [07:25:18] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.14, 1.74, 1.72 [07:25:55] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.54, 2.05, 1.74 [07:27:13] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.94, 1.52, 1.64 [07:27:54] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.70, 1.85, 1.70 [07:29:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.27, 1.28, 1.70 [07:29:53] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.30, 1.97, 1.76 [07:30:58] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [07:33:52] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.08, 1.76, 1.74 [07:39:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.84, 1.45, 1.63 [07:45:45] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.14, 1.62, 1.59 [07:47:45] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.68, 1.52, 1.55 [07:48:19] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.28, 1.76, 1.46 [07:50:14] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.35, 1.60, 1.43 [07:51:42] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.76, 1.64 [07:53:41] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.07, 1.83, 1.67 [07:55:40] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.60, 1.79, 1.68 [07:57:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.62, 1.40, 1.55 [08:01:25] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.45, 3.01 [08:02:46] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [08:03:23] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.06, 3.11, 2.95 [08:07:33] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.71, 1.91, 1.74 [08:09:32] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.39, 1.68, 1.68 [08:10:27] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ffe8704f6e29...b736eea253e2 [08:10:28] [url] Comparing ffe8704f6e29...b736eea253e2 · miraheze/mw-config · GitHub | github.com [08:10:29] [02miraheze/mw-config] 07Universal-Omega 03b736eea - Fix [08:11:39] miraheze/mw-config - Universal-Omega the build passed. [08:17:28] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.57, 1.72, 1.68 [08:19:27] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.54, 1.62 [08:20:40] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [08:27:51] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.67, 1.98, 1.62 [08:27:52] !log [@test131] starting deploy of {'config': True} to all [08:27:53] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [08:27:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.47, 3.57, 2.34 [08:28:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:28:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:31:40] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.39, 1.79, 1.64 [08:33:34] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [08:33:35] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.32, 1.65, 1.61 [08:33:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.87, 1.93, 2.00 [08:39:41] !log [@mwtask141] starting deploy of {'config': True} to all [08:39:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:39:50] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [08:39:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.84, 1.23, 1.67 [08:39:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:41:31] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [08:47:29] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [08:51:27] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [08:55:25] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [08:56:10] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.20, 2.04, 1.57 [08:57:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.18, 4.05, 2.61 [08:58:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.31, 3.60, 2.84 [08:59:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.18, 3.34, 2.52 [09:00:19] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.25, 3.17, 2.78 [09:03:22] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [09:04:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.86, 3.90, 3.19 [09:06:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 11.37, 6.24, 4.10 [09:07:20] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [09:08:05] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.33, 1.81, 1.86 [09:13:18] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [09:14:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.54, 3.94, 3.80 [09:16:01] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.95, 1.42, 1.69 [09:18:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.74, 2.64, 3.29 [09:19:15] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [09:23:14] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [09:23:59] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [09:28:52] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [09:31:10] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [09:33:10] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [09:38:09] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.61, 3.74, 2.50 [09:42:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.96, 4.54, 3.33 [09:45:05] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [09:46:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.49, 3.95, 3.41 [09:50:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.78, 3.19, 3.21 [09:51:18] PROBLEM - lore.checkit.ws - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - lore.checkit.ws All nameservers failed to answer the query. [09:51:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.84, 1.55, 1.92 [09:55:42] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.81, 1.76, 1.52 [09:59:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.83, 1.77, 1.52 [10:00:09] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:00:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.78, 4.15, 3.59 [10:01:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.49, 1.68, 1.52 [10:01:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.89, 1.37, 1.45 [10:02:08] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 2.091 second response time [10:03:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.40, 1.51, 1.70 [10:06:57] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [10:08:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.95, 3.95, 3.78 [10:11:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.79, 1.68, 1.58 [10:12:54] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [10:14:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.48, 2.99, 3.38 [10:15:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.44, 1.63, 1.59 [10:21:09] PROBLEM - lore.checkit.ws - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for lore.checkit.ws could not be found [10:22:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.43, 3.39 [10:24:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.06, 3.19, 3.30 [10:27:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.68, 1.76, 1.69 [10:28:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.59, 4.34, 3.73 [10:28:49] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [10:29:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.06, 1.83, 1.72 [10:31:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.07, 1.62, 1.65 [10:32:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.77, 3.68, 3.60 [10:32:27] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:34:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.72, 4.34, 3.84 [10:34:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.30, 3.27, 2.54 [10:36:28] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.26, 3.39, 2.68 [10:37:22] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36230 bytes in 0.042 second response time [10:38:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.39, 3.55 [10:38:45] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [10:39:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.89, 1.92, 1.77 [10:40:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.04, 2.95, 3.37 [10:42:24] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [10:44:22] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [10:46:18] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 2.00, 1.63 [10:47:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 17.90, 7.55, 3.80 [10:48:17] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.21, 1.72, 1.58 [10:50:16] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.01, 2.05, 1.73 [10:54:14] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.16, 1.78, 1.71 [10:54:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.88, 3.20, 2.99 [10:56:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.38, 2.82, 2.87 [10:56:39] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [10:58:13] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.64, 1.65, 1.67 [11:04:34] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [11:06:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.71, 3.19 [11:08:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.96, 4.05, 3.41 [11:08:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [11:11:02] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query roblox-wiki.tk. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:14:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.28, 3.87, 3.58 [11:16:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.47, 4.08, 3.66 [11:16:29] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [11:20:28] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [11:22:33] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:23:02] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 11.12, 6.58, 4.41 [11:24:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.04, 1.87, 1.57 [11:24:35] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 3.072 second response time [11:24:58] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.52, 5.79, 4.37 [11:26:00] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.87, 1.79, 1.57 [11:27:59] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.66, 1.64, 1.54 [11:28:51] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 4.04, 4.75, 4.26 [11:29:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.81, 1.64, 1.39 [11:29:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.10, 1.40, 1.91 [11:30:24] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [11:31:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.38, 1.53, 1.37 [11:31:57] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.01, 1.85, 1.64 [11:33:56] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.46, 1.75, 1.63 [11:35:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.35, 1.64, 1.82 [11:37:54] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.68, 1.91, 1.70 [11:39:53] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.23, 1.58, 1.60 [11:40:25] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp30.miraheze.org - NS RECORDS OK [11:41:05] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:43:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.29, 1.85, 1.97 [11:44:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.75, 3.40, 3.99 [11:48:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.19, 2.27, 3.40 [11:50:40] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36230 bytes in 3.193 second response time [11:52:23] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [11:52:46] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 1.72, 1.62 [11:54:45] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.38, 1.50 [11:56:23] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [11:57:01] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.57, 2.02, 1.66 [11:57:10] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.20, 4.50, 3.89 [11:57:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.94, 1.37, 1.64 [11:58:00] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [11:58:23] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [11:59:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.40, 3.85, 3.72 [11:59:55] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [12:01:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.37, 1.80, 1.67 [12:01:06] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.31, 4.11, 3.83 [12:02:23] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [12:02:40] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 2.14, 1.84 [12:03:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.22, 1.60, 1.61 [12:04:09] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 11.01, 6.76, 4.73 [12:05:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.66, 4.28, 2.98 [12:09:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.72, 3.71, 3.06 [12:09:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.97, 1.85, 1.74 [12:11:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.82, 4.23, 3.32 [12:11:55] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.15, 1.62, 1.68 [12:13:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.93, 3.77, 3.27 [12:13:51] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.13, 5.35, 5.42 [12:14:23] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [12:15:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.08, 3.75, 3.31 [12:15:48] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.26, 4.21, 4.99 [12:17:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.82, 3.57, 3.31 [12:19:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.31, 3.31, 3.23 [12:21:39] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 12.77, 7.35, 5.86 [12:21:55] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 4.42, 2.69, 2.03 [12:23:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.63, 3.38 [12:25:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.76, 4.46, 3.71 [12:25:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.64, 10.44, 9.27 [12:27:40] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.95, 11.64, 9.82 [12:28:20] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [12:29:26] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.39, 6.00, 5.97 [12:30:12] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.27, 14.36, 10.82 [12:31:22] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 12.84, 8.20, 6.75 [12:31:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.55, 3.90, 3.83 [12:32:18] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [12:33:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.84, 11.82, 10.72 [12:35:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.36, 11.65, 10.77 [12:35:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 9.93, 5.85, 4.55 [12:36:16] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [12:37:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.19, 11.04, 10.68 [12:39:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.88, 12.52, 11.28 [12:41:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.81, 5.36, 5.97 [12:46:13] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [12:48:50] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.05, 3.72, 4.97 [12:49:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.51, 11.71, 11.82 [12:51:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.00, 12.64, 12.15 [12:53:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.70, 11.82, 11.90 [12:55:26] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.97, 10.32, 11.65 [12:55:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.26, 12.40, 12.04 [12:56:08] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [12:57:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.05, 3.34, 3.92 [12:59:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.16, 11.79, 11.93 [13:05:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.73, 2.85, 3.40 [13:08:04] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [13:09:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.62, 8.51, 9.97 [13:09:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.28, 3.46 [13:11:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.61, 11.79, 11.23 [13:11:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 3.57, 3.55 [13:12:02] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [13:13:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.36, 11.44, 11.19 [13:15:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.83, 13.09, 11.90 [13:15:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.41, 3.25, 3.44 [13:17:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.91, 11.35, 11.42 [13:17:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.75, 3.04, 3.33 [13:20:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.74, 3.05, 3.94 [13:22:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.47, 3.81, 4.12 [13:23:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.45, 7.84, 9.80 [13:26:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.28, 3.24, 3.85 [13:26:20] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:31:17] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 1.060 second response time [13:34:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 9.97, 4.57, 3.93 [13:35:54] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [13:39:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.81, 1.72, 1.56 [13:39:06] PROBLEM - worldsanskrit.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'worldsanskrit.net' expires in 15 day(s) (Sat 24 Sep 2022 13:22:32 GMT +0000). [13:39:51] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [13:40:31] PROBLEM - www.fossilfighterswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.fossilfighterswiki.com' expires in 15 day(s) (Sat 24 Sep 2022 13:17:51 GMT +0000). [13:43:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.63, 1.69, 1.58 [13:43:48] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:44:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a83800336293...76bcfaeadf15 [13:44:46] [url] Comparing a83800336293...76bcfaeadf15 · miraheze/ssl · GitHub | github.com [13:44:47] [02miraheze/ssl] 07MirahezeSSLBot 0376bcfae - Bot: Update SSL cert for worldsanskrit.net [13:45:31] PROBLEM - wiki.aridia.space - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.aridia.space' expires in 15 day(s) (Sat 24 Sep 2022 13:17:20 GMT +0000). [13:45:43] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:48:23] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.96, 1.52, 1.97 [13:48:53] PROBLEM - celeste.ink - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'celeste.ink' expires in 15 day(s) (Sat 24 Sep 2022 13:42:28 GMT +0000). [13:49:48] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [13:50:49] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/76bcfaeadf15...ec1fda898f5b [13:50:50] [url] Comparing 76bcfaeadf15...ec1fda898f5b · miraheze/ssl · GitHub | github.com [13:50:51] [02miraheze/ssl] 07MirahezeSSLBot 03ec1fda8 - Bot: Update SSL cert for www.fossilfighterswiki.com [13:52:23] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.64, 2.04, 2.07 [13:53:46] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [13:57:35] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ec1fda898f5b...4ff05054cd1b [13:57:36] [url] Comparing ec1fda898f5b...4ff05054cd1b · miraheze/ssl · GitHub | github.com [13:57:37] [02miraheze/ssl] 07MirahezeSSLBot 034ff0505 - Bot: Update SSL cert for wiki.aridia.space [13:57:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 10.38, 9.37 [13:57:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4ff05054cd1b...b1634537763d [13:57:48] [url] Comparing 4ff05054cd1b...b1634537763d · miraheze/ssl · GitHub | github.com [13:57:49] [02miraheze/ssl] 07MirahezeSSLBot 03b163453 - Bot: Update SSL cert for celeste.ink [14:01:34] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.66, 11.21, 9.83 [14:01:43] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [14:02:01] PROBLEM - ethene.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ethene.wiki' expires in 15 day(s) (Sat 24 Sep 2022 13:58:45 GMT +0000). [14:02:48] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.40, 10.52, 9.05 [14:04:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b1634537763d...1924e7df5401 [14:04:25] [url] Comparing b1634537763d...1924e7df5401 · miraheze/ssl · GitHub | github.com [14:04:26] [02miraheze/ssl] 07MirahezeSSLBot 031924e7d - Bot: Update SSL cert for ethene.wiki [14:05:30] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.23, 10.99, 10.07 [14:06:39] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.63, 10.42, 9.48 [14:07:30] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.27, 9.98, 9.81 [14:08:59] RECOVERY - worldsanskrit.net - LetsEncrypt on sslhost is OK: OK - Certificate 'worldsanskrit.net' will expire on Wed 07 Dec 2022 12:44:39 GMT +0000. [14:09:39] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [14:10:30] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.84, 9.63, 9.39 [14:17:12] PROBLEM - www.ethene.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ethene.wiki' expires in 15 day(s) (Sat 24 Sep 2022 13:58:45 GMT +0000). [14:21:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.09, 3.89, 3.02 [14:23:17] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.17, 6.90, 4.88 [14:29:01] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.74, 1.41, 1.20 [14:29:32] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [14:29:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.95, 3.49 [14:30:33] RECOVERY - ethene.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ethene.wiki' will expire on Wed 07 Dec 2022 13:04:16 GMT +0000. [14:31:01] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.48, 1.42, 1.23 [14:31:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.57, 4.59, 3.80 [14:32:30] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.61, 11.22, 10.03 [14:32:47] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.98, 11.32, 10.32 [14:33:32] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [14:34:08] PROBLEM - wiki.zergmafia.top - reverse DNS on sslhost is WARNING: Timeout: The DNS operation timed out after 5.405359745025635 seconds [14:34:24] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.53, 11.32, 10.23 [14:35:30] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [14:36:18] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.52, 12.38, 10.76 [14:38:30] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36230 bytes in 3.094 second response time [14:39:28] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [14:39:31] RECOVERY - www.fossilfighterswiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.fossilfighterswiki.com' will expire on Wed 07 Dec 2022 12:50:43 GMT +0000. [14:41:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.62, 11.82, 11.05 [14:42:26] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 10.11, 10.03, 10.13 [14:43:31] RECOVERY - wiki.aridia.space - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.aridia.space' will expire on Wed 07 Dec 2022 12:57:30 GMT +0000. [14:45:55] RECOVERY - www.ethene.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'ethene.wiki' will expire on Wed 07 Dec 2022 13:04:16 GMT +0000. [14:46:53] RECOVERY - celeste.ink - LetsEncrypt on sslhost is OK: OK - Certificate 'celeste.ink' will expire on Wed 07 Dec 2022 12:57:41 GMT +0000. [14:47:17] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/4f3ecd9b7f9d...c6d7647d16fc [14:47:18] [url] Comparing 4f3ecd9b7f9d...c6d7647d16fc · miraheze/dns · GitHub | github.com [14:47:19] [02miraheze/dns] 07Reception123 03c6d7647 - add sparabel.wiki zone [14:47:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.88, 3.45, 3.86 [14:49:34] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.86, 9.10, 10.08 [14:51:37] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 3.93, 3.96 [14:53:39] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.84, 4.14, 4.02 [14:55:18] [02miraheze/ssl] 07Reception123 pushed 033 commits to 03master [+2/-0/±3] 13https://github.com/miraheze/ssl/compare/1924e7df5401...328c6bfd0260 [14:55:19] [url] Comparing 1924e7df5401...328c6bfd0260 · miraheze/ssl · GitHub | github.com [14:55:20] [02miraheze/ssl] 07Reception123 0352fee17 - add wiki.cryptoroyale.one cert [14:55:21] [02miraheze/ssl] 07Reception123 03ebdfb4b - refresh lore.checkit.ws cert [14:55:23] [02miraheze/ssl] 07Reception123 03328c6bf - add sparabel.wiki cert [14:55:39] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.34, 3.90, 3.95 [14:56:12] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.78, 3.28, 2.99 [14:56:20] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 16.20, 12.38, 11.01 [14:56:23] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.35, 9.62, 9.48 [14:57:38] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.56, 4.78, 4.28 [14:58:11] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 3.24, 3.37, 3.07 [14:58:19] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.38, 10.72, 9.88 [15:00:07] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [15:00:12] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.29, 11.38, 10.97 [15:02:12] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.00, 11.09, 10.20 [15:03:06] RECOVERY - wiki.zergmafia.top - reverse DNS on sslhost is OK: SSL OK - wiki.zergmafia.top reverse DNS resolves to cp31.miraheze.org - CNAME OK [15:04:09] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 20.39, 14.41, 11.51 [15:05:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.69, 3.64, 3.94 [15:06:01] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.13, 11.29, 10.93 [15:07:34] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.45, 1.53, 2.00 [15:07:36] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.61, 3.87 [15:07:58] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.43, 11.19, 10.93 [15:08:02] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.03, 11.66, 11.06 [15:09:10] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.32, 5.21, 5.97 [15:09:30] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.81, 1.72, 2.02 [15:09:36] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.62, 3.38, 3.77 [15:09:54] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.02, 11.97, 11.25 [15:10:00] PROBLEM - roblox-wiki.tk - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query roblox-wiki.tk. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:13:23] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.83, 1.76, 1.95 [15:13:46] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.87, 11.40, 11.25 [15:15:20] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.58, 2.03, 2.03 [15:21:32] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.21, 11.31, 11.25 [15:22:03] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [15:25:12] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.12, 2.91, 3.40 [15:25:18] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.92, 3.60, 4.70 [15:25:36] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.01, 1.76, 1.53 [15:25:54] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.03, 10.65, 10.99 [15:25:57] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 9.25, 9.52, 10.01 [15:29:51] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.76, 10.78, 10.45 [15:31:22] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.31, 1.80, 1.67 [15:31:49] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.65, 9.58, 10.05 [15:33:16] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.39, 1.71, 1.90 [15:33:18] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.29, 1.65, 1.63 [15:33:31] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.84, 8.90, 10.12 [15:34:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [15:35:12] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.60, 2.13, 2.03 [15:37:35] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:39:03] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.87, 1.97, 1.98 [15:39:52] RECOVERY - roblox-wiki.tk - reverse DNS on sslhost is OK: SSL OK - roblox-wiki.tk reverse DNS resolves to cp31.miraheze.org - NS RECORDS OK [15:40:07] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.55, 5.76, 5.17 [15:40:58] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.26, 2.03, 2.00 [15:42:05] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 3.42, 5.03, 4.98 [15:42:23] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [15:42:45] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 5.214 second response time [15:42:46] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:42:53] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.18, 1.76, 1.91 [15:47:42] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:48:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 4.28, 5.59, 5.32 [15:48:40] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.17, 1.96, 1.95 [15:48:59] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [15:50:09] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [15:50:35] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.88, 1.92, 1.94 [15:51:14] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 1.055 second response time [15:52:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 5.47, 6.47, 5.77 [15:52:31] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.47, 2.17, 2.03 [15:54:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.09, 5.27, 5.42 [15:56:05] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 1.81, 4.29, 5.05 [16:02:51] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:04:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 4.17, 6.28, 5.75 [16:04:50] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 0.046 second response time [16:06:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.03, 5.21, 5.42 [16:07:08] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.50, 10.45, 9.90 [16:09:06] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.72, 9.60, 9.64 [16:09:43] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [16:10:05] RECOVERY - gluster102 Current Load on gluster102 is OK: OK - load average: 2.74, 3.76, 4.77 [16:13:01] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.58, 10.03, 9.64 [16:14:14] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.10, 1.61, 1.97 [16:14:56] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.47, 8.97, 9.30 [16:19:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [16:22:14] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.28, 2.37, 2.13 [16:24:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 13.00, 8.20, 5.71 [16:24:13] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.59, 5.17, 3.68 [16:25:29] PROBLEM - gluster122 Current Load on gluster122 is CRITICAL: CRITICAL - load average: 4.30, 3.25, 2.26 [16:26:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.44, 10.34, 9.37 [16:27:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [16:27:49] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.74, 3.20, 2.70 [16:27:58] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.03, 11.12, 9.72 [16:29:29] RECOVERY - gluster122 Current Load on gluster122 is OK: OK - load average: 3.08, 3.40, 2.57 [16:29:56] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.96, 11.64, 10.10 [16:31:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [16:33:53] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.25, 11.93, 10.48 [16:37:30] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.23, 2.75, 2.87 [16:37:49] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.96, 11.57, 10.72 [16:38:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.04, 11.69, 11.10 [16:40:37] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.85, 11.95, 11.25 [16:41:45] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.38, 9.59, 10.08 [16:44:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 7.96, 10.54, 10.89 [16:52:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.52, 9.41, 10.17 [16:53:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.70, 3.52, 3.97 [16:57:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [16:57:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.46, 3.41, 3.79 [16:58:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.78, 1.67, 1.45 [16:58:46] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [16:59:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.40, 3.52, 3.79 [17:00:08] PROBLEM - cp30 Stunnel HTTP for mw142 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:24] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:00:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.40, 1.55, 1.43 [17:00:29] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:01:14] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:01:20] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:01:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [17:02:07] RECOVERY - cp30 Stunnel HTTP for mw142 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.263 second response time [17:02:33] PROBLEM - mw131 HTTPS on mw131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:03:15] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 6.350 second response time [17:03:20] PROBLEM - wiki.helsinginpoytaroolipelaajat.fi - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:03:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.92, 2.42, 3.25 [17:03:51] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:04:39] RECOVERY - mw131 HTTPS on mw131 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 545 bytes in 8.593 second response time [17:05:20] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.90, 10.20, 9.18 [17:05:27] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.493 second response time [17:05:50] PROBLEM - lgbta.miraheze.org - Sectigo on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:05:53] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 5.916 second response time [17:05:53] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36268 bytes in 1.058 second response time [17:06:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.82, 1.62, 1.48 [17:07:18] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.85, 8.77, 8.78 [17:08:33] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 2.748 second response time [17:09:23] PROBLEM - mw141 MediaWiki Rendering on mw141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:09:23] PROBLEM - mw131 MediaWiki Rendering on mw131 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:10:20] PROBLEM - mw132 MediaWiki Rendering on mw132 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:10:35] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:11:49] PROBLEM - cp30 Stunnel HTTP for mw132 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:11:51] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:12:20] RECOVERY - mw132 MediaWiki Rendering on mw132 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 2.015 second response time [17:12:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.33, 1.59, 1.52 [17:13:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [17:13:45] RECOVERY - cp30 Stunnel HTTP for mw132 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.238 second response time [17:13:52] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:14:40] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:14:47] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:15:38] RECOVERY - mw141 MediaWiki Rendering on mw141 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 1.113 second response time [17:16:35] PROBLEM - mw142 MediaWiki Rendering on mw142 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:16:45] PROBLEM - cp30 Stunnel HTTP for mw142 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:16:59] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 4 datacenters are down: 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [17:17:23] PROBLEM - polandball.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:17:30] PROBLEM - datacrondatabase.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:17:34] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 6 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb, 149.56.140.43/cpweb, 149.56.141.75/cpweb, 2607:5300:201:3100::929a/cpweb, 2607:5300:201:3100::5ebc/cpweb [17:18:00] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 2.084 second response time [17:18:07] PROBLEM - wiki-tintin.cf - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:18:43] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.233 second response time [17:18:46] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 0.318 second response time [17:18:54] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [17:19:00] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.248 second response time [17:19:29] PROBLEM - civclassic.miraheze.org - Sectigo on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:19:30] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 3 backends are down. mw121 mw122 mw132 [17:19:44] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 0.345 second response time [17:19:48] RECOVERY - mw131 MediaWiki Rendering on mw131 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 8.763 second response time [17:19:51] PROBLEM - wc.miraheze.org on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:20:03] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 3.463 second response time [17:20:30] PROBLEM - cp31 Stunnel HTTP for mw142 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:20:50] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 2 backends are down. mw122 mw132 [17:20:59] RECOVERY - cp30 Stunnel HTTP for mw142 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 7.473 second response time [17:21:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [17:21:48] RECOVERY - wc.miraheze.org on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [17:22:29] RECOVERY - cp31 Stunnel HTTP for mw142 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.256 second response time [17:22:39] RECOVERY - mw142 MediaWiki Rendering on mw142 is OK: HTTP OK: HTTP/1.1 200 OK - 29514 bytes in 0.335 second response time [17:22:50] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [17:23:29] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [17:24:21] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.94, 4.17, 3.56 [17:25:57] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:27:06] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.92, 10.11, 9.59 [17:27:56] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 0.064 second response time [17:28:26] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [17:29:01] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.94, 9.44, 9.43 [17:30:23] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [17:32:30] RECOVERY - wiki.helsinginpoytaroolipelaajat.fi - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.helsinginpoytaroolipelaajat.fi' will expire on Fri 11 Nov 2022 11:54:41 GMT +0000. [17:34:56] RECOVERY - lgbta.miraheze.org - Sectigo on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [17:38:07] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [17:38:08] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.74, 3.58, 3.92 [17:39:58] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.37, 10.83, 9.31 [17:40:01] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:40:06] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.31, 4.42, 4.15 [17:41:23] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.43, 12.55, 10.54 [17:43:54] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.25, 11.17, 9.79 [17:45:00] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 2.124 second response time [17:45:11] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.81, 11.67, 10.65 [17:46:34] RECOVERY - polandball.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'polandball.wiki' will expire on Fri 02 Dec 2022 09:31:43 GMT +0000. [17:46:36] RECOVERY - datacrondatabase.com - LetsEncrypt on sslhost is OK: OK - Certificate 'datacrondatabase.com' will expire on Fri 04 Nov 2022 15:56:48 GMT +0000. [17:47:55] RECOVERY - wiki-tintin.cf - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki-tintin.cf' will expire on Mon 10 Oct 2022 03:55:03 GMT +0000. [17:47:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.82, 3.80, 4.00 [17:49:00] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.28, 9.50, 10.04 [17:49:14] RECOVERY - civclassic.miraheze.org - Sectigo on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [17:49:49] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.99, 9.06, 9.40 [17:50:17] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:54:40] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36268 bytes in 0.054 second response time [17:57:06] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.74, 3.98, 3.19 [17:57:40] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:00:58] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.67, 3.83, 3.31 [18:01:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [18:02:46] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:02:55] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.77, 3.14, 3.12 [18:03:16] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.96, 10.19, 9.83 [18:03:44] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.07, 3.05, 3.37 [18:05:10] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.17, 9.50, 9.63 [18:07:43] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:09:05] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.26, 1.84, 1.46 [18:12:35] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.59, 3.77, 3.51 [18:12:47] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.80, 11.53, 10.44 [18:12:56] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.97, 1.91, 1.57 [18:14:33] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.10, 3.17, 3.32 [18:14:41] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.38, 11.71, 10.61 [18:14:51] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.41, 2.09, 1.68 [18:18:14] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.43, 1.79, 1.97 [18:18:28] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.65, 3.63, 3.49 [18:18:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.33, 11.81, 10.93 [18:20:14] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.86, 2.24, 2.12 [18:20:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.96, 3.33, 3.39 [18:22:44] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.87, 3.48, 3.07 [18:24:14] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.58, 1.88, 2.00 [18:24:44] PROBLEM - graylog121 Current Load on graylog121 is CRITICAL: CRITICAL - load average: 4.15, 3.58, 3.15 [18:26:13] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.51, 2.08, 2.05 [18:26:44] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.68, 3.18, 3.05 [18:28:17] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.52, 3.31, 3.21 [18:30:14] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.98, 1.76, 1.94 [18:30:15] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.79, 3.48, 3.28 [18:32:13] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.99, 3.25, 3.22 [18:34:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.24, 10.44, 9.66 [18:34:14] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.04, 1.78, 1.90 [18:36:02] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.51, 11.66, 10.18 [18:36:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.36, 11.40, 10.81 [18:38:00] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.58, 11.62, 10.36 [18:40:04] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.05, 4.40, 3.68 [18:43:54] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.30, 12.42, 11.11 [18:44:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.31, 10.78, 11.12 [18:45:59] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.78, 3.93, 3.70 [18:47:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.99, 5.52, 4.31 [18:48:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.63, 12.96, 11.93 [18:48:46] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:52:11] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [18:53:43] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:54:41] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36276 bytes in 0.043 second response time [18:55:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [18:58:27] [02puppet] 07Universal-Omega edited pull request 03#2809: CI: lint puppet code to standards - 13https://github.com/miraheze/puppet/pull/2809 [18:58:27] [url] Page not found · GitHub · GitHub | github.com [18:58:34] [02puppet] 07Universal-Omega closed pull request 03#2809: CI: lint puppet code to standards - 13https://github.com/miraheze/puppet/pull/2809 [18:58:35] [url] Page not found · GitHub · GitHub | github.com [18:58:36] [02miraheze/puppet] 07Universal-Omega deleted branch 03puppet-lint [18:58:37] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±80] 13https://github.com/miraheze/puppet/compare/786b0131f2e3...b86359f6ae9c [18:58:38] [url] Comparing 786b0131f2e3...b86359f6ae9c · miraheze/puppet · GitHub | github.com [18:58:39] [02miraheze/puppet] 07Universal-Omega 03b86359f - CI: lint puppet code to standards (#2809) [18:58:40] [02puppet] 07Universal-Omega deleted branch 03puppet-lint - 13https://github.com/miraheze/puppet [18:58:41] [url] Page not found · GitHub · GitHub | github.com [18:59:18] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.98, 3.75, 3.18 [18:59:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::1b80/cpweb [18:59:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.36, 3.79, 4.00 [19:00:02] PROBLEM - cp21 Stunnel HTTP for mon141 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:01:14] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.82, 3.33, 3.09 [19:02:09] RECOVERY - cp21 Stunnel HTTP for mon141 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 36261 bytes in 5.109 second response time [19:09:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.65, 4.06, 3.92 [19:15:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.65, 3.94, 3.96 [19:19:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.50, 3.63, 3.78 [19:21:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.89, 3.68, 3.78 [19:22:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 0.92, 1.29, 1.97 [19:23:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.77, 3.83, 3.81 [19:25:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.99, 3.72, 3.76 [19:27:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 3.99, 3.85 [19:30:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.07, 1.21, 1.64 [19:31:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.77, 3.14, 3.56 [19:32:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.56, 10.41, 11.72 [19:33:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.52, 2.83, 3.38 [19:34:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.96, 10.99, 11.74 [19:36:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.47, 10.50, 11.48 [19:38:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.89, 11.69, 11.76 [19:39:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.36, 4.43, 3.82 [19:42:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.38, 10.56, 11.34 [19:44:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.32, 10.99, 11.39 [19:45:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [19:46:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.56, 9.96, 11.01 [19:47:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 5.42, 9.45, 11.43 [19:48:45] PROBLEM - cp30 Stunnel HTTP for mw132 on cp30 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 201 bytes in 5.254 second response time [19:48:45] PROBLEM - mw132 MediaWiki Rendering on mw132 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:48:53] PROBLEM - cp31 Stunnel HTTP for mw132 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:49:20] PROBLEM - cp31 Stunnel HTTP for mw131 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:49:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 149.56.141.75/cpweb, 2607:5300:201:3100::5ebc/cpweb [19:50:07] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 149.56.140.43/cpweb, 2607:5300:201:3100::929a/cpweb [19:50:41] RECOVERY - cp30 Stunnel HTTP for mw132 on cp30 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.240 second response time [19:50:44] RECOVERY - mw132 MediaWiki Rendering on mw132 is OK: HTTP OK: HTTP/1.1 200 OK - 29522 bytes in 0.386 second response time [19:50:52] RECOVERY - cp31 Stunnel HTTP for mw132 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 0.238 second response time [19:51:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.19, 9.95, 11.10 [19:51:17] RECOVERY - cp31 Stunnel HTTP for mw131 on cp31 is OK: HTTP OK: HTTP/1.1 204 No Content - 352 bytes in 3.046 second response time [19:52:07] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [19:52:50] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw132 [19:53:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.75, 9.62, 10.87 [19:54:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.49, 10.62, 10.79 [19:54:50] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [19:57:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.56, 11.41, 11.32 [20:01:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.00, 11.56, 11.47 [20:05:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.77, 12.91, 11.97 [20:05:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [20:06:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.67, 11.68, 11.68 [20:07:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.01, 11.38, 11.52 [20:08:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.64, 11.75, 11.68 [20:10:33] PROBLEM - cp20 APT on cp20 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [20:12:32] RECOVERY - cp20 APT on cp20 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [20:13:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [20:18:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.71, 11.44, 11.82 [20:20:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.65, 13.38, 12.47 [20:21:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 17.41, 12.05, 11.23 [20:23:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.82, 11.52, 11.17 [20:24:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.61, 11.63, 11.94 [20:27:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.65, 11.98, 11.34 [20:28:26] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.24, 1.83, 1.53 [20:29:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.77, 11.69, 11.34 [20:30:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.66, 1.72, 1.53 [20:30:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.63, 11.52, 11.63 [20:32:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.38, 10.50, 11.23 [20:35:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.60, 3.04, 3.84 [20:38:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.66, 11.30, 11.14 [20:39:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.18, 11.49, 11.21 [20:39:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [20:40:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.22, 1.57, 1.56 [20:40:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.70, 11.34, 11.18 [20:42:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.17, 12.37, 11.60 [20:44:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.14, 11.64, 11.43 [20:47:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.77, 12.00, 11.83 [20:47:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.82, 2.84, 3.30 [20:53:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.68, 11.13, 11.42 [20:53:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.01, 3.58, 3.47 [20:54:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 17.18, 12.85, 11.84 [20:55:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 4.01, 3.65 [21:02:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.39, 11.53, 11.83 [21:04:30] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.56, 3.28, 3.01 [21:04:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.72, 12.69, 12.27 [21:05:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.11, 3.64, 3.76 [21:06:26] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 1.88, 2.83, 2.89 [21:08:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.41, 11.32, 11.81 [21:10:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 14.09, 11.90, 11.93 [21:11:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.74, 4.24, 3.96 [21:17:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.71, 3.80, 3.86 [21:17:52] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 1 datacenter is down: 2001:41d0:801:2000::4c25/cpweb [21:23:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.38, 3.50, 3.62 [21:26:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.70, 10.69, 11.81 [21:27:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.42, 10.72, 12.00 [21:27:38] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:27:40] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.58, 3.66 [21:28:26] PROBLEM - test131 Current Load on test131 is CRITICAL: CRITICAL - load average: 2.03, 1.56, 1.26 [21:30:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.56, 1.51, 1.28 [21:31:40] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.53, 2.93, 3.37 [21:35:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.23, 11.13, 11.47 [21:37:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.14, 10.38, 11.16 [21:38:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 18.35, 12.04, 11.39 [21:39:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.88, 11.18, 11.34 [21:41:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.79, 10.27, 11.00 [21:41:33] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 2001:41d0:801:2000::4c25/cpweb, 2001:41d0:801:2000::1b80/cpweb [21:42:08] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 2.75, 3.97, 5.95 [21:42:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.25, 10.58, 10.97 [21:45:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.19, 8.49, 10.10 [21:45:33] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [21:47:08] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b86359f6ae9c...fb9b39b0ce9d [21:47:08] [url] Comparing b86359f6ae9c...fb9b39b0ce9d · miraheze/puppet · GitHub | github.com [21:47:09] [02miraheze/puppet] 07Universal-Omega 03fb9b39b - graylog: upgrade to 4.3.6 [21:48:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.66, 5.57, 5.87 [21:49:12] [02puppet] 07Universal-Omega closed pull request 03#2770: Cleanup hieradata - 13https://github.com/miraheze/puppet/pull/2770 [21:49:12] [url] Page not found · GitHub · GitHub | github.com [21:51:01] @paladox: do you still need https://github.com/miraheze/puppet/pull/2706 ? [21:51:02] [url] mariadb: tweak config by paladox · Pull Request #2706 · miraheze/puppet · GitHub | github.com [21:51:18] [02puppet] 07paladox closed pull request 03#2706: mariadb: tweak config - 13https://github.com/miraheze/puppet/pull/2706 [21:51:19] [url] Page not found · GitHub · GitHub | github.com [21:51:19] If so, there is a conflict. [21:51:20] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [21:51:21] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [21:51:22] ... [21:51:26] oh, thanks! [21:53:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 10.63, 10.00, 10.10 [21:55:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.69, 9.97, 10.11 [21:56:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.36, 8.94, 9.99 [21:57:19] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.95, 3.60, 3.28 [21:58:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.97, 1.68, 1.39 [21:59:17] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.45, 3.21, 3.17 [22:00:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.44, 1.58, 1.39 [22:08:10] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.38, 10.85, 10.44 [22:11:59] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.49, 9.90, 10.16 [22:22:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.06, 5.61, 5.93 [22:24:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 10.45, 7.38, 6.53 [22:28:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.42, 4.99, 5.74 [22:29:14] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.76, 1.78, 1.60 [22:30:03] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.52, 11.00, 10.31 [22:30:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 6.29, 5.60, 5.87 [22:31:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.23, 10.97, 9.89 [22:31:09] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.42, 1.64, 1.57 [22:32:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.15, 5.49, 5.80 [22:32:40] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.99, 4.02, 3.37 [22:33:52] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 9.90, 10.10, 10.07 [22:34:38] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.87, 3.59, 3.29 [22:36:05] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 7.68, 5.68, 5.71 [22:36:37] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.94, 3.34, 3.24 [22:37:40] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.44, 11.36, 10.58 [22:39:34] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 11.04, 11.60, 10.79 [22:40:05] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 3.93, 5.18, 5.54 [22:41:29] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.92, 12.73, 11.30 [22:43:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.46, 11.57, 11.34 [22:43:28] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.78, 4.15, 3.67 [22:44:14] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 8.94, 5.52, 5.50 [22:45:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.86, 11.87, 11.44 [22:45:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.87, 3.95, 3.64 [22:46:13] PROBLEM - gluster102 Current Load on gluster102 is WARNING: WARNING - load average: 5.33, 5.47, 5.50 [22:47:24] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.80, 4.11, 3.72 [22:49:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.74, 11.35, 11.32 [22:50:11] PROBLEM - gluster102 Current Load on gluster102 is CRITICAL: CRITICAL - load average: 9.50, 6.29, 5.73 [22:50:59] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.96, 11.90, 11.69 [22:51:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 12.78, 11.79, 11.47 [22:51:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.75, 3.95, 3.75 [22:55:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 7.71, 10.96, 11.31 [22:58:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.01, 12.35, 11.83 [22:59:13] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.60, 3.96, 3.71 [23:00:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.56, 11.58, 11.60 [23:01:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.35, 3.75, 3.66 [23:04:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 13.35, 11.41, 11.40 [23:06:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.39, 10.79, 11.16 [23:07:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 13.07, 11.58, 11.47 [23:07:06] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.74, 2.70, 3.28 [23:09:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.39, 10.95, 11.27 [23:10:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 15.46, 12.83, 11.85 [23:10:44] PROBLEM - graylog121 Current Load on graylog121 is WARNING: WARNING - load average: 3.81, 3.32, 2.73 [23:12:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 10.47, 11.58, 11.50 [23:12:44] RECOVERY - graylog121 Current Load on graylog121 is OK: OK - load average: 2.17, 2.87, 2.63 [23:16:53] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.67, 3.87, 3.53 [23:18:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 12.43, 11.30, 11.35 [23:22:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.44, 11.57, 11.56 [23:28:26] PROBLEM - test131 Current Load on test131 is WARNING: WARNING - load average: 1.87, 1.54, 1.30 [23:28:43] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.38, 3.87, 3.94 [23:29:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.90, 1.56, 1.99 [23:30:26] RECOVERY - test131 Current Load on test131 is OK: OK - load average: 1.18, 1.43, 1.29 [23:31:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.13, 1.83, 2.04 [23:33:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 16.78, 12.32, 11.13 [23:33:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.80, 1.43, 1.86 [23:34:36] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.19, 8.69, 10.15 [23:35:04] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 11.32, 11.79, 11.09 [23:37:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.16, 1.57, 1.80 [23:38:33] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.64, 4.10, 3.90 [23:40:36] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 20.83, 13.47, 11.51 [23:41:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.99, 1.93, 1.91 [23:43:15] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 5.03, 2.64, 2.15 [23:45:23] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03nginx-cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/e8386059ff82 [23:45:24] [02miraheze/puppet] 07Universal-Omega 03e838605 - nginx: minor cleanup [23:45:26] [02puppet] 07Universal-Omega created branch 03nginx-cleanup - 13https://github.com/miraheze/puppet [23:45:26] [url] Page not found · GitHub · GitHub | github.com [23:45:27] [02puppet] 07Universal-Omega opened pull request 03#2815: nginx: minor cleanup - 13https://github.com/miraheze/puppet/pull/2815 [23:45:28] [url] Page not found · GitHub · GitHub | github.com [23:46:36] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.26, 11.53, 11.47 [23:48:24] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.38, 3.61, 3.88 [23:49:01] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03nginx-cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e8386059ff82...b2e7dc05b6d5 [23:49:02] [url] Comparing e8386059ff82...b2e7dc05b6d5 · miraheze/puppet · GitHub | github.com [23:49:03] [02miraheze/puppet] 07Universal-Omega 03b2e7dc0 - Update site.pp [23:49:04] [02puppet] 07Universal-Omega synchronize pull request 03#2815: nginx: minor cleanup - 13https://github.com/miraheze/puppet/pull/2815 [23:49:04] [url] Page not found · GitHub · GitHub | github.com [23:51:04] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.64, 9.10, 10.06 [23:55:04] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 14.52, 12.00, 11.04 [23:57:11] [02puppet] 07Universal-Omega synchronize pull request 03#2815: nginx: minor cleanup - 13https://github.com/miraheze/puppet/pull/2815 [23:57:11] [url] Page not found · GitHub · GitHub | github.com [23:57:12] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03nginx-cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b2e7dc05b6d5...90d93451cb89 [23:57:13] [url] Comparing b2e7dc05b6d5...90d93451cb89 · miraheze/puppet · GitHub | github.com [23:57:14] [02miraheze/puppet] 07Universal-Omega 0390d9345 - Update site.pp