[00:02:14] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.20, 6.49, 6.59 [00:02:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.25, 3.93, 3.77 [00:04:19] RECOVERY - cp21 Disk Space on cp21 is OK: DISK OK - free space: / 10334 MB (26% inode=96%); [00:07:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.96, 1.91 [00:09:27] RECOVERY - www.winenjoy.net - reverse DNS on sslhost is OK: SSL OK - www.winenjoy.net reverse DNS resolves to cp20.miraheze.org - CNAME OK [00:13:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.70, 1.95, 1.94 [00:16:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.48, 3.33, 3.85 [00:17:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.65, 2.13, 2.00 [00:20:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.66, 3.97 [00:20:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.47, 3.78 [00:22:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.92, 4.77, 4.33 [00:22:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.70, 3.64, 3.81 [00:25:13] [Grafana] !sre RESOLVED: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk [00:26:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.26, 3.78, 4.00 [00:28:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.04, 4.18, 4.11 [00:28:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.03, 2.96, 3.39 [00:31:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.81, 1.97 [00:32:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.80, 3.62, 3.93 [00:32:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.87, 3.97, 3.69 [00:36:19] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.99, 10.43, 9.25 [00:38:19] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.80, 9.58, 9.07 [00:38:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.27, 3.72, 3.83 [00:40:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.77, 3.65, 3.79 [00:40:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.05, 3.96, 3.86 [00:42:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.71, 4.41, 4.04 [00:46:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.30, 3.96, 3.94 [00:47:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.38, 1.89, 1.84 [00:48:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.33, 4.17, 3.96 [00:48:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 4.33, 4.09 [00:50:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.04, 3.79, 3.92 [00:53:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.56, 1.90, 1.90 [00:56:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.35, 3.78, 3.92 [00:58:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.15, 4.12, 4.04 [00:58:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.63, 3.91, 3.79 [01:00:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.69, 3.50, 3.82 [01:00:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.33, 3.58 [01:08:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.40, 3.05, 3.37 [01:09:53] [02puppet] 07Universal-Omega opened pull request 03#2637: grafana-webhook: use alertname for resolved alerts - 13https://github.com/miraheze/puppet/pull/2637 [01:09:54] [url] Page not found · GitHub · GitHub | github.com [01:10:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.10, 3.83, 3.73 [01:12:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.22, 3.59, 3.65 [01:12:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.13, 3.37, 3.45 [01:13:53] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.40, 1.55, 1.69 [01:16:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.26, 3.77, 3.59 [01:17:17] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.06, 3.40, 2.70 [01:18:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.85, 3.13, 3.40 [01:18:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.76, 3.36, 3.45 [01:19:13] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.65, 3.37, 2.77 [01:19:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.52, 2.07, 1.85 [01:20:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.00, 3.32, 3.88 [01:20:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.55, 3.08, 3.34 [01:21:08] PROBLEM - matomo101 Current Load on matomo101 is CRITICAL: CRITICAL - load average: 4.40, 3.68, 2.95 [01:22:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 3.53, 3.87 [01:23:03] PROBLEM - matomo101 Current Load on matomo101 is WARNING: WARNING - load average: 3.43, 3.45, 2.94 [01:23:07] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [01:23:07] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:23:55] [02ssl] 07Universal-Omega opened pull request 03#537: Redirect miraheze.wiki/Fundraiser to the Meta Fundraiser page - 13https://github.com/miraheze/ssl/pull/537 [01:23:55] [url] Page not found · GitHub · GitHub | github.com [01:24:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.47, 3.81 [01:24:59] RECOVERY - matomo101 Current Load on matomo101 is OK: OK - load average: 3.10, 3.26, 2.93 [01:26:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.71, 3.89, 3.92 [01:28:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.14, 3.51, 3.77 [01:28:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.90, 3.51, 3.40 [01:28:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.56, 3.36, 3.29 [01:29:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.41, 1.83, 1.92 [01:30:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.90, 3.92, 3.88 [01:30:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.94, 4.11, 3.64 [01:30:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.18, 3.36, 3.31 [01:31:02] [02ssl] 07Universal-Omega synchronize pull request 03#537: Redirect miraheze.wiki/Fundraiser to the Meta Fundraiser page - 13https://github.com/miraheze/ssl/pull/537 [01:31:02] [url] Page not found · GitHub · GitHub | github.com [01:31:35] [02ssl] 07Universal-Omega synchronize pull request 03#537: Redirect miraheze.wiki/Fundraiser to the Meta Fundraiser page - 13https://github.com/miraheze/ssl/pull/537 [01:31:35] ... [01:32:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.86, 3.86, 3.60 [01:34:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.63, 4.51, 3.86 [01:36:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.68, 4.44, 3.80 [01:38:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.48, 3.80, 3.71 [01:38:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.65, 4.00, 3.74 [01:40:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.30, 3.99, 3.78 [01:42:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.95, 3.95, 3.79 [01:44:18] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [01:45:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.84, 1.83 [01:46:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.50, 3.96, 3.77 [01:48:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.61, 3.34, 3.57 [01:50:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.96, 2.86, 3.33 [01:52:30] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [01:52:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.05, 3.47, 3.54 [01:54:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.89, 3.25, 3.45 [01:56:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 3.37, 3.90 [01:56:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.55, 3.00, 3.33 [01:58:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.56, 3.68, 3.94 [02:00:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.61, 3.52 [02:01:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.44, 1.76, 1.96 [02:06:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 3.93, 3.69 [02:10:17] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:14:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.43, 3.83, 3.61 [02:15:30] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.95, 3.77, 3.41 [02:16:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.90, 3.74, 3.59 [02:17:03] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.15, 1.77, 1.31 [02:17:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.90, 1.90 [02:18:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.35, 3.75, 3.60 [02:19:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.35, 1.50, 1.26 [02:19:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.45, 3.87, 3.50 [02:20:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.93, 3.70 [02:21:25] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.94, 3.41, 3.37 [02:25:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.86, 1.99, 1.98 [02:26:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.16, 3.83, 3.71 [02:28:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.87, 3.75 [02:30:06] [02miraheze/.github] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/.github/compare/aac18df32f0c...3bca4264a733 [02:30:06] [url] Comparing aac18df32f0c...3bca4264a733 · miraheze/.github · GitHub | github.com [02:30:07] [02miraheze/.github] 07Universal-Omega 033bca426 - Update cache action [02:30:25] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/259dd349e122...30397c0b7f12 [02:30:25] [url] Comparing 259dd349e122...30397c0b7f12 · miraheze/CreateWiki · GitHub | github.com [02:30:26] [02miraheze/CreateWiki] 07Universal-Omega 0330397c0 - Update workflow [02:30:28] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/764aa1d922e5...381c26aefa2c [02:30:28] [url] Comparing 764aa1d922e5...381c26aefa2c · miraheze/ManageWiki · GitHub | github.com [02:30:29] [02miraheze/ManageWiki] 07Universal-Omega 03381c26a - Update workflow [02:30:31] [02miraheze/WikiDiscover] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/3b6791d571ac...4acc72cd1dc3 [02:30:31] [url] Comparing 3b6791d571ac...4acc72cd1dc3 · miraheze/WikiDiscover · GitHub | github.com [02:30:32] [02miraheze/WikiDiscover] 07Universal-Omega 034acc72c - Update workflow [02:30:34] [02miraheze/IncidentReporting] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/fdc2a1f0de0f...7501dcf9ba22 [02:30:34] [url] Comparing fdc2a1f0de0f...7501dcf9ba22 · miraheze/IncidentReporting · GitHub | github.com [02:30:35] [02miraheze/IncidentReporting] 07Universal-Omega 037501dcf - Update workflow [02:30:37] [02miraheze/RottenLinks] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/a4eb9fa133f0...9643ad29c27f [02:30:37] [url] Comparing a4eb9fa133f0...9643ad29c27f · miraheze/RottenLinks · GitHub | github.com [02:30:38] [02miraheze/RottenLinks] 07Universal-Omega 039643ad2 - Update workflow [02:30:40] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/209b17824792...d0926073749d [02:30:40] [url] Comparing 209b17824792...d0926073749d · miraheze/MatomoAnalytics · GitHub | github.com [02:30:41] [02miraheze/MatomoAnalytics] 07Universal-Omega 03d092607 - Update workflow [02:30:43] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/2d47032a27a7...0e9a33907e4e [02:30:43] [url] Comparing 2d47032a27a7...0e9a33907e4e · miraheze/MirahezeMagic · GitHub | github.com [02:30:44] [02miraheze/MirahezeMagic] 07Universal-Omega 030e9a339 - Update workflow [02:30:46] [02miraheze/DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/3b481a28f0ae...b50590360c16 [02:30:46] [url] Comparing 3b481a28f0ae...b50590360c16 · miraheze/DataDump · GitHub | github.com [02:30:47] [02miraheze/DataDump] 07Universal-Omega 03b505903 - Update workflow [02:30:49] [02miraheze/PDFEmbed] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/PDFEmbed/compare/39ef02187646...a06c03fe6183 [02:30:49] [url] Comparing 39ef02187646...a06c03fe6183 · miraheze/PDFEmbed · GitHub | github.com [02:30:50] [02miraheze/PDFEmbed] 07Universal-Omega 03a06c03f - Update workflow [02:30:52] [02miraheze/RemovePII] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RemovePII/compare/6ea28561547e...84f7ff11946a [02:30:52] [url] Comparing 6ea28561547e...84f7ff11946a · miraheze/RemovePII · GitHub | github.com [02:30:53] [02miraheze/RemovePII] 07Universal-Omega 0384f7ff1 - Update workflow [02:30:55] [02miraheze/GlobalNewFiles] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/GlobalNewFiles/compare/5de47ae7e58d...d5a661229867 [02:30:55] [url] Comparing 5de47ae7e58d...d5a661229867 · miraheze/GlobalNewFiles · GitHub | github.com [02:30:56] [02miraheze/GlobalNewFiles] 07Universal-Omega 03d5a6612 - Update workflow [02:30:58] [02miraheze/YouTube] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/YouTube/compare/ea9091b18407...1ec02476605d [02:30:58] [url] Comparing ea9091b18407...1ec02476605d · miraheze/YouTube · GitHub | github.com [02:30:59] [02miraheze/YouTube] 07Universal-Omega 031ec0247 - Update workflow [02:31:01] [02miraheze/CustomHeader] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CustomHeader/compare/2dc33e3ddbab...2771c57eb706 [02:31:01] [url] Comparing 2dc33e3ddbab...2771c57eb706 · miraheze/CustomHeader · GitHub | github.com [02:31:02] [02miraheze/CustomHeader] 07Universal-Omega 032771c57 - Update workflow [02:31:04] [02miraheze/SpriteSheet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/3c16b75e80b1...4a6b7e54be8e [02:31:04] [url] Comparing 3c16b75e80b1...4a6b7e54be8e · miraheze/SpriteSheet · GitHub | github.com [02:31:05] [02miraheze/SpriteSheet] 07Universal-Omega 034a6b7e5 - Update workflow [02:33:13] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.95, 4.11, 3.73 [02:37:10] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.86, 3.92, 3.73 [02:44:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.46, 3.57 [02:46:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.20, 3.24, 3.46 [02:47:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.71, 2.07, 1.91 [02:48:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.18, 3.01, 3.36 [02:48:58] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.38, 3.00, 3.38 [02:52:08] PROBLEM - www.opendatascot.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'opendatascot.org' expires in 15 day(s) (Fri 17 Jun 2022 02:46:18 GMT +0000). [02:54:08] miraheze/ManageWiki - Universal-Omega the build passed. [02:55:18] PROBLEM - with.cpt-ra.bid - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'with.cpt-ra.bid' expires in 15 day(s) (Fri 17 Jun 2022 02:49:21 GMT +0000). [02:55:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.93, 1.95 [02:57:38] miraheze/CreateWiki - Universal-Omega the build passed. [02:58:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.57, 3.27, 3.89 [02:58:57] miraheze/MatomoAnalytics - Universal-Omega the build passed. [02:59:25] miraheze/IncidentReporting - Universal-Omega the build passed. [02:59:48] miraheze/PDFEmbed - Universal-Omega the build passed. [03:00:12] miraheze/WikiDiscover - Universal-Omega the build passed. [03:00:18] miraheze/YouTube - Universal-Omega the build passed. [03:00:27] miraheze/CustomHeader - Universal-Omega the build passed. [03:00:33] miraheze/RemovePII - Universal-Omega the build passed. [03:00:36] miraheze/GlobalNewFiles - Universal-Omega the build passed. [03:00:43] miraheze/SpriteSheet - Universal-Omega the build passed. [03:00:50] miraheze/RottenLinks - Universal-Omega the build passed. [03:01:23] miraheze/DataDump - Universal-Omega the build passed. [03:02:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.69, 3.89 [03:04:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.23, 3.42, 3.77 [03:05:08] miraheze/MirahezeMagic - Universal-Omega the build passed. [03:05:53] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.21, 1.44, 1.69 [03:08:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.79, 3.83 [03:09:17] PROBLEM - journeytheword.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.journeytheword.wiki' expires in 15 day(s) (Fri 17 Jun 2022 02:44:07 GMT +0000). [03:10:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.59, 3.35, 3.66 [03:12:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.54, 3.98, 3.86 [03:12:08] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.76, 1.44 [03:14:03] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.94, 1.43, 1.36 [03:14:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.98, 3.93, 3.85 [03:14:30] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.92, 3.86, 3.38 [03:14:56] PROBLEM - www.journeytheword.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.journeytheword.wiki' expires in 15 day(s) (Fri 17 Jun 2022 02:44:07 GMT +0000). [03:16:28] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.40, 3.20, 3.19 [03:17:04] PROBLEM - opendatascot.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'opendatascot.org' expires in 15 day(s) (Fri 17 Jun 2022 02:46:18 GMT +0000). [03:17:56] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.17, 1.79, 1.72 [03:20:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.48, 3.74, 3.75 [03:20:14] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 8.02, 6.46, 5.60 [03:22:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.76, 3.87, 3.81 [03:22:14] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.98, 6.32, 5.66 [03:23:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.55, 1.89, 1.83 [03:26:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.39, 4.07, 3.90 [03:26:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.30, 3.16 [03:27:18] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.66, 3.32, 3.11 [03:28:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.83, 3.60, 3.75 [03:28:16] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.54, 3.09, 3.11 [03:30:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.34, 3.76, 3.77 [03:31:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.38, 3.31, 3.15 [03:32:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.53, 3.65, 3.73 [03:33:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.29, 1.91, 1.83 [03:36:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.64, 4.53, 4.02 [03:38:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.42, 3.74, 3.43 [03:39:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.98, 1.90 [03:40:42] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.04, 3.25, 3.29 [03:42:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.88, 3.96, 3.95 [03:46:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.71, 3.92, 3.90 [03:46:29] [02puppet] 07Universal-Omega synchronize pull request 03#2637: grafana-webhook: use alertname for resolved alerts - 13https://github.com/miraheze/puppet/pull/2637 [03:46:29] [url] Page not found · GitHub · GitHub | github.com [03:46:43] [02puppet] 07Universal-Omega edited pull request 03#2637: Minor improvements to grafana-webhook - 13https://github.com/miraheze/puppet/pull/2637 [03:46:43] [url] Page not found · GitHub · GitHub | github.com [03:47:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.66, 2.03, 1.92 [03:49:39] [02puppet] 07Universal-Omega edited pull request 03#2637: Minor improvements to grafana-webhook - 13https://github.com/miraheze/puppet/pull/2637 [03:49:39] [url] Page not found · GitHub · GitHub | github.com [03:50:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.73, 3.46, 3.31 [03:56:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.34, 3.69, 3.94 [03:56:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.04, 3.35, 3.36 [03:57:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.54, 1.91, 1.98 [03:59:07] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.14, 1.69, 1.35 [04:00:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.58, 3.87, 3.61 [04:02:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.47, 3.84, 3.89 [04:02:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.11, 4.05, 3.71 [04:03:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.12, 1.51, 1.36 [04:04:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.51, 3.97, 3.73 [04:08:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.22, 3.86, 3.94 [04:10:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.89, 3.82, 3.70 [04:12:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.33, 4.69, 4.25 [04:14:25] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 4.06, 3.51 [04:14:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.08, 3.66, 3.69 [04:16:24] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.94, 3.53 [04:17:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.88, 1.85 [04:20:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.11, 4.18, 3.70 [04:20:37] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.94, 3.97, 3.73 [04:22:18] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.10, 3.74, 3.60 [04:22:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.84, 3.60, 3.61 [04:24:02] PROBLEM - cloud11 Puppet on cloud11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [04:24:16] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.84, 4.18, 3.75 [04:26:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.29, 3.61, 3.58 [04:26:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.00, 3.07, 3.39 [04:27:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.55, 1.86, 1.92 [04:32:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.52, 3.54, 3.98 [04:32:09] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.51, 2.89, 3.33 [04:34:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.38, 3.83, 4.02 [04:36:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.63, 3.69, 3.94 [04:40:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.79, 3.99, 3.98 [04:43:53] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.31, 1.54, 1.69 [04:44:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.72, 3.48 [04:46:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.57, 3.27, 3.34 [04:48:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.60, 3.66, 3.96 [04:49:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.87, 1.76, 1.72 [04:51:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.21, 1.92, 1.79 [04:52:01] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:53:53] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.86, 1.78 [04:54:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.69, 3.97, 3.90 [04:56:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.11, 3.61, 3.78 [04:58:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.83, 3.69, 3.77 [05:01:53] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.83, 2.10, 1.88 [05:02:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.17, 3.47, 3.66 [05:08:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.89, 3.07, 3.40 [05:16:18] [02ssl] 07Reception123 closed pull request 03#537: Redirect miraheze.wiki/Fundraiser to the Meta Fundraiser page - 13https://github.com/miraheze/ssl/pull/537 [05:16:18] [url] Page not found · GitHub · GitHub | github.com [05:16:20] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4b26ff17042b...a510ba9053da [05:16:20] [url] Comparing 4b26ff17042b...a510ba9053da · miraheze/ssl · GitHub | github.com [05:16:21] [02miraheze/ssl] 07Universal-Omega 03a510ba9 - Redirect miraheze.wiki/Fundraiser to the Meta Fundraiser page (#537) [05:18:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a7f549e00164...edb7aa0a75ee [05:18:40] [url] Comparing a7f549e00164...edb7aa0a75ee · miraheze/mw-config · GitHub | github.com [05:18:41] [02miraheze/mw-config] 07Universal-Omega 03edb7aa0 - Add ssl and letsencrypt to disallowed subdomains [05:18:56] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.91, 4.15, 3.70 [05:19:25] [02puppet] 07Universal-Omega opened pull request 03#2638: varnish: use letsencrypt instead of sslrequest - 13https://github.com/miraheze/puppet/pull/2638 [05:19:25] [url] Page not found · GitHub · GitHub | github.com [05:19:37] [02puppet] 07Universal-Omega synchronize pull request 03#2638: varnish: use letsencrypt instead of sslrequest - 13https://github.com/miraheze/puppet/pull/2638 [05:19:37] ... [05:19:46] [02puppet] 07Universal-Omega synchronize pull request 03#2638: varnish: use letsencrypt instead of sslrequest - 13https://github.com/miraheze/puppet/pull/2638 [05:19:46] ... [05:19:50] miraheze/mw-config - Universal-Omega the build passed. [05:22:21] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=darkwitchwiki --report 1 --no-updates --username-prefix=wikia:darkwitch /home/reception/darkwitch_pages_full.xml (START) [05:22:22] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=darkwitchwiki --report 1 --no-updates --username-prefix=wikia:darkwitch /home/reception/darkwitch_pages_full.xml (END - exit=2) [05:22:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:22:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:22:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.51, 3.40, 3.52 [05:22:54] PROBLEM - cp30 Puppet on cp30 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:24:11] !log [@test101] starting deploy of {'config': True} to all [05:24:12] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [05:24:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:24:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:24:39] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tf2tagswikiwiki --report 1 --no-updates --username-prefix=wikia:tf2tags-lore /home/reception/tf2tagslore_pages_full.xml (START) [05:24:40] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tf2tagswikiwiki --report 1 --no-updates --username-prefix=wikia:tf2tags-lore /home/reception/tf2tagslore_pages_full.xml (END - exit=2) [05:24:42] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:24:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:24:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:25:32] [02puppet] 07Reception123 closed pull request 03#2638: varnish: use letsencrypt instead of sslrequest - 13https://github.com/miraheze/puppet/pull/2638 [05:25:32] [url] Page not found · GitHub · GitHub | github.com [05:25:34] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/44f62a03c738...8964b48952d6 [05:25:34] PROBLEM - puppet111 Puppet on puppet111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:25:34] [url] Comparing 44f62a03c738...8964b48952d6 · miraheze/puppet · GitHub | github.com [05:25:35] [02miraheze/puppet] 07Universal-Omega 038964b48 - varnish: use letsencrypt instead of sslrequest (#2638) [05:26:35] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.68, 3.14, 3.39 [05:27:45] PROBLEM - cp20 Puppet on cp20 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:27:52] PROBLEM - cp21 Puppet on cp21 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:29:12] [02mediawiki] 07Reception123 closed pull request 03#7237: Bump skins/Citizen from `fae6062` to `da03fa5` - 13https://github.com/miraheze/mediawiki/pull/7237 [05:29:12] [url] Page not found · GitHub · GitHub | github.com [05:29:13] [02mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_38/skins/Citizen-da03fa5 - 13https://github.com/miraheze/mediawiki [05:29:14] [url] Page not found · GitHub · GitHub | github.com [05:29:15] [02miraheze/mediawiki] 07Reception123 pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/fa91c692a9e9...4e21c5f609c2 [05:29:15] [url] Comparing fa91c692a9e9...4e21c5f609c2 · miraheze/mediawiki · GitHub | github.com [05:29:16] [02miraheze/mediawiki] 07dependabot[bot] 034e21c5f - Bump skins/Citizen from `fae6062` to `da03fa5` (#7237) [05:29:18] [02miraheze/mediawiki] 07Reception123 deleted branch 03dependabot/submodules/REL1_38/skins/Citizen-da03fa5 [05:29:43] !log [reception@mwtask111] starting deploy of {'world': True} to all [05:29:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:30:25] Reception123: you merged to 1.38, but are running deploy on 1.37 (mwtask111) [05:31:33] CosmicAlpha: I just realised... so I guess we don't have updates for 1.37 anymore with dependabot [05:31:37] I'll have to do it manually then [05:32:16] Reception123: no, I turned them off. We have no way to test 1.37 anymore. [05:32:36] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:32:53] CosmicAlpha: ah, well for Citizen I think it would be ok since it's just a skin, and I wouldn't want to keep that user waiting for weeks [05:33:55] We need to really get a move on with 1.38, 1.38 is releasing stable very soon probably. It was supposed to in May I thought, but didn't seem to happen. [05:34:03] Reception123: ^ [05:34:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.67, 3.51 [05:34:25] But yeah it's fine for that one probably [05:35:07] CosmicAlpha: yeah, I'll need to get back to testing extensions [05:35:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.55, 3.45, 3.06 [05:35:45] Also 1.39 is a nightmare already. DPL3 in particular is already failing on 1.39 due to the dropping of revision_actor_temp tables. [05:36:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.53, 3.79, 3.56 [05:36:08] PROBLEM - mw122 Puppet on mw122 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:36:14] And a ton of other extensions also still use the temp tables. [05:36:22] Social tools extensions do I think. [05:36:29] And even some of our own stuff. [05:37:47] !log [reception@mwtask111] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [05:37:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:38:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.56, 3.48, 3.48 [05:38:08] PROBLEM - mw111 Puppet on mw111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:39:07] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.38, 3.31, 3.08 [05:42:49] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=darkwitchwiki (START) [05:42:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:43:04] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.79, 1.48, 1.12 [05:43:08] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tf2tagswikiwikiwiki (START) [05:43:09] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tf2tagswikiwikiwiki (END - exit=65280) [05:43:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:43:14] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tf2tagswikiwiki (START) [05:43:15] PROBLEM - mw102 Puppet on mw102 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:43:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:43:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:43:35] PROBLEM - mw121 Puppet on mw121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:44:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.75, 3.18, 3.36 [05:45:04] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.16, 1.37, 1.12 [05:48:51] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=darkwitchwiki (END - exit=0) [05:48:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:50:57] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/runJobs.php --wiki=darkwitchwiki (END - exit=0) [05:51:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:51:21] PROBLEM - mw112 Puppet on mw112 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:51:44] PROBLEM - cp31 Puppet on cp31 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:52:38] PROBLEM - test101 Puppet on test101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [05:54:46] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.77, 3.50 [05:55:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.53, 3.51, 3.19 [05:57:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.35, 3.54, 2.93 [05:58:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.65, 3.71, 3.58 [06:01:08] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.98, 3.27, 3.20 [06:01:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.84, 3.29, 2.97 [06:02:24] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.37, 3.13, 3.37 [06:07:31] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tf2tagswikiwiki (END - exit=0) [06:07:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:14:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.41, 3.74, 3.49 [06:22:28] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.92, 3.20, 3.33 [06:25:03] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.45, 3.12, 2.83 [06:26:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.10, 3.61, 3.51 [06:27:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.62, 2.85, 2.76 [06:28:12] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.62, 2.91, 3.26 [06:50:31] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.82, 1.53, 1.15 [06:52:27] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.65, 1.48, 1.17 [07:02:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.16, 3.19, 3.06 [07:04:30] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.98, 3.25, 3.11 [07:08:41] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.81, 3.30, 2.89 [07:09:00] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.56, 1.74, 1.47 [07:10:14] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.72, 3.60, 3.28 [07:10:37] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.43, 3.04, 2.85 [07:10:46] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.60, 3.45, 3.00 [07:10:56] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.45, 1.70, 1.49 [07:12:09] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.61, 3.81, 3.38 [07:12:46] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.39, 3.29, 3.01 [07:13:25] [02puppet] 07Universal-Omega synchronize pull request 03#2637: Minor improvements to grafana-webhook - 13https://github.com/miraheze/puppet/pull/2637 [07:13:25] [url] Page not found · GitHub · GitHub | github.com [07:14:08] [02puppet] 07Universal-Omega edited pull request 03#2637: Minor improvements to grafana-webhook - 13https://github.com/miraheze/puppet/pull/2637 [07:14:08] [url] Page not found · GitHub · GitHub | github.com [07:18:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.34, 3.79, 3.53 [07:22:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.09, 4.02, 3.64 [07:28:07] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.15, 3.82, 3.69 [07:38:07] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.92, 2.97, 3.33 [07:39:13] [02ssl] 07Universal-Omega opened pull request 03#538: Fix path_redirects for Fundraiser - 13https://github.com/miraheze/ssl/pull/538 [07:39:13] [url] Page not found · GitHub · GitHub | github.com [07:46:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.59, 4.32, 3.65 [07:52:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.93, 3.64, 3.67 [07:58:19] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.54, 2.75, 3.29 [08:20:23] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.67, 6.54, 5.52 [08:22:44] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in