[00:01:18] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:28] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [00:03:07] RECOVERY - swiftac1 Puppet on swiftac1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:03:18] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:05:58] RECOVERY - services1 Puppet on services1 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [00:05:59] RECOVERY - graylog1 Puppet on graylog1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:06:45] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [00:06:45] RECOVERY - mem1 Puppet on mem1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:10:22] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:10:29] RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [00:11:57] RECOVERY - phorge1 Puppet on phorge1 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [00:12:03] RECOVERY - prometheus1 Puppet on prometheus1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:43] RECOVERY - ldap1 Puppet on ldap1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:31] RECOVERY - swiftobject1 Puppet on swiftobject1 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [00:19:06] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:19:13] RECOVERY - jobchron1 Puppet on jobchron1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:19:40] RECOVERY - puppet1 Puppet on puppet1 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [00:19:51] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [00:20:30] RECOVERY - bots1 Puppet on bots1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:21:13] RECOVERY - matomo1 Puppet on matomo1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:21:44] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:22:15] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:23:43] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:25:52] RECOVERY - db1 Puppet on db1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:26:08] RECOVERY - cp1 Puppet on cp1 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [00:26:27] RECOVERY - bast1 Puppet on bast1 is OK: OK: Puppet is currently enabled, last run 58 seconds ago with 0 failures [00:29:54] RECOVERY - swiftproxy1 Puppet on swiftproxy1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:44:59] RECOVERY - wc.wikitide.org on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Mon 18 Nov 2024 11:59:59 PM GMT +0000. [02:48:12] [02WikiTideOrg/mw-config] 07dependabot[bot] pushed 031 commit to 03dependabot/github_actions/styfle/cancel-workflow-action-0.12.1 [+0/-0/±1] 13https://github.com/WikiTideOrg/mw-config/commit/03226a8f24c0 [02:48:15] [02WikiTideOrg/mw-config] 07dependabot[bot] 0303226a8 - Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.12.0 to 0.12.1. - [Release notes](https://github.com/styfle/cancel-workflow-action/releases) - [Commits](https://github.com/styfle/cancel-workflow-action/compare/0.12.0...0.12.1) --- updated-dependencies: - dependency-name: styfle/canc [02:48:16] [02mw-config] 07dependabot[bot] created branch 03dependabot/github_actions/styfle/cancel-workflow-action-0.12.1 - 13https://github.com/WikiTideOrg/mw-config [02:48:19] [02mw-config] 07dependabot[bot] opened pull request 03#30: Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 - 13https://github.com/WikiTideOrg/mw-config/pull/30 [02:48:21] [02mw-config] 07dependabot[bot] labeled pull request 03#30: Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 - 13https://github.com/WikiTideOrg/mw-config/pull/30 [02:48:23] [02mw-config] 07dependabot[bot] labeled pull request 03#30: Bump styfle/cancel-workflow-action from 0.12.0 to 0.12.1 - 13https://github.com/WikiTideOrg/mw-config/pull/30 [02:48:59] PROBLEM - wc.wikitide.org on sslhost is WARNING: WARNING - Certificate 'wikitide.org' expires in 25 day(s) (Tue 20 Feb 2024 12:20:10 PM GMT +0000). [02:49:11] WikiTideOrg/mw-config - dependabot[bot] the build passed. [06:34:28] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:37:58] PROBLEM - services1 Puppet on services1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:38:45] PROBLEM - mw1 Puppet on mw1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:42:04] PROBLEM - prometheus1 Puppet on prometheus1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:42:26] PROBLEM - cp4 Puppet on cp4 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:42:29] PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [06:43:58] PROBLEM - phorge1 Puppet on phorge1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [07:02:28] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 53 seconds ago with 0 failures [07:05:58] RECOVERY - services1 Puppet on services1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:07:59] PROBLEM - graylog1 Puppet on graylog1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [07:14:43] PROBLEM - ldap1 Puppet on ldap1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [07:34:28] PROBLEM - mw2 Puppet on mw2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [07:35:59] RECOVERY - graylog1 Puppet on graylog1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:46:31] PROBLEM - swiftobject1 Puppet on swiftobject1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:49:13] PROBLEM - jobchron1 Puppet on jobchron1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:49:27] PROBLEM - cp6 Puppet on cp6 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:49:51] PROBLEM - jobrunner1 Puppet on jobrunner1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:51:41] PROBLEM - puppet1 Puppet on puppet1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:52:30] PROBLEM - bots1 Puppet on bots1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:53:13] PROBLEM - matomo1 Puppet on matomo1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:53:45] PROBLEM - cp3 Puppet on cp3 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:54:15] PROBLEM - cp2 Puppet on cp2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:55:02] PROBLEM - cp5 Puppet on cp5 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:57:52] PROBLEM - db1 Puppet on db1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:58:28] PROBLEM - bast1 Puppet on bast1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:00:08] PROBLEM - cp1 Puppet on cp1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:01:54] PROBLEM - swiftproxy1 Puppet on swiftproxy1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:03:07] PROBLEM - swiftac1 Puppet on swiftac1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:06:45] PROBLEM - mem1 Puppet on mem1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:07:58] PROBLEM - services1 Puppet on services1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:07:59] PROBLEM - graylog1 Puppet on graylog1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:07:48] PROBLEM - www.greatamerica.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.greatamerica.wiki All nameservers failed to answer the query. [14:09:58] PROBLEM - hsck.lophocmatngu.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - hsck.lophocmatngu.wiki All nameservers failed to answer the query. [14:14:25] PROBLEM - wikitide.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikitide.net All nameservers failed to answer the query. [14:15:02] PROBLEM - wiki.myehs.eu - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.myehs.eu All nameservers failed to answer the query. [14:15:04] PROBLEM - www.polandballwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.polandballwiki.com All nameservers failed to answer the query. [14:21:49] PROBLEM - wiki.colleimadcat.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.colleimadcat.com All nameservers failed to answer the query. [14:21:53] PROBLEM - www.lgbtqia.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.lgbtqia.wiki All nameservers failed to answer the query. [14:37:48] PROBLEM - www.greatamerica.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['john.ns.cloudflare.com.', 'vida.ns.cloudflare.com.'], 'CNAME': 'mw-lb.wikitide.org.'} [14:39:58] PROBLEM - hsck.lophocmatngu.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['lia.ns.cloudflare.com.', 'gerald.ns.cloudflare.com.'], 'CNAME': 'cf-lb.wikitide.org.'} [14:44:25] PROBLEM - wikitide.net - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikitide.net could not be found [14:45:03] PROBLEM - wiki.myehs.eu - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['fred.ns.cloudflare.com.', 'ivy.ns.cloudflare.com.'], 'CNAME': 'cf-lb.wikitide.org.'} [14:45:04] PROBLEM - www.polandballwiki.com - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['john.ns.cloudflare.com.', 'vida.ns.cloudflare.com.'], 'CNAME': 'mw-lb.wikitide.org.'} [14:51:50] PROBLEM - wiki.colleimadcat.com - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dns30.hichina.com.', 'dns29.hichina.com.'], 'CNAME': 'cf-lb.wikitide.org.'} [14:51:53] PROBLEM - www.lgbtqia.wiki - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['grannbo.ns.cloudflare.com.', 'rodrigo.ns.cloudflare.com.'], 'CNAME': 'cf-lb.wikitide.org.'}