[00:02:10] PROBLEM - test131 Puppet on test131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:03:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a5eee7face5b...e2d1005326d6 [00:03:30] [02miraheze/puppet] 07paladox 03e2d1005 - Fix [00:03:33] [02puppet] 07paladox closed pull request 03#3326: Fix hiera - 13https://github.com/miraheze/puppet/pull/3326 [00:03:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e2d1005326d6...fc15f4981898 [00:03:37] [02miraheze/puppet] 07paladox 03fc15f49 - Fix hiera (#3326) [00:03:39] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [00:03:40] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:06:35] RECOVERY - puppet141 Puppet on puppet141 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [00:07:31] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:07:32] PROBLEM - matomo121 Puppet on matomo121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:07:45] PROBLEM - swiftobject111 Puppet on swiftobject111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:07:51] PROBLEM - mw134 Puppet on mw134 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:08:24] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:08:27] PROBLEM - mw143 Puppet on mw143 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 3 minutes ago with 1104 failures [00:09:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fc15f4981898...7eb2ce52df2a [00:09:04] [02miraheze/puppet] 07paladox 037eb2ce5 - base::postfix: replace legacy variable [00:09:31] PROBLEM - cloud12 Puppet on cloud12 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [00:09:46] PROBLEM - swiftproxy131 Puppet on swiftproxy131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [00:09:51] PROBLEM - cloud13 Puppet on cloud13 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 19 minutes ago with 0 failures [00:09:56] PROBLEM - db112 Puppet on db112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [00:09:56] PROBLEM - mem141 Puppet on mem141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 26 minutes ago with 0 failures [00:09:57] PROBLEM - puppetdb121 Puppet on puppetdb121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [00:09:57] PROBLEM - bast121 Puppet on bast121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [00:10:00] PROBLEM - swiftobject101 Puppet on swiftobject101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [00:10:00] PROBLEM - db142 Puppet on db142 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [00:10:01] PROBLEM - mon141 Puppet on mon141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [00:10:01] PROBLEM - cloud11 Puppet on cloud11 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 20 minutes ago with 0 failures [00:10:02] PROBLEM - mem131 Puppet on mem131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [00:10:03] PROBLEM - db101 Puppet on db101 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 19 minutes ago with 0 failures [00:10:06] PROBLEM - test131 Puppet on test131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 1 minute ago with 0 failures [00:10:14] PROBLEM - db131 Puppet on db131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [00:10:17] PROBLEM - mw142 Puppet on mw142 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 8 minutes ago with 0 failures [00:10:25] PROBLEM - cloud10 Puppet on cloud10 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 31 minutes ago with 0 failures [00:10:26] PROBLEM - phab121 Puppet on phab121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 32 minutes ago with 0 failures [00:10:26] PROBLEM - puppet141 Puppet on puppet141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 4 minutes ago with 0 failures [00:10:27] PROBLEM - ldap141 Puppet on ldap141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 7 minutes ago with 0 failures [00:10:27] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [00:10:28] PROBLEM - cloud14 Puppet on cloud14 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 19 minutes ago with 0 failures [00:10:35] PROBLEM - jobchron121 Puppet on jobchron121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 24 minutes ago with 0 failures [00:10:35] PROBLEM - swiftac111 Puppet on swiftac111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [00:10:36] PROBLEM - prometheus131 Puppet on prometheus131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 17 minutes ago with 0 failures [00:10:36] PROBLEM - mw131 Puppet on mw131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 28 minutes ago with 0 failures [00:10:36] PROBLEM - mw133 Puppet on mw133 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 10 minutes ago with 0 failures [00:10:39] PROBLEM - swiftproxy111 Puppet on swiftproxy111 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 29 minutes ago with 0 failures [00:10:40] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:10:43] PROBLEM - cp33 Puppet on cp33 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [00:10:44] PROBLEM - ns1 Puppet on ns1 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 15 minutes ago with 0 failures [00:10:48] PROBLEM - os141 Puppet on os141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 2 minutes ago with 0 failures [00:10:51] PROBLEM - mwtask141 Puppet on mwtask141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 30 minutes ago with 0 failures [00:10:59] PROBLEM - ns2 Puppet on ns2 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 22 minutes ago with 0 failures [00:11:02] PROBLEM - graylog131 Puppet on graylog131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 9 minutes ago with 0 failures [00:11:06] PROBLEM - swiftobject121 Puppet on swiftobject121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 21 minutes ago with 0 failures [00:11:07] PROBLEM - swiftobject112 Puppet on swiftobject112 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 25 minutes ago with 0 failures [00:11:13] PROBLEM - db121 Puppet on db121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 22 minutes ago with 0 failures [00:11:15] PROBLEM - mail121 Puppet on mail121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 28 minutes ago with 0 failures [00:11:15] PROBLEM - cp22 Puppet on cp22 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 28 minutes ago with 0 failures [00:11:16] PROBLEM - os131 Puppet on os131 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [00:11:20] PROBLEM - bast141 Puppet on bast141 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 12 minutes ago with 0 failures [00:11:20] PROBLEM - cp23 Puppet on cp23 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [00:11:25] PROBLEM - swiftobject113 Puppet on swiftobject113 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 27 minutes ago with 0 failures [00:11:28] PROBLEM - swiftobject122 Puppet on swiftobject122 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 21 minutes ago with 0 failures [00:11:31] PROBLEM - reports121 Puppet on reports121 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 14 minutes ago with 0 failures [00:12:05] RECOVERY - test131 Puppet on test131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:19] PROBLEM - cp32 Puppet on cp32 is WARNING: WARNING: Puppet is currently disabled, message: paladox, last run 36 seconds ago with 584 failures [00:12:27] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [00:12:33] RECOVERY - jobchron121 Puppet on jobchron121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:12:36] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:12:36] RECOVERY - mw133 Puppet on mw133 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:12:37] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [00:12:44] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:12:51] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:12:59] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:01] RECOVERY - graylog131 Puppet on graylog131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:06] RECOVERY - swiftobject121 Puppet on swiftobject121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:13] RECOVERY - db121 Puppet on db121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:15] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:15] RECOVERY - os131 Puppet on os131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:15] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:18] RECOVERY - bast141 Puppet on bast141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:20] RECOVERY - cp23 Puppet on cp23 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:26] RECOVERY - swiftobject113 Puppet on swiftobject113 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:26] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:29] RECOVERY - swiftobject122 Puppet on swiftobject122 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:31] RECOVERY - reports121 Puppet on reports121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:44] RECOVERY - swiftproxy131 Puppet on swiftproxy131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:50] RECOVERY - cloud13 Puppet on cloud13 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:52] RECOVERY - bast121 Puppet on bast121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:55] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:13:55] RECOVERY - db112 Puppet on db112 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:13:56] RECOVERY - mem141 Puppet on mem141 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [00:13:59] RECOVERY - swiftobject101 Puppet on swiftobject101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:00] RECOVERY - db142 Puppet on db142 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:01] RECOVERY - mon141 Puppet on mon141 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:01] RECOVERY - mem131 Puppet on mem131 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:02] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:03] RECOVERY - db101 Puppet on db101 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:12] RECOVERY - db131 Puppet on db131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:14] RECOVERY - mw142 Puppet on mw142 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:21] RECOVERY - puppet141 Puppet on puppet141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:21] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:22] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Puppet has 584 failures. Last run 2 minutes ago with 584 failures. Failed resources (up to 3 shown): File[/etc/default/ferm],File[/usr/lib/nagios/plugins/check_conntrack],File[/usr/lib/nagios/plugins/check_ferm],File[/etc/nagios/nrpe.cfg] [00:14:22] RECOVERY - ldap141 Puppet on ldap141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:26] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:14:27] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:28] PROBLEM - mw143 Puppet on mw143 is CRITICAL: CRITICAL: Puppet has 1104 failures. Last run 1 minute ago with 1104 failures. Failed resources (up to 3 shown): File[/etc/apt/trusted.gpg.d/puppetlabs.gpg],File[/usr/local/bin/puppet-enabled],File[/etc/rsyslog.d],File[/etc/rsyslog.conf] [00:14:30] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:33] RECOVERY - prometheus131 Puppet on prometheus131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:34] RECOVERY - swiftproxy111 Puppet on swiftproxy111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:40] RECOVERY - cp33 Puppet on cp33 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:14:48] RECOVERY - os141 Puppet on os141 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:15:07] RECOVERY - swiftobject112 Puppet on swiftobject112 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:17:59] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.37, 3.66, 2.94 [00:19:54] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.90, 3.40, 2.93 [00:20:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/8204a91d6efe [00:20:20] [02miraheze/puppet] 07paladox 038204a91 - Switch legacy facts off by default [00:20:22] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:20:24] [02puppet] 07paladox opened pull request 03#3327: Switch legacy facts off by default - 13https://github.com/miraheze/puppet/pull/3327 [00:20:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8204a91d6efe...75759b65a2ba [00:20:42] [02miraheze/puppet] 07paladox 0375759b6 - Update puppet.conf.erb [00:20:44] [02puppet] 07paladox synchronize pull request 03#3327: Switch legacy facts off by default - 13https://github.com/miraheze/puppet/pull/3327 [00:20:46] [02puppet] 07paladox closed pull request 03#3327: Switch legacy facts off by default - 13https://github.com/miraheze/puppet/pull/3327 [00:20:49] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/7eb2ce52df2a...bfcd89afc320 [00:20:52] [02miraheze/puppet] 07paladox 03bfcd89a - Switch legacy facts off by default (#3327) [00:20:54] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [00:20:56] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:24:01] PROBLEM - mon141 Puppet on mon141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:24:23] PROBLEM - cloud14 Puppet on cloud14 is CRITICAL: CRITICAL: Puppet has 8 failures. Last run 2 minutes ago with 8 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/wildcard.miraheze.org-2020-2.crt],File[/etc/ssl/private/wildcard.miraheze.org-2020-2.key],File[/etc/ferm/conf.d/02_main],File[/etc/sudoers] [00:26:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/e4078b7d1b1b [00:26:11] [02miraheze/puppet] 07paladox 03e4078b7 - Replace sshecdsakey legacy variable [00:26:13] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:26:14] [02puppet] 07paladox opened pull request 03#3328: Replace sshecdsakey legacy variable - 13https://github.com/miraheze/puppet/pull/3328 [00:26:29] [02puppet] 07paladox closed pull request 03#3328: Replace sshecdsakey legacy variable - 13https://github.com/miraheze/puppet/pull/3328 [00:26:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/bfcd89afc320...a2a977d05fd0 [00:26:33] [02miraheze/puppet] 07paladox 03a2a977d - Replace sshecdsakey legacy variable (#3328) [00:26:35] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [00:26:38] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:35:27] RECOVERY - swiftobject111 Puppet on swiftobject111 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [00:36:01] RECOVERY - mon141 Puppet on mon141 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [00:36:53] RECOVERY - mw143 Puppet on mw143 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:37:31] RECOVERY - mw132 Puppet on mw132 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:37:32] RECOVERY - matomo121 Puppet on matomo121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:37:40] RECOVERY - mw134 Puppet on mw134 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:37:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/8fce3eef6bbc [00:37:52] [02miraheze/puppet] 07paladox 038fce3ee - Replace lsbdistcodename legacy fact [00:37:55] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:37:57] [02puppet] 07paladox opened pull request 03#3329: Replace lsbdistcodename legacy fact - 13https://github.com/miraheze/puppet/pull/3329 [00:38:20] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [00:38:21] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8fce3eef6bbc...f4c3de670d69 [00:38:24] [02miraheze/puppet] 07paladox 03f4c3de6 - Update init.pp [00:38:24] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:38:25] [02puppet] 07paladox synchronize pull request 03#3329: Replace lsbdistcodename legacy fact - 13https://github.com/miraheze/puppet/pull/3329 [00:40:01] PROBLEM - mon141 Puppet on mon141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:40:34] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f4c3de670d69...496dc9296aef [00:40:37] [02miraheze/puppet] 07paladox 03496dc92 - Update init.pp [00:40:40] [02puppet] 07paladox synchronize pull request 03#3329: Replace lsbdistcodename legacy fact - 13https://github.com/miraheze/puppet/pull/3329 [00:40:43] [02puppet] 07paladox closed pull request 03#3329: Replace lsbdistcodename legacy fact - 13https://github.com/miraheze/puppet/pull/3329 [00:40:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/a2a977d05fd0...44b1a624bdb6 [00:40:47] [02miraheze/puppet] 07paladox 0344b1a62 - Replace lsbdistcodename legacy fact (#3329) [00:40:50] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [00:40:53] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [00:42:01] RECOVERY - mon141 Puppet on mon141 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [00:46:24] PROBLEM - ambient.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ambient.wiki All nameservers failed to answer the query. [00:46:56] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/44b1a624bdb6...73113a9f83f0 [00:46:58] [02miraheze/puppet] 07paladox 0373113a9 - openldap: remove deadcode [00:50:16] PROBLEM - ldap141 Puppet on ldap141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:51:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.69, 3.07, 3.87 [00:51:44] PROBLEM - elephants.ggreig.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query ggreig.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:52:16] RECOVERY - ldap141 Puppet on ldap141 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [00:52:17] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:53:26] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 7.08, 4.77, 4.39 [00:53:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/73113a9f83f0...0b0ce61eb777 [00:53:52] [02miraheze/puppet] 07paladox 030b0ce61 - openldap: Update params.pp [00:53:54] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0b0ce61eb777...65d5c4a89e6a [00:53:55] [02miraheze/puppet] 07paladox 0365d5c4a - Update params.pp [00:55:08] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/65d5c4a89e6a...ba4dc08a9a8d [00:55:09] [02miraheze/puppet] 07paladox 03ba4dc08 - Update config.pp [00:55:14] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.43, 3.85, 3.34 [00:55:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ba4dc08a9a8d...2d8119169a20 [00:55:19] [02miraheze/puppet] 07paladox 032d81191 - Replace legacy fact in openldap [00:56:08] PROBLEM - wikien.wildterra2.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wikien.wildterra2.com. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:57:14] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.44, 3.62, 3.31 [00:57:26] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.39, 3.27, 3.84 [00:59:37] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2d8119169a20...1c11f016512b [00:59:40] [02miraheze/puppet] 07paladox 031c11f01 - Update openldap.pp [01:00:55] RECOVERY - td10.org - LetsEncrypt on sslhost is OK: OK - Certificate 'td10.org' will expire on Sat 21 Oct 2023 23:16:58 GMT +0000. [01:01:14] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.83, 3.17, 3.20 [01:01:24] RECOVERY - ldap141 LDAP on ldap141 is OK: LDAP OK - 0.044 seconds response time [01:01:43] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1c11f016512b...a60be23bdcb3 [01:01:46] [02miraheze/puppet] 07paladox 03a60be23 - Update cadvisor.pp [01:03:26] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.16, 2.76, 3.40 [01:05:02] PROBLEM - graylog131 Puppet on graylog131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [01:07:14] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.15, 3.53, 3.36 [01:09:14] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.85, 3.31, 3.30 [01:12:20] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/23619137e59a [01:12:23] [02miraheze/puppet] 07paladox 032361913 - Replace osfamily legacy variable [01:12:25] [02puppet] 07paladox created branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [01:12:28] [02puppet] 07paladox opened pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:12:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/23619137e59a...d4f06fec888a [01:12:51] [02miraheze/puppet] 07paladox 03d4f06fe - Update schema.pp [01:12:52] [02puppet] 07paladox synchronize pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:12:58] PROBLEM - arru.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - arru.xyz All nameservers failed to answer the query. [01:13:23] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d4f06fec888a...85fdd8d5dda4 [01:13:24] [02miraheze/puppet] 07paladox 0385fdd8d - Update init.pp [01:13:25] [02puppet] 07paladox synchronize pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:14:10] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/85fdd8d5dda4...e9f50e1a2836 [01:14:12] [02miraheze/puppet] 07paladox 03e9f50e1 - Update install.pp [01:14:13] [02puppet] 07paladox synchronize pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:14:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e9f50e1a2836...986ddc9b5e9d [01:14:40] [02miraheze/puppet] 07paladox 03986ddc9 - Update database.pp [01:14:42] [02puppet] 07paladox synchronize pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:15:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-7 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/986ddc9b5e9d...c3ef15ccf93a [01:15:03] [02miraheze/puppet] 07paladox 03c3ef15c - Update slapdconf.pp [01:15:06] [02puppet] 07paladox synchronize pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:15:12] [02puppet] 07paladox closed pull request 03#3330: Replace osfamily legacy variable - 13https://github.com/miraheze/puppet/pull/3330 [01:15:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±6] 13https://github.com/miraheze/puppet/compare/a60be23bdcb3...160ddbbb685f [01:15:18] [02miraheze/puppet] 07paladox 03160ddbb - Replace osfamily legacy variable (#3330) [01:15:20] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-7 [01:15:22] [02puppet] 07paladox deleted branch 03paladox-patch-7 - 13https://github.com/miraheze/puppet [01:16:15] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.33, 3.69, 3.41 [01:16:18] PROBLEM - ambient.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ambient.wiki could not be found [01:17:02] RECOVERY - graylog131 Puppet on graylog131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [01:20:06] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 3.16, 3.26 [01:20:19] PROBLEM - redkrayolawiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - redkrayolawiki.org All nameservers failed to answer the query. [01:20:57] !log reboot cp33 [01:21:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:21:32] RECOVERY - elephants.ggreig.com - reverse DNS on sslhost is OK: SSL OK - elephants.ggreig.com reverse DNS resolves to cp23.miraheze.org - CNAME OK [01:21:52] !log reboot cp32 [01:22:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:22:03] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.26, 0.36, 0.13 [01:22:30] PROBLEM - www.vanillaanarchy.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query vanillaanarchy.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:23:00] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.67, 3.42, 3.34 [01:24:29] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 12.72, 7.03, 4.48 [01:25:01] PROBLEM - ldap141 LDAP on ldap141 is UNKNOWN: check_ldap: Please specify the host nameUsage: check_ldap -H -b [-p ] [-a ] [-D ] [-P ] [-w ] [-c ] [-t timeout] [-2|-3] [-4|-6] [01:25:17] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.92, 5.42, 3.88 [01:25:32] RECOVERY - wikien.wildterra2.com - reverse DNS on sslhost is OK: SSL OK - wikien.wildterra2.com reverse DNS resolves to cp22.miraheze.org - CNAME OK [01:25:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 15.36, 11.70, 7.00 [01:25:54] PROBLEM - swiftobject121 Current Load on swiftobject121 is WARNING: WARNING - load average: 6.84, 6.65, 4.86 [01:26:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/160ddbbb685f...901ac9c09dd6 [01:26:23] [02miraheze/puppet] 07paladox 03901ac9c - Update openldap.pp [01:26:52] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.07, 3.29, 3.32 [01:27:12] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 4.76, 5.04, 3.92 [01:28:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.92, 7.42, 5.27 [01:29:11] RECOVERY - ldap141 LDAP on ldap141 is OK: LDAP OK - 0.028 seconds response time [01:30:25] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.33, 7.80, 5.63 [01:31:04] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.25, 3.78, 3.52 [01:32:21] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.67, 6.97, 5.59 [01:33:02] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.00, 3.51, 3.46 [01:33:15] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 4.90, 7.72, 6.99 [01:33:33] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.93, 6.47, 5.51 [01:34:20] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.23, 3.98, 2.09 [01:36:13] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 3.87, 6.22, 5.68 [01:36:19] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.42, 3.66, 2.20 [01:38:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.31, 4.65, 2.80 [01:40:18] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.38, 3.79, 2.71 [01:40:55] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.40, 3.24, 3.39 [01:41:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.53, 8.20, 7.29 [01:42:02] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.98, 7.60, 6.35 [01:42:18] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.63, 4.01, 2.91 [01:42:22] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [01:42:39] PROBLEM - arru.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for arru.xyz could not be found [01:42:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.20, 7.79, 7.26 [01:43:58] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.03, 6.75, 6.19 [01:44:17] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.84, 3.51, 2.86 [01:46:15] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [01:48:16] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.35, 3.21, 2.94 [01:48:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.49, 8.31, 7.44 [01:48:45] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.95, 3.80, 3.50 [01:50:19] PROBLEM - redkrayolawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for redkrayolawiki.org could not be found [01:50:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.32, 7.23, 7.14 [01:50:43] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.03, 3.55, 3.45 [01:51:15] PROBLEM - www.vanillaanarchy.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.vanillaanarchy.net All nameservers failed to answer the query. [01:54:40] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.11, 3.82, 3.59 [01:56:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03openldap-upgrade [+35/-23/±78] 13https://github.com/miraheze/puppet/commit/e4b209cfc936 [01:56:10] [02miraheze/puppet] 07paladox 03e4b209c - Update openldap to 7.0 [01:56:11] [02puppet] 07paladox created branch 03openldap-upgrade - 13https://github.com/miraheze/puppet [01:56:18] [02puppet] 07paladox opened pull request 03#3331: Update openldap to 7.0 - 13https://github.com/miraheze/puppet/pull/3331 [01:56:22] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.11, 5.38, 3.60 [01:56:30] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.25, 5.78, 6.55 [01:58:36] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 2.98, 3.56, 3.56 [02:00:15] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.99, 3.73, 3.30 [02:00:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/901ac9c09dd6...b7d9e2948e04 [02:00:45] [02miraheze/puppet] 07paladox 03b7d9e29 - Replace deprecated merge() in ssl::hiera [02:02:32] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [02:04:17] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.67, 7.08, 6.65 [02:04:30] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.41, 3.60, 3.56 [02:06:06] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.43, 3.25, 3.23 [02:06:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.83, 6.69, 6.54 [02:06:28] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.23, 3.45, 3.51 [02:08:10] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 3.24, 5.35, 6.07 [02:08:26] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.59, 3.12, 3.38 [02:12:33] PROBLEM - wikipanel.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikipanel.org All nameservers failed to answer the query. [02:20:42] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.hsdfz.org.cn All nameservers failed to answer the query. [02:22:55] PROBLEM - elephants.ggreig.com - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query ggreig.com. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:26:40] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.38, 3.83, 3.33 [02:28:37] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.87, 3.38, 3.22 [02:31:30] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [02:41:01] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.96, 3.15, 2.56 [02:42:11] PROBLEM - wikipanel.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wikipanel.org could not be found [02:43:01] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.21, 2.88, 2.55 [02:43:15] PROBLEM - redkrayolawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'redkrayolawiki.org' expires in 11 day(s) (Fri 04 Aug 2023 07:27:51 GMT +0000). [02:44:42] RECOVERY - hkrail.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hkrail.wiki' will expire on Tue 05 Sep 2023 13:55:12 GMT +0000. [02:49:40] PROBLEM - wiki.hsdfz.org.cn - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['emily.ns.cloudflare.com.', 'lloyd.ns.cloudflare.com.'], 'CNAME': 'hsdfz.miraheze.org.'} [02:49:42] PROBLEM - wiki.ravynos.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ravynos.com All nameservers failed to answer the query. [02:52:31] RECOVERY - elephants.ggreig.com - reverse DNS on sslhost is OK: SSL OK - elephants.ggreig.com reverse DNS resolves to cp22.miraheze.org - CNAME OK [02:56:17] PROBLEM - hkrail.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - hkrail.wiki All nameservers failed to answer the query. [03:05:54] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.20, 3.29, 2.84 [03:07:54] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.50, 2.96, 2.78 [03:09:51] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.72, 3.16 [03:11:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.39, 3.32, 3.08 [03:12:02] PROBLEM - redkrayolawiki.org - LetsEncrypt on sslhost is CRITICAL: connect to address redkrayolawiki.org and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:16:39] PROBLEM - hkrail.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address hkrail.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [03:16:46] RECOVERY - www.vanillaanarchy.net - reverse DNS on sslhost is OK: SSL OK - www.vanillaanarchy.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [03:20:49] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.64, 3.72, 3.33 [03:22:49] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.46, 2.90, 3.07 [03:25:30] PROBLEM - hkrail.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for hkrail.wiki could not be found [03:28:47] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.48, 3.35, 3.18 [03:29:38] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:30:46] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.13, 3.27, 3.17 [03:36:04] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 5.00, 4.04, 3.04 [03:36:40] PROBLEM - puritwiki.p-e.kr - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query p-e.kr. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:36:44] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 4.08, 3.45, 3.21 [03:38:04] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.15, 3.76, 3.07 [03:38:44] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 3.77, 3.48, 3.25 [03:40:04] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 1.89, 3.04, 2.89 [03:40:43] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 3.02, 3.28, 3.20 [03:46:38] RECOVERY - hkrail.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hkrail.wiki' will expire on Tue 05 Sep 2023 13:55:12 GMT +0000. [03:49:41] PROBLEM - wiki.ravynos.com - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.ravynos.com could not be found [03:59:12] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [04:05:19] RECOVERY - puritwiki.p-e.kr - reverse DNS on sslhost is OK: SSL OK - puritwiki.p-e.kr reverse DNS resolves to cp22.miraheze.org - CNAME OK [04:10:37] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [04:16:51] PROBLEM - redkrayolawiki.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - redkrayolawiki.org All nameservers failed to answer the query. [04:18:08] RECOVERY - perilous-times.com - LetsEncrypt on sslhost is OK: OK - Certificate 'perilous-times.com' will expire on Wed 30 Aug 2023 14:08:32 GMT +0000. [04:18:35] PROBLEM - hkrail.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address hkrail.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:25:30] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.37, 4.60, 3.28 [04:27:30] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 2.60, 3.83, 3.16 [04:29:29] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 1.91, 3.09, 2.97 [04:40:16] PROBLEM - open-justice.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'open-justice.info' expires in 15 day(s) (Wed 09 Aug 2023 04:10:03 GMT +0000). [04:40:29] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [04:41:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/24f91eb69594...6dc52df99828 [04:42:00] [02miraheze/ssl] 07MirahezeSSLBot 036dc52df - Bot: Update SSL cert for open-justice.info [04:46:09] PROBLEM - redkrayolawiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for redkrayolawiki.org could not be found [04:47:24] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 5.37, 3.64, 2.93 [04:49:24] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 2.08, 2.99, 2.78 [04:49:56] PROBLEM - perilous-times.com - LetsEncrypt on sslhost is CRITICAL: connect to address perilous-times.com and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:57:42] PROBLEM - mirawww.glitchcity.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - mirawww.glitchcity.wiki All nameservers failed to answer the query. [05:06:41] PROBLEM - wiki.litek.top - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.litek.top All nameservers failed to answer the query. [05:09:19] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 15.29, 12.54, 6.65 [05:14:04] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [05:15:32] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:17:06] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 13137 MB (9% inode=99%); [05:17:28] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [05:17:30] PROBLEM - wiki.pulsus.cc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pulsus.cc' expires in 15 day(s) (Wed 09 Aug 2023 04:51:43 GMT +0000). [05:18:41] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.146 seconds response time. miraheze.org returns 108.175.15.182,2607:f1c0:1800:26f::1,2607:f1c0:1800:8100::1,74.208.203.152 [05:19:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6dc52df99828...3d091da5b88d [05:19:37] [02miraheze/ssl] 07MirahezeSSLBot 033d091da - Bot: Update SSL cert for wiki.pulsus.cc [05:21:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.25, 6.51, 4.56 [05:23:03] PROBLEM - perilous-times.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - perilous-times.com All nameservers failed to answer the query. [05:23:32] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.14, 5.81, 4.54 [05:24:43] PROBLEM - hkrail.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - hkrail.wiki All nameservers failed to answer the query. [05:25:11] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [05:27:34] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:27:37] PROBLEM - mirawww.glitchcity.wiki - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for mirawww.glitchcity.wiki could not be found [05:27:53] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [05:27:58] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [05:29:53] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:29:54] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [05:30:24] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:31:10] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:32:22] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 4.601 seconds response time. miraheze.org returns 2607:f1c0:1800:26f::1,74.208.203.152 [05:32:30] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 15 backends are healthy [05:33:13] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 25 minutes ago with 0 failures [05:33:37] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [05:36:33] PROBLEM - wiki.litek.top - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['dry.dnspod.net.', 'python.dnspod.net.'], 'CNAME': 'kahmiz.miraheze.org.'} [05:37:06] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7627 MB (5% inode=99%); [05:37:16] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.42, 3.12, 2.66 [05:37:31] RECOVERY - open-justice.info - LetsEncrypt on sslhost is OK: OK - Certificate 'open-justice.info' will expire on Sun 22 Oct 2023 03:41:44 GMT +0000. [05:39:13] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.88, 2.71, 2.57 [05:40:47] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [05:40:49] PROBLEM - redkrayolawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'redkrayolawiki.org' expires in 11 day(s) (Fri 04 Aug 2023 07:27:51 GMT +0000). [05:42:34] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 108.175.15.182/cpweb, 2607:f1c0:1800:8100::1/cpweb [05:42:38] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [05:42:53] PROBLEM - cp32 HTTPS on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:43:48] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:44:30] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [05:44:35] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:44:45] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [05:45:01] RECOVERY - cp32 HTTPS on cp32 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3727 bytes in 8.311 second response time [05:45:40] PROBLEM - cp32 ferm_active on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:45:46] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:47:44] PROBLEM - pyramidgames.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pyramidgames.wiki. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:48:26] RECOVERY - cp32 ferm_active on cp32 is OK: OK ferm input default policy is set [05:50:57]