[00:03:49] PROBLEM - cp22 Current Load on cp22 is CRITICAL: LOAD CRITICAL - total load average: 33.22, 32.25, 14.22 [00:03:58] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 18.48, 26.20, 12.61 [00:04:33] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.38, 3.89, 3.56 [00:06:33] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 2.31, 3.33, 3.39 [00:11:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 4.95, 6.74, 7.90 [00:15:36] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 11.70, 7.90, 7.99 [00:19:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.86, 7.20, 7.71 [00:21:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.53, 7.18, 7.96 [00:25:58] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 0.23, 0.83, 3.92 [00:29:58] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.40, 0.68, 3.15 [00:31:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.96, 7.67, 7.49 [00:33:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.74, 7.29, 7.36 [00:33:43] PROBLEM - cp22 Current Load on cp22 is WARNING: LOAD WARNING - total load average: 0.20, 0.71, 3.87 [00:35:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.09, 6.83, 7.11 [00:37:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.34, 6.98, 7.13 [00:37:42] RECOVERY - cp22 Current Load on cp22 is OK: LOAD OK - total load average: 0.21, 0.44, 3.04 [00:43:21] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.85, 5.99, 6.66 [00:47:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.06, 6.70, 6.93 [00:49:35] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.71, 6.31, 6.76 [00:57:32] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1015877128 secs [01:02:09] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [01:12:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.61, 6.70, 6.41 [01:16:35] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.88, 6.28, 6.35 [01:32:09] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [01:50:55] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.38, 7.33, 6.66 [01:52:52] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.56, 6.29, 6.35 [02:32:10] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [02:45:51] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.28, 6.67, 5.91 [02:47:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.05, 6.89, 6.06 [02:49:49] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.70, 6.22, 5.93 [03:02:10] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [03:08:52] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 2.33, 3.86, 2.25 [03:10:52] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 0.68, 2.70, 2.02 [04:35:28] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query zulaclub.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:41:20] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.18, 3.70, 3.03 [04:43:15] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.72, 3.42, 3.01 [04:45:11] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.75, 2.94, 2.89 [05:05:25] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [06:12:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.69, 3.36, 2.67 [06:14:36] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 6.91, 5.16, 3.49 [06:16:35] PROBLEM - swiftac111 Current Load on swiftac111 is CRITICAL: CRITICAL - load average: 15.14, 8.67, 4.96 [06:17:21] PROBLEM - swiftac111 SSH on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:17:34] PROBLEM - swiftac111 PowerDNS Recursor on swiftac111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [06:21:11] PROBLEM - swiftac111 Puppet on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:24:10] PROBLEM - swiftac111 Backups Swift Account Container on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:24:18] PROBLEM - swiftproxy111 HTTP on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [06:24:19] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [06:25:11] PROBLEM - swiftproxy131 HTTPS on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 401 Unauthorized [06:25:21] PROBLEM - swiftproxy131 HTTP on swiftproxy131 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host: HTTP/1.1 401 Unauthorized [06:25:47] PROBLEM - swiftac111 ferm_active on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:26:14] PROBLEM - swiftac111 Swift Account Service on swiftac111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:26:19] PROBLEM - swiftac111 NTP time on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:38] PROBLEM - swiftac111 conntrack_table_size on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:58] PROBLEM - cp23 Current Load on cp23 is CRITICAL: LOAD CRITICAL - total load average: 4.31, 5.16, 2.53 [06:28:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.83, 3.40 [06:29:58] PROBLEM - cp23 Current Load on cp23 is WARNING: LOAD WARNING - total load average: 1.23, 3.68, 2.30 [06:30:11] PROBLEM - swiftac111 Disk Space on swiftac111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:31:58] RECOVERY - cp23 Current Load on cp23 is OK: LOAD OK - total load average: 0.22, 2.49, 2.03 [06:32:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.60, 3.76, 3.47 [06:38:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.05, 3.36, 3.40 [06:42:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.71, 3.76, 3.57 [06:52:02] RECOVERY - swiftac111 ferm_active on swiftac111 is OK: OK ferm input default policy is set [06:52:12] RECOVERY - swiftac111 NTP time on swiftac111 is OK: NTP OK: Offset 0.0003197491169 secs [06:52:22] RECOVERY - swiftac111 Puppet on swiftac111 is OK: OK: Puppet is currently enabled, last run 58 minutes ago with 0 failures [06:52:29] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:52:41] RECOVERY - swiftproxy131 HTTP on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.042 second response time [06:52:45] RECOVERY - swiftac111 Disk Space on swiftac111 is OK: DISK OK - free space: / 110858 MB (89% inode=95%); [06:52:50] RECOVERY - swiftproxy131 HTTPS on swiftproxy131 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.031 second response time [06:53:11] RECOVERY - swiftac111 SSH on swiftac111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [06:53:19] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.037 second response time [06:53:22] RECOVERY - swiftac111 PowerDNS Recursor on swiftac111 is OK: DNS OK: 0.114 seconds response time. miraheze.org returns 109.228.51.216,217.174.247.33,2a00:da00:1800:326::1,2a00:da00:1800:328::1 [06:53:24] RECOVERY - swiftac111 conntrack_table_size on swiftac111 is OK: OK: nf_conntrack is 10 % full [06:53:27] RECOVERY - swiftac111 Backups Swift Account Container on swiftac111 is OK: FILE_AGE OK: /var/log/swift-account-container-backup.log is 3206 seconds old and 0 bytes [06:53:29] RECOVERY - swiftac111 Swift Account Service on swiftac111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:202 port 6002 [06:53:30] RECOVERY - swiftproxy111 HTTP on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.050 second response time [06:56:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.42, 3.95, 3.67 [07:00:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.75, 3.90, 3.72 [07:06:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.71, 2.63, 3.25 [07:21:45] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [07:23:26] PROBLEM - cloud14 Puppet on cloud14 is UNKNOWN: NRPE: Unable to read output [07:30:31] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 2.06, 1.75, 7.40 [07:32:31] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 2.56, 1.98, 6.79 [07:51:26] RECOVERY - cloud14 Puppet on cloud14 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [08:22:16] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:51:33] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [09:22:08] PROBLEM - cloud11 Puppet on cloud11 is UNKNOWN: NRPE: Unable to read output [09:50:08] RECOVERY - cloud11 Puppet on cloud11 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [10:02:10] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [10:32:09] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [10:53:43] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:23:00] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [12:08:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.17, 2.97, 2.35 [12:10:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 4.00, 3.32, 2.56 [12:14:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.72, 3.82, 2.93 [12:22:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.51, 3.99, 3.46 [12:24:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.21, 4.04, 3.54 [12:27:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ssl/compare/fceca6247257...b4fb5bb1efc1 [12:27:48] [02miraheze/ssl] 07MirahezeSSLBot 03b4fb5bb - Bot: Add SSL cert for glitchcity.wiki [12:28:13] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b4fb5bb1efc1...8d6399720019 [12:28:16] [02miraheze/ssl] 07paladox 038d63997 - Update certs.yaml [12:28:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.65, 3.95, 3.62 [12:28:50] miraheze/ssl - MirahezeSSLBot the build has errored. [12:30:26] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.96, 4.45, 3.86 [12:32:00] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/35a014ec7e7b [12:32:01] [02miraheze/ssl] 07paladox 0335a014e - Remove wiki.randomstuffimade.page [12:32:02] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:32:03] [02ssl] 07paladox opened pull request 03#716: Remove wiki.randomstuffimade.page - 13https://github.com/miraheze/ssl/pull/716 [12:32:22] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/35a014ec7e7b...6556c1272140 [12:32:24] [02miraheze/ssl] 07paladox 036556c12 - Delete certificates/wiki.randomstuffimade.page.crt [12:32:25] [02ssl] 07paladox synchronize pull request 03#716: Remove wiki.randomstuffimade.page - 13https://github.com/miraheze/ssl/pull/716 [12:32:29] [02ssl] 07paladox closed pull request 03#716: Remove wiki.randomstuffimade.page - 13https://github.com/miraheze/ssl/pull/716 [12:32:31] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/8d6399720019...4b4c43080028 [12:32:32] [02miraheze/ssl] 07paladox 034b4c430 - Remove wiki.randomstuffimade.page (#716) [12:32:33] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [12:32:35] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:34:22] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/e4cdc2aa5363 [12:34:25] [02miraheze/ssl] 07paladox 03e4cdc2a - Remove wiki.chouverse.cludisciples.cyou [12:34:26] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:34:28] [02ssl] 07paladox opened pull request 03#717: Remove wiki.chouverse.cludisciples.cyou - 13https://github.com/miraheze/ssl/pull/717 [12:34:43] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/e4cdc2aa5363...49c2f3f7b830 [12:34:45] [02miraheze/ssl] 07paladox 0349c2f3f - Delete certificates/wiki.chouverse.cludisciples.cyou.crt [12:34:47] [02ssl] 07paladox synchronize pull request 03#717: Remove wiki.chouverse.cludisciples.cyou - 13https://github.com/miraheze/ssl/pull/717 [12:34:50] [02ssl] 07paladox closed pull request 03#717: Remove wiki.chouverse.cludisciples.cyou - 13https://github.com/miraheze/ssl/pull/717 [12:34:52] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/4b4c43080028...5c224d728a31 [12:34:53] [02miraheze/ssl] 07paladox 035c224d7 - Remove wiki.chouverse.cludisciples.cyou (#717) [12:34:55] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [12:34:57] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:37:03] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/04f8d9be3ffb [12:37:04] [02miraheze/ssl] 07paladox 0304f8d9b - Remove arcanewaterswiki.org [12:37:07] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:37:08] [02ssl] 07paladox opened pull request 03#718: Remove arcanewaterswiki.org - 13https://github.com/miraheze/ssl/pull/718 [12:37:25] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/04f8d9be3ffb...b1cc8a322a71 [12:37:26] [02miraheze/ssl] 07paladox 03b1cc8a3 - Delete certificates/arcanewaterswiki.org.crt [12:37:29] [02ssl] 07paladox synchronize pull request 03#718: Remove arcanewaterswiki.org - 13https://github.com/miraheze/ssl/pull/718 [12:37:45] [02ssl] 07paladox closed pull request 03#718: Remove arcanewaterswiki.org - 13https://github.com/miraheze/ssl/pull/718 [12:37:48] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/5c224d728a31...8072031b7381 [12:37:51] [02miraheze/ssl] 07paladox 038072031 - Remove arcanewaterswiki.org (#718) [12:37:52] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [12:37:55] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:40:36] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/commit/dacf7f2f952b [12:40:37] [02miraheze/ssl] 07paladox 03dacf7f2 - Remove rosettacode.org [12:40:38] [02ssl] 07paladox created branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:40:41] [02ssl] 07paladox opened pull request 03#719: Remove rosettacode.org - 13https://github.com/miraheze/ssl/pull/719 [12:40:56] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dacf7f2f952b...5a8ec445b1b3 [12:40:57] [02miraheze/ssl] 07paladox 035a8ec44 - Update redirects.yaml [12:40:58] [02ssl] 07paladox synchronize pull request 03#719: Remove rosettacode.org - 13https://github.com/miraheze/ssl/pull/719 [12:41:05] [02miraheze/ssl] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/compare/5a8ec445b1b3...a3c013766500 [12:41:07] [02miraheze/ssl] 07paladox 03a3c0137 - Delete certificates/rosettacode.org.crt [12:41:10] [02ssl] 07paladox synchronize pull request 03#719: Remove rosettacode.org - 13https://github.com/miraheze/ssl/pull/719 [12:41:12] [02ssl] 07paladox closed pull request 03#719: Remove rosettacode.org - 13https://github.com/miraheze/ssl/pull/719 [12:41:14] [02miraheze/ssl] 07paladox pushed 031 commit to 03master [+0/-1/±2] 13https://github.com/miraheze/ssl/compare/8072031b7381...41ba088f78a4 [12:41:15] [02miraheze/ssl] 07paladox 0341ba088 - Remove rosettacode.org (#719) [12:41:18] [02miraheze/ssl] 07paladox deleted branch 03paladox-patch-1 [12:41:21] [02ssl] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/ssl [12:48:26] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.50, 3.77, 4.00 [12:49:07] RECOVERY - wiki.randomstuffimade.page - LetsEncrypt on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Sat 11 Nov 2023 23:59:59 GMT +0000. [12:53:21] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/336b21d7039c...1373aa0138b8 [12:53:22] [02miraheze/dns] 07paladox 031373aa0 - Add acme challenge for betaheze [12:54:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/41ba088f78a4...38215c7d06ee [12:54:19] [02miraheze/ssl] 07MirahezeSSLBot 0338215c7 - Bot: Update SSL cert for betaheze.org [12:57:56] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.32, 3.96, 3.98 [12:59:54] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.67, 3.87, 3.95 [13:01:51] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.65, 4.06, 4.00 [13:03:22] RECOVERY - betaheze.org - LetsEncrypt on sslhost is OK: OK - Certificate 'betaheze.org' will expire on Sat 02 Dec 2023 11:54:11 GMT +0000. [13:05:47] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.11, 3.31, 3.74 [13:09:42] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.46, 2.71, 3.38 [13:30:36] RECOVERY - prometheus131 APT on prometheus131 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:32:28] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fb04fa06b2fc...381ac537170c [13:32:29] [02miraheze/puppet] 07paladox 03381ac53 - grafana: update apt key [13:33:17] RECOVERY - puppet141 APT on puppet141 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:36:38] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:36:42] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:37:08] PROBLEM - swiftobject111 Puppet on swiftobject111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:37:42] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:38:01] PROBLEM - mw134 Puppet on mw134 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:40:33] PROBLEM - os141 Puppet on os141 is UNKNOWN: NRPE: Unable to read output [13:40:48] PROBLEM - puppet141 puppetdb on puppet141 is CRITICAL: connect to address 2a10:6740::6:507 and port 8081: Connection refused [13:40:57] PROBLEM - puppet141 puppetserver on puppet141 is CRITICAL: connect to address 2a10:6740::6:507 and port 8140: Connection refused [13:41:21] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:41:59] PROBLEM - puppet141 Puppet on puppet141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:01] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:29] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [13:42:34] PROBLEM - mwtask141 Puppet on mwtask141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:44] RECOVERY - puppet141 puppetdb on puppet141 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:507 port 8081 [13:42:56] RECOVERY - puppet141 puppetserver on puppet141 is OK: TCP OK - 0.000 second response time on 2a10:6740::6:507 port 8140 [13:42:56] PROBLEM - phab121 Puppet on phab121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:42:57] PROBLEM - bast121 Puppet on bast121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:17] PROBLEM - db142 Puppet on db142 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:43:18] PROBLEM - mw131 Puppet on mw131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:44:23] RECOVERY - mon141 APT on mon141 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [13:44:27] PROBLEM - mail121 Puppet on mail121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:44:34] PROBLEM - swiftproxy111 Puppet on swiftproxy111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:46:03] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:46:21] PROBLEM - mem141 Puppet on mem141 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:46:21] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [14:05:08] RECOVERY - swiftobject111 Puppet on swiftobject111 is OK: OK: Puppet is currently enabled, last run 3 seconds ago with 0 failures [14:05:49] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:06:01] RECOVERY - mw134 Puppet on mw134 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [14:06:38] RECOVERY - mw132 Puppet on mw132 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:06:42] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:08:33] RECOVERY - os141 Puppet on os141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:08:57] RECOVERY - phab121 Puppet on phab121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:09:35] RECOVERY - puppet141 Puppet on puppet141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:10:01] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [14:10:16] RECOVERY - mwtask141 Puppet on mwtask141 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [14:10:52] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:10:57] RECOVERY - bast121 Puppet on bast121 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [14:11:04] RECOVERY - db142 Puppet on db142 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:11:18] RECOVERY - mw131 Puppet on mw131 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [14:12:03] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [14:12:34] RECOVERY - swiftproxy111 Puppet on swiftproxy111 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [14:14:21] RECOVERY - mem141 Puppet on mem141 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:14:27] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:19:42] PROBLEM - cp32 Current Load on cp32 is CRITICAL: LOAD CRITICAL - total load average: 7.37, 4.17, 1.97 [14:21:42] RECOVERY - cp32 Current Load on cp32 is OK: LOAD OK - total load average: 1.80, 3.15, 1.86 [14:24:23] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [14:24:26] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/abdf3ca0d0a9...53b3a0855902 [14:24:27] [02miraheze/puppet] 07paladox 0353b3a08 - Update default.vcl [14:29:04] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [14:29:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/53b3a0855902...160150b5483f [14:29:06] [02miraheze/puppet] 07paladox 03160150b - Update default.vcl [14:30:26] PROBLEM - test131 Puppet on test131 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:30:53] PROBLEM - reports121 Puppet on reports121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:30:56] RECOVERY - mon141 NTP time on mon141 is OK: NTP OK: Offset 0.001024395227 secs [14:38:18] RECOVERY - puppet141 Puppet on puppet141 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [14:58:03] RECOVERY - test131 Puppet on test131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [14:58:38] RECOVERY - reports121 Puppet on reports121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:15:31] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/381ac537170c...1849edd288ed [15:15:34] [02miraheze/puppet] 07paladox 031849edd - graylog: upgrade mongodb to 5.0.20 [15:15:43] RECOVERY - reports121 APT on reports121 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:16:20] PROBLEM - matomo121 APT on matomo121 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [15:16:34] RECOVERY - ldap141 APT on ldap141 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:18:19] PROBLEM - matomo121 APT on matomo121 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (22 critical updates). [15:18:59] RECOVERY - graylog131 APT on graylog131 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:20:19] RECOVERY - matomo121 APT on matomo121 is OK: APT OK: 2 packages available for upgrade (0 critical updates). [15:22:03] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.07596710324 secs [15:23:13] RECOVERY - ns2 APT on ns2 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:24:25] PROBLEM - graylog131 HTTPS on graylog131 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 295 bytes in 0.014 second response time [15:26:24] RECOVERY - graylog131 HTTPS on graylog131 is OK: HTTP OK: HTTP/1.1 200 OK - 2086 bytes in 0.063 second response time [15:27:03] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset -0.308487922 secs [15:27:13] RECOVERY - ns1 APT on ns1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [15:28:01] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: CRITICAL - load average: 7.42, 3.63, 1.45 [15:30:01] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.48, 3.54, 1.70 [15:32:00] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 2.70, 3.25, 1.81 [15:41:27] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/160150b5483f...1bb0e8b129eb [15:41:29] [02miraheze/puppet] 07paladox 031bb0e8b - Update default.vcl [15:41:31] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [15:44:03] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [15:44:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1bb0e8b129eb...1ccbd26cc6ee [15:44:08] [02miraheze/puppet] 07paladox 031ccbd26 - Update default.vcl [15:56:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1ccbd26cc6ee...c930c1a9d88b [15:56:47] [02miraheze/puppet] 07paladox 03c930c1a - Update init.pp [15:56:48] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [15:58:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c930c1a9d88b...4bc9be70553e [15:58:04] [02miraheze/puppet] 07paladox 034bc9be7 - Update default.vcl [15:58:05] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [16:00:34] PROBLEM - cp22 Puppet on cp22 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:01:25] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:01:47] PROBLEM - cp23 Puppet on cp23 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:02:17] PROBLEM - cp33 Puppet on cp33 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [16:02:55] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/4bc9be70553e...57a6cd4c12d9 [16:02:57] [02miraheze/puppet] 07paladox 0357a6cd4 - Update init.pp [16:02:58] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [16:03:45] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/57a6cd4c12d9...cfbe04a4521b [16:03:48] [02miraheze/puppet] 07paladox 03cfbe04a - Update init.pp [16:03:51] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [16:04:34] RECOVERY - cp22 Puppet on cp22 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [16:05:26] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [16:05:46] RECOVERY - cp23 Puppet on cp23 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:06:16] RECOVERY - cp33 Puppet on cp33 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [16:10:22] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cfbe04a4521b...817321ee175d [16:10:23] [02miraheze/puppet] 07paladox 03817321e - Update init.pp [16:10:24] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [16:13:32] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/817321ee175d...95b6a9b0035e [16:13:34] [02miraheze/puppet] 07paladox 0395b6a9b - Update init.pp [16:13:37] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [16:17:17] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.35, 6.79, 5.64 [16:19:13] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.34, 6.11, 5.53 [18:04:12] [02puppet] 07paladox edited pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [18:04:23] [02puppet] 07paladox edited pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [18:10:20] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.52, 3.11, 2.64 [18:14:15] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.12, 3.29, 2.83 [18:17:48] [02puppet] 07paladox synchronize pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [18:17:50] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/95b6a9b0035e...37acb56a7415 [18:17:53] [02miraheze/puppet] 07paladox 0337acb56 - Update default.vcl [18:18:08] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.22, 3.75, 3.12 [18:22:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.90, 3.93, 3.34 [18:24:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 3.86, 4.02, 3.45 [18:25:59] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.89, 3.98, 3.51 [18:34:03] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset -0.09949210286 secs [18:37:45] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.24, 3.87, 3.65 [19:03:26] [02puppet] 07paladox closed pull request 03#3398: varnish: Improve static.miraheze.org - 13https://github.com/miraheze/puppet/pull/3398 [19:03:29] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/1849edd288ed...0ddd03cbf5cd [19:03:30] [02miraheze/puppet] 07paladox 030ddd03c - varnish: Improve static.miraheze.org (#3398) [19:03:33] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [19:03:34] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [19:05:04] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [19:05:21] [02DataDump] 07paladox synchronize pull request 03#61: Add new status field to table - 13https://github.com/miraheze/DataDump/pull/61 [19:05:22] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/69f68d79102d...56d61d42c1f9 [19:05:23] [02miraheze/DataDump] 07paladox 0356d61d4 - Update DataDumpGenerateJob.php [19:07:10] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.24, 3.09, 3.90 [19:08:14] miraheze/DataDump - paladox the build has errored. [19:09:01] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [19:13:04] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.83, 2.48, 3.38 [19:16:38] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=shipsthatpasswiki --no-updates --username-prefix=wikia:shipsthatpass /home/root/ImportDump/shipsthatpasswiki-20230824233048.xml (START) [19:16:39] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=shipsthatpasswiki --no-updates --username-prefix=wikia:shipsthatpass /home/root/ImportDump/shipsthatpasswiki-20230824233048.xml (END - exit=256) [19:16:40] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shipsthatpasswiki (START) [19:16:41] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shipsthatpasswiki (END - exit=0) [19:16:43] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=shipsthatpasswiki --active --update (END - exit=0) [19:16:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:47] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=shipsthatpasswiki --no-updates --username-prefix=wikia:shipsthatpass /home/paladox/ImportDump/shipsthatpasswiki-20230824233048.xml (START) [19:16:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:22] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tbotvwiki --no-updates --username-prefix=wikia:voidexp /home/paladox/ImportDump/tbotvwiki-20230825205830.xml (START) [19:17:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:27] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=shipsthatpasswiki --no-updates --username-prefix=wikia:shipsthatpass /home/paladox/ImportDump/shipsthatpasswiki-20230824233048.xml (END - exit=0) [19:17:28] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shipsthatpasswiki (START) [19:17:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:34] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=minewatchwiki --no-updates --username-prefix=wikia:minewatch /home/paladox/ImportDump/minewatchwiki-20230825222505.xml (START) [19:17:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:17:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:18:01] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=shipsthatpasswiki (END - exit=0) [19:18:02] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=shipsthatpasswiki --active --update (END - exit=0) [19:18:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:18:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:18:26] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=alphaverwiki --no-updates --username-prefix=wikia:alphaver /home/paladox/ImportDump/alphaverwiki-20230826181640.xml (START) [19:18:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:19:31] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=minewatchwiki --no-updates --username-prefix=wikia:minewatch /home/paladox/ImportDump/minewatchwiki-20230825222505.xml (END - exit=0) [19:19:32] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=minewatchwiki (START) [19:19:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:19:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:27] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=minewatchwiki (END - exit=0) [19:20:28] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=minewatchwiki --active --update (END - exit=0) [19:20:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:58] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deadboltwiki --no-updates --username-prefix=wikia:deadbolt /home/paladox/ImportDump/deadboltwiki-20230828181123.xml (START) [19:21:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:21:28] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=dimension20wiki --no-updates --username-prefix=wikia:dimension20 /home/paladox/ImportDump/dimension20wiki-20230829043033.xml (START) [19:21:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:22:19] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=tbotvwiki --no-updates --username-prefix=wikia:voidexp /home/paladox/ImportDump/tbotvwiki-20230825205830.xml (END - exit=0) [19:22:20] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tbotvwiki (START) [19:22:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:22:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:24:24] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deadboltwiki --no-updates --username-prefix=wikia:deadbolt /home/paladox/ImportDump/deadboltwiki-20230828181123.xml (END - exit=0) [19:24:25] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deadboltwiki (START) [19:24:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:24:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:31] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deadboltwiki (END - exit=0) [19:25:32] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=deadboltwiki --active --update (END - exit=0) [19:25:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:55] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tbotvwiki (END - exit=0) [19:25:56] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=tbotvwiki --active --update (END - exit=0) [19:26:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:26:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:03] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=nothanscydiawiki --no-updates --username-prefix=w /home/paladox/ImportDump/nothanscydiawiki-20230830095642.xml (START) [19:28:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:17] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deathstrandingwiki --no-updates --username-prefix=wikia:deathstranding /home/paladox/ImportDump/deathstrandingwiki-20230830161131.xml (START) [19:28:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:28:27] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deadboltwiki --no-updates --username-prefix=wikia:deadbolt /home/paladox/ImportDump/deadboltwiki-20230901205706.xml (START) [19:28:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:17] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=alphaverwiki --no-updates --username-prefix=wikia:alphaver /home/paladox/ImportDump/alphaverwiki-20230826181640.xml (END - exit=0) [19:29:18] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=alphaverwiki (START) [19:29:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:29:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:26] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deadboltwiki --no-updates --username-prefix=wikia:deadbolt /home/paladox/ImportDump/deadboltwiki-20230901205706.xml (END - exit=0) [19:30:27] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deadboltwiki (START) [19:30:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:30:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:31:07] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deadboltwiki (END - exit=0) [19:31:10] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=deadboltwiki --active --update (END - exit=0) [19:31:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:31:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:32:22] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=alphaverwiki (END - exit=0) [19:32:23] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=alphaverwiki --active --update (END - exit=0) [19:32:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:32:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:33:04] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.85, 3.69, 3.25 [19:34:33] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=nothanscydiawiki --no-updates --username-prefix=w /home/paladox/ImportDump/nothanscydiawiki-20230830095642.xml (END - exit=0) [19:34:34] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=nothanscydiawiki (START) [19:34:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:34:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:35:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.69, 3.73, 3.32 [19:37:04] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.45, 3.95, 3.45 [19:42:51] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=deathstrandingwiki --no-updates --username-prefix=wikia:deathstranding /home/paladox/ImportDump/deathstrandingwiki-20230830161131.xml (END - exit=0) [19:42:52] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deathstrandingwiki (START) [19:42:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:11] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=nothanscydiawiki (END - exit=0) [19:45:12] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=nothanscydiawiki --active --update (END - exit=0) [19:45:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:24] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=deathstrandingwiki (END - exit=0) [19:53:25] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=deathstrandingwiki --active --update (END - exit=0) [19:53:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:59:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.43, 2.84, 3.92 [20:05:04] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.40, 3.78 [20:07:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.01, 3.27, 3.69 [20:07:53] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=dimension20wiki --no-updates --username-prefix=wikia:dimension20 /home/paladox/ImportDump/dimension20wiki-20230829043033.xml (END - exit=0) [20:07:54] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=dimension20wiki (START) [20:07:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:08:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:17:04] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.06, 3.78, 3.75 [20:19:04] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.51, 3.69, 3.72 [20:30:56] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=dimension20wiki (END - exit=0) [20:30:57] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=dimension20wiki --active --update (END - exit=0) [20:31:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:31:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:37:04] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.70, 2.98, 3.33 [21:11:31] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.92, 3.45, 3.21 [21:13:29] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.25, 3.34, 3.19 [21:38:30] PROBLEM - wiki.helsinginpoytaroolipelaajat.fi - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.helsinginpoytaroolipelaajat.fi. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:26:00] PROBLEM - graylog131 Current Load on graylog131 is WARNING: WARNING - load average: 3.57, 2.90, 2.36 [22:28:01] RECOVERY - graylog131 Current Load on graylog131 is OK: OK - load average: 2.82, 2.94, 2.45 [22:36:30] PROBLEM - wiki.helsinginpoytaroolipelaajat.fi - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.helsinginpoytaroolipelaajat.fi All nameservers failed to answer the query. [22:37:48] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query en.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:49:30] PROBLEM - cp33 Current Load on cp33 is WARNING: LOAD WARNING - total load average: 1.77, 3.74, 2.48 [22:51:30] RECOVERY - cp33 Current Load on cp33 is OK: LOAD OK - total load average: 0.70, 2.72, 2.26 [22:51:46] PROBLEM - betaheze.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - betaheze.org reverse DNS resolves to 2607-f1c0-100f-f000-0000-0000-0000-024d.elastic-ssl.ui-r.com [22:53:32] PROBLEM - betaheze.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Cannot make SSL connection. [23:04:04] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 12.35, 7.11, 3.25 [23:06:28] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - en.religiononfire.mar.in.ua All nameservers failed to answer the query. [23:09:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.82, 6.90, 6.12 [23:10:04] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.10, 7.42, 5.07 [23:11:14] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.99, 6.33, 5.99 [23:12:04] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 0.48, 5.04, 4.48 [23:35:08] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [23:36:30] RECOVERY - wiki.helsinginpoytaroolipelaajat.fi - reverse DNS on sslhost is OK: SSL OK - wiki.helsinginpoytaroolipelaajat.fi reverse DNS resolves to cp22.miraheze.org - CNAME OK [23:46:01] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - uk.religiononfire.mar.in.ua All nameservers failed to answer the query.