[00:04:29] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.45, 3.18, 3.17 [00:09:23] RECOVERY - cp33 Disk Space on cp33 is OK: DISK OK - free space: / 13030 MB (17% inode=98%); [00:12:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.66, 3.47, 3.30 [00:16:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.18, 3.36, 3.29 [00:16:22] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.37, 3.56, 2.58 [00:18:22] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.16, 3.06, 2.52 [00:22:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.81, 3.92, 3.52 [00:24:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.73, 3.51, 3.42 [00:26:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.68, 2.98, 3.24 [00:27:16] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.93, 3.42, 2.85 [00:29:14] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.49, 3.76, 3.04 [00:33:10] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.65, 3.76, 3.21 [00:42:59] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.06, 3.39, 3.28 [00:52:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.62, 3.37, 3.93 [00:54:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 8.25, 5.06, 4.47 [01:02:37] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.04, 3.25, 3.01 [01:06:33] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.91, 3.70, 3.24 [01:08:31] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.91, 3.26, 3.13 [01:40:04] PROBLEM - cp32 PowerDNS Recursor on cp32 is CRITICAL: CRITICAL - Plugin timed out while executing system call [01:40:37] PROBLEM - cp32 Current Load on cp32 is CRITICAL: CRITICAL - load average: 40.95, 19.91, 8.00 [01:40:43] PROBLEM - cp32 SSH on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:41:18] PROBLEM - cp32 HTTPS on cp32 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:41:27] PROBLEM - cp32 ferm_active on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:42:22] PROBLEM - cp32 conntrack_table_size on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:43:01] PROBLEM - cp32 Varnish Backends on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:43:05] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:43:30] PROBLEM - cp32 HTTP 4xx/5xx ERROR Rate on cp32 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:44:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.26, 2.71, 3.95 [01:46:52] RECOVERY - cp32 SSH on cp32 is OK: SSH OK - OpenSSH_8.4p1 Debian-5+deb11u1 (protocol 2.0) [01:46:53] RECOVERY - cp32 ferm_active on cp32 is OK: OK ferm input default policy is set [01:47:04] RECOVERY - cp32 PowerDNS Recursor on cp32 is OK: DNS OK: 0.092 seconds response time. miraheze.org returns 2607:f1c0:1800:26f::1,74.208.203.152 [01:47:21] RECOVERY - cp32 HTTPS on cp32 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3732 bytes in 0.429 second response time [01:47:25] RECOVERY - cp32 conntrack_table_size on cp32 is OK: OK: nf_conntrack is 0 % full [01:48:01] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [01:48:05] RECOVERY - cp32 Varnish Backends on cp32 is OK: All 15 backends are healthy [01:48:38] RECOVERY - cp32 HTTP 4xx/5xx ERROR Rate on cp32 is OK: OK - NGINX Error Rate is 3% [01:50:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.03, 3.32, 3.78 [01:54:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.61, 3.95, 3.94 [01:56:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.05, 3.87, 3.90 [01:58:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.22, 3.70, 3.84 [02:02:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.67, 3.79, 3.81 [02:04:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.98, 3.10, 3.55 [02:06:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.56, 3.45, 3.61 [02:14:23] PROBLEM - cp32 Current Load on cp32 is WARNING: WARNING - load average: 0.01, 0.22, 3.86 [02:16:22] RECOVERY - cp32 Current Load on cp32 is OK: OK - load average: 0.07, 0.16, 3.39 [02:18:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.34, 3.70, 3.94 [02:20:40] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.58, 4.85, 4.36 [02:23:34] [02puppet] 07The-Voidwalker closed pull request 03#3347: miraheze-backup: reusable pca_connection - 13https://github.com/miraheze/puppet/pull/3347 [02:31:07] [02miraheze/puppet] 07The-Voidwalker pushed 031 commit to 03The-Voidwalker-patch-2 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/62c1e4226e74 [02:31:08] [02miraheze/puppet] 07The-Voidwalker 0362c1e42 - miraheze-backup discard IO errors on put [02:31:09] [02puppet] 07The-Voidwalker created branch 03The-Voidwalker-patch-2 - 13https://github.com/miraheze/puppet [02:31:11] [02puppet] 07The-Voidwalker opened pull request 03#3348: miraheze-backup discard IO errors on put - 13https://github.com/miraheze/puppet/pull/3348 [02:44:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.71, 2.84, 3.83 [02:46:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.77, 3.98, 4.12 [02:59:53] PROBLEM - rct.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query rct.wiki. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [03:04:20] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [03:29:17] RECOVERY - rct.wiki - reverse DNS on sslhost is OK: SSL OK - rct.wiki reverse DNS resolves to cp23.miraheze.org - NS RECORDS OK [03:31:55] [02puppet] 07paladox synchronize pull request 03#3346: miraheze-backup: use subprocess rather than os - 13https://github.com/miraheze/puppet/pull/3346 [03:31:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/eb85261ee59a...7a330d491dff [03:31:58] [02miraheze/puppet] 07paladox 037a330d4 - Update miraheze-backup.py.erb [03:32:05] [02puppet] 07paladox synchronize pull request 03#3346: miraheze-backup: use subprocess rather than os - 13https://github.com/miraheze/puppet/pull/3346 [03:32:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7a330d491dff...0a4a67b93a44 [03:32:10] [02miraheze/puppet] 07paladox 030a4a67b - Update backup.pp [03:32:13] [02puppet] 07paladox synchronize pull request 03#3346: miraheze-backup: use subprocess rather than os - 13https://github.com/miraheze/puppet/pull/3346 [03:32:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/0a4a67b93a44...21e32c9bb433 [03:32:17] [02miraheze/puppet] 07paladox 0321e32c9 - Delete miraheze-mariadb-dump.sh.erb [03:34:06] [02puppet] 07paladox synchronize pull request 03#3346: miraheze-backup: use subprocess rather than os - 13https://github.com/miraheze/puppet/pull/3346 [03:34:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/21e32c9bb433...9ae82669a5b9 [03:34:08] [02miraheze/puppet] 07paladox 039ae8266 - Update miraheze-backup.py.erb [03:34:13] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [04:12:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.50, 3.41, 3.97 [04:14:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.20, 3.74, 4.01 [04:16:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.46, 3.52, 3.90 [04:18:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.30, 3.58, 3.86 [04:20:26] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 13423 MB (10% inode=99%); [04:27:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [04:32:56] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.73, 7.35, 5.57 [04:34:51] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 4.63, 6.71, 5.57 [04:48:26] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7722 MB (5% inode=99%); [04:50:26] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 17172 MB (12% inode=99%); [04:53:15] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 8.16, 7.13, 6.09 [04:55:11] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.75, 5.45, 5.60 [04:57:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [04:59:10] PROBLEM - en.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:00:26] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 13913 MB (10% inode=99%); [05:05:36] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.religiononfire.mar.in.ua. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:05:52] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 7.25, 8.37, 7.04 [05:06:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.15, 3.21, 3.96 [05:07:48] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.70, 6.38, 6.47 [05:10:26] RECOVERY - db112 Disk Space on db112 is OK: DISK OK - free space: / 19160 MB (14% inode=99%); [05:16:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.39, 3.21, 3.54 [05:18:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.64, 2.93, 3.40 [05:22:08] [02puppet] 07paladox closed pull request 03#3348: miraheze-backup discard IO errors on put - 13https://github.com/miraheze/puppet/pull/3348 [05:22:09] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cae863714d3c...f7361da9d4ac [05:22:12] [02miraheze/puppet] 07The-Voidwalker 03f7361da - miraheze-backup discard IO errors on put (#3348) [05:22:13] [02miraheze/puppet] 07paladox deleted branch 03The-Voidwalker-patch-2 [05:22:15] [02puppet] 07paladox deleted branch 03The-Voidwalker-patch-2 - 13https://github.com/miraheze/puppet [05:22:22] [02puppet] 07paladox closed pull request 03#3346: miraheze-backup: use subprocess rather than os - 13https://github.com/miraheze/puppet/pull/3346 [05:22:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f7361da9d4ac...41ec6d052e6d [05:22:26] [02miraheze/puppet] 07paladox 0341ec6d0 - miraheze-backup: use subprocess rather than os (#3346) [05:22:27] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [05:22:29] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [05:22:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.41, 4.67, 4.02 [05:25:07] RECOVERY - db131 Puppet on db131 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:29:02] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [05:34:41] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp22.miraheze.org - CNAME OK [05:42:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.34, 3.94 [05:48:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.60, 3.59, 3.81 [05:50:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.11, 3.89, 3.94 [05:52:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.33, 4.49, 4.12 [05:56:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.16, 3.57, 3.86 [06:00:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.45, 2.49, 3.35 [06:04:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.39, 3.17, 3.47 [06:06:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.11, 3.57, 3.58 [06:08:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.25, 3.46, 3.54 [06:10:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.11, 3.07, 3.40 [06:12:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.38, 3.50, 2.64 [06:14:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.56, 2.77 [06:16:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.91, 3.32, 2.78 [06:33:57] PROBLEM - cp23 NTP time on cp23 is WARNING: NTP WARNING: Offset 0.1174446642 secs [06:36:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.33, 3.06 [06:40:24] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.39, 3.84, 3.32 [06:42:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.88, 3.86, 3.40 [06:44:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.01, 3.94, 3.48 [06:45:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.75, 3.63, 3.28 [06:46:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.68, 3.79, 3.48 [06:46:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.95, 2.95, 3.07 [06:48:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.48, 4.05, 3.61 [06:54:48] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.24, 3.84, 3.32 [06:56:45] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.06, 3.49, 3.25 [06:58:42] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 3.13, 3.14, 3.14 [07:01:57] RECOVERY - cp23 NTP time on cp23 is OK: NTP OK: Offset 0.08200272918 secs [07:08:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 2.88, 3.81, 3.95 [07:16:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 1.97, 2.44, 3.28 [07:54:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.22, 3.87, 2.99 [07:56:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.93, 3.42, 2.93 [07:57:57] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.46, 3.07, 2.86 [08:08:22] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.80, 3.24, 2.28 [08:10:22] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.97, 3.14, 2.36 [10:35:35] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:54:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.13, 3.48, 2.85 [10:56:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.62, 3.44, 2.91 [10:58:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.57, 3.16, 2.87 [11:04:41] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [11:12:45] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±16] 13https://github.com/miraheze/CreateWiki/compare/1b01b8e08c5e...251ae85e0144 [11:12:47] [02miraheze/CreateWiki] 07translatewiki 03251ae85 - Localisation updates from https://translatewiki.net. [11:12:49] [02miraheze/ImportDump] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ImportDump/compare/522c3394261f...bb3a96c821ab [11:12:51] [02miraheze/ImportDump] 07translatewiki 03bb3a96c - Localisation updates from https://translatewiki.net. [11:12:54] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/ManageWiki/compare/045c9eadb436...a5e33fe580be [11:12:56] [02miraheze/ManageWiki] 07translatewiki 03a5e33fe - Localisation updates from https://translatewiki.net. [11:12:58] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±9] 13https://github.com/miraheze/landing/compare/b444433ac3d5...a1446c765124 [11:13:01] [02miraheze/landing] 07translatewiki 03a1446c7 - Localisation updates from https://translatewiki.net. [11:13:04] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±32] 13https://github.com/miraheze/MirahezeMagic/compare/513db7676d0e...a99f0f983e5c [11:13:05] [02miraheze/MirahezeMagic] 07translatewiki 03a99f0f9 - Localisation updates from https://translatewiki.net. [11:13:53] miraheze/landing - translatewiki the build passed. [11:20:20] miraheze/ManageWiki - translatewiki the build passed. [11:24:02] miraheze/ImportDump - translatewiki the build has errored. [11:25:36] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.72, 3.24, 2.21 [11:26:30] miraheze/CreateWiki - translatewiki the build passed. [11:27:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.98, 2.99, 2.23 [11:57:50] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [12:19:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.45, 3.51, 2.98 [12:22:51] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.13, 3.67, 3.15 [12:24:46] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.74, 3.70, 3.22 [12:27:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [12:36:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.82, 3.30, 3.30 [12:40:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 5.07, 4.12, 3.63 [12:42:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.58, 3.97, 3.65 [12:54:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.46, 3.00, 3.30 [13:02:45] PROBLEM - bast141 Current Load on bast141 is CRITICAL: CRITICAL - load average: 2.23, 1.29, 0.55 [13:04:45] RECOVERY - bast141 Current Load on bast141 is OK: OK - load average: 0.34, 0.88, 0.49 [13:35:35] PROBLEM - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query mar.in.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:03:57] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 7.79, 4.22, 2.83 [14:04:41] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp23.miraheze.org - CNAME OK [14:05:54] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.58, 3.56, 2.74 [14:07:51] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.39, 3.74, 2.89 [14:09:48] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.02, 3.05, 2.74 [14:27:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [14:57:50] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [15:57:30] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.36, 4.18, 3.33 [15:59:27] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.43, 3.73, 3.26 [16:01:24] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.20, 4.24, 3.49 [16:08:22] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.39, 3.45, 2.40 [16:10:22] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.89, 3.62, 2.59 [16:11:10] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.35, 3.67, 3.72 [16:12:22] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.86, 3.86, 2.80 [16:12:53] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=soothouseocswiki --no-updates --username-prefix=wikia:soothouse-ocs /home/macfan/ImportDump/soothouseocswiki-20230712212532.xml (START) [16:12:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:14:22] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 3.91, 3.72, 2.87 [16:17:01] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.73, 4.11, 3.84 [16:18:58] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 1.80, 3.30, 3.58 [16:22:22] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 4.13, 3.92, 3.32 [16:22:52] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.97, 3.89, 3.69 [16:23:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=soothouseocswiki --no-updates --username-prefix=wikia:soothouse-ocs /home/macfan/ImportDump/soothouseocswiki-20230712212532.xml (END - exit=0) [16:23:59] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=soothouseocswiki (START) [16:24:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:24:49] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.26, 3.66, 3.63 [16:26:22] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 4.00, 3.96, 3.48 [16:30:22] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 2.84, 3.31, 3.31 [16:30:40] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.22, 2.98, 3.37 [16:31:32] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=soothouseocswiki (END - exit=0) [16:31:33] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=soothouseocswiki --active --update (END - exit=0) [16:31:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:31:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:35:24] PROBLEM - cp33 Disk Space on cp33 is WARNING: DISK WARNING - free space: / 8216 MB (10% inode=98%); [16:46:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.93, 5.36, 4.08 [16:50:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.00, 3.83, 3.73 [16:56:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.76, 2.88, 3.39 [17:00:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/17cb9ffd1335 [17:00:19] [02miraheze/puppet] 07paladox 0317cb9ff - miraheze-backup: fix swift account container name [17:00:21] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [17:00:22] [02puppet] 07paladox opened pull request 03#3349: miraheze-backup: fix swift account container name - 13https://github.com/miraheze/puppet/pull/3349 [17:00:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/17cb9ffd1335...191745906988 [17:00:50] [02miraheze/puppet] 07paladox 031917459 - Update ac.pp [17:00:51] [02puppet] 07paladox synchronize pull request 03#3349: miraheze-backup: fix swift account container name - 13https://github.com/miraheze/puppet/pull/3349 [17:01:04] [02puppet] 07paladox closed pull request 03#3349: miraheze-backup: fix swift account container name - 13https://github.com/miraheze/puppet/pull/3349 [17:01:05] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/41ec6d052e6d...303904ddab4f [17:01:07] [02miraheze/puppet] 07paladox 03303904d - miraheze-backup: fix swift account container name (#3349) [17:01:09] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [17:01:11] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [17:07:45] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: CRITICAL - load average: 5.41, 3.80, 3.32 [17:07:48] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/09c2391d79e0 [17:07:51] [02miraheze/puppet] 07paladox 0309c2391 - swift: when backing up ac also log any errors using 2>&1 [17:07:53] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [17:07:54] [02puppet] 07paladox opened pull request 03#3350: swift: when backing up ac also log any errors using 2>&1 - 13https://github.com/miraheze/puppet/pull/3350 [17:07:59] [02puppet] 07paladox closed pull request 03#3350: swift: when backing up ac also log any errors using 2>&1 - 13https://github.com/miraheze/puppet/pull/3350 [17:08:00] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-8 [17:08:02] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/303904ddab4f...ca7f584ead8c [17:08:05] [02miraheze/puppet] 07paladox 03ca7f584 - swift: when backing up ac also log any errors using 2>&1 (#3350) [17:08:06] [02puppet] 07paladox deleted branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [17:09:43] PROBLEM - matomo121 Current Load on matomo121 is WARNING: WARNING - load average: 2.86, 3.43, 3.25 [17:11:41] RECOVERY - matomo121 Current Load on matomo121 is OK: OK - load average: 3.39, 3.30, 3.21 [17:13:52] miraheze/MirahezeMagic - translatewiki the build has errored. [17:22:19] RECOVERY - cloud11 IPMI Sensors on cloud11 is OK: IPMI Status: OK [17:25:21] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 9.82, 5.08, 3.71 [17:26:12] PROBLEM - cloud11 IPMI Sensors on cloud11 is CRITICAL: IPMI Status: Critical [Cntlr 2 Bay 8 = Critical] [17:27:18] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ca7f584ead8c...234202f95a2e [17:27:19] [02miraheze/puppet] 07paladox 03234202f - Phabricator: when backing up make sure to log errors [17:31:12] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.10, 3.62, 3.51 [17:31:22] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [17:33:09] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.97, 3.31, 3.40 [17:36:42] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1028323174 secs [17:39:03] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 5.26, 4.76, 3.98 [17:39:50] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 3.55, 2.25, 1.19 [17:40:21] [02puppet] 07paladox synchronize pull request 03#3310: salt: Set ssh_max_procs to how many cpu cores that the host has - 13https://github.com/miraheze/puppet/pull/3310 [17:40:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/84b64ef2354f...061bffa4f9c9 [17:40:26] [02miraheze/puppet] 07paladox 03061bffa - Update roster.erb [17:41:00] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.12, 3.83, 3.74 [17:46:51] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.84, 2.89, 3.36 [17:48:17] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query zulaclub.net. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:51:50] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.63, 1.99, 1.72 [17:54:42] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.09019550681 secs [17:55:50] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.31, 2.04, 1.79 [17:57:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.nj.cn.eu.org All nameservers failed to answer the query. [17:57:50] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.88, 1.96, 1.79 [18:01:23] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [18:07:22] PROBLEM - phab121 Disk Space on phab121 is WARNING: DISK WARNING - free space: / 4306 MB (9% inode=91%); [18:09:51] PROBLEM - phab121 Current Load on phab121 is CRITICAL: CRITICAL - load average: 2.26, 1.99, 1.85 [18:10:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.39, 3.21, 2.33 [18:10:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.46, 3.59, 3.27 [18:12:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.04, 3.06, 2.38 [18:12:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 10.97, 5.59, 3.98 [18:13:22] PROBLEM - phab121 Disk Space on phab121 is CRITICAL: DISK CRITICAL - free space: / 2592 MB (5% inode=91%); [18:18:00] PROBLEM - wiki.zulaclub.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.zulaclub.net All nameservers failed to answer the query. [18:18:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.01, 3.72, 3.65 [18:19:50] PROBLEM - phab121 Current Load on phab121 is WARNING: WARNING - load average: 1.90, 2.00, 1.93 [18:25:50] RECOVERY - phab121 Current Load on phab121 is OK: OK - load average: 0.88, 1.23, 1.61 [18:26:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 6.73, 5.00, 4.10 [18:27:49] PROBLEM - wiki.nj.cn.eu.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.nj.cn.eu.org could not be found [18:28:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/3a41dfc2bbbf [18:28:34] [02miraheze/puppet] 07paladox 033a41dfc - salt: Use roster_defaults to set some params [18:28:36] [02puppet] 07paladox created branch 03paladox-patch-8 - 13https://github.com/miraheze/puppet [18:28:38] [02puppet] 07paladox opened pull request 03#3351: salt: Use roster_defaults to set some params - 13https://github.com/miraheze/puppet/pull/3351 [18:29:59] [02puppet] 07paladox synchronize pull request 03#3351: salt: Use roster_defaults to set some params - 13https://github.com/miraheze/puppet/pull/3351 [18:30:01] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-8 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3a41dfc2bbbf...84987b77050e [18:30:03] [02miraheze/puppet] 07paladox 0384987b7 - Update roster.erb [18:34:57] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.04, 3.45, 3.11 [18:36:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.63, 3.60, 3.89 [18:36:52] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 3.32, 3.34, 3.11 [18:42:35] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.77, 3.70, 3.33 [18:42:39] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 1.50, 2.30, 3.21 [18:47:44] RECOVERY - wiki.zulaclub.net - reverse DNS on sslhost is OK: SSL OK - wiki.zulaclub.net reverse DNS resolves to cp22.miraheze.org - CNAME OK [18:52:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.29, 3.83, 3.52 [18:54:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.37, 3.70, 3.51 [19:08:18] PROBLEM - mw141 Puppet on mw141 is CRITICAL: CRITICAL: Puppet has 160 failures. Last run 2 minutes ago with 160 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/wiki.hsdfz.org.cn.key],File[/etc/ssl/localcerts/beamngmodding.com.crt],File[/etc/ssl/private/beamngmodding.com.key],File[/etc/ssl/localcerts/studio.niuboss123.com.crt] [19:08:19] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 2.29, 3.06, 3.32 [19:08:36] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [19:09:22] RECOVERY - phab121 Disk Space on phab121 is OK: DISK OK - free space: / 12944 MB (28% inode=91%); [19:09:27] PROBLEM - mw134 Puppet on mw134 is CRITICAL: CRITICAL: Puppet has 464 failures. Last run 3 minutes ago with 464 failures. Failed resources (up to 3 shown): File[/etc/ssl/private/perpustaxaan.wiki.key],File[/etc/ssl/localcerts/istpcomputing.com.crt],File[/etc/ssl/private/istpcomputing.com.key],File[/etc/ssl/localcerts/replication-ops.com.crt] [19:09:36] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.11, 3.55, 3.09 [19:09:44] PROBLEM - cp32 Puppet on cp32 is CRITICAL: CRITICAL: Puppet has 247 failures. Last run 3 minutes ago with 247 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/wonderfuleveryday.org.crt],File[/etc/ssl/private/wonderfuleveryday.org.key],File[/etc/ssl/localcerts/wiki.3point0.science.crt],File[/etc/ssl/private/wiki.3point0.science.key] [19:09:50] PROBLEM - mw132 Puppet on mw132 is CRITICAL: CRITICAL: Puppet has 653 failures. Last run 3 minutes ago with 653 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/www.sidem.wiki.crt],File[/etc/ssl/private/www.sidem.wiki.key],File[/etc/ssl/localcerts/wiki.wisrail.de.crt],File[/etc/ssl/private/wiki.wisrail.de.key] [19:11:33] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.18, 2.98, 2.93 [19:23:18] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.85, 4.19, 3.33 [19:27:12] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.42, 3.89, 3.47 [19:31:06] RECOVERY - cp33 Current Load on cp33 is OK: OK - load average: 2.97, 3.33, 3.33 [19:36:20] RECOVERY - mw141 Puppet on mw141 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [19:37:21] RECOVERY - mw134 Puppet on mw134 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [19:37:43] RECOVERY - cp32 Puppet on cp32 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [19:37:52] RECOVERY - mw132 Puppet on mw132 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:38:36] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:39:05] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/root/ImportDump/uewikiwiki-20230713184342.xml (START) [19:39:06] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/root/ImportDump/uewikiwiki-20230713184342.xml (END - exit=256) [19:39:07] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (START) [19:39:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:34] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (END - exit=2) [19:39:35] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=uewikiwiki --active --update (END - exit=2) [19:39:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:39:54] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/paladox/ImportDump/uewikiwiki-20230713184342.xml (START) [19:39:55] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/paladox/ImportDump/uewikiwiki-20230713184342.xml (END - exit=256) [19:39:56] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (START) [19:39:57] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (END - exit=2) [19:39:58] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=uewikiwiki --active --update (END - exit=0) [19:39:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:24] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/paladox/ImportDump/uewikiwiki-20230713184342.xml (START) [19:40:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:29] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=uewikiwiki --no-updates /home/paladox/ImportDump/uewikiwiki-20230713184342.xml (END - exit=0) [19:41:30] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (START) [19:41:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:20] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=uewikiwiki (END - exit=0) [19:43:21] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=uewikiwiki --active --update (END - exit=0) [19:43:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:44:19] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=anotherworldwiki --no-updates --username-prefix=w /home/paladox/ImportDump/anotherworldwiki-20230713201656.xml (START) [19:44:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:09] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=anotherworldwiki --no-updates --username-prefix=w /home/paladox/ImportDump/anotherworldwiki-20230713201656.xml (END - exit=0) [19:45:10] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=anotherworldwiki (START) [19:45:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:47:03] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 7.32, 4.23, 3.15 [19:52:40] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 10.82, 5.40, 3.86 [19:52:40] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=wingsoffiretheredskyprophecywiki --no-updates --username-prefix=wikia:wingsoffirefanfic /home/paladox/ImportDump/wingsoffiretheredskyprophecywiki-20230713230258.xml (START) [19:52:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:43] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=anotherworldwiki (END - exit=0) [19:53:44] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=anotherworldwiki --active --update (END - exit=0) [19:53:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:54] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=wingsoffiretheredskyprophecywiki --no-updates --username-prefix=wikia:wingsoffirefanfic /home/paladox/ImportDump/wingsoffiretheredskyprophecywiki-20230713230258.xml (END - exit=0) [19:53:55] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=wingsoffiretheredskyprophecywiki (START) [19:53:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:54:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:54:36] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=wingsoffiretheredskyprophecywiki (END - exit=0) [19:54:37] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=wingsoffiretheredskyprophecywiki --active --update (END - exit=0) [19:54:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:54:42] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=robloxhourswiki --no-updates --username-prefix=wikia:hours-roblox /home/paladox/ImportDump/robloxhourswiki-20230715034330.xml (START) [19:54:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:54:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:55:45] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=dayofduskwiki --no-updates --username-prefix=wikia:day-of-dusk-rbx /home/paladox/ImportDump/dayofduskwiki-20230715180423.xml (START) [19:55:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:58:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.35, 3.72, 3.60 [20:02:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.61, 3.58, 3.52 [20:03:18] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=dayofduskwiki --no-updates --username-prefix=wikia:day-of-dusk-rbx /home/paladox/ImportDump/dayofduskwiki-20230715180423.xml (END - exit=0) [20:03:19] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=dayofduskwiki (START) [20:03:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:03:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:04:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 3.69, 3.42, 3.47 [20:06:11] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=dayofduskwiki (END - exit=0) [20:06:12] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=dayofduskwiki --active --update (END - exit=0) [20:06:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:06:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:08:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.00, 3.18, 3.85 [20:08:39] PROBLEM - cp33 Current Load on cp33 is CRITICAL: CRITICAL - load average: 4.51, 3.74, 3.56 [20:09:02] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ryfeliswiki --no-updates --username-prefix=wikia:rise-and-fall-ii /home/paladox/ImportDump/ryfeliswiki-20230719161231.xml (START) [20:09:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:08] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=ryfeliswiki --no-updates --username-prefix=wikia:rise-and-fall-ii /home/paladox/ImportDump/ryfeliswiki-20230719161231.xml (END - exit=0) [20:10:09] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ryfeliswiki (START) [20:10:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:10:19] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 4.27, 3.56, 3.90 [20:11:15] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ryfeliswiki (END - exit=0) [20:11:16] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=ryfeliswiki --active --update (END - exit=0) [20:11:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:11:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:11:54] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=starwarsroleplayingwiki --no-updates --username-prefix=wikia:star-wars-roleplaying /home/paladox/ImportDump/starwarsroleplayingwiki-20230719202828.xml (START) [20:11:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:12:19] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 3.29, 3.60, 3.89 [20:13:17] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=exeindexwiki --no-updates --username-prefix=wikia:exe-index /home/paladox/ImportDump/exeindexwiki-20230722184147.xml (START) [20:13:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:13:54] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=exeindexwiki --no-updates --username-prefix=wikia:exe-index /home/paladox/ImportDump/exeindexwiki-20230722184147.xml (END - exit=0) [20:13:55] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=exeindexwiki (START) [20:13:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:15:09] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=robloxhourswiki --no-updates --username-prefix=wikia:hours-roblox /home/paladox/ImportDump/robloxhourswiki-20230715034330.xml (END - exit=0) [20:15:10] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=robloxhourswiki (START) [20:15:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:15:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:16:28] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=exeindexwiki (END - exit=0) [20:16:29] !log [paladox@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=exeindexwiki --active --update (END - exit=0) [20:16:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:16:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:20:39] PROBLEM - cp33 Current Load on cp33 is WARNING: WARNING - load average: 2.14, 3.39, 3.76 [20:20:59]