[00:02:31] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.38, 6.79, 6.68 [00:03:16] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 14.42, 6.92, 3.12 [00:05:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.76, 7.52, 7.36 [00:19:16] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 2.67, 7.02, 7.54 [00:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.98, 7.87, 7.38 [00:23:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.63, 7.62, 7.38 [00:25:16] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.17, 5.25, 6.63 [00:27:18] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.95, 3.29, 2.34 [00:29:16] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.94, 3.06, 2.37 [00:31:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.36, 3.53, 3.05 [00:31:03] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [00:33:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.16, 5.98, 6.65 [00:37:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.12, 3.40, 3.14 [00:37:17] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 3.47, 2.84 [00:39:16] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.80, 3.64, 2.99 [00:43:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.88, 6.69, 6.69 [00:45:15] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.14, 3.17, 3.02 [00:45:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.77, 6.58, 6.64 [00:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.05, 7.10, 6.85 [00:53:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.18, 6.76, 6.76 [00:59:39] PROBLEM - swiftobject122 NTP time on swiftobject122 is WARNING: NTP WARNING: Offset -0.1055406928 secs [01:07:07] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:09:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.72, 6.87, 6.67 [01:11:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.18, 8.06, 7.10 [01:12:58] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.32, 7.00, 6.31 [01:14:54] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.50, 6.25, 6.11 [01:15:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.24, 6.81, 6.84 [01:16:11] RECOVERY - swiftobject122 NTP time on swiftobject122 is OK: NTP OK: Offset -0.04554110765 secs [01:19:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.91, 7.59, 7.08 [01:21:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.42, 7.23, 7.00 [01:26:46] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.009464651346 secs [01:29:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.67, 6.50, 6.79 [01:35:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.61, 6.75, 6.83 [01:36:08] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [01:37:04] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.74, 3.25, 3.07 [01:39:01] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.21, 3.49, 3.17 [01:39:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.16, 6.62, 6.77 [01:41:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.64, 3.48, 3.20 [01:43:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.33, 3.39, 3.20 [02:02:08] PROBLEM - wiki.veloren.net - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.veloren.net. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:17:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [02:30:46] RECOVERY - wiki.veloren.net - reverse DNS on sslhost is OK: SSL OK - wiki.veloren.net reverse DNS resolves to cp25.miraheze.org - CNAME OK [02:37:10] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [02:45:53] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [02:58:46] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1450665891 secs [03:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [03:34:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [03:37:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.83, 7.24, 6.69 [03:44:57] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.05, 6.52, 6.60 [03:50:44] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.39, 7.38, 6.90 [03:54:34] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.33, 6.62, 6.72 [03:58:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.02, 6.98, 6.84 [04:00:22] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.36, 6.56, 6.69 [04:04:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:22:36] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.94, 6.79, 6.67 [04:24:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.03, 7.79, 7.06 [04:26:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.88, 7.07, 6.88 [04:30:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.10, 6.54, 6.71 [04:35:03] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [04:36:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.55, 6.63, 6.63 [04:39:55] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.05, 6.37, 6.52 [04:46:09] PROBLEM - phab121 Current Load on phab121 is CRITICAL: LOAD CRITICAL - total load average: 3.02, 1.92, 0.96 [04:49:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.03, 7.33, 6.88 [04:51:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.59, 7.09, 6.85 [04:52:06] PROBLEM - phab121 Current Load on phab121 is WARNING: LOAD WARNING - total load average: 0.85, 1.80, 1.26 [04:53:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.36, 7.15, 6.87 [04:54:05] RECOVERY - phab121 Current Load on phab121 is OK: LOAD OK - total load average: 0.43, 1.29, 1.14 [04:55:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.18, 6.99, 6.84 [04:57:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.52, 6.34, 6.61 [05:03:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.21, 6.98, 6.72 [05:05:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.78, 6.84, 6.70 [05:06:06] PROBLEM - phab121 Current Load on phab121 is CRITICAL: LOAD CRITICAL - total load average: 2.17, 1.68, 1.23 [05:06:46] PROBLEM - ns1 NTP time on ns1 is CRITICAL: NTP CRITICAL: Offset 0.5037696362 secs [05:07:03] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:07:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.21, 6.61, 6.63 [05:08:06] RECOVERY - phab121 Current Load on phab121 is OK: LOAD OK - total load average: 0.64, 1.27, 1.13 [05:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.82, 7.50, 6.94 [05:21:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.13, 7.09, 6.87 [05:23:56] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7532 MB (5% inode=99%); [05:25:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.58, 6.75 [05:25:56] PROBLEM - db112 Disk Space on db112 is WARNING: DISK WARNING - free space: / 8131 MB (6% inode=99%); [05:31:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.21, 7.18, 6.95 [05:33:51] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.00, 6.60, 6.76 [05:36:06] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [05:39:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.31, 8.98, 7.59 [05:40:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.49, 7.45, 6.05 [05:42:22] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.35, 6.57, 5.90 [05:43:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.40, 7.58, 7.36 [05:51:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.29, 6.30, 6.73 [06:04:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.85, 8.52, 7.34 [06:08:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.41, 7.91, 7.36 [06:31:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.85, 6.24, 6.70 [06:58:11] PROBLEM - wiki.globasa.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.403 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [07:04:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:27:06] RECOVERY - wiki.globasa.net - reverse DNS on sslhost is OK: SSL OK - wiki.globasa.net reverse DNS resolves to cp24.miraheze.org - CNAME OK [07:34:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [08:04:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [08:07:02] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [08:09:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.72, 7.46, 6.46 [08:11:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.47, 6.73, 6.32 [08:34:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [08:36:05] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [09:58:46] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.005640387535 secs [10:16:12] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query franchising.org.ua. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [10:45:23] RECOVERY - franchise.franchising.org.ua - reverse DNS on sslhost is OK: SSL OK - franchise.franchising.org.ua reverse DNS resolves to cp24.miraheze.org - CNAME OK [11:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [12:21:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.79, 6.92, 6.45 [12:23:25] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.77, 6.79, 6.45 [12:26:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.03, 6.39, 5.73 [12:27:19] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.22, 7.25, 6.73 [12:28:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.36, 6.13, 5.72 [12:31:09] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.36, 6.55, 6.55 [12:41:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 7.05, 6.63 [12:43:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.80, 6.50, 6.47 [12:49:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.49, 7.08, 6.69 [12:51:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.33, 6.43, 6.50 [13:12:46] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1014810801 secs [13:20:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.90, 6.65, 6.30 [13:22:22] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.64, 6.41, 6.24 [13:29:39] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [13:44:47] PROBLEM - wiki.seamly.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.seamly.net All nameservers failed to answer the query. [13:50:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.81, 7.72, 6.71 [13:52:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.69, 7.16, 6.62 [13:56:18] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.24, 6.56, 6.48 [13:58:40] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: SSL OK - housing.wiki reverse DNS resolves to cp34.miraheze.org - CNAME FLAT [14:03:56] PROBLEM - db112 Disk Space on db112 is CRITICAL: DISK CRITICAL - free space: / 7967 MB (5% inode=99%); [14:13:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.88, 6.58, 6.40 [14:14:53] PROBLEM - wiki.seamly.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [14:15:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.54, 6.65, 6.40 [14:17:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.65, 6.41, 6.34 [14:21:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.01, 6.96, 6.63 [14:25:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.64, 6.55, 6.56 [14:53:14] PROBLEM - mw143 Current Load on mw143 is WARNING: LOAD WARNING - total load average: 10.77, 7.08, 4.67 [14:55:12] RECOVERY - mw143 Current Load on mw143 is OK: LOAD OK - total load average: 6.25, 7.49, 5.18 [15:04:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [15:34:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [15:45:14] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-15 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/cccd47dd6f07 [15:45:15] [02miraheze/puppet] 07paladox 03cccd47d - matomo: upgrade to 5.0.0 [15:45:17] [02puppet] 07paladox created branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [15:45:19] [02puppet] 07paladox opened pull request 03#3573: matomo: upgrade to 5.0.0 - 13https://github.com/miraheze/puppet/pull/3573 [15:45:39] [02puppet] 07paladox closed pull request 03#3573: matomo: upgrade to 5.0.0 - 13https://github.com/miraheze/puppet/pull/3573 [15:45:42] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-15 [15:45:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5bf70ef55b27...e595ec124d90 [15:45:46] [02miraheze/puppet] 07paladox 03e595ec1 - matomo: upgrade to 5.0.0 (#3573) [15:45:47] [02puppet] 07paladox deleted branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [15:47:07] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.98, 6.45, 5.87 [15:47:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.13, 6.74, 6.37 [15:49:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 3.88, 5.54, 5.61 [15:51:01] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.96, 4.81, 2.77 [15:51:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.53, 6.41, 6.35 [16:01:31] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.19, 9.29, 6.55 [16:03:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.71, 3.29, 3.27 [16:03:32] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 6.59, 8.19, 6.49 [16:09:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 7.37, 6.65 [16:11:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.54, 8.07, 7.01 [16:21:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.00, 7.37, 7.31 [16:25:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.30, 6.14, 6.80 [16:37:51] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 6.84, 4.19, 2.97 [16:41:07] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e595ec124d90...2360eb6d3084 [16:41:10] [02miraheze/puppet] 07paladox 032360eb6 - matomo: Install Provider plugin [16:41:41] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.80, 3.77, 3.05 [16:42:56] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=complexicadndwiki --no-updates --username-prefix=wikia:complexicadnd /home/macfan/ImportDump/complexicadndwiki-20231210071120.xml (START) [16:43:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:43:49] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=complexicadndwiki --no-updates --username-prefix=wikia:complexicadnd /home/macfan/ImportDump/complexicadndwiki-20231210071120.xml (END - exit=0) [16:43:50] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=complexicadndwiki (START) [16:43:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:36] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=complexicadndwiki (END - exit=0) [16:44:37] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=complexicadndwiki --active --update (END - exit=0) [16:44:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:45:17] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bettyhackswiki --no-updates /home/macfan/ImportDump/bettyhackswiki-20231210131938.xml (START) [16:45:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:47:13] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bettyhackswiki --no-updates /home/macfan/ImportDump/bettyhackswiki-20231210131938.xml (END - exit=0) [16:47:13] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bettyhackswiki (START) [16:47:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:47:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:47:37] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bettyhackswiki (END - exit=0) [16:47:37] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=bettyhackswiki --active --update (END - exit=0) [16:47:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:47:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:49:20] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.17, 3.87, 3.35 [16:51:15] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.97, 3.85, 3.40 [16:55:04] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 5.90, 4.26, 3.62 [17:01:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.52, 3.88, 3.70 [17:07:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.98, 6.88, 6.68 [17:17:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.06, 6.58, 6.66 [17:21:24] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bukwiki --no-updates --username-prefix=w /home/macfan/ImportDump/bukwiki-20231210220845.xml (START) [17:21:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:22:23] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bukwiki --no-updates --username-prefix=w /home/macfan/ImportDump/bukwiki-20231210220845.xml (END - exit=0) [17:22:24] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bukwiki (START) [17:22:27] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.02, 6.82, 6.26 [17:22:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:22:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:23:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.92, 3.12, 3.38 [17:24:23] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.09, 6.06, 6.04 [17:29:57] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bukwiki (END - exit=0) [17:29:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=bukwiki --active --update (END - exit=0) [17:30:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:30:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:30:29] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.69, 6.97, 6.80 [17:32:23] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=asoiafvisualreferencewiki --no-updates --username-prefix=wikia:asoiafvisualreference /home/macfan/ImportDump/asoiafvisualreferencewiki-20231211101103.xml (START) [17:32:24] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.56, 6.46, 6.63 [17:32:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:35:44] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=asoiafvisualreferencewiki --no-updates --username-prefix=wikia:asoiafvisualreference /home/macfan/ImportDump/asoiafvisualreferencewiki-20231211101103.xml (END - exit=0) [17:35:45] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=asoiafvisualreferencewiki (START) [17:35:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:35:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:36:59] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=asoiafvisualreferencewiki (END - exit=0) [17:37:00] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=asoiafvisualreferencewiki --active --update (END - exit=0) [17:37:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:37:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:37:31] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.60, 3.31, 3.30 [17:41:15] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.95, 2.93, 1.66 [17:41:20] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.92, 2.87, 3.14 [17:43:15] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.90, 2.12, 1.51 [17:50:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.18, 6.80, 6.59 [17:52:37] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.71, 6.41, 6.47 [18:01:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.65, 6.90, 6.68 [18:07:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.14, 7.14, 6.83 [18:08:59] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.55, 6.59, 6.66 [18:22:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.73, 7.15, 6.85 [18:24:23] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.94, 6.65, 6.70 [18:31:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.33, 7.43, 7.00 [18:33:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.16, 6.98, 6.89 [18:34:59] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.91, 6.67, 6.79 [18:38:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.80, 6.93, 6.88 [18:40:46] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.92, 6.68, 6.80 [18:51:34] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=pixellandwiki --no-updates --username-prefix=wikia:pixelsland /home/macfan/ImportDump/pixellandwiki-20231212063214.xml (START) [18:51:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:52:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.68, 7.09, 6.92 [18:56:12] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.62, 7.81, 7.20 [18:58:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.99, 7.53, 7.15 [19:01:57] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.63, 7.82, 7.31 [19:02:02] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=pixellandwiki --no-updates --username-prefix=wikia:pixelsland /home/macfan/ImportDump/pixellandwiki-20231212063214.xml (END - exit=0) [19:02:03] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pixellandwiki (START) [19:02:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:02:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:03:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.81, 7.45, 7.24 [19:04:58] [02mw-config] 07R4356th opened pull request 03#5408: Add $wgDiscordName to ManageWiki (T11459) - 13https://github.com/miraheze/mw-config/pull/5408 [19:06:06] miraheze/mw-config - R4356th the build has errored. [19:07:03] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=pixellandwiki (END - exit=0) [19:07:04] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=pixellandwiki --active --update (END - exit=0) [19:07:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:07:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.98, 7.99, 7.48 [19:27:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.56, 7.38, 7.48 [19:29:26] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=modernphenotypeswiki --no-updates --username-prefix=wikia:modernphenotypes /home/macfan/ImportDump/modernphenotypeswiki-20231212094610.xml (START) [19:29:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:12] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=modernphenotypeswiki --no-updates --username-prefix=wikia:modernphenotypes /home/macfan/ImportDump/modernphenotypeswiki-20231212094610.xml (END - exit=0) [19:41:13] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=modernphenotypeswiki (START) [19:41:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:22] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=modernphenotypeswiki (END - exit=256) [19:41:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:23] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=modernphenotypeswiki --active --update (END - exit=0) [19:41:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:41:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:45:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.18, 6.05, 6.72 [19:46:55] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.45, 7.06, 6.56 [19:47:58] [02mw-config] 07OAuthority commented on pull request 03#5408: Add $wgDiscordName to ManageWiki (T11459) - 13https://github.com/miraheze/mw-config/pull/5408#issuecomment-1870582848 [19:49:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 7.25, 7.09 [19:51:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.09, 7.15, 7.07 [19:52:45] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.78, 6.58, 6.52 [19:55:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.44, 6.39, 6.74 [20:01:32] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.60, 6.89, 6.69 [20:01:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 6.98, 6.90 [20:03:29] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.68, 6.78, 6.67 [20:05:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.91, 6.45, 6.71 [20:15:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.02, 7.18, 6.92 [20:17:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.98, 7.41, 7.03 [20:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.91, 8.03, 7.28 [20:20:03] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.23, 7.53, 6.88 [20:21:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.86, 7.46, 7.16 [20:24:03] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.34, 6.64, 6.64 [20:29:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 7.10, 6.97 [20:31:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.54, 6.68, 6.85 [20:52:30] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=eternalbabyconflictwiki --no-updates --username-prefix=wikia:eternalbabyconflict-lore /home/macfan/ImportDump/eternalbabyconflictwiki-20231213002749.xml (END - exit=0) [20:52:31] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=eternalbabyconflictwiki (START) [20:52:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:52:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.50, 7.54, 7.33 [20:58:41] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.01, 6.59, 6.35 [20:59:50] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.93, 6.90, 7.12 [21:00:35] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.37, 6.25, 6.23 [21:09:15] PROBLEM - wiki.grab-tools.live - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.grab-tools.live' expires in 15 day(s) (Fri 12 Jan 2024 20:53:42 GMT +0000). [21:09:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/595371320a4a...1d72d561aa29 [21:09:33] [02miraheze/ssl] 07MirahezeSSLBot 031d72d56 - Bot: Update SSL cert for wiki.grab-tools.live [21:09:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.86, 6.32, 6.72 [21:10:22] PROBLEM - wiki.cutefame.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.cutefame.net' expires in 15 day(s) (Fri 12 Jan 2024 20:52:10 GMT +0000). [21:10:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1d72d561aa29...1154e522cad1 [21:10:37] [02miraheze/ssl] 07MirahezeSSLBot 031154e52 - Bot: Update SSL cert for wiki.cutefame.net [21:13:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.97, 7.16, 6.96 [21:15:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.19, 6.70, 6.80 [21:21:56] PROBLEM - gogigantic.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'gogigantic.wiki' expires in 15 day(s) (Fri 12 Jan 2024 20:53:08 GMT +0000). [21:22:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1154e522cad1...3339274df847 [21:22:11] [02miraheze/ssl] 07MirahezeSSLBot 033339274 - Bot: Update SSL cert for gogigantic.wiki [21:28:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.52, 6.91, 6.79 [21:32:22] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.59, 6.51, 6.69 [21:37:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.74, 3.02, 2.67 [21:38:49] RECOVERY - wiki.grab-tools.live - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.grab-tools.live' will expire on Tue 26 Mar 2024 20:09:24 GMT +0000. [21:39:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.23, 3.00, 2.70 [21:39:43] RECOVERY - wiki.cutefame.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cutefame.net' will expire on Tue 26 Mar 2024 20:10:28 GMT +0000. [21:43:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.46, 7.00, 6.82 [21:45:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.33, 7.87, 7.15 [21:53:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.19, 7.62, 7.44 [21:57:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.17, 7.48, 7.38 [21:59:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.18, 7.34, 7.33 [22:06:10] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bewitchmentwiki --no-updates --username-prefix=wikia:bewitchment /home/macfan/ImportDump/bewitchmentwiki-20231213180926.xml (START) [22:06:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:07:01] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.89, 3.54, 2.79 [22:09:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.55, 3.50, 2.87 [22:09:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.68, 6.14, 6.72 [22:10:15] PROBLEM - pandorastale.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pandorastale.wiki' expires in 15 day(s) (Fri 12 Jan 2024 22:02:42 GMT +0000). [22:10:19] PROBLEM - phab121 Current Load on phab121 is WARNING: LOAD WARNING - total load average: 1.98, 1.66, 1.08 [22:10:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3339274df847...a039c2a2a63a [22:10:30] [02miraheze/ssl] 07MirahezeSSLBot 03a039c2a - Bot: Update SSL cert for pandorastale.wiki [22:11:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.70, 3.37, 2.91 [22:12:18] RECOVERY - phab121 Current Load on phab121 is OK: LOAD OK - total load average: 1.11, 1.45, 1.07 [22:13:02] [02miraheze/mw-config] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e6414126dc89...3cece00a8c80 [22:13:03] [02miraheze/mw-config] 07MacFan4000 033cece00 - add fr wikipedia to wgImportSources for shemwiki [22:13:56] !log [macfan@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [22:13:59] miraheze/mw-config - MacFan4000 the build passed. [22:14:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:14:23] !log [macfan@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 27s [22:14:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:15:01] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.04, 3.90, 3.24 [22:17:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.27, 3.81, 3.29 [22:17:05] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bewitchmentwiki --no-updates --username-prefix=wikia:bewitchment /home/macfan/ImportDump/bewitchmentwiki-20231213180926.xml (END - exit=0) [22:17:06] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bewitchmentwiki (START) [22:17:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:17:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:17:53] PROBLEM - os141 APT on os141 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [22:18:18] PROBLEM - mwtask141 MediaWiki Rendering on mwtask141 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [22:19:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 6.77, 6.64 [22:19:52] PROBLEM - os141 APT on os141 is CRITICAL: APT CRITICAL: 37 packages available for upgrade (12 critical updates). [22:19:59] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [22:20:16] RECOVERY - mwtask141 MediaWiki Rendering on mwtask141 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.410 second response time [22:20:47] PROBLEM - mockgovernments.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mockgovernments.com' expires in 15 day(s) (Fri 12 Jan 2024 22:02:12 GMT +0000). [22:21:03] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a039c2a2a63a...3417e3cb7064 [22:21:04] [02miraheze/ssl] 07MirahezeSSLBot 033417e3c - Bot: Update SSL cert for mockgovernments.com [22:21:54] RECOVERY - gogigantic.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'gogigantic.wiki' will expire on Tue 26 Mar 2024 20:22:01 GMT +0000. [22:22:29] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bewitchmentwiki (END - exit=0) [22:22:30] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=bewitchmentwiki --active --update (END - exit=0) [22:22:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:22:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:23:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.54, 3.16, 3.19 [22:24:28] PROBLEM - worldtobuild.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'worldtobuild.wiki' expires in 15 day(s) (Fri 12 Jan 2024 22:03:27 GMT +0000). [22:24:41] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3417e3cb7064...a4caa65263f1 [22:24:43] [02miraheze/ssl] 07MirahezeSSLBot 03a4caa65 - Bot: Update SSL cert for worldtobuild.wiki [22:25:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.38, 6.69, 6.69 [22:26:29] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=micronationalitywiki --no-updates /home/macfan/ImportDump/micronationalitywiki-20231215024122.xml (START) [22:26:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:26] !log [@test131] starting deploy of {'config': True} to all [22:27:28] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 2s [22:27:32] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=micronationalitywiki --no-updates /home/macfan/ImportDump/micronationalitywiki-20231215024122.xml (END - exit=0) [22:27:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:27:33] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=micronationalitywiki (START) [22:27:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:42] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=micronationalitywiki (END - exit=0) [22:28:44] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=micronationalitywiki --active --update (END - exit=0) [22:28:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:28:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:29:01] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.30, 3.62, 3.39 [22:30:05] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 11.14, 8.78, 6.64 [22:31:01] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.54, 3.25, 3.28 [22:31:42] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=exeoraclewiki --no-updates --username-prefix=wikia:exe-oracle /home/macfan/ImportDump/exeoraclewiki-20231218000232.xml (START) [22:31:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:31:59] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 5.27, 7.38, 6.37 [22:32:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=exeoraclewiki --no-updates --username-prefix=wikia:exe-oracle /home/macfan/ImportDump/exeoraclewiki-20231218000232.xml (END - exit=0) [22:32:59] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=exeoraclewiki (START) [22:33:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:24] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=exeoraclewiki (END - exit=0) [22:33:25] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=exeoraclewiki --active --update (END - exit=0) [22:33:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:39:31] RECOVERY - pandorastale.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'pandorastale.wiki' will expire on Tue 26 Mar 2024 21:10:22 GMT +0000. [22:39:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.01, 7.25, 6.86 [22:43:16] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 7.56, 4.10, 2.19 [22:43:50] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.84, 6.56, 6.66 [22:45:16] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.03, 3.29, 2.14 [22:50:05] RECOVERY - mockgovernments.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mockgovernments.com' will expire on Tue 26 Mar 2024 21:20:56 GMT +0000. [22:52:41] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.27, 6.88, 6.82 [22:54:26] RECOVERY - worldtobuild.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'worldtobuild.wiki' will expire on Tue 26 Mar 2024 21:24:34 GMT +0000. [22:56:32] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.82, 6.70, 6.79 [23:00:24] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.54, 6.77, 6.82 [23:04:15] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.52, 6.62, 6.75 [23:04:34] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [23:07:59] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:08:07] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.64, 7.11, 6.93 [23:09:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bcrwikiwiki --no-updates --username-prefix=wikia:ball-card-rng-guides-info /home/macfan/ImportDump/bcrwikiwiki-20231218172307.xml (START) [23:10:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:11:00] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=bcrwikiwiki --no-updates --username-prefix=wikia:ball-card-rng-guides-info /home/macfan/ImportDump/bcrwikiwiki-20231218172307.xml (END - exit=0) [23:11:01] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bcrwikiwiki (START) [23:11:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:11:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:11:25] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=bcrwikiwiki (END - exit=0) [23:11:26]