[00:02:17] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 6.95, 6.68 [00:03:25] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 15.19, 8.08, 3.68 [00:06:05] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.10, 6.44, 6.53 [00:07:37] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 5.59, 3.49, 1.82 [00:08:51] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.66, 3.57, 2.38 [00:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [00:09:35] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.91, 3.41, 1.99 [00:10:50] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.72, 3.67, 2.57 [00:11:33] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.45, 3.04, 2.02 [00:12:49] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.41, 3.28, 2.57 [00:12:57] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.66, 3.27, 2.29 [00:13:25] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 3.05, 7.55, 6.33 [00:14:54] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.03, 3.15, 2.37 [00:15:25] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 4.28, 6.61, 6.13 [00:31:29] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.16, 3.57, 2.73 [00:33:28] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.85, 3.62, 2.85 [00:35:27] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 1.90, 3.03, 2.73 [00:37:14] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.17, 3.43, 2.93 [00:39:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.28, 3.31, 2.94 [00:43:04] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.92, 3.67, 3.18 [00:46:57] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 1.87, 3.09, 3.08 [00:53:33] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.71, 3.61, 3.10 [00:55:27] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.10, 3.47, 3.12 [00:57:23] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.18, 3.60, 3.20 [01:01:17] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.96, 3.48, 3.27 [01:03:13] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.81, 3.27, 3.22 [01:35:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.01, 7.13, 6.79 [01:36:03] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.68, 3.86, 3.30 [01:39:52] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.28, 3.37, 3.32 [01:41:32] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.94, 6.78, 6.78 [01:45:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.73, 7.73, 7.15 [01:47:15] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.49, 7.71, 7.21 [02:00:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.35, 7.46, 7.18 [02:02:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.94, 7.56, 7.25 [02:10:36] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [02:10:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.56, 6.39, 6.75 [02:13:49] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [02:20:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 7.18, 6.89 [02:22:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.27, 6.75, 6.76 [02:32:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.82, 7.31, 6.98 [02:36:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.61, 6.61, 6.79 [02:38:23] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [02:40:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.56, 6.93, 6.89 [02:42:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.78, 6.65, 6.80 [02:46:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.88, 6.89, 6.88 [02:58:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.04, 6.56, 6.77 [03:03:12] PROBLEM - db142 Backups SQL on db142 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864191 seconds old and 0 bytes [03:07:16] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [03:07:46] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [03:11:18] PROBLEM - cloud12 Puppet on cloud12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [03:21:23] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.52, 6.78, 6.18 [03:23:20] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.46, 6.26, 6.07 [03:35:26] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.19, 6.97, 6.63 [03:38:23] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [03:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 13 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [03:39:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.20, 7.60, 6.97 [03:39:18] RECOVERY - cloud12 Puppet on cloud12 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [03:43:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.61, 7.43, 7.05 [03:58:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.15, 6.28, 6.70 [04:07:16] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [04:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [04:26:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.41, 7.44, 6.71 [04:28:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.87, 6.72, 6.53 [05:15:14] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [05:22:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.54, 6.96, 6.55 [05:24:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.59, 6.41, 6.39 [05:38:26] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.andreijiroh.uk.eu.org. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:45:14] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:56:18] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.98, 7.05, 6.48 [05:58:13] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.80, 6.76, 6.43 [06:02:01] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.52, 8.79, 7.26 [06:05:27] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.18, 8.34, 6.69 [06:10:52] PROBLEM - housing.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query housing.wiki. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:15:36] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [06:15:48] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [06:19:04] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.31, 7.86, 7.50 [06:24:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.13, 7.90, 7.56 [06:26:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.68, 7.37, 7.42 [06:32:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 9.13, 7.40, 7.42 [06:34:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.11, 7.27, 7.36 [06:36:23] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [06:38:57] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.16, 5.87, 6.79 [06:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 12 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [06:40:46] RECOVERY - housing.wiki - reverse DNS on sslhost is OK: SSL OK - housing.wiki reverse DNS resolves to cp35.miraheze.org - CNAME FLAT [06:53:43] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.53, 7.06, 6.72 [06:56:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.16, 7.01, 7.83 [06:57:37] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 3.47, 5.67, 6.28 [07:04:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.26, 6.76, 7.15 [07:07:47] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 39 seconds ago with 0 failures [07:08:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 6.92, 7.16 [07:12:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.25, 7.78, 7.40 [07:14:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.46, 6.87, 7.12 [07:32:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.12, 7.47, 7.40 [07:34:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.70, 7.14, 7.28 [07:38:23] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:42:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.62, 8.00, 7.51 [07:44:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.62, 7.41, 7.35 [07:52:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.11, 8.24, 7.58 [07:56:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.26, 7.28, 7.34 [07:58:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.14, 8.35, 7.75 [08:04:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.35, 7.71, 7.65 [08:07:16] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [08:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [08:22:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.54, 8.37, 7.68 [08:24:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.58, 7.57, 7.47 [08:36:23] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [08:40:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.16, 6.30, 6.78 [09:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 12 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [09:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [09:39:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.87, 7.28, 6.76 [09:45:00] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.80, 7.63, 7.01 [09:46:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.53, 6.88, 6.79 [09:52:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.36, 6.64 [10:08:24] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [10:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 12 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [10:15:48] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [10:20:36] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [10:20:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.42, 7.08, 6.71 [10:22:52] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.29, 6.33, 6.48 [10:25:36] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [10:27:51] RECOVERY - ns1 NTP time on ns1 is OK: NTP OK: Offset 0.002642184496 secs [10:37:17] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [10:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:41:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.89, 7.31, 6.76 [10:43:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.07, 7.73, 6.96 [10:45:25] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.70, 6.90, 6.74 [10:47:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.86, 6.21, 6.51 [10:55:54] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.57, 7.18, 6.84 [10:59:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.12, 6.37, 6.60 [11:06:23] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.88, 7.63, 7.00 [11:07:46] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [11:10:11] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.62, 7.55, 7.10 [11:19:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.31, 8.28, 7.43 [11:23:31] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.06, 7.34, 7.26 [11:40:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.59, 6.22, 6.66 [11:50:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.45, 7.14, 6.87 [11:52:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.08, 6.51, 6.68 [12:10:48] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.62, 3.18, 1.81 [12:12:48] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.69, 2.20, 1.61 [13:02:43] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/4726f498e655...bc0974ceb147 [13:02:46] [02miraheze/RottenLinks] 07paladox 03bc0974c - Update SpecialRottenLinks.php [13:02:48] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:02:57] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/bc0974ceb147...3ce99b2e755e [13:02:58] [02miraheze/RottenLinks] 07paladox 033ce99b2 - Update de.json [13:03:01] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:03:08] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/3ce99b2e755e...49dc0cdbac27 [13:03:10] [02miraheze/RottenLinks] 07paladox 0349dc0cd - Update en.json [13:03:11] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:03:30] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/49dc0cdbac27...bf4ede1b5c35 [13:03:32] [02miraheze/RottenLinks] 07paladox 03bf4ede1 - Update fi.json [13:03:34] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:03:41] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/bf4ede1b5c35...e86323d82495 [13:03:44] [02miraheze/RottenLinks] 07paladox 03e86323d - Update fr.json [13:03:46] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:03:54] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/e86323d82495...1f80cdcbaa9a [13:03:56] [02miraheze/RottenLinks] 07paladox 031f80cdc - Update he.json [13:03:58] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:04:07] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/1f80cdcbaa9a...a62ddd3e51db [13:04:10] [02miraheze/RottenLinks] 07paladox 03a62ddd3 - Update hu.json [13:04:13] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:04:18] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/a62ddd3e51db...f0d8e545ce02 [13:04:19] [02miraheze/RottenLinks] 07paladox 03f0d8e54 - Update ia.json [13:04:22] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:04:34] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/f0d8e545ce02...c4639f7ab000 [13:04:35] [02miraheze/RottenLinks] 07paladox 03c4639f7 - Update ja.json [13:04:36] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:04:54] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/c4639f7ab000...217d25f3f3e1 [13:04:55] [02miraheze/RottenLinks] 07paladox 03217d25f - Update lt.json [13:04:56] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:05:05] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/217d25f3f3e1...04b07c433912 [13:05:07] [02miraheze/RottenLinks] 07paladox 0304b07c4 - Update mk.json [13:05:10] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:05:23] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/04b07c433912...6bb8a4ac3b71 [13:05:26] [02miraheze/RottenLinks] 07paladox 036bb8a4a - Update nl.json [13:05:27] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:05:32] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/6bb8a4ac3b71...ebc35fcd9287 [13:05:33] [02miraheze/RottenLinks] 07paladox 03ebc35fc - Update pl.json [13:05:34] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:05:44] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/ebc35fcd9287...2acd51a39ae0 [13:05:47] [02miraheze/RottenLinks] 07paladox 032acd51a - Update pt-br.json [13:05:48] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:05:59] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/2acd51a39ae0...3a440a0f9af5 [13:06:02] [02miraheze/RottenLinks] 07paladox 033a440a0 - Update pt.json [13:06:03] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:07:36] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/3a440a0f9af5...d6975c47d08b [13:07:38] [02miraheze/RottenLinks] 07paladox 03d6975c4 - Update qqq.json [13:07:41] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:07:48] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/d6975c47d08b...f34a14d2fc36 [13:07:49] [02miraheze/RottenLinks] 07paladox 03f34a14d - Update qqq.json [13:07:50] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:07:59] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/f34a14d2fc36...a6028b7399cc [13:08:00] [02miraheze/RottenLinks] 07paladox 03a6028b7 - Update ru.json [13:08:02] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:08:19] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/a6028b7399cc...b9b846f73863 [13:08:21] [02miraheze/RottenLinks] 07paladox 03b9b846f - Update sh-cyrl.json [13:08:23] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:08:30] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/b9b846f73863...a4f0846e4e8e [13:08:31] [02miraheze/RottenLinks] 07paladox 03a4f0846 - Update sh-latn.json [13:08:32] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:08:47] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/a4f0846e4e8e...f99ccc91ed04 [13:08:50] [02miraheze/RottenLinks] 07paladox 03f99ccc9 - Update sl.json [13:08:53] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:09:00] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/f99ccc91ed04...d9edeeb6580f [13:09:02] [02miraheze/RottenLinks] 07paladox 03d9edeeb - Update sv.json [13:09:05] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:09:20] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/d9edeeb6580f...5fa8ab2127a3 [13:09:23] [02miraheze/RottenLinks] 07paladox 035fa8ab2 - Update tr.json [13:09:26] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:09:31] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/5fa8ab2127a3...12d4af7685d0 [13:09:34] [02miraheze/RottenLinks] 07paladox 0312d4af7 - Update zh-hans.json [13:09:37] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:09:42] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/12d4af7685d0...b5280a201611 [13:09:44] [02miraheze/RottenLinks] 07paladox 03b5280a2 - Update zh-hant.json [13:09:45] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:11:03] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/b5280a201611...513610bcf634 [13:11:04] [02miraheze/RottenLinks] 07paladox 03513610b - Update updateExternalLinks.php [13:11:06] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:12:35] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:12:36] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/513610bcf634...e4e2dd04f407 [13:12:39] [02miraheze/RottenLinks] 07paladox 03e4e2dd0 - Update CHANGELOG.md [13:12:55] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:12:56] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/e4e2dd04f407...454e260389fb [13:12:57] [02miraheze/RottenLinks] 07paladox 03454e260 - Update extension.json [13:18:16] miraheze/RottenLinks - paladox the build passed. [13:24:32] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [13:24:33] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/454e260389fb...38a3cd179983 [13:24:34] [02miraheze/RottenLinks] 07paladox 0338a3cd1 - Update RottenLinksPager.php [13:29:55] miraheze/RottenLinks - paladox the build passed. [13:41:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 7.37, 6.79 [13:51:03] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.77, 6.33, 6.66 [13:51:51] PROBLEM - ns1 NTP time on ns1 is WARNING: NTP WARNING: Offset 0.1015518606 secs [14:01:20] [02RottenLinks] 07paladox synchronize pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [14:01:21] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://github.com/miraheze/RottenLinks/compare/38a3cd179983...d6907f8bce0d [14:01:24] [02miraheze/RottenLinks] 07paladox 03d6907f8 - Update RottenLinksJob.php [14:06:45] miraheze/RottenLinks - paladox the build passed. [14:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 12 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [14:12:21] [02RottenLinks] 07paladox closed pull request 03#73: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/miraheze/RottenLinks/pull/73 [14:12:22] [02miraheze/RottenLinks] 07paladox pushed 031 commit to 03master [+2/-0/±32] 13https://github.com/miraheze/RottenLinks/compare/56e4017c0af0...a005bff89d62 [14:12:23] [02miraheze/RottenLinks] 07paladox 03a005bff - Redesign RottenLinks to not depend on a maintenance script (#73) [14:12:25] [02miraheze/RottenLinks] 07paladox deleted branch 03paladox-patch-1 [14:12:27] [02RottenLinks] 07paladox deleted branch 03paladox-patch-1 - 13https://github.com/miraheze/RottenLinks [14:13:03] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/a09f587a910a...d56a2503a289 [14:13:05] [02miraheze/mediawiki] 07paladox 03d56a250 - Update RottenLinks [14:13:36] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [14:13:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:14:34] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_41 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/86d7f83e084a...90dc04f53c20 [14:14:37] [02miraheze/mediawiki] 07paladox 0390dc04f - Update RottenLinks [14:15:24] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [14:15:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:15:48] PROBLEM - puppetdb121 Puppet on puppetdb121 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppetdb] [14:18:39] miraheze/RottenLinks - paladox the build passed. [14:20:06] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.95, 8.08, 6.94 [14:22:57] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.86, 7.32, 6.54 [14:23:55] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.76, 7.92, 7.12 [14:24:57] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.58, 6.65, 6.38 [14:32:04] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 1000s [14:32:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:37:16] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.48, 6.03, 6.58 [14:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [14:39:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 14.05, 10.13, 6.66 [14:40:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-15 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/132a4626c37c [14:40:09] [02miraheze/puppet] 07paladox 03132a462 - runner: purge rottenlinks cron [14:40:11] [02puppet] 07paladox created branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [14:40:12] [02puppet] 07paladox opened pull request 03#3569: runner: purge rottenlinks cron - 13https://github.com/miraheze/puppet/pull/3569 [14:40:18] [02puppet] 07paladox closed pull request 03#3569: runner: purge rottenlinks cron - 13https://github.com/miraheze/puppet/pull/3569 [14:40:19] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0b3109da3781...2b99cb839008 [14:40:20] [02miraheze/puppet] 07paladox 032b99cb8 - runner: purge rottenlinks cron (#3569) [14:40:21] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-15 [14:40:23] [02puppet] 07paladox deleted branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [14:41:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 11.29, 10.74, 7.33 [14:42:21] PROBLEM - mw133 Current Load on mw133 is CRITICAL: LOAD CRITICAL - total load average: 13.02, 10.85, 7.24 [14:42:59] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.84, 6.79, 6.76 [14:44:21] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 9.12, 9.98, 7.35 [14:44:53] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.44, 6.67, 6.72 [14:44:58] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.84, 10.69, 7.32 [14:45:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 8.39, 9.62, 7.68 [14:48:57] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 11.05, 11.65, 8.55 [14:49:38] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 5.38, 4.51, 2.83 [14:49:46] PROBLEM - mw132 Current Load on mw132 is CRITICAL: LOAD CRITICAL - total load average: 16.48, 12.18, 9.07 [14:50:21] PROBLEM - mw133 Current Load on mw133 is WARNING: LOAD WARNING - total load average: 11.61, 11.02, 8.56 [14:50:57] PROBLEM - mw134 Current Load on mw134 is CRITICAL: LOAD CRITICAL - total load average: 12.58, 12.60, 9.31 [14:51:36] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 1.85, 3.64, 2.72 [14:51:46] PROBLEM - mw132 Current Load on mw132 is WARNING: LOAD WARNING - total load average: 9.62, 11.58, 9.27 [14:52:21] RECOVERY - mw133 Current Load on mw133 is OK: LOAD OK - total load average: 5.34, 9.31, 8.27 [14:52:57] PROBLEM - mw134 Current Load on mw134 is WARNING: LOAD WARNING - total load average: 6.15, 10.29, 8.87 [14:53:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.72, 6.80, 6.82 [14:53:35] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 1.11, 2.76, 2.51 [14:53:46] RECOVERY - mw132 Current Load on mw132 is OK: LOAD OK - total load average: 4.67, 9.02, 8.61 [14:54:57] RECOVERY - mw134 Current Load on mw134 is OK: LOAD OK - total load average: 5.69, 8.66, 8.44 [14:56:51] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.35, 3.70, 2.66 [15:00:10] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 9.33, 10.40, 7.59 [15:01:09] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.07, 6.55, 6.78 [15:02:10] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 8.98, 9.85, 7.73 [15:06:25] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: LOAD WARNING - total load average: 11.32, 9.81, 8.36 [15:07:46] RECOVERY - puppetdb121 Puppet on puppetdb121 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [15:10:10] PROBLEM - mw141 Current Load on mw141 is WARNING: LOAD WARNING - total load average: 10.64, 10.38, 8.76 [15:10:26] RECOVERY - mwtask141 Current Load on mwtask141 is OK: LOAD OK - total load average: 9.68, 9.75, 8.68 [15:11:38] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.24, 6.78, 6.75 [15:12:11] RECOVERY - mw141 Current Load on mw141 is OK: LOAD OK - total load average: 7.68, 9.62, 8.69 [15:13:33] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.75, 6.45, 6.63 [15:14:25] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: LOAD WARNING - total load average: 10.22, 10.46, 9.26 [15:15:40] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [15:16:25] RECOVERY - mwtask141 Current Load on mwtask141 is OK: LOAD OK - total load average: 8.48, 9.60, 9.09 [15:16:48] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 1.79, 2.83, 3.76 [15:20:48] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.80, 1.84, 3.17 [15:20:59] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-3 [+0/-0/±1] 13https://github.com/miraheze/DataDump/commit/e37024e262ba [15:21:01] [02miraheze/DataDump] 07paladox 03e37024e - insertMissingDumps: Use dumps_status column [15:21:03] [02DataDump] 07paladox created branch 03paladox-patch-3 - 13https://github.com/miraheze/DataDump [15:21:05] [02DataDump] 07paladox opened pull request 03#71: insertMissingDumps: Use dumps_status column - 13https://github.com/miraheze/DataDump/pull/71 [15:23:03] [02DataDump] 07paladox closed pull request 03#71: insertMissingDumps: Use dumps_status column - 13https://github.com/miraheze/DataDump/pull/71 [15:23:06] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/f54c4f72e86b...718d81c8418a [15:23:07] [02miraheze/DataDump] 07paladox 03718d81c - insertMissingDumps: Use dumps_status column (#71) [15:23:49] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d56a2503a289...dc21abf84fcd [15:23:50] [02miraheze/mediawiki] 07paladox 03dc21abf - Update DataDump [15:26:56] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/DataDump/commit/5639adad9b66 [15:26:57] [02miraheze/DataDump] 07paladox 035639ada - Add drop dumps_failed and dumps_completed sql patches [15:26:58] [02DataDump] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/DataDump [15:26:59] [02DataDump] 07paladox opened pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:27:34] miraheze/DataDump - paladox the build passed. [15:28:00] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/DataDump/compare/5639adad9b66...df329fd5f503 [15:28:02] [02miraheze/DataDump] 07paladox 03df329fd - Create patch-drop-dumps_completed.sql [15:28:05] [02DataDump] 07paladox synchronize pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:28:15] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/df329fd5f503...a57aaa51d8e7 [15:28:16] [02miraheze/DataDump] 07paladox 03a57aaa5 - Update DataDumpHooks.php [15:28:18] [02DataDump] 07paladox synchronize pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:28:27] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/a57aaa51d8e7...649631cf665d [15:28:27] miraheze/DataDump - paladox the build passed. [15:28:28] [02miraheze/DataDump] 07paladox 03649631c - Update patch-drop-dumps_completed.sql [15:28:31] [02DataDump] 07paladox synchronize pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:28:50] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 7.39, 4.32, 3.63 [15:29:00] [02miraheze/DataDump] 07paladox pushed 031 commit to 03paladox-patch-4 [+1/-0/±0] 13https://github.com/miraheze/DataDump/compare/649631cf665d...61c7552546f5 [15:29:01] [02miraheze/DataDump] 07paladox 0361c7552 - Create patch-drop-dumps_failed.sql [15:29:03] [02DataDump] 07paladox synchronize pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:29:19] PROBLEM - os141 Puppet on os141 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [15:30:29] PROBLEM - mw142 Current Load on mw142 is WARNING: LOAD WARNING - total load average: 10.52, 8.97, 7.24 [15:31:22] [02DataDump] 07paladox closed pull request 03#72: Add drop dumps_failed and dumps_completed sql patches - 13https://github.com/miraheze/DataDump/pull/72 [15:31:24] [02miraheze/DataDump] 07paladox pushed 031 commit to 03master [+2/-0/±1] 13https://github.com/miraheze/DataDump/compare/718d81c8418a...a77bbdb1cfcf [15:31:26] [02miraheze/DataDump] 07paladox 03a77bbdb - Add drop dumps_failed and dumps_completed sql patches (#72) [15:31:34] RECOVERY - os141 Puppet on os141 is OK: OK: Puppet is currently enabled, last run 18 minutes ago with 0 failures [15:31:54] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_41 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/90dc04f53c20...928e2ac16888 [15:31:57] [02miraheze/mediawiki] 07paladox 03928e2ac - Update DataDump [15:32:29] RECOVERY - mw142 Current Load on mw142 is OK: LOAD OK - total load average: 6.04, 7.73, 6.99 [15:32:33] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/dc21abf84fcd...0b74e4f687d3 [15:32:35] [02miraheze/mediawiki] 07paladox 030b74e4f - Update DataDump [15:32:48] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.27, 3.66, 3.57 [15:32:59] !log [paladox@test131] starting deploy of {'world': True, 'l10n': True} to all [15:33:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:33:17] !log [paladox@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 4779s [15:33:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:33:56] !log [paladox@mwtask141] starting deploy of {'world': True, 'l10n': True} to all [15:34:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:34:25] miraheze/DataDump - paladox the build passed. [15:34:48] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.24, 3.16, 3.40 [15:35:47] !log [paladox@test131] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 168s [15:35:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:47] miraheze/DataDump - paladox the build passed. [15:37:30] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.41, 2.86, 2.18 [15:39:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wows.wiki' expires in 12 day(s) (Sat 30 Dec 2023 06:14:45 GMT +0000). [15:39:30] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.81, 2.41, 2.10 [15:43:18] [02miraheze/DataDump] 07paladox deleted branch 03paladox-patch-3 [15:43:19] [02DataDump] 07paladox deleted branch 03paladox-patch-3 - 13https://github.com/miraheze/DataDump [15:46:39] !log [paladox@mwtask141] finished deploy of {'world': True, 'l10n': True} to all - SUCCESS in 762s [15:46:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:57:03] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.92, 6.68, 6.39 [16:04:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.02, 6.37, 6.37 [16:06:02] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.46, 3.02, 2.07 [16:08:02] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 3.30, 3.03, 2.19 [16:09:08] PROBLEM - wows.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address wows.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [16:11:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-15 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/cf5cfd9573e3 [16:11:52] [02miraheze/puppet] 07paladox 03cf5cfd9 - prometheus-jobqueue-stats: add RottenLinksJob to list [16:11:54] [02puppet] 07paladox created branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [16:11:56] [02puppet] 07paladox opened pull request 03#3570: prometheus-jobqueue-stats: add RottenLinksJob to list - 13https://github.com/miraheze/puppet/pull/3570 [16:12:13] [02puppet] 07paladox synchronize pull request 03#3570: prometheus-jobqueue-stats: add RottenLinksJob to list - 13https://github.com/miraheze/puppet/pull/3570 [16:12:16] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-15 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cf5cfd9573e3...d68bce106edc [16:12:17] [02miraheze/puppet] 07paladox 03d68bce1 - Update prometheus-jobqueue-stats.py.erb [16:12:41] [02puppet] 07paladox closed pull request 03#3570: prometheus-jobqueue-stats: add RottenLinksJob to list - 13https://github.com/miraheze/puppet/pull/3570 [16:12:44] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2b99cb839008...622aed83a304 [16:12:46] [02miraheze/puppet] 07paladox 03622aed8 - prometheus-jobqueue-stats: add RottenLinksJob to list (#3570) [16:12:48] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-15 [16:12:49] [02puppet] 07paladox deleted branch 03paladox-patch-15 - 13https://github.com/miraheze/puppet [16:13:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.78, 7.40, 6.76 [16:15:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.37, 7.43, 6.85 [16:17:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.65, 7.64, 6.98 [16:19:16] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.06, 7.27, 6.91 [16:24:58] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 8.03, 7.37 [16:26:12] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.01, 7.15, 6.54 [16:28:09] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.03, 6.30, 6.29 [16:30:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.95, 7.97, 7.58 [16:44:48] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 5.02, 3.05, 1.86 [16:46:48] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.46, 2.41, 1.78 [16:48:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.88, 7.77, 7.34 [16:52:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.10, 7.35, 7.31 [17:03:47] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.43, 7.13, 6.67 [17:05:44] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.49, 6.68, 6.55 [17:08:37] [02miraheze/ManageWiki] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/commit/aa4ca2ba2409 [17:08:38] [02miraheze/ManageWiki] 07paladox 03aa4ca2b - Update ManageWikiTypes: Fix "Unsupported operand types: string * int" for type integer [17:08:41] [02ManageWiki] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/ManageWiki [17:08:42] [02ManageWiki] 07paladox opened pull request 03#437: Update ManageWikiTypes: Fix "Unsupported operand types: string * int"… - 13https://github.com/miraheze/ManageWiki/pull/437 [17:08:48] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.05, 7.41, 7.16 [17:08:49] [02ManageWiki] 07paladox edited pull request 03#437: Update ManageWikiTypes: Fix "Unsupported operand types: string * int" for type integer - 13https://github.com/miraheze/ManageWiki/pull/437 [17:08:52]