[00:03:55] PROBLEM - cp37 HTTP 4xx/5xx ERROR Rate on cp37 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:05:27] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.02, 21.26, 19.97 [00:21:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.88, 23.74, 22.94 [00:35:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.81, 23.11, 22.40 [00:37:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.87, 21.83, 22.09 [00:47:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.46, 17.23, 19.74 [00:55:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.28, 19.82, 19.64 [00:59:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.22, 19.54, 19.63 [01:08:14] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.03, 21.42, 20.49 [01:15:57] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.98, 19.99, 20.29 [01:33:21] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.62, 20.61, 19.47 [01:35:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.33, 20.03, 19.41 [01:39:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.58, 21.27, 20.21 [01:49:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.49, 20.03, 20.23 [01:53:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.28, 21.89, 21.08 [01:59:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 28.17, 23.87, 22.03 [02:01:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.98, 22.66, 21.82 [02:09:02] [02mw-config] 07BlankEclair opened pull request 03#5599: T12286: Add $wgRSSAllowLinkTag and $wgRSSItemMaxLength - 13https://github.com/miraheze/mw-config/pull/5599 [02:09:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.13, 23.59, 22.40 [02:10:02] miraheze/mw-config - BlankEclair the build passed. [02:11:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.34, 23.14, 22.39 [02:11:25] PROBLEM - www.mwcosmos.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mwcosmos.com' expires in 15 day(s) (Mon 22 Jul 2024 01:54:35 AM GMT +0000). [02:13:41] [02mediawiki-repos] 07BlankEclair opened pull request 03#25: T12262: Add TimezoneConverter - 13https://github.com/miraheze/mediawiki-repos/pull/25 [02:13:45] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request 03#25: T12262: Add TimezoneConverter - 13https://github.com/miraheze/mediawiki-repos/pull/25#issuecomment-2211587440 [02:14:27] miraheze/mediawiki-repos - BlankEclair the build has errored. [02:17:18] [02mediawiki-repos] 07BlankEclair synchronize pull request 03#25: T12262: Add TimezoneConverter - 13https://github.com/miraheze/mediawiki-repos/pull/25 [02:20:54] [02mw-config] 07BlankEclair opened pull request 03#5600: T12262: Add TimezoneConverter - 13https://github.com/miraheze/mw-config/pull/5600 [02:21:49] miraheze/mw-config - BlankEclair the build passed. [02:22:12] PROBLEM - mwcosmos.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mwcosmos.com' expires in 15 day(s) (Mon 22 Jul 2024 01:54:35 AM GMT +0000). [02:22:29] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/41acdc69ff48...74438c5d33d0 [02:22:32] [02ssl] 07WikiTideSSLBot 0374438c5 - Bot: Update SSL cert for mwcosmos.com [02:25:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 28.91, 23.56, 22.31 [02:31:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.64, 23.92, 23.04 [02:33:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.70, 24.78, 23.44 [02:51:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.63, 22.58, 23.81 [02:51:24] RECOVERY - mwcosmos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mwcosmos.com' will expire on Fri 04 Oct 2024 01:22:23 AM GMT +0000. [02:55:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.86, 22.95, 23.53 [03:01:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.97, 22.69, 23.41 [03:05:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.81, 22.47, 23.07 [03:09:25] RECOVERY - www.mwcosmos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mwcosmos.com' will expire on Fri 04 Oct 2024 01:22:23 AM GMT +0000. [03:15:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.75, 23.95, 23.91 [03:17:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.80, 24.50, 24.10 [03:47:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.38, 21.12, 23.50 [03:57:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 12.82, 16.50, 20.20 [03:57:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [04:11:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.46, 18.51, 18.11 [04:17:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.69, 19.81, 18.99 [04:25:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.34, 20.95, 19.76 [04:32:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [04:41:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.80, 21.52, 20.82 [04:43:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.81, 21.93, 21.07 [04:53:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.66, 19.00, 20.26 [05:05:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.02, 22.01, 21.02 [05:07:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.36, 21.93, 21.11 [05:15:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.51, 19.22, 20.16 [05:33:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.38, 21.66, 20.70 [05:37:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.48, 21.79, 21.04 [05:39:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.14, 22.33, 21.27 [05:41:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.56, 22.52, 21.47 [05:51:21] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.70, 19.41, 20.36 [06:05:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.24, 20.01, 19.39 [06:07:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.48, 20.31, 19.57 [06:11:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.58, 21.90, 20.34 [06:15:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.46, 22.07, 20.86 [06:23:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.93, 23.18, 21.70 [06:31:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.71, 23.38, 22.68 [06:45:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.60, 21.90, 21.72 [06:47:21] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.36, 21.64, 21.67 [06:51:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.18, 18.00, 20.13 [07:49:48] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.52, 19.32, 18.05 [08:05:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.21, 18.73, 19.21 [10:17:24] PROBLEM - cloud17 Puppet on cloud17 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [10:39:57] PROBLEM - cheeseepedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'cheeseepedia.org' expires in 15 day(s) (Mon 22 Jul 2024 10:31:56 AM GMT +0000). [10:40:10] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/74438c5d33d0...1256e5d308d7 [10:40:11] [02ssl] 07WikiTideSSLBot 031256e5d - Bot: Update SSL cert for cheeseepedia.org [10:43:51] PROBLEM - db151 Backups SQL on db151 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864221 seconds old and 134446 bytes [10:45:10] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [10:45:24] RECOVERY - cloud17 Puppet on cloud17 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [10:59:14] [02puppet] 07redbluegreenhat closed pull request 03#3876: mediawiki: prevent mwdeploy from upgrading extensions configured to checkout a specific commit - 13https://github.com/miraheze/puppet/pull/3876 [11:15:17] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:37:57] RECOVERY - cheeseepedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'cheeseepedia.org' will expire on Fri 04 Oct 2024 09:40:03 AM GMT +0000. [11:45:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.71, 20.17, 18.47 [11:51:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.38, 22.07, 19.74 [11:53:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.64, 21.34, 19.77 [12:01:45] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/VoteNY'} to test151 [12:01:46] !log [alex@test151] finished deploy of {'folders': '1.42/extensions/VoteNY'} to test151 - SUCCESS in 0s [12:01:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:02:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:02:56] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/AjaxPoll'} to test151 [12:03:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:07] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/AJAXPoll'} to test151 [12:03:08] !log [alex@test151] finished deploy of {'folders': '1.42/extensions/AJAXPoll'} to test151 - SUCCESS in 0s [12:03:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:18] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/Comments'} to test151 [12:03:19] !log [alex@test151] finished deploy of {'folders': '1.42/extensions/Comments'} to test151 - SUCCESS in 0s [12:03:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:33] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/QuizGame'} to test151 [12:03:33] !log [alex@test151] finished deploy of {'folders': '1.42/extensions/QuizGame'} to test151 - SUCCESS in 0s [12:03:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:44] !log [alex@test151] starting deploy of {'folders': '1.42/extensions/WikiForum'} to test151 [12:03:45] !log [alex@test151] finished deploy of {'folders': '1.42/extensions/WikiForum'} to test151 - SUCCESS in 0s [12:03:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:03:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:04:35] !log [alex@test151] starting deploy of {'world': True, 'versions': '1.42'} to test151 [12:04:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:04:56] !log [alex@test151] finished deploy of {'world': True, 'versions': '1.42'} to test151 - SUCCESS in 21s [12:05:01] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.665 second response time [12:05:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:05:39] !log [alex@test151] starting deploy of {'upgrade_vendor': True, 'versions': '1.42'} to test151 [12:05:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:05:56] !log [alex@test151] finished deploy of {'upgrade_vendor': True, 'versions': '1.42'} to test151 - SUCCESS in 17s [12:06:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:13:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.13, 23.36, 21.79 [12:15:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.48, 21.23, 21.17 [12:21:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.64, 18.38, 20.06 [12:40:14] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [12:45:40] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.60, 18.31, 17.90 [12:47:36] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.45, 17.42, 17.62 [13:10:15] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [14:19:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.20, 19.82, 16.91 [14:21:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.30, 19.28, 17.07 [14:22:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [14:25:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.54, 20.99, 18.24 [14:27:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.77, 19.41, 18.00 [14:48:47] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 28.30, 21.48, 18.92 [14:50:43] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.04, 22.03, 19.48 [14:54:35] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.68, 22.31, 20.07 [14:58:26] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.20, 21.96, 20.46 [15:08:05] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.57, 19.52, 20.17 [15:19:44] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.49, 21.69, 20.62 [15:23:36] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.47, 22.98, 21.34 [15:25:32] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 18.34, 21.57, 21.05 [15:37:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.80, 18.36, 19.87 [15:57:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [16:07:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.66, 21.10, 19.02 [16:09:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.54, 19.79, 18.79 [16:25:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.54, 19.64, 18.57 [16:27:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.66, 18.88, 18.43 [16:59:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.20, 21.24, 19.70 [17:01:23] PROBLEM - lophocmatngu.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lophocmatngu.wiki' expires in 15 day(s) (Mon 22 Jul 2024 04:53:21 PM GMT +0000). [17:01:40] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1256e5d308d7...0fe565c7fe6c [17:01:41] [02ssl] 07WikiTideSSLBot 030fe565c - Bot: Update SSL cert for lophocmatngu.wiki [17:07:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.04, 18.90, 19.50 [17:16:40] PROBLEM - sdiy.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sdiy.info' expires in 15 day(s) (Mon 22 Jul 2024 05:01:38 PM GMT +0000). [17:16:51] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0fe565c7fe6c...de261c410ffc [17:16:54] [02ssl] 07WikiTideSSLBot 03de261c4 - Bot: Update SSL cert for sdiy.info [17:31:19] RECOVERY - lophocmatngu.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'lophocmatngu.wiki' will expire on Fri 04 Oct 2024 04:01:28 PM GMT +0000. [17:46:26] RECOVERY - sdiy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'sdiy.info' will expire on Fri 04 Oct 2024 04:16:45 PM GMT +0000. [17:51:24] PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:53:21] RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.0008661150932 secs [18:12:14] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.46, 20.26, 17.89 [18:14:09] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.77, 18.96, 17.68 [18:47:13] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_dns] [19:11:19] PROBLEM - www.samafia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.samafia.org' expires in 15 day(s) (Mon 22 Jul 2024 07:01:36 PM GMT +0000). [19:11:30] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/de261c410ffc...bc12fa91ee63 [19:11:32] [02ssl] 07WikiTideSSLBot 03bc12fa9 - Bot: Update SSL cert for www.samafia.org [19:15:35] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [19:21:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.18, 20.01, 16.32 [19:23:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.84, 19.50, 16.61 [19:27:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.40, 22.63, 18.56 [19:29:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.20, 21.65, 18.70 [19:33:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.79, 20.05, 18.82 [20:09:19] RECOVERY - www.samafia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.samafia.org' will expire on Fri 04 Oct 2024 06:11:24 PM GMT +0000. [20:12:13] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 18.69, 20.64, 20.17 [20:14:09] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.03, 20.08, 20.04 [20:23:50] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.47, 21.32, 20.49 [20:29:38] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.15, 18.90, 19.96 [20:52:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [22:25:21] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.45, 20.02, 18.16 [22:26:29] PROBLEM - wiki.jill-jimmy.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.jill-jimmy.com' expires in 15 day(s) (Mon 22 Jul 2024 10:18:43 PM GMT +0000). [22:26:42] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bc12fa91ee63...da30525491df [22:26:43] [02ssl] 07WikiTideSSLBot 03da30525 - Bot: Update SSL cert for wiki.jill-jimmy.com [22:31:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.03, 18.55, 18.28 [22:56:23] RECOVERY - wiki.jill-jimmy.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jill-jimmy.com' will expire on Fri 04 Oct 2024 09:26:34 PM GMT +0000. [23:25:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.75, 22.15, 19.74 [23:27:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.63, 19.25, 18.93