[00:00:53] Naleksuh: we don't use update.php, update.php does run it, but even if we did, we would want to before, or Miraheze would be down for months during upgrade to finish running it. [00:01:30] Is it not on a per-wiki basis? [00:01:56] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.96, 1.85 [00:02:22] Naleksuh: it is. I am running it on a loop over all wikis, but it can't be ran on all wikis at once, it is wiki-by-wiki, is why it takes so long. [00:02:33] I meant is upgrading not on a per wiki basis? [00:03:15] Naleksuh: no, all wikis are upgraded at once. [00:03:45] Its a shared file system, we don't support multi versions, it is another thing we eventually want, but low priority right now. [00:04:08] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.81, 1.70, 1.56 [00:04:37] I asked about it once but everyone squawked about how it would break commons and other stuff [00:04:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.09, 3.45, 3.94 [00:06:05] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.90, 1.47, 1.49 [00:06:22] Naleksuh: it probably would if not done right, but multiversion is complex, so doing it will take a long time to get it right, also requiring more disk space on our servers, and very low priority. [00:06:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 3.91, 4.06 [00:07:56] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.95, 1.98, 1.90 [00:08:36] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-7 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/ee7111cf1f76 [00:08:38] [02miraheze/mw-config] 07Universal-Omega 03ee7111c - Remove the `$wgCosmosUseWVUISearch` config [00:08:39] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-7 - 13https://github.com/miraheze/mw-config [00:08:40] [url] Page not found · GitHub · GitHub | github.com [00:08:41] [02mw-config] 07Universal-Omega opened pull request 03#4807: Remove the `$wgCosmosUseWVUISearch` config - 13https://github.com/miraheze/mw-config/pull/4807 [00:08:41] [url] Page not found · GitHub · GitHub | github.com [00:09:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-7 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ee7111cf1f76...0edf49230e3c [00:09:14] [url] Comparing ee7111cf1f76...0edf49230e3c · miraheze/mw-config · GitHub | github.com [00:09:15] [02miraheze/mw-config] 07Universal-Omega 030edf492 - Update ManageWikiSettings.php [00:09:16] [02mw-config] 07Universal-Omega synchronize pull request 03#4807: Remove the `$wgCosmosUseWVUISearch` config - 13https://github.com/miraheze/mw-config/pull/4807 [00:09:17] [url] Page not found · GitHub · GitHub | github.com [00:10:04] miraheze/mw-config - Universal-Omega the build passed. [00:10:25] miraheze/mw-config - Universal-Omega the build passed. [00:12:22] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:30] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 1595 bytes in 0.333 second response time [00:12:34] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:39] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 3.66, 2.27, 1.22 [00:12:41] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:45] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:12:49] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:07] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:07] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:09] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:27] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:13:55] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:14:02] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:14:24] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 2 backends are down. mw121 mw122 [00:14:38] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.08, 1.76, 1.16 [00:14:47] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 3 backends are down. mw102 mw111 mw112 [00:14:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.95, 3.45, 3.93 [00:15:03] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.792 second response time [00:15:23] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 3 backends are down. mw102 mw111 mw121 [00:15:55] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.313 second response time [00:16:22] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.313 second response time [00:16:36] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.356 second response time [00:16:37] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.38, 1.30, 1.07 [00:16:42] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.017 second response time [00:16:43] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.311 second response time [00:16:47] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [00:16:47] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.024 second response time [00:16:51] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.312 second response time [00:17:16] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 6.736 second response time [00:17:18] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 5.229 second response time [00:17:22] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [00:17:34] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.320 second response time [00:17:56] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.37, 1.84, 1.82 [00:18:11] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 1.255 second response time [00:18:24] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [00:18:25] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:18:43] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:18:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.38, 4.65, 4.27 [00:19:28] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:20:19] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.603 second response time [00:20:43] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 1.017 second response time [00:21:32] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.978 second response time [00:23:57] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.96, 1.90 [00:24:33] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:35] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:36] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:48] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:24:56] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:09] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:36] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:46] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:25:51] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [00:27:15] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 7.196 second response time [00:27:38] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.006 second response time [00:28:38] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.481 second response time [00:29:31] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:29:56] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.27, 1.48, 1.70 [00:30:36] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:32:52] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:34:51] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:35:08] PROBLEM - mwtask111 NTP time on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:35:08] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:35:18] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.09, 1.76, 1.23 [00:35:45] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:35:53] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:37:10] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:37:16] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.89, 1.41, 1.16 [00:37:39] PROBLEM - mwtask111 Check Gluster Clients on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:37:40] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:37:42] PROBLEM - mwtask111 JobRunner Service on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:37:58] PROBLEM - mwtask111 Disk Space on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:38:00] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [00:38:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.09, 2.91, 3.88 [00:42:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.31, 3.01, 3.67 [00:42:53] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [00:42:53] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [00:42:54] RECOVERY - mwtask111 Disk Space on mwtask111 is OK: DISK OK - free space: / 30739 MB (46% inode=89%); [00:42:55] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [00:42:55] RECOVERY - mwtask111 JobRunner Service on mwtask111 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [00:42:59] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 5.798 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [00:42:59] RECOVERY - mwtask111 NTP time on mwtask111 is OK: NTP OK: Offset -0.00120139122 secs [00:43:09] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [00:44:19] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:30] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: HTTP CRITICAL - No data received from host [00:44:38] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:45:08] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:45:12] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:46:19] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.105 second response time [00:46:28] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.284 second response time [00:46:29] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 55 minutes ago with 0 failures [00:46:33] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.320 second response time [00:46:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.57, 3.49, 3.97 [00:47:01] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [00:47:10] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 4.694 second response time [00:47:12] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.971 second response time [00:47:50] PROBLEM - cp20 APT on cp20 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [00:47:51] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.854 second response time RECOVERY - cp20 HTTPS on cp20 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3401 bytes in 0.015 second response time [00:47:51] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:47:52] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [00:48:06] RECOVERY - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is OK: OK - NGINX Error Rate is 2% [00:48:30] PROBLEM - cp20 APT on cp20 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [00:48:30] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:48:31] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.670 second response time [00:48:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.91, 3.24, 3.66 [00:48:50] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: HTTP CRITICAL - No data received from host [00:48:51] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 0.693 second response time [00:49:36] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.40, 1.92, 1.75 [00:49:38] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15942 bytes in 0.796 second response time [00:49:43] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15942 bytes in 0.340 second response time [00:50:25] PROBLEM - cp20 Varnish Backends on cp20 is CRITICAL: 1 backends are down. mw111 [00:50:47] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 6.755 second response time [00:50:51] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.013 second response time [00:50:55] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.730 second response time [00:50:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.09, 3.93, 4.00 [00:51:13] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.109 second response time [00:51:16] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.421 second response time [00:52:20] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 14 backends are healthy [00:52:29] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 7.985 second response time [00:52:29] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.246 second response time [00:52:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.28, 3.79, 3.78 [00:52:40] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 5.723 second response time [00:52:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.62, 3.69, 3.90 [00:53:29] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.77, 1.95, 1.81 [00:54:17] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.467 second response time [00:54:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.14, 3.64, 3.84 [00:59:18] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.90, 1.83 [01:00:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.97, 3.85, 3.87 [01:00:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.61, 3.78 [01:01:14] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.96, 1.90, 1.83 [01:03:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.07, 1.98, 1.87 [01:05:07] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.66, 1.90, 1.86 [01:06:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.04, 3.29, 3.55 [01:06:57] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.36, 2.73, 3.33 [01:09:30] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.82, 1.58, 1.23 [01:10:56] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.08, 1.83, 1.83 [01:11:27] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.26, 1.36, 1.19 [01:12:52] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.93, 1.85, 1.83 [01:12:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.67, 3.55 [01:14:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.26, 3.39, 3.45 [01:16:45] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.52, 2.05, 1.91 [01:16:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.92, 3.64 [01:19:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.63, 3.49, 3.94 [01:21:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.92, 4.02, 4.07 [01:22:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.10, 3.80, 3.96 [01:22:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.99, 3.79, 3.73 [01:23:15] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.44, 1.72, 1.47 [01:24:30] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.46, 1.92, 1.96 [01:24:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.13, 4.19, 4.07 [01:24:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 4.01, 3.82 [01:25:12] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.28, 1.58, 1.45 [01:26:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.93, 3.99 [01:27:14] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.02, 1.41, 3.82 [01:28:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.79, 3.77 [01:31:15] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 2.31, 1.72, 3.37 [01:34:48] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:35:08] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:35:13] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:35:15] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:36:11] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:36:52] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:38:46] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:39:07] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:00] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:01] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.44, 1.50, 1.69 [01:40:20] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:32] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.20, 2.63, 3.32 [01:40:34] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:40:45] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [01:41:23] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [01:42:02] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [01:42:48] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [01:43:31] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [01:43:32] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [01:44:52] [02mw-config] 07Universal-Omega closed pull request 03#4798: Add settings for Share extension - 13https://github.com/miraheze/mw-config/pull/4798 [01:44:53] [url] Page not found · GitHub · GitHub | github.com [01:44:54] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/mw-config/compare/11b92183600b...36250f20e39d [01:44:54] [url] Comparing 11b92183600b...36250f20e39d · miraheze/mw-config · GitHub | github.com [01:44:55] [02miraheze/mw-config] 07AgentIsai 0336250f2 - Add settings for Share extension (#4798) [01:44:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.26, 4.12, 3.73 [01:45:22] !log [universalomega@test101] starting deploy of {'pull': 'config', 'config': True} to all [01:45:25] !log [universalomega@test101] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 3s [01:45:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:45:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:45:50] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [01:46:00] miraheze/mw-config - Universal-Omega the build passed. [01:46:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.62, 4.30, 3.81 [01:49:57] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.76, 2.91, 2.20 [01:50:07] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 25 minutes ago with 0 failures [01:52:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.08, 3.88, 3.84 [01:54:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.94, 3.86 [01:54:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.21, 3.66, 3.86 [01:56:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 4.49, 4.15 [01:58:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.20, 3.82, 3.85 [01:59:31] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.89, 1.64, 1.49 [01:59:53] !log [@mwtask111] starting deploy of {'config': True} to all [01:59:56] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 0.28, 1.44, 1.86 [01:59:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:00:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.79, 3.38, 3.80 [02:01:23] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 1 minute ago with 2 failures. Failed resources (up to 3 shown): Service[nginx],Exec[MediaWiki Config Sync] [02:01:28] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.86, 1.34, 1.39 [02:01:56] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 0.13, 0.99, 1.64 [02:10:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.44, 3.72, 3.71 [02:12:57] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.38, 4.15, 3.96 [02:14:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.95, 3.52, 3.75 [02:15:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.25, 3.21, 3.98 [02:15:51] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 1.851 second response time [02:16:01] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.147 second response time [02:16:20] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.013 second response time [02:16:20] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.026 second response time [02:16:24] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.332 second response time [02:16:27] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.321 second response time [02:16:32] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.82, 3.67, 3.75 [02:17:19] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:20:32] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.57, 2.71, 3.35 [02:20:57] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.31, 2.89, 3.40 [02:22:35] !log [universalomega@mwtask111] starting deploy of {'pull': 'config', 'config': True} to all [02:23:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:23:13] !log [universalomega@mwtask111] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 38s [02:23:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:23:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.14, 4.46, 4.20 [02:26:04] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:26:27] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:26:48] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:28:00] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:28:01] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:28:44] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.281 second response time [02:29:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.73, 3.43, 3.89 [02:29:58] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.318 second response time [02:30:00] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.319 second response time [02:30:05] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.287 second response time [02:30:27] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.021 second response time [02:31:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 3.87, 4.00 [02:32:00] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/wiki.infomedia.co.uk.crt] [02:32:11] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:22] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:26] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:32:31] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:33:51] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.74, 3.47, 3.84 [02:34:14] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.247 second response time [02:34:24] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.795 second response time [02:34:30] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.541 second response time [02:34:34] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 4.693 second response time [02:39:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.01, 4.65, 4.11 [02:40:14] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:15] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:25] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:32] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:42] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.06, 4.29, 3.59 [02:40:52] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:40:55] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.68, 4.82, 3.76 [02:40:58] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:41:14] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:41:26] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:41:57] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:14] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [02:43:16] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:31] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.485 second response time [02:43:50] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:50] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:43:59] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:46:10] PROBLEM - cp20 Stunnel HTTP for mw121 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:46:20] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:46:34] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.67, 3.70, 3.68 [02:46:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 1.70, 3.56, 3.62 [02:46:44] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: HTTP CRITICAL - No data received from host [02:47:14] PROBLEM - cp21 Stunnel HTTP for mw121 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:47:18] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:47:33] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:47:49] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:47:55] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:48:02] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:48:03] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.430 second response time [02:48:04] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:48:13] RECOVERY - cp20 Stunnel HTTP for mw121 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.409 second response time [02:48:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.91, 4.20, 3.84 [02:48:31] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:48:47] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.198 second response time [02:49:08] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:49:10] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:49:12] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 5.225 second response time [02:49:17] RECOVERY - cp21 Stunnel HTTP for mw121 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.968 second response time [02:49:22] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 9.389 second response time [02:49:35] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:49:52] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.724 second response time [02:49:55] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.640 second response time [02:49:57] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.018 second response time [02:50:12] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.975 second response time [02:50:31] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.579 second response time [02:50:33] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.520 second response time [02:50:33] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 8.62, 5.39, 4.25 [02:50:41] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.878 second response time [02:50:46] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.021 second response time [02:50:54] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 1.719 second response time [02:51:07] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 2.843 second response time [02:51:09] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 1.445 second response time [02:51:35] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.961 second response time [02:51:42] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 1.690 second response time [02:52:29] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 2.985 second response time [02:53:31] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 5.216 second response time [02:53:45] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 4.781 second response time [02:54:07] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.937 second response time [02:54:18] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:54:19] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 6.871 second response time [02:54:19] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 5.883 second response time [02:54:53] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:07] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:07] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:10] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:21] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:21] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:21] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:29] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:31] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:37] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:55:39] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:56:01] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:56:08] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:56:17] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:57:00] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:57:06] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [02:58:22] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.816 second response time [02:58:26] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 7.775 second response time [02:58:59] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 0.920 second response time [02:59:03] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.467 second response time [02:59:20] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 1.787 second response time [02:59:21] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 2.267 second response time [02:59:58] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:04] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [03:00:06] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.120 second response time [03:00:17] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:27] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:35] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:46] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:00:56] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:01:50] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+1/-0/±0] 13https://github.com/miraheze/ssl/commit/e1b473035014 [03:01:51] [02miraheze/ssl] 07Universal-Omega 03e1b4730 - Add puritwiki.p-e.kr cert [03:01:53] [02ssl] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [03:01:53] [url] Page not found · GitHub · GitHub | github.com [03:01:54] [02ssl] 07Universal-Omega opened pull request 03#559: Add puritwiki.p-e.kr cert - 13https://github.com/miraheze/ssl/pull/559 [03:01:55] ... [03:02:13] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 3.487 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [03:02:30] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: connect to address 2a10:6740::6:208 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [03:02:57] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [03:03:07] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e1b473035014...82aea6a08b78 [03:03:07] [url] Comparing e1b473035014...82aea6a08b78 · miraheze/ssl · GitHub | github.com [03:03:08] [02miraheze/ssl] 07Universal-Omega 0382aea6a - Update certs.yaml [03:03:08] miraheze/ssl - Universal-Omega the build has errored. [03:03:10] [02ssl] 07Universal-Omega synchronize pull request 03#559: Add puritwiki.p-e.kr cert - 13https://github.com/miraheze/ssl/pull/559 [03:03:10] [url] Page not found · GitHub · GitHub | github.com [03:04:14] [02ssl] 07Universal-Omega closed pull request 03#559: Add puritwiki.p-e.kr cert - 13https://github.com/miraheze/ssl/pull/559 [03:04:14] [url] Page not found · GitHub · GitHub | github.com [03:04:15] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/df5607149fa9...585c72e95ae6 [03:04:16] [url] Comparing df5607149fa9...585c72e95ae6 · miraheze/ssl · GitHub | github.com [03:04:17] [02miraheze/ssl] 07Universal-Omega 03585c72e - Add puritwiki.p-e.kr cert (#559) [03:04:18] [02miraheze/ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [03:04:20] [02ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [03:04:20] [url] Page not found · GitHub · GitHub | github.com [03:04:24] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:04:30] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:04:31] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:05:56] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.169 second response time [03:05:56] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 6.005 second response time [03:05:57] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.817 second response time [03:06:21] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.045 second response time [03:06:30] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 7.716 second response time [03:06:37] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.838 second response time [03:06:38] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.907 second response time [03:06:46] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 5.106 second response time [03:06:51] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27107 bytes in 2.091 second response time [03:06:56] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.379 second response time [03:07:08] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.328 second response time [03:07:13] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [03:07:35] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.025 second response time [03:07:36] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.323 second response time [03:07:46] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.023 second response time [03:10:53] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:11:12] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:11:26] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:11:31] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:12:24] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 1 backends are down. mw111 [03:12:44] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:12:46] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:12:51] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:13:09] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:13:09] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:13:34] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:14:37] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.805 second response time [03:14:41] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.845 second response time [03:14:45] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.481 second response time [03:14:59] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:15:03] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 5.686 second response time [03:15:31] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.356 second response time [03:16:24] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [03:16:59] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.636 second response time [03:19:03] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 4.665 second response time [03:19:14] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.020 second response time [03:19:24] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.334 second response time [03:19:25] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.024 second response time [03:19:38] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.86, 1.82, 1.56 [03:19:38] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.403 second response time [03:19:50] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.553 second response time [03:19:50] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.760 second response time [03:19:52] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.002 second response time [03:20:52] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.701 second response time [03:21:18] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.241 second response time [03:21:35] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.23, 1.67, 1.53 [03:22:34] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:22:48] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:23:09] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:23:32] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:23:55] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:24:04] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:24:35] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:24:47] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:24:49] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:24:58] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:25:32] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.59, 1.82, 1.64 [03:25:58] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:26:28] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:27:13] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:27:18] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:27:24] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:27:25] PROBLEM - mwtask111 JobRunner Service on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:27:38] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:27:38] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:27:39] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:27:58] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:30:37] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.330 second response time [03:30:51] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 1.470 second response time [03:30:52] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:30:59] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.602 second response time [03:30:59] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.927 second response time [03:31:12] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 5.398 second response time [03:31:19] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.69, 1.24, 1.45 [03:31:25] PROBLEM - mwtask111 Check Gluster Clients on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:31:26] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [03:32:52] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: connect to address 2a10:6740::6:208 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [03:32:52] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [03:33:40] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [03:33:40] RECOVERY - mwtask111 JobRunner Service on mwtask111 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [03:33:40] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [03:33:43] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [03:35:00] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [03:35:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [03:35:03] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 28 minutes ago with 0 failures [03:36:19] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 0.629 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [03:38:23] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.55, 3.54, 4.00 [03:38:39] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.781 second response time [03:40:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.01, 3.82, 4.04 [03:41:51] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:42:22] PROBLEM - mw122 MediaWiki Rendering on mw122 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:43:10] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:43:22] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:43:55] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.593 second response time [03:44:23] RECOVERY - mw122 MediaWiki Rendering on mw122 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 3.435 second response time [03:44:57] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.820 second response time [03:45:06] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.021 second response time [03:45:08] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.352 second response time [03:45:18] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.329 second response time [03:45:20] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [03:45:30] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.024 second response time [03:45:39] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.033 second response time [03:48:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.42, 3.63, 3.93 [03:50:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.42, 4.00, 4.03 [03:53:39] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:53:53] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:53:58] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:54:03] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:54:12] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [03:55:38] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27107 bytes in 1.943 second response time [03:55:51] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.777 second response time [03:55:54] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.398 second response time [03:56:01] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15942 bytes in 0.067 second response time [03:56:10] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.903 second response time [03:58:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.69, 3.67, 3.91 [04:00:02] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:00:30] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:01:21] PROBLEM - db121 NTP time on db121 is WARNING: NTP WARNING: Offset -0.2081386745 secs [04:02:00] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.538 second response time [04:02:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.82, 3.36, 3.85 [04:02:31] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 4.749 second response time [04:06:23] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:06:51] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:07:27] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:07:28] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset -0.005266517401 secs [04:07:28] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:07:34] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:08:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.04, 2.93, 3.34 [04:10:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [04:10:13] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.300 second response time [04:10:23] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.322 second response time [04:10:33] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:10:53] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 9.664 second response time [04:11:59] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [04:12:32] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 3.344 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [04:13:22] PROBLEM - cp21 Stunnel HTTP for mw122 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:13:29] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:13:30] PROBLEM - cp20 Stunnel HTTP for mw122 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:13:53] PROBLEM - cp31 Stunnel HTTP for mw122 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:14:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.30, 3.76, 3.64 [04:14:31] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:14:40] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:15:15] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:15:25] RECOVERY - cp21 Stunnel HTTP for mw122 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 7.898 second response time [04:15:32] RECOVERY - cp20 Stunnel HTTP for mw122 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.406 second response time [04:15:33] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 5.774 second response time [04:15:54] RECOVERY - cp31 Stunnel HTTP for mw122 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.314 second response time [04:16:14] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:16:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.35, 3.50, 3.39 [04:18:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.61, 3.59 [04:20:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.34, 3.71, 3.50 [04:20:21] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [04:21:11] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27107 bytes in 3.730 second response time [04:22:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.95, 3.52, 3.46 [04:22:24] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 5.136 second response time [04:22:42] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.333 second response time [04:22:47] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.263 second response time [04:22:57] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 7.927 second response time [04:24:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.06, 3.90, 3.70 [04:24:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.32, 4.49, 3.84 [04:25:34] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:26:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.51, 3.59 [04:26:49] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:26:52] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:27:00] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:27:04] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:27:09] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:27:25] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:28:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.65, 3.73, 3.69 [04:28:46] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:28:59] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 3.117 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [04:29:37] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: connect to address 2a10:6740::6:208 and port 443: Connection refusedHTTP CRITICAL - Unable to open TCP socket [04:31:03] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:31:44] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:31:47] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:32:01] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 7.977 second response time [04:32:57] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [04:33:03] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.713 second response time [04:33:06] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 1.210 second response time [04:33:11] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.719 second response time [04:33:45] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.093 second response time [04:33:49] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.024 second response time [04:33:55] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 8.637 second response time [04:34:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.63, 2.62, 3.21 [04:36:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.75, 2.71, 3.26 [04:36:20] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:37:13] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.011 second response time [04:37:21] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:37:39] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:38:17] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:38:25] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:39:22] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:39:50] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:40:52] PROBLEM - mwtask111 Check Gluster Clients on mwtask111 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [04:40:54] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [04:41:04] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [04:41:56] [02mediawiki] 07dependabot[bot] opened pull request 03#7869: Bump extensions/JsonConfig from `5ac98d8` to `efba2ee` - 13https://github.com/miraheze/mediawiki/pull/7869 [04:41:57] [url] Page not found · GitHub · GitHub | github.com [04:41:58] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/JsonConfig-efba2ee [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/3401bbb63843 [04:41:59] [02miraheze/mediawiki] 07dependabot[bot] 033401bbb - Bump extensions/JsonConfig from `5ac98d8` to `efba2ee` [04:42:01] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/TabberNeue-1d26cfd [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/6626dbeff4b6 [04:42:02] [02miraheze/mediawiki] 07dependabot[bot] 036626dbe - Bump extensions/TabberNeue from `6b53029` to `1d26cfd` [04:42:04] [02mediawiki] 07dependabot[bot] commented on pull request 03#7775: Bump extensions/JsonConfig from `5ac98d8` to `45533ef` - 13https://github.com/miraheze/mediawiki/pull/7775#issuecomment-1171928518 [04:42:04] [url] Bump extensions/JsonConfig from `5ac98d8` to `45533ef` by dependabot[bot] · Pull Request #7775 · miraheze/mediawiki · GitHub | github.com [04:42:05] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/JsonConfig-efba2ee - 13https://github.com/miraheze/mediawiki [04:42:05] [url] Page not found · GitHub · GitHub | github.com [04:42:07] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/TabberNeue-1d26cfd - 13https://github.com/miraheze/mediawiki [04:42:07] ... [04:42:08] [02mediawiki] 07dependabot[bot] opened pull request 03#7870: Bump extensions/TabberNeue from `6b53029` to `1d26cfd` - 13https://github.com/miraheze/mediawiki/pull/7870 [04:42:08] ... [04:42:10] [02mediawiki] 07dependabot[bot] closed pull request 03#7775: Bump extensions/JsonConfig from `5ac98d8` to `45533ef` - 13https://github.com/miraheze/mediawiki/pull/7775 [04:42:10] [url] Page not found · GitHub · GitHub | github.com [04:42:11] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/JsonConfig-45533ef [04:42:13] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/JsonConfig-45533ef - 13https://github.com/miraheze/mediawiki [04:42:13] [url] Page not found · GitHub · GitHub | github.com [04:42:14] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Vector-d1ae39c [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/b9a177df7b70 [04:42:16] [02miraheze/mediawiki] 07dependabot[bot] 03b9a177d - Bump skins/Vector from `a7b3595` to `d1ae39c` [04:42:17] [02mediawiki] 07dependabot[bot] commented on pull request 03#7852: Bump extensions/TabberNeue from `6b53029` to `bba0f8b` - 13https://github.com/miraheze/mediawiki/pull/7852#issuecomment-1171928553 [04:42:18] [url] Bump extensions/TabberNeue from `6b53029` to `bba0f8b` by dependabot[bot] · Pull Request #7852 · miraheze/mediawiki · GitHub | github.com [04:42:19] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Vector-d1ae39c - 13https://github.com/miraheze/mediawiki [04:42:19] [url] Page not found · GitHub · GitHub | github.com [04:42:20] [02mediawiki] 07dependabot[bot] opened pull request 03#7871: Bump skins/Vector from `a7b3595` to `d1ae39c` - 13https://github.com/miraheze/mediawiki/pull/7871 [04:42:21] [url] Page not found · GitHub · GitHub | github.com [04:42:22] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TabberNeue-bba0f8b [04:42:23] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TabberNeue-bba0f8b - 13https://github.com/miraheze/mediawiki [04:42:23] ... [04:42:25] [02mediawiki] 07dependabot[bot] closed pull request 03#7852: Bump extensions/TabberNeue from `6b53029` to `bba0f8b` - 13https://github.com/miraheze/mediawiki/pull/7852 [04:42:25] [url] Page not found · GitHub · GitHub | github.com [04:42:26] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CentralNotice-cfc4f85 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/2f764455ffff [04:42:28] [02miraheze/mediawiki] 07dependabot[bot] 032f76445 - Bump extensions/CentralNotice from `7dc40b8` to `cfc4f85` [04:42:29] [02mediawiki] 07dependabot[bot] opened pull request 03#7872: Bump extensions/CentralNotice from `7dc40b8` to `cfc4f85` - 13https://github.com/miraheze/mediawiki/pull/7872 [04:42:30] ... [04:42:31] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CentralNotice-cfc4f85 - 13https://github.com/miraheze/mediawiki [04:42:31] ... [04:42:32] [02mediawiki] 07dependabot[bot] commented on pull request 03#7779: Bump skins/Vector from `a7b3595` to `ca069b1` - 13https://github.com/miraheze/mediawiki/pull/7779#issuecomment-1171928574 [04:42:33] [url] Bump skins/Vector from `a7b3595` to `ca069b1` by dependabot[bot] · Pull Request #7779 · miraheze/mediawiki · GitHub | github.com [04:42:34] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/Vector-ca069b1 [04:42:35] [02mediawiki] 07dependabot[bot] closed pull request 03#7779: Bump skins/Vector from `a7b3595` to `ca069b1` - 13https://github.com/miraheze/mediawiki/pull/7779 [04:42:36] [url] Page not found · GitHub · GitHub | github.com [04:42:37] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/TimedMediaHandler-079184a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c14ff7587ed2 [04:42:38] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 7.737 second response time [04:42:38] [02miraheze/mediawiki] 07dependabot[bot] 03c14ff75 - Bump extensions/TimedMediaHandler from `d37f60b` to `079184a` [04:42:40] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/Vector-ca069b1 - 13https://github.com/miraheze/mediawiki [04:42:40] [url] Page not found · GitHub · GitHub | github.com [04:42:41] [02mediawiki] 07dependabot[bot] commented on pull request 03#7780: Bump extensions/CentralNotice from `7dc40b8` to `0ddd33f` - 13https://github.com/miraheze/mediawiki/pull/7780#issuecomment-1171928594 [04:42:42] [url] Bump extensions/CentralNotice from `7dc40b8` to `0ddd33f` by dependabot[bot] · Pull Request #7780 · miraheze/mediawiki · GitHub | github.com [04:42:43] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/TimedMediaHandler-079184a - 13https://github.com/miraheze/mediawiki [04:42:43] [url] Page not found · GitHub · GitHub | github.com [04:42:44] [02mediawiki] 07dependabot[bot] opened pull request 03#7873: Bump extensions/TimedMediaHandler from `d37f60b` to `079184a` - 13https://github.com/miraheze/mediawiki/pull/7873 [04:42:45] [url] Page not found · GitHub · GitHub | github.com [04:42:46] [02mediawiki] 07dependabot[bot] closed pull request 03#7780: Bump extensions/CentralNotice from `7dc40b8` to `0ddd33f` - 13https://github.com/miraheze/mediawiki/pull/7780 [04:42:46] [url] Page not found · GitHub · GitHub | github.com [04:42:47] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CentralNotice-0ddd33f [04:42:49] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CentralNotice-0ddd33f - 13https://github.com/miraheze/mediawiki [04:42:49] ... [04:42:50] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/SemanticMediaWiki-da63f63 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/86261fc64d29 [04:42:52] [02miraheze/mediawiki] 07dependabot[bot] 0386261fc - Bump extensions/SemanticMediaWiki from `48d6275` to `da63f63` [04:42:53] [02mediawiki] 07dependabot[bot] commented on pull request 03#7791: Bump extensions/TimedMediaHandler from `d37f60b` to `e303a3a` - 13https://github.com/miraheze/mediawiki/pull/7791#issuecomment-1171928622 [04:42:54] [url] Bump extensions/TimedMediaHandler from `d37f60b` to `e303a3a` by dependabot[bot] · Pull Request #7791 · miraheze/mediawiki · GitHub | github.com [04:42:55] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/SemanticMediaWiki-da63f63 - 13https://github.com/miraheze/mediawiki [04:42:55] ... [04:42:55] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:42:56] [02mediawiki] 07dependabot[bot] closed pull request 03#7791: Bump extensions/TimedMediaHandler from `d37f60b` to `e303a3a` - 13https://github.com/miraheze/mediawiki/pull/7791 [04:42:56] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:42:56] [url] Page not found · GitHub · GitHub | github.com [04:42:58] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TimedMediaHandler-e303a3a [04:42:59] PROBLEM - db121 NTP time on db121 is WARNING: NTP WARNING: Offset -0.1216976047 secs [04:42:59] [02mediawiki] 07dependabot[bot] opened pull request 03#7874: Bump extensions/SemanticMediaWiki from `48d6275` to `da63f63` - 13https://github.com/miraheze/mediawiki/pull/7874 [04:43:00] [url] Page not found · GitHub · GitHub | github.com [04:43:01] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Cargo-80c7a39 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/87648835d354 [04:43:02] [02miraheze/mediawiki] 07dependabot[bot] 038764883 - Bump extensions/Cargo from `876d2fa` to `80c7a39` [04:43:04] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TimedMediaHandler-e303a3a - 13https://github.com/miraheze/mediawiki [04:43:04] ... [04:43:05] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Cargo-80c7a39 - 13https://github.com/miraheze/mediawiki [04:43:05] [url] Page not found · GitHub · GitHub | github.com [04:43:06] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:43:07] [02mediawiki] 07dependabot[bot] opened pull request 03#7875: Bump extensions/Cargo from `876d2fa` to `80c7a39` - 13https://github.com/miraheze/mediawiki/pull/7875 [04:43:07] [url] Page not found · GitHub · GitHub | github.com [04:43:08] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PageTriage-4fe0290 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/6360821e3dac [04:43:09] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [04:43:10] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [04:43:10] [02miraheze/mediawiki] 07dependabot[bot] 036360821 - Bump extensions/PageTriage from `f7c61b6` to `4fe0290` [04:43:11] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PageTriage-4fe0290 - 13https://github.com/miraheze/mediawiki [04:43:11] [url] Page not found · GitHub · GitHub | github.com [04:43:13] [02mediawiki] 07dependabot[bot] opened pull request 03#7876: Bump extensions/PageTriage from `f7c61b6` to `4fe0290` - 13https://github.com/miraheze/mediawiki/pull/7876 [04:43:13] ... [04:43:14] [02mediawiki] 07dependabot[bot] commented on pull request 03#7778: Bump extensions/Cargo from `876d2fa` to `08a007a` - 13https://github.com/miraheze/mediawiki/pull/7778#issuecomment-1171928656 [04:43:14] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.022 second response time [04:43:15] [url] Bump extensions/Cargo from `876d2fa` to `08a007a` by dependabot[bot] · Pull Request #7778 · miraheze/mediawiki · GitHub | github.com [04:43:16] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Cargo-08a007a [04:43:17] [02mediawiki] 07dependabot[bot] closed pull request 03#7778: Bump extensions/Cargo from `876d2fa` to `08a007a` - 13https://github.com/miraheze/mediawiki/pull/7778 [04:43:18] ... [04:43:19] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Wikibase-4a45d49 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/38f617335a65 [04:43:20] [02miraheze/mediawiki] 07dependabot[bot] 0338f6173 - Bump extensions/Wikibase from `53ad322` to `4a45d49` [04:43:22] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Cargo-08a007a - 13https://github.com/miraheze/mediawiki [04:43:22] [url] Page not found · GitHub · GitHub | github.com [04:43:23] PROBLEM - cp30 Stunnel HTTP for mw122 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:43:23] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.020 second response time [04:43:23] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Wikibase-4a45d49 - 13https://github.com/miraheze/mediawiki [04:43:24] [url] Page not found · GitHub · GitHub | github.com [04:43:25] [02mediawiki] 07dependabot[bot] opened pull request 03#7877: Bump extensions/Wikibase from `53ad322` to `4a45d49` - 13https://github.com/miraheze/mediawiki/pull/7877 [04:43:25] ... [04:43:26] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/DynamicPageList3-bbd4923 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/9af02c808b72 [04:43:28] [02miraheze/mediawiki] 07dependabot[bot] 039af02c8 - Bump extensions/DynamicPageList3 from `8e5ae9c` to `bbd4923` [04:43:29] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/DynamicPageList3-bbd4923 - 13https://github.com/miraheze/mediawiki [04:43:29] [url] Page not found · GitHub · GitHub | github.com [04:43:31] [02mediawiki] 07dependabot[bot] commented on pull request 03#7859: Bump extensions/Wikibase from `53ad322` to `d7d2656` - 13https://github.com/miraheze/mediawiki/pull/7859#issuecomment-1171928696 [04:43:31] [url] Bump extensions/Wikibase from `53ad322` to `d7d2656` by dependabot[bot] · Pull Request #7859 · miraheze/mediawiki · GitHub | github.com [04:43:32] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Wikibase-d7d2656 [04:43:34] [02mediawiki] 07dependabot[bot] opened pull request 03#7878: Bump extensions/DynamicPageList3 from `8e5ae9c` to `bbd4923` - 13https://github.com/miraheze/mediawiki/pull/7878 [04:43:34] [url] Page not found · GitHub · GitHub | github.com [04:43:35] [02mediawiki] 07dependabot[bot] closed pull request 03#7859: Bump extensions/Wikibase from `53ad322` to `d7d2656` - 13https://github.com/miraheze/mediawiki/pull/7859 [04:43:36] [url] Page not found · GitHub · GitHub | github.com [04:43:36] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:43:37] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Wikibase-d7d2656 - 13https://github.com/miraheze/mediawiki [04:43:37] ... [04:43:38] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/MassMessage-3490174 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/5f3e68f5d32b [04:43:40] [02miraheze/mediawiki] 07dependabot[bot] 035f3e68f - Bump extensions/MassMessage from `7a9dd62` to `3490174` [04:43:41] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/MassMessage-3490174 - 13https://github.com/miraheze/mediawiki [04:43:41] ... [04:43:43] [02mediawiki] 07dependabot[bot] commented on pull request 03#7784: Bump extensions/DynamicPageList3 from `8e5ae9c` to `64cf6c5` - 13https://github.com/miraheze/mediawiki/pull/7784#issuecomment-1171928720 [04:43:43] [url] Bump extensions/DynamicPageList3 from `8e5ae9c` to `64cf6c5` by dependabot[bot] · Pull Request #7784 · miraheze/mediawiki · GitHub | github.com [04:43:44] [02mediawiki] 07dependabot[bot] opened pull request 03#7879: Bump extensions/MassMessage from `7a9dd62` to `3490174` - 13https://github.com/miraheze/mediawiki/pull/7879 [04:43:56] [02mediawiki] 07dependabot[bot] opened pull request 03#7880: Bump extensions/UrlShortener from `61f50bd` to `473cad5` - 13https://github.com/miraheze/mediawiki/pull/7880 [04:43:57] [url] Page not found · GitHub · GitHub | github.com [04:43:57] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.323 second response time [04:43:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.05, 3.05, 4.00 [04:43:58] [02mediawiki] 07github-actions[bot] labeled pull request 03#7870: Bump extensions/TabberNeue from `6b53029` to `1d26cfd` - 13https://github.com/miraheze/mediawiki/pull/7870 [04:43:58] [url] Page not found · GitHub · GitHub | github.com [04:43:59] [02mediawiki] 07dependabot[bot] closed pull request 03#7862: Bump extensions/MassMessage from `7a9dd62` to `7926810` - 13https://github.com/miraheze/mediawiki/pull/7862 [04:44:00] ... [04:44:01] [02mediawiki] 07dependabot[bot] commented on pull request 03#7862: Bump extensions/MassMessage from `7a9dd62` to `7926810` - 13https://github.com/miraheze/mediawiki/pull/7862#issuecomment-1171928748 [04:44:01] [url] Bump extensions/MassMessage from `7a9dd62` to `7926810` by dependabot[bot] · Pull Request #7862 · miraheze/mediawiki · GitHub | github.com [04:44:02] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MassMessage-7926810 [04:44:04] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CentralAuth-62327ad [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ed882bd76021 [04:44:05] [02miraheze/mediawiki] 07dependabot[bot] 03ed882bd - Bump extensions/CentralAuth from `6e7a049` to `62327ad` [04:44:07] [02mediawiki] 07github-actions[bot] labeled pull request 03#7870: Bump extensions/TabberNeue from `6b53029` to `1d26cfd` - 13https://github.com/miraheze/mediawiki/pull/7870 [04:44:07] [url] Page not found · GitHub · GitHub | github.com [04:44:08] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MassMessage-7926810 - 13https://github.com/miraheze/mediawiki [04:44:09] [url] Page not found · GitHub · GitHub | github.com [04:44:10] [02mediawiki] 07dependabot[bot] opened pull request 03#7881: Bump extensions/CentralAuth from `6e7a049` to `62327ad` - 13https://github.com/miraheze/mediawiki/pull/7881 [04:44:10] ... [04:44:11] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CentralAuth-62327ad - 13https://github.com/miraheze/mediawiki [04:44:11] ... [04:44:13] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikiEditor-cbb34d5 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/3066491b95cb [04:44:14] [02miraheze/mediawiki] 07dependabot[bot] 033066491 - Bump extensions/WikiEditor from `8f1d9bd` to `cbb34d5` [04:44:15] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:44:16] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikiEditor-cbb34d5 - 13https://github.com/miraheze/mediawiki [04:44:16] [url] Page not found · GitHub · GitHub | github.com [04:44:17] [02mediawiki] 07github-actions[bot] labeled pull request 03#7872: Bump extensions/CentralNotice from `7dc40b8` to `cfc4f85` - 13https://github.com/miraheze/mediawiki/pull/7872 [04:44:18] [url] Page not found · GitHub · GitHub | github.com [04:44:19] [02mediawiki] 07github-actions[bot] labeled pull request 03#7873: Bump extensions/TimedMediaHandler from `d37f60b` to `079184a` - 13https://github.com/miraheze/mediawiki/pull/7873 [04:44:19] [url] Page not found · GitHub · GitHub | github.com [04:44:20] [02mediawiki] 07dependabot[bot] closed pull request 03#7816: Bump extensions/CentralAuth from `6e7a049` to `3a07dd6` - 13https://github.com/miraheze/mediawiki/pull/7816 [04:44:21] ... [04:44:22] [02mediawiki] 07dependabot[bot] opened pull request 03#7882: Bump extensions/WikiEditor from `8f1d9bd` to `cbb34d5` - 13https://github.com/miraheze/mediawiki/pull/7882 [04:44:23] [02mediawiki] 07dependabot[bot] commented on pull request 03#7816: Bump extensions/CentralAuth from `6e7a049` to `3a07dd6` - 13https://github.com/miraheze/mediawiki/pull/7816#issuecomment-1171928783 [04:44:23] [url] Bump extensions/CentralAuth from `6e7a049` to `3a07dd6` by dependabot[bot] · Pull Request #7816 · miraheze/mediawiki · GitHub | github.com [04:44:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7871: Bump skins/Vector from `a7b3595` to `d1ae39c` - 13https://github.com/miraheze/mediawiki/pull/7871 [04:44:25] [url] Page not found · GitHub · GitHub | github.com [04:44:26] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CentralAuth-3a07dd6 [04:44:27] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 2.176 second response time [04:44:28] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CentralAuth-3a07dd6 - 13https://github.com/miraheze/mediawiki [04:44:28] [url] Page not found · GitHub · GitHub | github.com [04:44:29] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Renameuser-f7767b2 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/8c09add1409e [04:44:31] [02miraheze/mediawiki] 07dependabot[bot] 038c09add - Bump extensions/Renameuser from `55ecf26` to `f7767b2` [04:44:32] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Renameuser-f7767b2 - 13https://github.com/miraheze/mediawiki [04:44:32] ... [04:44:34] [02mediawiki] 07dependabot[bot] commented on pull request 03#7827: Bump extensions/WikiEditor from `8f1d9bd` to `850a937` - 13https://github.com/miraheze/mediawiki/pull/7827#issuecomment-1171928805 [04:44:35] [url] Bump extensions/WikiEditor from `8f1d9bd` to `850a937` by dependabot[bot] · Pull Request #7827 · miraheze/mediawiki · GitHub | github.com [04:44:35] [02mediawiki] 07github-actions[bot] labeled pull request 03#7875: Bump extensions/Cargo from `876d2fa` to `80c7a39` - 13https://github.com/miraheze/mediawiki/pull/7875 [04:44:36] [url] Page not found · GitHub · GitHub | github.com [04:44:37] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/WikiEditor-850a937 [04:44:38] [02mediawiki] 07dependabot[bot] opened pull request 03#7883: Bump extensions/Renameuser from `55ecf26` to `f7767b2` - 13https://github.com/miraheze/mediawiki/pull/7883 [04:44:39] [url] Page not found · GitHub · GitHub | github.com [04:44:40] [02mediawiki] 07github-actions[bot] labeled pull request 03#7875: Bump extensions/Cargo from `876d2fa` to `80c7a39` - 13https://github.com/miraheze/mediawiki/pull/7875 [04:44:40] [url] Page not found · GitHub · GitHub | github.com [04:44:41] [02mediawiki] 07dependabot[bot] closed pull request 03#7827: Bump extensions/WikiEditor from `8f1d9bd` to `850a937` - 13https://github.com/miraheze/mediawiki/pull/7827 [04:44:42] ... [04:44:43] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/SpriteSheet-55c5f2a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ab6e12186c50 [04:44:44] [02miraheze/mediawiki] 07dependabot[bot] 03ab6e121 - Bump extensions/SpriteSheet from `e2d996d` to `55c5f2a` [04:44:46] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/WikiEditor-850a937 - 13https://github.com/miraheze/mediawiki [04:44:46] ... [04:44:47] [02mediawiki] 07dependabot[bot] opened pull request 03#7884: Bump extensions/SpriteSheet from `e2d996d` to `55c5f2a` - 13https://github.com/miraheze/mediawiki/pull/7884 [04:44:48] [url] Page not found · GitHub · GitHub | github.com [04:44:49] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/SpriteSheet-55c5f2a - 13https://github.com/miraheze/mediawiki [04:44:49] [url] Page not found · GitHub · GitHub | github.com [04:44:50] [02mediawiki] 07dependabot[bot] commented on pull request 03#7832: Bump extensions/Renameuser from `55ecf26` to `1a60579` - 13https://github.com/miraheze/mediawiki/pull/7832#issuecomment-1171928828 [04:44:51] [url] Bump extensions/Renameuser from `55ecf26` to `1a60579` by dependabot[bot] · Pull Request #7832 · miraheze/mediawiki · GitHub | github.com [04:44:52] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Renameuser-1a60579 [04:44:53] [02mediawiki] 07dependabot[bot] closed pull request 03#7832: Bump extensions/Renameuser from `55ecf26` to `1a60579` - 13https://github.com/miraheze/mediawiki/pull/7832 [04:44:54] ... [04:44:55] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CategoryTree-60f0307 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/24838d91397a [04:44:56] [02miraheze/mediawiki] 07dependabot[bot] 0324838d9 - Bump extensions/CategoryTree from `1e06722` to `60f0307` [04:44:58] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Renameuser-1a60579 - 13https://github.com/miraheze/mediawiki [04:44:58] [url] Page not found · GitHub · GitHub | github.com [04:44:59] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CategoryTree-60f0307 - 13https://github.com/miraheze/mediawiki [04:44:59] [url] Page not found · GitHub · GitHub | github.com [04:45:01] [02mediawiki] 07dependabot[bot] opened pull request 03#7885: Bump extensions/CategoryTree from `1e06722` to `60f0307` - 13https://github.com/miraheze/mediawiki/pull/7885 [04:45:01] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset -0.002906411886 secs [04:45:01] [url] Page not found · GitHub · GitHub | github.com [04:45:02] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/MobileFrontend-611c162 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/19125ac2f9f0 [04:45:04] [02miraheze/mediawiki] 07dependabot[bot] 0319125ac - Bump extensions/MobileFrontend from `88c1c73` to `611c162` [04:45:04] PROBLEM - cp31 Stunnel HTTP for mw121 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:45:05] [02mediawiki] 07github-actions[bot] labeled pull request 03#7876: Bump extensions/PageTriage from `f7c61b6` to `4fe0290` - 13https://github.com/miraheze/mediawiki/pull/7876 [04:45:06] ... [04:45:07] [02mediawiki] 07dependabot[bot] commented on pull request 03#7849: Bump extensions/CategoryTree from `1e06722` to `421fc1c` - 13https://github.com/miraheze/mediawiki/pull/7849#issuecomment-1171928858 [04:45:07] [url] Bump extensions/CategoryTree from `1e06722` to `421fc1c` by dependabot[bot] · Pull Request #7849 · miraheze/mediawiki · GitHub | github.com [04:45:08] [02mediawiki] 07dependabot[bot] opened pull request 03#7886: Bump extensions/MobileFrontend from `88c1c73` to `611c162` - 13https://github.com/miraheze/mediawiki/pull/7886 [04:45:09] [url] Page not found · GitHub · GitHub | github.com [04:45:10] PROBLEM - cp30 Stunnel HTTP for mw121 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:45:10] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/MobileFrontend-611c162 - 13https://github.com/miraheze/mediawiki [04:45:10] [url] Page not found · GitHub · GitHub | github.com [04:45:11] [02mediawiki] 07dependabot[bot] closed pull request 03#7849: Bump extensions/CategoryTree from `1e06722` to `421fc1c` - 13https://github.com/miraheze/mediawiki/pull/7849 [04:45:22] [02mediawiki] 07dependabot[bot] commented on pull request 03#7845: Bump extensions/MobileFrontend from `88c1c73` to `c64840b` - 13https://github.com/miraheze/mediawiki/pull/7845#issuecomment-1171928875 [04:45:22] [url] Bump extensions/MobileFrontend from `88c1c73` to `c64840b` by dependabot[bot] · Pull Request #7845 · miraheze/mediawiki · GitHub | github.com [04:45:23] [02mediawiki] 07github-actions[bot] labeled pull request 03#7874: Bump extensions/SemanticMediaWiki from `48d6275` to `da63f63` - 13https://github.com/miraheze/mediawiki/pull/7874 [04:45:24] [url] Page not found · GitHub · GitHub | github.com [04:45:25] [02mediawiki] 07dependabot[bot] closed pull request 03#7845: Bump extensions/MobileFrontend from `88c1c73` to `c64840b` - 13https://github.com/miraheze/mediawiki/pull/7845 [04:45:25] [url] Page not found · GitHub · GitHub | github.com [04:45:26] RECOVERY - cp30 Stunnel HTTP for mw122 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.755 second response time [04:45:26] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MobileFrontend-c64840b [04:45:28] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CheckUser-b6a15e0 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/12026fc9f71c [04:45:29] [02miraheze/mediawiki] 07dependabot[bot] 0312026fc - Bump extensions/CheckUser from `49b4b6f` to `b6a15e0` [04:45:31] [02mediawiki] 07github-actions[bot] labeled pull request 03#7874: Bump extensions/SemanticMediaWiki from `48d6275` to `da63f63` - 13https://github.com/miraheze/mediawiki/pull/7874 [04:45:31] [url] Page not found · GitHub · GitHub | github.com [04:45:32] [02mediawiki] 07github-actions[bot] labeled pull request 03#7878: Bump extensions/DynamicPageList3 from `8e5ae9c` to `bbd4923` - 13https://github.com/miraheze/mediawiki/pull/7878 [04:45:33] ... [04:45:34] [02mediawiki] 07github-actions[bot] labeled pull request 03#7874: Bump extensions/SemanticMediaWiki from `48d6275` to `da63f63` - 13https://github.com/miraheze/mediawiki/pull/7874 [04:45:34] ... [04:45:35] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [04:45:35] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MobileFrontend-c64840b - 13https://github.com/miraheze/mediawiki [04:45:36] [url] Page not found · GitHub · GitHub | github.com [04:45:37] [02mediawiki] 07dependabot[bot] opened pull request 03#7888: Bump extensions/CheckUser from `49b4b6f` to `b6a15e0` - 13https://github.com/miraheze/mediawiki/pull/7888 [04:45:37] [url] Page not found · GitHub · GitHub | github.com [04:45:38] [02mediawiki] 07dependabot[bot] commented on pull request 03#7818: Bump extensions/Disambiguator from `d4dcfd5` to `dfb90da` - 13https://github.com/miraheze/mediawiki/pull/7818#issuecomment-1171928897 [04:45:39] [url] Bump extensions/Disambiguator from `d4dcfd5` to `dfb90da` by dependabot[bot] · Pull Request #7818 · miraheze/mediawiki · GitHub | github.com [04:45:39] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [04:45:40] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CheckUser-b6a15e0 - 13https://github.com/miraheze/mediawiki [04:45:40] ... [04:45:41] [02mediawiki] 07dependabot[bot] closed pull request 03#7818: Bump extensions/Disambiguator from `d4dcfd5` to `dfb90da` - 13https://github.com/miraheze/mediawiki/pull/7818 [04:45:42] [url] Page not found · GitHub · GitHub | github.com [04:45:43] [02mediawiki] 07github-actions[bot] labeled pull request 03#7878: Bump extensions/DynamicPageList3 from `8e5ae9c` to `bbd4923` - 13https://github.com/miraheze/mediawiki/pull/7878 [04:45:43] [url] Page not found · GitHub · GitHub | github.com [04:45:44] [02mediawiki] 07github-actions[bot] labeled pull request 03#7879: Bump extensions/MassMessage from `7a9dd62` to `3490174` - 13https://github.com/miraheze/mediawiki/pull/7879 [04:45:45] [url] Page not found · GitHub · GitHub | github.com [04:45:46] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Disambiguator-dfb90da [04:45:47] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikibaseQualityConstraints-c6fbf36 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/eaaadc045d20 [04:45:49] [02miraheze/mediawiki] 07dependabot[bot] 03eaaadc0 - Bump extensions/WikibaseQualityConstraints from `0024a67` to `c6fbf36` [04:45:50] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Disambiguator-dfb90da - 13https://github.com/miraheze/mediawiki [04:45:50] ... [04:45:52] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikibaseQualityConstraints-c6fbf36 - 13https://github.com/miraheze/mediawiki [04:45:52] ... [04:45:53] [02mediawiki] 07dependabot[bot] opened pull request 03#7889: Bump extensions/WikibaseQualityConstraints from `0024a67` to `c6fbf36` - 13https://github.com/miraheze/mediawiki/pull/7889 [04:45:54] [url] Page not found · GitHub · GitHub | github.com [04:45:55] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikiDiscover-0bd18ba [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/76a2a1d3637c [04:45:56] [02miraheze/mediawiki] 07dependabot[bot] 0376a2a1d - Bump extensions/WikiDiscover from `0fcef42` to `0bd18ba` [04:45:58] [02mediawiki] 07dependabot[bot] commented on pull request 03#7803: Bump extensions/CheckUser from `49b4b6f` to `8007a79` - 13https://github.com/miraheze/mediawiki/pull/7803#issuecomment-1171928914 [04:45:58] [url] Bump extensions/CheckUser from `49b4b6f` to `8007a79` by dependabot[bot] · Pull Request #7803 · miraheze/mediawiki · GitHub | github.com [04:45:59] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CheckUser-8007a79 [04:46:01] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikiDiscover-0bd18ba - 13https://github.com/miraheze/mediawiki [04:46:01] [url] Page not found · GitHub · GitHub | github.com [04:46:02] [02mediawiki] 07dependabot[bot] closed pull request 03#7803: Bump extensions/CheckUser from `49b4b6f` to `8007a79` - 13https://github.com/miraheze/mediawiki/pull/7803 [04:46:03] ... [04:46:04] [02mediawiki] 07dependabot[bot] opened pull request 03#7890: Bump extensions/WikiDiscover from `0fcef42` to `0bd18ba` - 13https://github.com/miraheze/mediawiki/pull/7890 [04:46:04] [url] Page not found · GitHub · GitHub | github.com [04:46:05] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CheckUser-8007a79 - 13https://github.com/miraheze/mediawiki [04:46:05] [url] Page not found · GitHub · GitHub | github.com [04:46:07] [02mediawiki] 07dependabot[bot] commented on pull request 03#7804: Bump extensions/WikibaseQualityConstraints from `0024a67` to `438a4a3` - 13https://github.com/miraheze/mediawiki/pull/7804#issuecomment-1171928928 [04:46:07] [url] Bump extensions/WikibaseQualityConstraints from `0024a67` to `438a4a3` by dependabot[bot] · Pull Request #7804 · miraheze/mediawiki · GitHub | github.com [04:46:08] [02mediawiki] 07github-actions[bot] labeled pull request 03#7883: Bump extensions/Renameuser from `55ecf26` to `f7767b2` - 13https://github.com/miraheze/mediawiki/pull/7883 [04:46:09] [url] Page not found · GitHub · GitHub | github.com [04:46:10] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/LoginNotify-d146e97 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/1b80e197b6af [04:46:11] [02miraheze/mediawiki] 07dependabot[bot] 031b80e19 - Bump extensions/LoginNotify from `c968ccb` to `d146e97` [04:46:13] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/WikibaseQualityConstraints-438a4a3 [04:46:14] [02mediawiki] 07dependabot[bot] closed pull request 03#7804: Bump extensions/WikibaseQualityConstraints from `0024a67` to `438a4a3` - 13https://github.com/miraheze/mediawiki/pull/7804 [04:46:15] ... [04:46:16] [02mediawiki] 07github-actions[bot] labeled pull request 03#7880: Bump extensions/UrlShortener from `61f50bd` to `473cad5` - 13https://github.com/miraheze/mediawiki/pull/7880 [04:46:16] [url] Page not found · GitHub · GitHub | github.com [04:46:17] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/LoginNotify-d146e97 - 13https://github.com/miraheze/mediawiki [04:46:18] ... [04:46:19] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/WikibaseQualityConstraints-438a4a3 - 13https://github.com/miraheze/mediawiki [04:46:19] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 7.300 second response time [04:46:19] [url] Page not found · GitHub · GitHub | github.com [04:46:20] [02mediawiki] 07dependabot[bot] opened pull request 03#7891: Bump extensions/LoginNotify from `c968ccb` to `d146e97` - 13https://github.com/miraheze/mediawiki/pull/7891 [04:46:21] ... [04:46:22] [02mediawiki] 07github-actions[bot] labeled pull request 03#7884: Bump extensions/SpriteSheet from `e2d996d` to `55c5f2a` - 13https://github.com/miraheze/mediawiki/pull/7884 [04:46:34] [02miraheze/mediawiki] 07dependabot[bot] 03bac1696 - Bump extensions/QuizGame from `7a1c5f9` to `00b2fea` [04:46:35] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/QuizGame-00b2fea - 13https://github.com/miraheze/mediawiki [04:46:36] [url] Page not found · GitHub · GitHub | github.com [04:46:37] [02mediawiki] 07dependabot[bot] opened pull request 03#7893: Bump extensions/QuizGame from `7a1c5f9` to `00b2fea` - 13https://github.com/miraheze/mediawiki/pull/7893 [04:46:38] [02mediawiki] 07dependabot[bot] closed pull request 03#7843: Bump skins/CologneBlue from `ebbcfbc` to `d8f044c` - 13https://github.com/miraheze/mediawiki/pull/7843 [04:46:40] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Translate-1b89bc6 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c6cad6c66fde [04:46:41] [02miraheze/mediawiki] 07dependabot[bot] 03c6cad6c - Bump extensions/Translate from `508542e` to `1b89bc6` [04:46:43] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/CologneBlue-d8f044c [04:46:44] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Translate-1b89bc6 - 13https://github.com/miraheze/mediawiki [04:46:46] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/CologneBlue-d8f044c - 13https://github.com/miraheze/mediawiki [04:46:47] [02mediawiki] 07dependabot[bot] opened pull request 03#7894: Bump extensions/Translate from `508542e` to `1b89bc6` - 13https://github.com/miraheze/mediawiki/pull/7894 [04:46:49] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ReplaceText-53e9890 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/3ab0d536d995 [04:46:50] [02miraheze/mediawiki] 07dependabot[bot] 033ab0d53 - Bump extensions/ReplaceText from `67bca38` to `53e9890` [04:46:52] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ReplaceText-53e9890 - 13https://github.com/miraheze/mediawiki [04:46:53] [02mediawiki] 07dependabot[bot] opened pull request 03#7895: Bump extensions/ReplaceText from `67bca38` to `53e9890` - 13https://github.com/miraheze/mediawiki/pull/7895 [04:46:55] [02mediawiki] 07github-actions[bot] labeled pull request 03#7877: Bump extensions/Wikibase from `53ad322` to `4a45d49` - 13https://github.com/miraheze/mediawiki/pull/7877 [04:46:56] [02mediawiki] 07dependabot[bot] commented on pull request 03#7788: Bump extensions/Translate from `508542e` to `2910b57` - 13https://github.com/miraheze/mediawiki/pull/7788#issuecomment-1171929020 [04:46:57] [url] Bump extensions/Translate from `508542e` to `2910b57` by dependabot[bot] · Pull Request #7788 · miraheze/mediawiki · GitHub | github.com [04:46:58] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/TemplateData-987cd90 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/e42451d79b50 [04:46:59] [02miraheze/mediawiki] 07dependabot[bot] 03e42451d - Bump extensions/TemplateData from `9f361df` to `987cd90` [04:47:01] [02mediawiki] 07github-actions[bot] labeled pull request 03#7877: Bump extensions/Wikibase from `53ad322` to `4a45d49` - 13https://github.com/miraheze/mediawiki/pull/7877 [04:47:01] [url] Page not found · GitHub · GitHub | github.com [04:47:02] [02mediawiki] 07dependabot[bot] closed pull request 03#7788: Bump extensions/Translate from `508542e` to `2910b57` - 13https://github.com/miraheze/mediawiki/pull/7788 [04:47:03] ... [04:47:04] [02mediawiki] 07github-actions[bot] labeled pull request 03#7877: Bump extensions/Wikibase from `53ad322` to `4a45d49` - 13https://github.com/miraheze/mediawiki/pull/7877 [04:47:04] [url] Page not found · GitHub · GitHub | github.com [04:47:04] RECOVERY - cp31 Stunnel HTTP for mw121 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.988 second response time [04:47:05] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/TemplateData-987cd90 - 13https://github.com/miraheze/mediawiki [04:47:06] ... [04:47:07] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Translate-2910b57 [04:47:07] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 9.161 second response time [04:47:08] [02mediawiki] 07dependabot[bot] opened pull request 03#7896: Bump extensions/TemplateData from `9f361df` to `987cd90` - 13https://github.com/miraheze/mediawiki/pull/7896 [04:47:09] [url] Page not found · GitHub · GitHub | github.com [04:47:10] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Translate-2910b57 - 13https://github.com/miraheze/mediawiki [04:47:10] ... [04:47:11] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.693 second response time [04:47:11] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 6.458 second response time [04:47:11] RECOVERY - cp30 Stunnel HTTP for mw121 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 2.227 second response time [04:47:11] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/GlobalPreferences-a85f6a6 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/608dda8ce943 [04:47:13] [02miraheze/mediawiki] 07dependabot[bot] 03608dda8 - Bump extensions/GlobalPreferences from `17ef045` to `a85f6a6` [04:47:14] [02mediawiki] 07github-actions[bot] labeled pull request 03#7881: Bump extensions/CentralAuth from `6e7a049` to `62327ad` - 13https://github.com/miraheze/mediawiki/pull/7881 [04:47:15] [url] Page not found · GitHub · GitHub | github.com [04:47:16] [02mediawiki] 07dependabot[bot] commented on pull request 03#7857: Bump extensions/TemplateData from `9f361df` to `78dcb72` - 13https://github.com/miraheze/mediawiki/pull/7857#issuecomment-1171929046 [04:47:16] [url] Bump extensions/TemplateData from `9f361df` to `78dcb72` by dependabot[bot] · Pull Request #7857 · miraheze/mediawiki · GitHub | github.com [04:47:17] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/GlobalPreferences-a85f6a6 - 13https://github.com/miraheze/mediawiki [04:47:18] [url] Page not found · GitHub · GitHub | github.com [04:47:19] [02mediawiki] 07dependabot[bot] opened pull request 03#7897: Bump extensions/GlobalPreferences from `17ef045` to `a85f6a6` - 13https://github.com/miraheze/mediawiki/pull/7897 [04:47:19] [url] Page not found · GitHub · GitHub | github.com [04:47:20] [02mediawiki] 07github-actions[bot] labeled pull request 03#7887: Bump extensions/Disambiguator from `d4dcfd5` to `b123fa9` - 13https://github.com/miraheze/mediawiki/pull/7887 [04:47:21] ... [04:47:22] [02mediawiki] 07dependabot[bot] closed pull request 03#7857: Bump extensions/TemplateData from `9f361df` to `78dcb72` - 13https://github.com/miraheze/mediawiki/pull/7857 [04:47:22] [url] Page not found · GitHub · GitHub | github.com [04:47:23] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TemplateData-78dcb72 [04:47:25] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Tweeki-4e5f2af [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/4f7acf158d2a [04:47:26] [02miraheze/mediawiki] 07dependabot[bot] 034f7acf1 - Bump skins/Tweeki from `09124c1` to `4e5f2af` [04:47:28] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TemplateData-78dcb72 - 13https://github.com/miraheze/mediawiki [04:47:28] ... [04:47:29] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Tweeki-4e5f2af - 13https://github.com/miraheze/mediawiki [04:47:30] [url] Page not found · GitHub · GitHub | github.com [04:47:31] [02mediawiki] 07dependabot[bot] opened pull request 03#7898: Bump skins/Tweeki from `09124c1` to `4e5f2af` - 13https://github.com/miraheze/mediawiki/pull/7898 [04:47:31] ... [04:47:32] [02mediawiki] 07dependabot[bot] commented on pull request 03#7810: Bump extensions/GlobalPreferences from `17ef045` to `c11525d` - 13https://github.com/miraheze/mediawiki/pull/7810#issuecomment-1171929069 [04:47:33] [url] Bump extensions/GlobalPreferences from `17ef045` to `c11525d` by dependabot[bot] · Pull Request #7810 · miraheze/mediawiki · GitHub | github.com [04:47:34] [02mediawiki] 07github-actions[bot] labeled pull request 03#7888: Bump extensions/CheckUser from `49b4b6f` to `b6a15e0` - 13https://github.com/miraheze/mediawiki/pull/7888 [04:47:34] [url] Page not found · GitHub · GitHub | github.com [04:47:35] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GlobalPreferences-c11525d [04:47:36] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 8.533 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [04:47:37] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Math-fb3b474 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/fc3e6ad921bd [04:47:37] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 2.689 second response time [04:47:38] [02miraheze/mediawiki] 07dependabot[bot] 03fc3e6ad - Bump extensions/Math from `1d0c78b` to `fb3b474` [04:47:40] [02mediawiki] 07dependabot[bot] closed pull request 03#7810: Bump extensions/GlobalPreferences from `17ef045` to `c11525d` - 13https://github.com/miraheze/mediawiki/pull/7810 [04:47:40] [url] Page not found · GitHub · GitHub | github.com [04:47:41] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Math-fb3b474 - 13https://github.com/miraheze/mediawiki [04:47:42] [url] Page not found · GitHub · GitHub | github.com [04:47:43] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GlobalPreferences-c11525d - 13https://github.com/miraheze/mediawiki [04:47:43] ... [04:47:44] [02mediawiki] 07dependabot[bot] opened pull request 03#7899: Bump extensions/Math from `1d0c78b` to `fb3b474` - 13https://github.com/miraheze/mediawiki/pull/7899 [04:47:45] [url] Page not found · GitHub · GitHub | github.com [04:47:46] [02mediawiki] 07github-actions[bot] labeled pull request 03#7886: Bump extensions/MobileFrontend from `88c1c73` to `611c162` - 13https://github.com/miraheze/mediawiki/pull/7886 [04:47:46] ... [04:47:47] [02mediawiki] 07github-actions[bot] labeled pull request 03#7886: Bump extensions/MobileFrontend from `88c1c73` to `611c162` - 13https://github.com/miraheze/mediawiki/pull/7886 [04:47:48] [url] Page not found · GitHub · GitHub | github.com [04:47:49] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/GlobalNewFiles-846d817 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/a1b9aaef3ed5 [04:47:50] [02miraheze/mediawiki] 07dependabot[bot] 03a1b9aae - Bump extensions/GlobalNewFiles from `b314d6b` to `846d817` [04:47:52] [02mediawiki] 07github-actions[bot] labeled pull request 03#7890: Bump extensions/WikiDiscover from `0fcef42` to `0bd18ba` - 13https://github.com/miraheze/mediawiki/pull/7890 [04:47:52] ... [04:47:53] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/GlobalNewFiles-846d817 - 13https://github.com/miraheze/mediawiki [04:47:55] [02mediawiki] 07dependabot[bot] opened pull request 03#7900: Bump extensions/GlobalNewFiles from `b314d6b` to `846d817` - 13https://github.com/miraheze/mediawiki/pull/7900 [04:47:56] [02mediawiki] 07github-actions[bot] labeled pull request 03#7890: Bump extensions/WikiDiscover from `0fcef42` to `0bd18ba` - 13https://github.com/miraheze/mediawiki/pull/7890 [04:47:58] [02mediawiki] 07github-actions[bot] labeled pull request 03#7892: Bump skins/CologneBlue from `ebbcfbc` to `aba298e` - 13https://github.com/miraheze/mediawiki/pull/7892 [04:47:59] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Echo-a548d30 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/99e1983b56cf [04:48:00] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [04:48:01] [02miraheze/mediawiki] 07dependabot[bot] 0399e1983 - Bump extensions/Echo from `e6ccb44` to `a548d30` [04:48:02] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Echo-a548d30 - 13https://github.com/miraheze/mediawiki [04:48:04] [02mediawiki] 07dependabot[bot] opened pull request 03#7901: Bump extensions/Echo from `e6ccb44` to `a548d30` - 13https://github.com/miraheze/mediawiki/pull/7901 [04:48:05] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/BlueSky-467e015 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/32f131dfc5bc [04:48:07] [02miraheze/mediawiki] 07dependabot[bot] 0332f131d - Bump skins/BlueSky from `05d9b59` to `467e015` [04:48:08] [02mediawiki] 07github-actions[bot] labeled pull request 03#7889: Bump extensions/WikibaseQualityConstraints from `0024a67` to `c6fbf36` - 13https://github.com/miraheze/mediawiki/pull/7889 [04:48:10] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/BlueSky-467e015 - 13https://github.com/miraheze/mediawiki [04:48:11] [02mediawiki] 07dependabot[bot] commented on pull request 03#7854: Bump extensions/Echo from `e6ccb44` to `d0988cd` - 13https://github.com/miraheze/mediawiki/pull/7854#issuecomment-1171929146 [04:48:12] [url] Bump extensions/Echo from `e6ccb44` to `d0988cd` by dependabot[bot] · Pull Request #7854 · miraheze/mediawiki · GitHub | github.com [04:48:13] [02mediawiki] 07dependabot[bot] opened pull request 03#7902: Bump skins/BlueSky from `05d9b59` to `467e015` - 13https://github.com/miraheze/mediawiki/pull/7902 [04:48:13] [url] Page not found · GitHub · GitHub | github.com [04:48:14] [02mediawiki] 07dependabot[bot] closed pull request 03#7854: Bump extensions/Echo from `e6ccb44` to `d0988cd` - 13https://github.com/miraheze/mediawiki/pull/7854 [04:48:15] [url] Page not found · GitHub · GitHub | github.com [04:48:16] [02mediawiki] 07github-actions[bot] labeled pull request 03#7891: Bump extensions/LoginNotify from `c968ccb` to `d146e97` - 13https://github.com/miraheze/mediawiki/pull/7891 [04:48:16] [url] Page not found · GitHub · GitHub | github.com [04:48:17] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/apex-de72363 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/27b27f4ec389 [04:48:19] [02miraheze/mediawiki] 07dependabot[bot] 0327b27f4 - Bump skins/apex from `ab93aff` to `de72363` [04:48:20] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Echo-d0988cd [04:48:22] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Echo-d0988cd - 13https://github.com/miraheze/mediawiki [04:48:22] ... [04:48:23] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/apex-de72363 - 13https://github.com/miraheze/mediawiki [04:48:24] [url] Page not found · GitHub · GitHub | github.com [04:48:25] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:48:25] [02mediawiki] 07dependabot[bot] opened pull request 03#7903: Bump skins/apex from `ab93aff` to `de72363` - 13https://github.com/miraheze/mediawiki/pull/7903 [04:48:25] ... [04:48:26] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Report-2161d9a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/506f5816cb05 [04:48:28] [02miraheze/mediawiki] 07dependabot[bot] 03506f581 - Bump extensions/Report from `54b823d` to `2161d9a` [04:48:29] [02mediawiki] 07github-actions[bot] labeled pull request 03#7893: Bump extensions/QuizGame from `7a1c5f9` to `00b2fea` - 13https://github.com/miraheze/mediawiki/pull/7893 [04:48:30] [url] Page not found · GitHub · GitHub | github.com [04:48:31] [02mediawiki] 07dependabot[bot] commented on pull request 03#7825: Bump skins/BlueSky from `05d9b59` to `6c57a31` - 13https://github.com/miraheze/mediawiki/pull/7825#issuecomment-1171929176 [04:48:32] [url] Bump skins/BlueSky from `05d9b59` to `6c57a31` by dependabot[bot] · Pull Request #7825 · miraheze/mediawiki · GitHub | github.com [04:48:32] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Report-2161d9a - 13https://github.com/miraheze/mediawiki [04:48:33] ... [04:48:34] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/BlueSky-6c57a31 [04:48:35] [02mediawiki] 07dependabot[bot] closed pull request 03#7825: Bump skins/BlueSky from `05d9b59` to `6c57a31` - 13https://github.com/miraheze/mediawiki/pull/7825 [04:48:36] [url] Page not found · GitHub · GitHub | github.com [04:48:37] [02mediawiki] 07dependabot[bot] opened pull request 03#7904: Bump extensions/Report from `54b823d` to `2161d9a` - 13https://github.com/miraheze/mediawiki/pull/7904 [04:48:37] [url] Page not found · GitHub · GitHub | github.com [04:48:38] [02mediawiki] 07dependabot[bot] closed pull request 03#7793: Bump skins/apex from `ab93aff` to `fcc7566` - 13https://github.com/miraheze/mediawiki/pull/7793 [04:48:39] [url] Page not found · GitHub · GitHub | github.com [04:48:40] [02mediawiki] 07dependabot[bot] commented on pull request 03#7793: Bump skins/apex from `ab93aff` to `fcc7566` - 13https://github.com/miraheze/mediawiki/pull/7793#issuecomment-1171929185 [04:48:40] [url] Bump skins/apex from `ab93aff` to `fcc7566` by dependabot[bot] · Pull Request #7793 · miraheze/mediawiki · GitHub | github.com [04:48:41] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/apex-fcc7566 [04:48:43] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/apex-fcc7566 - 13https://github.com/miraheze/mediawiki [04:48:43] [url] Page not found · GitHub · GitHub | github.com [04:48:44] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/skins/BlueSky-6c57a31 - 13https://github.com/miraheze/mediawiki [04:48:45] ... [04:48:46] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/UploadWizard-2bc8638 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/697961c8bd03 [04:48:47] [02miraheze/mediawiki] 07dependabot[bot] 03697961c - Bump extensions/UploadWizard from `50d14f6` to `2bc8638` [04:48:49] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/UploadWizard-2bc8638 - 13https://github.com/miraheze/mediawiki [04:48:49] [url] Page not found · GitHub · GitHub | github.com [04:48:50] [02mediawiki] 07dependabot[bot] opened pull request 03#7905: Bump extensions/UploadWizard from `50d14f6` to `2bc8638` - 13https://github.com/miraheze/mediawiki/pull/7905 [04:48:51] ... [04:48:52] [02mediawiki] 07github-actions[bot] labeled pull request 03#7895: Bump extensions/ReplaceText from `67bca38` to `53e9890` - 13https://github.com/miraheze/mediawiki/pull/7895 [04:48:52] ... [04:48:53] [02mediawiki] 07dependabot[bot] commented on pull request 03#7812: Bump extensions/UploadWizard from `50d14f6` to `b8f2d78` - 13https://github.com/miraheze/mediawiki/pull/7812#issuecomment-1171929247 [04:48:54] [url] Bump extensions/UploadWizard from `50d14f6` to `b8f2d78` by dependabot[bot] · Pull Request #7812 · miraheze/mediawiki · GitHub | github.com [04:48:55] [02mediawiki] 07github-actions[bot] labeled pull request 03#7896: Bump extensions/TemplateData from `9f361df` to `987cd90` - 13https://github.com/miraheze/mediawiki/pull/7896 [04:48:55] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:48:55] [url] Page not found · GitHub · GitHub | github.com [04:48:56] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Maps-4cd83f9 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/946012165cb2 [04:48:58] [02miraheze/mediawiki] 07dependabot[bot] 039460121 - Bump extensions/Maps from `df2d815` to `4cd83f9` [04:48:58] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:48:59] [02mediawiki] 07dependabot[bot] closed pull request 03#7812: Bump extensions/UploadWizard from `50d14f6` to `b8f2d78` - 13https://github.com/miraheze/mediawiki/pull/7812 [04:49:00] [url] Page not found · GitHub · GitHub | github.com [04:49:01] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Maps-4cd83f9 - 13https://github.com/miraheze/mediawiki [04:49:01] [url] Page not found · GitHub · GitHub | github.com [04:49:02] [02mediawiki] 07dependabot[bot] opened pull request 03#7906: Bump extensions/Maps from `df2d815` to `4cd83f9` - 13https://github.com/miraheze/mediawiki/pull/7906 [04:49:03] [url] Page not found · GitHub · GitHub | github.com [04:49:04] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UploadWizard-b8f2d78 [04:49:05] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UploadWizard-b8f2d78 - 13https://github.com/miraheze/mediawiki [04:49:06] ... [04:49:07] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Newsletter-b85b4ea [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/a652cb89be5f [04:49:08] [02miraheze/mediawiki] 07dependabot[bot] 03a652cb8 - Bump extensions/Newsletter from `96198d3` to `b85b4ea` [04:49:10] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Newsletter-b85b4ea - 13https://github.com/miraheze/mediawiki [04:49:10] [url] Page not found · GitHub · GitHub | github.com [04:49:11] [02mediawiki] 07dependabot[bot] opened pull request 03#7907: Bump extensions/Newsletter from `96198d3` to `b85b4ea` - 13https://github.com/miraheze/mediawiki/pull/7907 [04:49:12] ... [04:49:13] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/3D-d532e9c [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/bad807532ef9 [04:49:14] [02miraheze/mediawiki] 07dependabot[bot] 03bad8075 - Bump extensions/3D from `73a6dd9` to `d532e9c` [04:49:16] [02mediawiki] 07dependabot[bot] commented on pull request 03#7802: Bump extensions/Newsletter from `96198d3` to `7d62e6a` - 13https://github.com/miraheze/mediawiki/pull/7802#issuecomment-1171929286 [04:49:16] [url] Bump extensions/Newsletter from `96198d3` to `7d62e6a` by dependabot[bot] · Pull Request #7802 · miraheze/mediawiki · GitHub | github.com [04:49:17] [02mediawiki] 07github-actions[bot] labeled pull request 03#7898: Bump skins/Tweeki from `09124c1` to `4e5f2af` - 13https://github.com/miraheze/mediawiki/pull/7898 [04:49:18] ... [04:49:19] [02mediawiki] 07dependabot[bot] opened pull request 03#7908: Bump extensions/3D from `73a6dd9` to `d532e9c` - 13https://github.com/miraheze/mediawiki/pull/7908 [04:49:19] [url] Page not found · GitHub · GitHub | github.com [04:49:20] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/3D-d532e9c - 13https://github.com/miraheze/mediawiki [04:49:21] [url] Page not found · GitHub · GitHub | github.com [04:49:22] [02mediawiki] 07dependabot[bot] closed pull request 03#7802: Bump extensions/Newsletter from `96198d3` to `7d62e6a` - 13https://github.com/miraheze/mediawiki/pull/7802 [04:49:22] [url] Page not found · GitHub · GitHub | github.com [04:49:34] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/AbuseFilter-9074d0a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/705d3454673a [04:49:35] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:49:35] [02miraheze/mediawiki] 07dependabot[bot] 03705d345 - Bump extensions/AbuseFilter from `111cfc9` to `9074d0a` [04:49:37] [02mediawiki] 07github-actions[bot] labeled pull request 03#7894: Bump extensions/Translate from `508542e` to `1b89bc6` - 13https://github.com/miraheze/mediawiki/pull/7894 [04:49:37] [url] Page not found · GitHub · GitHub | github.com [04:49:38] [02mediawiki] 07github-actions[bot] labeled pull request 03#7894: Bump extensions/Translate from `508542e` to `1b89bc6` - 13https://github.com/miraheze/mediawiki/pull/7894 [04:49:39] ... [04:49:40] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/AbuseFilter-9074d0a - 13https://github.com/miraheze/mediawiki [04:49:40] [url] Page not found · GitHub · GitHub | github.com [04:49:41] [02mediawiki] 07dependabot[bot] commented on pull request 03#7782: Bump extensions/MultimediaViewer from `1c2c576` to `893a56b` - 13https://github.com/miraheze/mediawiki/pull/7782#issuecomment-1171929317 [04:49:42] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:49:42] [url] Bump extensions/MultimediaViewer from `1c2c576` to `893a56b` by dependabot[bot] · Pull Request #7782 · miraheze/mediawiki · GitHub | github.com [04:49:43] [02mediawiki] 07github-actions[bot] labeled pull request 03#7897: Bump extensions/GlobalPreferences from `17ef045` to `a85f6a6` - 13https://github.com/miraheze/mediawiki/pull/7897 [04:49:43] ... [04:49:44] [02mediawiki] 07dependabot[bot] opened pull request 03#7910: Bump extensions/AbuseFilter from `111cfc9` to `9074d0a` - 13https://github.com/miraheze/mediawiki/pull/7910 [04:49:45] ... [04:49:46] [02mediawiki] 07dependabot[bot] closed pull request 03#7782: Bump extensions/MultimediaViewer from `1c2c576` to `893a56b` - 13https://github.com/miraheze/mediawiki/pull/7782 [04:49:46] [url] Page not found · GitHub · GitHub | github.com [04:49:47] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/DataDump-e8bf0d3 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/afdff64ae989 [04:49:49] [02miraheze/mediawiki] 07dependabot[bot] 03afdff64 - Bump extensions/DataDump from `0c39251` to `e8bf0d3` [04:49:50] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-893a56b [04:49:51] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.02, 4.02, 4.02 [04:49:52] [02mediawiki] 07github-actions[bot] labeled pull request 03#7900: Bump extensions/GlobalNewFiles from `b314d6b` to `846d817` - 13https://github.com/miraheze/mediawiki/pull/7900 [04:49:52] [url] Page not found · GitHub · GitHub | github.com [04:49:53] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/DataDump-e8bf0d3 - 13https://github.com/miraheze/mediawiki [04:49:54] [url] Page not found · GitHub · GitHub | github.com [04:49:55] [02mediawiki] 07dependabot[bot] opened pull request 03#7911: Bump extensions/DataDump from `0c39251` to `e8bf0d3` - 13https://github.com/miraheze/mediawiki/pull/7911 [04:49:55] [url] Page not found · GitHub · GitHub | github.com [04:49:57] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MultimediaViewer-893a56b - 13https://github.com/miraheze/mediawiki [04:49:57] ... [04:49:58] [02mediawiki] 07github-actions[bot] labeled pull request 03#7900: Bump extensions/GlobalNewFiles from `b314d6b` to `846d817` - 13https://github.com/miraheze/mediawiki/pull/7900 [04:49:58] [url] Page not found · GitHub · GitHub | github.com [04:50:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Moderation-ee20787 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ea56c5ae6ce5 [04:50:01] [02miraheze/mediawiki] 07dependabot[bot] 03ea56c5a - Bump extensions/Moderation from `20f6879` to `ee20787` [04:50:02] [02mediawiki] 07dependabot[bot] commented on pull request 03#7842: Bump extensions/AbuseFilter from `111cfc9` to `3b7733b` - 13https://github.com/miraheze/mediawiki/pull/7842#issuecomment-1171929342 [04:50:03] [url] Bump extensions/AbuseFilter from `111cfc9` to `3b7733b` by dependabot[bot] · Pull Request #7842 · miraheze/mediawiki · GitHub | github.com [04:50:04] [02mediawiki] 07dependabot[bot] opened pull request 03#7912: Bump extensions/Moderation from `20f6879` to `ee20787` - 13https://github.com/miraheze/mediawiki/pull/7912 [04:50:05] ... [04:50:05] [02mediawiki] 07github-actions[bot] labeled pull request 03#7899: Bump extensions/Math from `1d0c78b` to `fb3b474` - 13https://github.com/miraheze/mediawiki/pull/7899 [04:50:06] ... [04:50:07] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/AbuseFilter-3b7733b [04:50:09] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Moderation-ee20787 - 13https://github.com/miraheze/mediawiki [04:50:09] [url] Page not found · GitHub · GitHub | github.com [04:50:10] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/AbuseFilter-3b7733b - 13https://github.com/miraheze/mediawiki [04:50:10] [url] Page not found · GitHub · GitHub | github.com [04:50:12] [02mediawiki] 07dependabot[bot] closed pull request 03#7842: Bump extensions/AbuseFilter from `111cfc9` to `3b7733b` - 13https://github.com/miraheze/mediawiki/pull/7842 [04:50:12] [url] Page not found · GitHub · GitHub | github.com [04:50:13] [02mediawiki] 07github-actions[bot] labeled pull request 03#7902: Bump skins/BlueSky from `05d9b59` to `467e015` - 13https://github.com/miraheze/mediawiki/pull/7902 [04:50:13] [url] Page not found · GitHub · GitHub | github.com [04:50:15] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Moderation-a8f1ccb [04:50:16] [02mediawiki] 07dependabot[bot] commented on pull request 03#7844: Bump extensions/Moderation from `20f6879` to `a8f1ccb` - 13https://github.com/miraheze/mediawiki/pull/7844#issuecomment-1171929369 [04:50:17] [url] Bump extensions/Moderation from `20f6879` to `a8f1ccb` by dependabot[bot] · Pull Request #7844 · miraheze/mediawiki · GitHub | github.com [04:50:17] [02mediawiki] 07github-actions[bot] labeled pull request 03#7903: Bump skins/apex from `ab93aff` to `de72363` - 13https://github.com/miraheze/mediawiki/pull/7903 [04:50:18] [url] Page not found · GitHub · GitHub | github.com [04:50:19] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ManageWiki-b0139c2 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/465659ac8cd7 [04:50:20] [02miraheze/mediawiki] 07dependabot[bot] 03465659a - Bump extensions/ManageWiki from `e654457` to `b0139c2` [04:50:22] [02mediawiki] 07dependabot[bot] closed pull request 03#7844: Bump extensions/Moderation from `20f6879` to `a8f1ccb` - 13https://github.com/miraheze/mediawiki/pull/7844 [04:50:22] ... [04:50:23] [02mediawiki] 07github-actions[bot] labeled pull request 03#7903: Bump skins/apex from `ab93aff` to `de72363` - 13https://github.com/miraheze/mediawiki/pull/7903 [04:50:24] ... [04:50:25] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Moderation-a8f1ccb - 13https://github.com/miraheze/mediawiki [04:50:25] ... [04:50:26] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ManageWiki-b0139c2 - 13https://github.com/miraheze/mediawiki [04:50:27] [url] Page not found · GitHub · GitHub | github.com [04:50:28] [02mediawiki] 07dependabot[bot] opened pull request 03#7913: Bump extensions/ManageWiki from `e654457` to `b0139c2` - 13https://github.com/miraheze/mediawiki/pull/7913 [04:50:28] [url] Page not found · GitHub · GitHub | github.com [04:50:40] [02mediawiki] 07github-actions[bot] labeled pull request 03#7901: Bump extensions/Echo from `e6ccb44` to `a548d30` - 13https://github.com/miraheze/mediawiki/pull/7901 [04:50:40] [url] Page not found · GitHub · GitHub | github.com [04:50:41] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Flow-0b2ea10 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/2749121b4a29 [04:50:43] [02miraheze/mediawiki] 07dependabot[bot] 032749121 - Bump extensions/Flow from `c6f25fd` to `0b2ea10` [04:50:44] [02mediawiki] 07github-actions[bot] labeled pull request 03#7909: Bump extensions/MultimediaViewer from `1c2c576` to `2221f84` - 13https://github.com/miraheze/mediawiki/pull/7909 [04:50:45] [url] Page not found · GitHub · GitHub | github.com [04:50:46] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Flow-0b2ea10 - 13https://github.com/miraheze/mediawiki [04:50:47] [02mediawiki] 07dependabot[bot] opened pull request 03#7915: Bump extensions/Flow from `c6f25fd` to `0b2ea10` - 13https://github.com/miraheze/mediawiki/pull/7915 [04:50:49] [02mediawiki] 07github-actions[bot] labeled pull request 03#7905: Bump extensions/UploadWizard from `50d14f6` to `2bc8638` - 13https://github.com/miraheze/mediawiki/pull/7905 [04:50:51] [02mediawiki] 07dependabot[bot] commented on pull request 03#7799: Bump extensions/ConfirmEdit from `9ae04a5` to `50f4dfd` - 13https://github.com/miraheze/mediawiki/pull/7799#issuecomment-1171929441 [04:50:51] [url] Bump extensions/ConfirmEdit from `9ae04a5` to `50f4dfd` by dependabot[bot] · Pull Request #7799 · miraheze/mediawiki · GitHub | github.com [04:50:52] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/ConfirmEdit-50f4dfd [04:50:54] [02mediawiki] 07dependabot[bot] closed pull request 03#7799: Bump extensions/ConfirmEdit from `9ae04a5` to `50f4dfd` - 13https://github.com/miraheze/mediawiki/pull/7799 [04:50:54] [url] Page not found · GitHub · GitHub | github.com [04:50:55] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Citizen-e53afa5 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ebeb2a1b29dd [04:50:57] [02miraheze/mediawiki] 07dependabot[bot] 03ebeb2a1 - Bump skins/Citizen from `38c560d` to `e53afa5` [04:50:58] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/ConfirmEdit-50f4dfd - 13https://github.com/miraheze/mediawiki [04:50:58] ... [04:50:59] [02mediawiki] 07github-actions[bot] labeled pull request 03#7911: Bump extensions/DataDump from `0c39251` to `e8bf0d3` - 13https://github.com/miraheze/mediawiki/pull/7911 [04:51:00] ... [04:51:01] [02mediawiki] 07dependabot[bot] opened pull request 03#7916: Bump skins/Citizen from `38c560d` to `e53afa5` - 13https://github.com/miraheze/mediawiki/pull/7916 [04:51:02] ... [04:51:03] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Citizen-e53afa5 - 13https://github.com/miraheze/mediawiki [04:51:03] [url] Page not found · GitHub · GitHub | github.com [04:51:04] [02mediawiki] 07dependabot[bot] closed pull request 03#7821: Bump extensions/Flow from `c6f25fd` to `636c70b` - 13https://github.com/miraheze/mediawiki/pull/7821 [04:51:04] [url] Page not found · GitHub · GitHub | github.com [04:51:05] [02mediawiki] 07github-actions[bot] labeled pull request 03#7911: Bump extensions/DataDump from `0c39251` to `e8bf0d3` - 13https://github.com/miraheze/mediawiki/pull/7911 [04:51:06] [url] Page not found · GitHub · GitHub | github.com [04:51:07] [02mediawiki] 07dependabot[bot] commented on pull request 03#7821: Bump extensions/Flow from `c6f25fd` to `636c70b` - 13https://github.com/miraheze/mediawiki/pull/7821#issuecomment-1171929460 [04:51:07] [url] Bump extensions/Flow from `c6f25fd` to `636c70b` by dependabot[bot] · Pull Request #7821 · miraheze/mediawiki · GitHub | github.com [04:51:08] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Flow-636c70b [04:51:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.89, 3.66, 3.26 [04:51:10] [02mediawiki] 07github-actions[bot] labeled pull request 03#7908: Bump extensions/3D from `73a6dd9` to `d532e9c` - 13https://github.com/miraheze/mediawiki/pull/7908 [04:51:10] [url] Page not found · GitHub · GitHub | github.com [04:51:11] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Flow-636c70b - 13https://github.com/miraheze/mediawiki [04:51:12] [url] Page not found · GitHub · GitHub | github.com [04:51:13] [02mediawiki] 07github-actions[bot] labeled pull request 03#7908: Bump extensions/3D from `73a6dd9` to `d532e9c` - 13https://github.com/miraheze/mediawiki/pull/7908 [04:51:13] ... [04:51:14] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ShortDescription-bf89cf9 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/14d3ca15b7f5 [04:51:16] [02miraheze/mediawiki] 07dependabot[bot] 0314d3ca1 - Bump extensions/ShortDescription from `0e4139f` to `bf89cf9` [04:51:17] [02mediawiki] 07dependabot[bot] opened pull request 03#7917: Bump extensions/ShortDescription from `0e4139f` to `bf89cf9` - 13https://github.com/miraheze/mediawiki/pull/7917 [04:51:18] ... [04:51:19] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ShortDescription-bf89cf9 - 13https://github.com/miraheze/mediawiki [04:51:19] ... [04:51:20] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/YouTube-b199c95 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/74d9621a62ed [04:51:22] [02miraheze/mediawiki] 07dependabot[bot] 0374d9621 - Bump extensions/YouTube from `01ce5f3` to `b199c95` [04:51:23] [02mediawiki] 07dependabot[bot] opened pull request 03#7918: Bump extensions/YouTube from `01ce5f3` to `b199c95` - 13https://github.com/miraheze/mediawiki/pull/7918 [04:51:24] [url] Page not found · GitHub · GitHub | github.com [04:51:25] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:51:25] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/YouTube-b199c95 - 13https://github.com/miraheze/mediawiki [04:51:25] [url] Page not found · GitHub · GitHub | github.com [04:51:27] [02mediawiki] 07dependabot[bot] commented on pull request 03#7831: Bump extensions/ShortDescription from `0e4139f` to `5043f94` - 13https://github.com/miraheze/mediawiki/pull/7831#issuecomment-1171929523 [04:51:27] [url] Bump extensions/ShortDescription from `0e4139f` to `5043f94` by dependabot[bot] · Pull Request #7831 · miraheze/mediawiki · GitHub | github.com [04:51:28] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/StructuredNavigation-310e3ef [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/5a0f8959acaf [04:51:28] [url] Page not found · GitHub · GitHub | github.com [04:51:30] [02miraheze/mediawiki] 07dependabot[bot] 035a0f895 - Bump extensions/StructuredNavigation from `005f244` to `310e3ef` [04:51:31] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/ShortDescription-5043f94 [04:51:33] [02mediawiki] 07dependabot[bot] closed pull request 03#7831: Bump extensions/ShortDescription from `0e4139f` to `5043f94` - 13https://github.com/miraheze/mediawiki/pull/7831 [04:51:34] [url] Page not found · GitHub · GitHub | github.com [04:51:34] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/StructuredNavigation-310e3ef - 13https://github.com/miraheze/mediawiki [04:51:34] [url] Page not found · GitHub · GitHub | github.com [04:51:35] [02mediawiki] 07dependabot[bot] opened pull request 03#7919: Bump extensions/StructuredNavigation from `005f244` to `310e3ef` - 13https://github.com/miraheze/mediawiki/pull/7919 [04:51:36] ... [04:51:37] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/ShortDescription-5043f94 - 13https://github.com/miraheze/mediawiki [04:51:37] ... [04:51:38] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PDFEmbed-d31da31 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/3e765242aaae [04:51:40] [02miraheze/mediawiki] 07dependabot[bot] 033e76524 - Bump extensions/PDFEmbed from `1170024` to `d31da31` [04:51:42] [02mediawiki] 07dependabot[bot] opened pull request 03#7920: Bump extensions/PDFEmbed from `1170024` to `d31da31` - 13https://github.com/miraheze/mediawiki/pull/7920 [04:51:42] ... [04:51:43] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PDFEmbed-d31da31 - 13https://github.com/miraheze/mediawiki [04:51:43] [url] Page not found · GitHub · GitHub | github.com [04:51:44] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Thanks-eda0459 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/030d9dc690d4 [04:51:46] [02miraheze/mediawiki] 07dependabot[bot] 03030d9dc - Bump extensions/Thanks from `14a1bad` to `eda0459` [04:51:48] [02mediawiki] 07github-actions[bot] labeled pull request 03#7912: Bump extensions/Moderation from `20f6879` to `ee20787` - 13https://github.com/miraheze/mediawiki/pull/7912 [04:51:48] [url] Page not found · GitHub · GitHub | github.com [04:51:49] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Thanks-eda0459 - 13https://github.com/miraheze/mediawiki [04:51:51] [02mediawiki] 07dependabot[bot] opened pull request 03#7921: Bump extensions/Thanks from `14a1bad` to `eda0459` - 13https://github.com/miraheze/mediawiki/pull/7921 [04:51:52] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/BetaFeatures-2719ddc [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c9bbf8fc4b41 [04:51:54] [02miraheze/mediawiki] 07dependabot[bot] 03c9bbf8f - Bump extensions/BetaFeatures from `3ff0bb3` to `2719ddc` [04:51:55] [02mediawiki] 07dependabot[bot] opened pull request 03#7922: Bump extensions/BetaFeatures from `3ff0bb3` to `2719ddc` - 13https://github.com/miraheze/mediawiki/pull/7922 [04:51:57] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/BetaFeatures-2719ddc - 13https://github.com/miraheze/mediawiki [04:51:58] [02mediawiki] 07dependabot[bot] commented on pull request 03#7858: Bump extensions/Thanks from `14a1bad` to `6a9f14e` - 13https://github.com/miraheze/mediawiki/pull/7858#issuecomment-1171929590 [04:51:59] [url] Bump extensions/Thanks from `14a1bad` to `6a9f14e` by dependabot[bot] · Pull Request #7858 · miraheze/mediawiki · GitHub | github.com [04:52:00] [02mediawiki] 07github-actions[bot] labeled pull request 03#7910: Bump extensions/AbuseFilter from `111cfc9` to `9074d0a` - 13https://github.com/miraheze/mediawiki/pull/7910 [04:52:00] [url] Page not found · GitHub · GitHub | github.com [04:52:01] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Thanks-6a9f14e [04:52:03] [02mediawiki] 07dependabot[bot] closed pull request 03#7858: Bump extensions/Thanks from `14a1bad` to `6a9f14e` - 13https://github.com/miraheze/mediawiki/pull/7858 [04:52:03] [url] Page not found · GitHub · GitHub | github.com [04:52:04] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CodeEditor-dffd7cf [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/b4620748b880 [04:52:06] [02miraheze/mediawiki] 07dependabot[bot] 03b462074 - Bump extensions/CodeEditor from `0468e3c` to `dffd7cf` [04:52:07] [02mediawiki] 07github-actions[bot] labeled pull request 03#7910: Bump extensions/AbuseFilter from `111cfc9` to `9074d0a` - 13https://github.com/miraheze/mediawiki/pull/7910 [04:52:07] [url] Page not found · GitHub · GitHub | github.com [04:52:09] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Thanks-6a9f14e - 13https://github.com/miraheze/mediawiki [04:52:09] ... [04:52:10] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CodeEditor-dffd7cf - 13https://github.com/miraheze/mediawiki [04:52:10] ... [04:52:12] [02mediawiki] 07dependabot[bot] opened pull request 03#7923: Bump extensions/CodeEditor from `0468e3c` to `dffd7cf` - 13https://github.com/miraheze/mediawiki/pull/7923 [04:52:12] ... [04:52:13] [02mediawiki] 07github-actions[bot] labeled pull request 03#7913: Bump extensions/ManageWiki from `e654457` to `b0139c2` - 13https://github.com/miraheze/mediawiki/pull/7913 [04:52:13] [url] Page not found · GitHub · GitHub | github.com [04:52:15] [02mediawiki] 07github-actions[bot] labeled pull request 03#7913: Bump extensions/ManageWiki from `e654457` to `b0139c2` - 13https://github.com/miraheze/mediawiki/pull/7913 [04:52:15] [url] Page not found · GitHub · GitHub | github.com [04:52:16] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/SimpleChanges-78d7ab4 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/eba50e3377cd [04:52:18] [02miraheze/mediawiki] 07dependabot[bot] 03eba50e3 - Bump extensions/SimpleChanges from `c510c1c` to `78d7ab4` [04:52:19] [02mediawiki] 07dependabot[bot] opened pull request 03#7924: Bump extensions/SimpleChanges from `c510c1c` to `78d7ab4` - 13https://github.com/miraheze/mediawiki/pull/7924 [04:52:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/SimpleChanges-78d7ab4 - 13https://github.com/miraheze/mediawiki [04:52:22] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/GlobalBlocking-07a59e7 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/742efb308791 [04:52:24] [02miraheze/mediawiki] 07dependabot[bot] 03742efb3 - Bump extensions/GlobalBlocking from `99bb1ba` to `07a59e7` [04:52:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7918: Bump extensions/YouTube from `01ce5f3` to `b199c95` - 13https://github.com/miraheze/mediawiki/pull/7918 [04:52:27] [02mediawiki] 07dependabot[bot] opened pull request 03#7925: Bump extensions/GlobalBlocking from `99bb1ba` to `07a59e7` - 13https://github.com/miraheze/mediawiki/pull/7925 [04:52:28] [02mediawiki] 07github-actions[bot] labeled pull request 03#7915: Bump extensions/Flow from `c6f25fd` to `0b2ea10` - 13https://github.com/miraheze/mediawiki/pull/7915 [04:52:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/GlobalBlocking-07a59e7 - 13https://github.com/miraheze/mediawiki [04:52:31] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/RemovePII-3be5c88 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/6a7042019dbe [04:52:33] [02miraheze/mediawiki] 07dependabot[bot] 036a70420 - Bump extensions/RemovePII from `20686b4` to `3be5c88` [04:52:34] [02mediawiki] 07dependabot[bot] opened pull request 03#7926: Bump extensions/RemovePII from `20686b4` to `3be5c88` - 13https://github.com/miraheze/mediawiki/pull/7926 [04:52:36] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/RemovePII-3be5c88 - 13https://github.com/miraheze/mediawiki [04:52:37] [02mediawiki] 07github-actions[bot] labeled pull request 03#7917: Bump extensions/ShortDescription from `0e4139f` to `bf89cf9` - 13https://github.com/miraheze/mediawiki/pull/7917 [04:52:39] [02mediawiki] 07dependabot[bot] commented on pull request 03#7813: Bump extensions/GlobalBlocking from `99bb1ba` to `3a69630` - 13https://github.com/miraheze/mediawiki/pull/7813#issuecomment-1171929658 [04:52:39] [url] Bump extensions/GlobalBlocking from `99bb1ba` to `3a69630` by dependabot[bot] · Pull Request #7813 · miraheze/mediawiki · GitHub | github.com [04:52:40] [02mediawiki] 07github-actions[bot] labeled pull request 03#7920: Bump extensions/PDFEmbed from `1170024` to `d31da31` - 13https://github.com/miraheze/mediawiki/pull/7920 [04:52:41] [url] Page not found · GitHub · GitHub | github.com [04:52:42] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GlobalBlocking-3a69630 [04:52:43] [02mediawiki] 07github-actions[bot] labeled pull request 03#7914: Bump extensions/ConfirmEdit from `9ae04a5` to `9159047` - 13https://github.com/miraheze/mediawiki/pull/7914 [04:52:44] [url] Page not found · GitHub · GitHub | github.com [04:52:45] [02mediawiki] 07dependabot[bot] closed pull request 03#7813: Bump extensions/GlobalBlocking from `99bb1ba` to `3a69630` - 13https://github.com/miraheze/mediawiki/pull/7813 [04:52:45] [url] Page not found · GitHub · GitHub | github.com [04:52:46] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/MediaWikiChat-ddfa325 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/e9c288e3f958 [04:52:48] [02miraheze/mediawiki] 07dependabot[bot] 03e9c288e - Bump extensions/MediaWikiChat from `c2398b2` to `ddfa325` [04:52:49] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GlobalBlocking-3a69630 - 13https://github.com/miraheze/mediawiki [04:52:49] ... [04:52:50] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:52:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#7916: Bump skins/Citizen from `38c560d` to `e53afa5` - 13https://github.com/miraheze/mediawiki/pull/7916 [04:52:51] ... [04:52:52] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/MediaWikiChat-ddfa325 - 13https://github.com/miraheze/mediawiki [04:52:52] ... [04:52:54] [02mediawiki] 07github-actions[bot] labeled pull request 03#7916: Bump skins/Citizen from `38c560d` to `e53afa5` - 13https://github.com/miraheze/mediawiki/pull/7916 [04:52:54] [url] Page not found · GitHub · GitHub | github.com [04:52:55] [02mediawiki] 07dependabot[bot] opened pull request 03#7927: Bump extensions/MediaWikiChat from `c2398b2` to `ddfa325` - 13https://github.com/miraheze/mediawiki/pull/7927 [04:52:55] [url] Page not found · GitHub · GitHub | github.com [04:52:57] [02mediawiki] 07github-actions[bot] labeled pull request 03#7919: Bump extensions/StructuredNavigation from `005f244` to `310e3ef` - 13https://github.com/miraheze/mediawiki/pull/7919 [04:52:58] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/TranslationNotifications-60ffe02 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/b592e299ae4f [04:53:00] [02miraheze/mediawiki] 07dependabot[bot] 03b592e29 - Bump extensions/TranslationNotifications from `54a8197` to `60ffe02` [04:53:01] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/TranslationNotifications-60ffe02 - 13https://github.com/miraheze/mediawiki [04:53:03] [02mediawiki] 07dependabot[bot] opened pull request 03#7928: Bump extensions/TranslationNotifications from `54a8197` to `60ffe02` - 13https://github.com/miraheze/mediawiki/pull/7928 [04:53:04] [02mediawiki] 07dependabot[bot] commented on pull request 03#7824: Bump extensions/MediaWikiChat from `c2398b2` to `8ff9578` - 13https://github.com/miraheze/mediawiki/pull/7824#issuecomment-1171929681 [04:53:05] [url] Bump extensions/MediaWikiChat from `c2398b2` to `8ff9578` by dependabot[bot] · Pull Request #7824 · miraheze/mediawiki · GitHub | github.com [04:53:06] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MediaWikiChat-8ff9578 [04:53:07] [02mediawiki] 07dependabot[bot] closed pull request 03#7824: Bump extensions/MediaWikiChat from `c2398b2` to `8ff9578` - 13https://github.com/miraheze/mediawiki/pull/7824 [04:53:07] [url] Page not found · GitHub · GitHub | github.com [04:53:09] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/MediaWikiChat-8ff9578 - 13https://github.com/miraheze/mediawiki [04:53:09] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [04:53:09] [url] Page not found · GitHub · GitHub | github.com [04:53:10] [02mediawiki] 07dependabot[bot] commented on pull request 03#7830: Bump extensions/TranslationNotifications from `54a8197` to `12c1ff5` - 13https://github.com/miraheze/mediawiki/pull/7830#issuecomment-1171929690 [04:53:11] [url] Bump extensions/TranslationNotifications from `54a8197` to `12c1ff5` by dependabot[bot] · Pull Request #7830 · miraheze/mediawiki · GitHub | github.com [04:53:12] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Linter-9bacd76 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/194197ae2af5 [04:53:13] [02miraheze/mediawiki] 07dependabot[bot] 03194197a - Bump extensions/Linter from `36320b4` to `9bacd76` [04:53:15] [02mediawiki] 07dependabot[bot] closed pull request 03#7830: Bump extensions/TranslationNotifications from `54a8197` to `12c1ff5` - 13https://github.com/miraheze/mediawiki/pull/7830 [04:53:15] [url] Page not found · GitHub · GitHub | github.com [04:53:16] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Linter-9bacd76 - 13https://github.com/miraheze/mediawiki [04:53:16] ... [04:53:18] [02mediawiki] 07dependabot[bot] opened pull request 03#7929: Bump extensions/Linter from `36320b4` to `9bacd76` - 13https://github.com/miraheze/mediawiki/pull/7929 [04:53:18] ... [04:53:19] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TranslationNotifications-12c1ff5 [04:53:21] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/TranslationNotifications-12c1ff5 - 13https://github.com/miraheze/mediawiki [04:53:21] [url] Page not found · GitHub · GitHub | github.com [04:53:22] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Cite-7395f4d [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/20a00ad84240 [04:53:24] [02miraheze/mediawiki] 07dependabot[bot] 0320a00ad - Bump extensions/Cite from `7fdd57d` to `7395f4d` [04:53:25] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Cite-7395f4d - 13https://github.com/miraheze/mediawiki [04:53:25] [url] Page not found · GitHub · GitHub | github.com [04:53:27] [02mediawiki] 07dependabot[bot] opened pull request 03#7930: Bump extensions/Cite from `7fdd57d` to `7395f4d` - 13https://github.com/miraheze/mediawiki/pull/7930 [04:53:27] ... [04:53:28] [02mediawiki] 07dependabot[bot] commented on pull request 03#7800: Bump extensions/Linter from `36320b4` to `61ac080` - 13https://github.com/miraheze/mediawiki/pull/7800#issuecomment-1171929712 [04:53:29] [url] Bump extensions/Linter from `36320b4` to `61ac080` by dependabot[bot] · Pull Request #7800 · miraheze/mediawiki · GitHub | github.com [04:53:30] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Linter-61ac080 [04:53:31] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Linter-61ac080 - 13https://github.com/miraheze/mediawiki [04:53:31] [url] Page not found · GitHub · GitHub | github.com [04:53:33] [02mediawiki] 07dependabot[bot] closed pull request 03#7800: Bump extensions/Linter from `36320b4` to `61ac080` - 13https://github.com/miraheze/mediawiki/pull/7800 [04:53:33] [url] Page not found · GitHub · GitHub | github.com [04:53:34] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/MirahezeMagic-515f04f [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/394fa0df556f [04:53:36] [02miraheze/mediawiki] 07dependabot[bot] 03394fa0d - Bump extensions/MirahezeMagic from `d38b6ae` to `515f04f` [04:53:37] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/MirahezeMagic-515f04f - 13https://github.com/miraheze/mediawiki [04:53:37] [url] Page not found · GitHub · GitHub | github.com [04:53:39] [02mediawiki] 07dependabot[bot] opened pull request 03#7931: Bump extensions/MirahezeMagic from `d38b6ae` to `515f04f` - 13https://github.com/miraheze/mediawiki/pull/7931 [04:53:39] ... [04:53:40] [02mediawiki] 07dependabot[bot] commented on pull request 03#7851: Bump extensions/Cite from `7fdd57d` to `d40993e` - 13https://github.com/miraheze/mediawiki/pull/7851#issuecomment-1171929730 [04:53:41] [url] Bump extensions/Cite from `7fdd57d` to `d40993e` by dependabot[bot] · Pull Request #7851 · miraheze/mediawiki · GitHub | github.com [04:53:42] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Kartographer-bcb0b66 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/aae8039e0b97 [04:53:43] [02miraheze/mediawiki] 07dependabot[bot] 03aae8039 - Bump extensions/Kartographer from `67f9065` to `bcb0b66` [04:53:45] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Cite-d40993e [04:53:46] [02mediawiki] 07github-actions[bot] labeled pull request 03#7921: Bump extensions/Thanks from `14a1bad` to `eda0459` - 13https://github.com/miraheze/mediawiki/pull/7921 [04:53:47] [url] Page not found · GitHub · GitHub | github.com [04:53:48] [02mediawiki] 07dependabot[bot] closed pull request 03#7851: Bump extensions/Cite from `7fdd57d` to `d40993e` - 13https://github.com/miraheze/mediawiki/pull/7851 [04:53:48] [url] Page not found · GitHub · GitHub | github.com [04:53:49] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Kartographer-bcb0b66 - 13https://github.com/miraheze/mediawiki [04:53:49] ... [04:53:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#7923: Bump extensions/CodeEditor from `0468e3c` to `dffd7cf` - 13https://github.com/miraheze/mediawiki/pull/7923 [04:53:51] ... [04:53:52] [02mediawiki] 07dependabot[bot] opened pull request 03#7932: Bump extensions/Kartographer from `67f9065` to `bcb0b66` - 13https://github.com/miraheze/mediawiki/pull/7932 [04:53:52] [url] Page not found · GitHub · GitHub | github.com [04:53:54] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Cite-d40993e - 13https://github.com/miraheze/mediawiki [04:53:54] [url] Page not found · GitHub · GitHub | github.com [04:53:55] [02mediawiki] 07github-actions[bot] labeled pull request 03#7922: Bump extensions/BetaFeatures from `3ff0bb3` to `2719ddc` - 13https://github.com/miraheze/mediawiki/pull/7922 [04:53:56] [url] Page not found · GitHub · GitHub | github.com [04:53:57] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikimediaIncubator-1e9870d [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/9ca057fbcbdc [04:53:58] [02miraheze/mediawiki] 07dependabot[bot] 039ca057f - Bump extensions/WikimediaIncubator from `98caf22` to `1e9870d` [04:54:00] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikimediaIncubator-1e9870d - 13https://github.com/miraheze/mediawiki [04:54:00] ... [04:54:01] [02mediawiki] 07dependabot[bot] opened pull request 03#7933: Bump extensions/WikimediaIncubator from `98caf22` to `1e9870d` - 13https://github.com/miraheze/mediawiki/pull/7933 [04:54:03] [02mediawiki] 07dependabot[bot] commented on pull request 03#7814: Bump extensions/Kartographer from `67f9065` to `ddd5253` - 13https://github.com/miraheze/mediawiki/pull/7814#issuecomment-1171929767 [04:54:03] [url] Bump extensions/Kartographer from `67f9065` to `ddd5253` by dependabot[bot] · Pull Request #7814 · miraheze/mediawiki · GitHub | github.com [04:54:04] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Kartographer-ddd5253 [04:54:06] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PortableInfobox-abc29f3 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/e5128a0fac5b [04:54:07] [02miraheze/mediawiki] 07dependabot[bot] 03e5128a0 - Bump extensions/PortableInfobox from `6e0bc5e` to `abc29f3` [04:54:09] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Kartographer-ddd5253 - 13https://github.com/miraheze/mediawiki [04:54:09] [url] Page not found · GitHub · GitHub | github.com [04:54:10] [02mediawiki] 07dependabot[bot] closed pull request 03#7814: Bump extensions/Kartographer from `67f9065` to `ddd5253` - 13https://github.com/miraheze/mediawiki/pull/7814 [04:54:11] [url] Page not found · GitHub · GitHub | github.com [04:54:12] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PortableInfobox-abc29f3 - 13https://github.com/miraheze/mediawiki [04:54:12] ... [04:54:13] [02mediawiki] 07dependabot[bot] opened pull request 03#7934: Bump extensions/PortableInfobox from `6e0bc5e` to `abc29f3` - 13https://github.com/miraheze/mediawiki/pull/7934 [04:54:14] ... [04:54:15] [02mediawiki] 07github-actions[bot] labeled pull request 03#7924: Bump extensions/SimpleChanges from `c510c1c` to `78d7ab4` - 13https://github.com/miraheze/mediawiki/pull/7924 [04:54:15] [url] Page not found · GitHub · GitHub | github.com [04:54:16] [02mediawiki] 07github-actions[bot] labeled pull request 03#7928: Bump extensions/TranslationNotifications from `54a8197` to `60ffe02` - 13https://github.com/miraheze/mediawiki/pull/7928 [04:54:17] [url] Page not found · GitHub · GitHub | github.com [04:54:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/SimpleTooltip-383fa88 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/6fb7e3cb9058 [04:54:19] [02miraheze/mediawiki] 07dependabot[bot] 036fb7e3c - Bump extensions/SimpleTooltip from `2b8c3d0` to `383fa88` [04:54:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/SimpleTooltip-383fa88 - 13https://github.com/miraheze/mediawiki [04:54:21] [url] Page not found · GitHub · GitHub | github.com [04:54:22] [02mediawiki] 07dependabot[bot] opened pull request 03#7935: Bump extensions/SimpleTooltip from `2b8c3d0` to `383fa88` - 13https://github.com/miraheze/mediawiki/pull/7935 [04:54:23] ... [04:54:24] [02mediawiki] 07github-actions[bot] labeled pull request 03#7925: Bump extensions/GlobalBlocking from `99bb1ba` to `07a59e7` - 13https://github.com/miraheze/mediawiki/pull/7925 [04:54:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7927: Bump extensions/MediaWikiChat from `c2398b2` to `ddfa325` - 13https://github.com/miraheze/mediawiki/pull/7927 [04:54:27] [02mediawiki] 07github-actions[bot] labeled pull request 03#7926: Bump extensions/RemovePII from `20686b4` to `3be5c88` - 13https://github.com/miraheze/mediawiki/pull/7926 [04:54:28] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Nuke-aeb2a1a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/37cd11c908e8 [04:54:30] [02miraheze/mediawiki] 07dependabot[bot] 0337cd11c - Bump extensions/Nuke from `1f07b7a` to `aeb2a1a` [04:54:31] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Nuke-aeb2a1a - 13https://github.com/miraheze/mediawiki [04:54:33] [02mediawiki] 07dependabot[bot] opened pull request 03#7936: Bump extensions/Nuke from `1f07b7a` to `aeb2a1a` - 13https://github.com/miraheze/mediawiki/pull/7936 [04:54:34] [02mediawiki] 07github-actions[bot] labeled pull request 03#7926: Bump extensions/RemovePII from `20686b4` to `3be5c88` - 13https://github.com/miraheze/mediawiki/pull/7926 [04:54:36] [02mediawiki] 07github-actions[bot] labeled pull request 03#7930: Bump extensions/Cite from `7fdd57d` to `7395f4d` - 13https://github.com/miraheze/mediawiki/pull/7930 [04:54:37] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/OAuth-40a1df3 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/99fc33b4eabb [04:54:39] [02miraheze/mediawiki] 07dependabot[bot] 0399fc33b - Bump extensions/OAuth from `b088139` to `40a1df3` [04:54:40] [02mediawiki] 07dependabot[bot] opened pull request 03#7937: Bump extensions/OAuth from `b088139` to `40a1df3` - 13https://github.com/miraheze/mediawiki/pull/7937 [04:54:42] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/OAuth-40a1df3 - 13https://github.com/miraheze/mediawiki [04:54:43] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/GrowthExperiments-472a196 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/863e93a0e957 [04:54:45] [02miraheze/mediawiki] 07dependabot[bot] 03863e93a - Bump extensions/GrowthExperiments from `8413d8b` to `472a196` [04:54:46] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/GrowthExperiments-472a196 - 13https://github.com/miraheze/mediawiki [04:54:48] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.06, 3.69, 3.51 [04:54:48] [02mediawiki] 07dependabot[bot] opened pull request 03#7938: Bump extensions/GrowthExperiments from `8413d8b` to `472a196` - 13https://github.com/miraheze/mediawiki/pull/7938 [04:54:49] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-6ce773e [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/89afe8a0f865 [04:54:51] [02miraheze/mediawiki] 07dependabot[bot] 0389afe8a - Bump extensions/UniversalLanguageSelector from `7262dda` to `6ce773e` [04:54:52] [02mediawiki] 07dependabot[bot] commented on pull request 03#7837: Bump extensions/OAuth from `b088139` to `519e4e5` - 13https://github.com/miraheze/mediawiki/pull/7837#issuecomment-1171929891 [04:54:53] [url] Bump extensions/OAuth from `b088139` to `519e4e5` by dependabot[bot] · Pull Request #7837 · miraheze/mediawiki · GitHub | github.com [04:54:54] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/OAuth-519e4e5 [04:54:55] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-6ce773e - 13https://github.com/miraheze/mediawiki [04:54:55] [url] Page not found · GitHub · GitHub | github.com [04:54:57] [02mediawiki] 07dependabot[bot] closed pull request 03#7837: Bump extensions/OAuth from `b088139` to `519e4e5` - 13https://github.com/miraheze/mediawiki/pull/7837 [04:54:57] [url] Page not found · GitHub · GitHub | github.com [04:54:58] [02mediawiki] 07dependabot[bot] commented on pull request 03#7811: Bump extensions/GrowthExperiments from `8413d8b` to `fc94744` - 13https://github.com/miraheze/mediawiki/pull/7811#issuecomment-1171929909 [04:54:59] [url] Bump extensions/GrowthExperiments from `8413d8b` to `fc94744` by dependabot[bot] · Pull Request #7811 · miraheze/mediawiki · GitHub | github.com [04:55:00] [02mediawiki] 07dependabot[bot] opened pull request 03#7939: Bump extensions/UniversalLanguageSelector from `7262dda` to `6ce773e` - 13https://github.com/miraheze/mediawiki/pull/7939 [04:55:00] ... [04:55:01] [02mediawiki] 07github-actions[bot] labeled pull request 03#7929: Bump extensions/Linter from `36320b4` to `9bacd76` - 13https://github.com/miraheze/mediawiki/pull/7929 [04:55:02] [url] Page not found · GitHub · GitHub | github.com [04:55:03] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PageSchemas-1d16b69 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/9caeea78920a [04:55:03] [url] Page not found · GitHub · GitHub | github.com [04:55:04] [02miraheze/mediawiki] 07dependabot[bot] 039caeea7 - Bump extensions/PageSchemas from `589aaa2` to `1d16b69` [04:55:05] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.62, 3.25, 3.19 [04:55:06] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/OAuth-519e4e5 - 13https://github.com/miraheze/mediawiki [04:55:06] [url] Page not found · GitHub · GitHub | github.com [04:55:07] [02mediawiki] 07dependabot[bot] closed pull request 03#7811: Bump extensions/GrowthExperiments from `8413d8b` to `fc94744` - 13https://github.com/miraheze/mediawiki/pull/7811 [04:55:08] ... [04:55:09] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PageSchemas-1d16b69 - 13https://github.com/miraheze/mediawiki [04:55:09] ... [04:55:10] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GrowthExperiments-fc94744 [04:55:12] [02mediawiki] 07dependabot[bot] opened pull request 03#7940: Bump extensions/PageSchemas from `589aaa2` to `1d16b69` - 13https://github.com/miraheze/mediawiki/pull/7940 [04:55:12] [url] Page not found · GitHub · GitHub | github.com [04:55:13] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/GrowthExperiments-fc94744 - 13https://github.com/miraheze/mediawiki [04:55:13] [url] Page not found · GitHub · GitHub | github.com [04:55:15] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/CreateWiki-31ac68c [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/9ee9c9c9b189 [04:55:16] [02miraheze/mediawiki] 07dependabot[bot] 039ee9c9c - Bump extensions/CreateWiki from `e281042` to `31ac68c` [04:55:18] [02mediawiki] 07dependabot[bot] commented on pull request 03#7868: Bump extensions/UniversalLanguageSelector from `7262dda` to `c9fef46` - 13https://github.com/miraheze/mediawiki/pull/7868#issuecomment-1171929922 [04:55:18] [url] Bump extensions/UniversalLanguageSelector from `7262dda` to `c9fef46` by dependabot[bot] · Pull Request #7868 · miraheze/mediawiki · GitHub | github.com [04:55:19] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-c9fef46 [04:55:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/CreateWiki-31ac68c - 13https://github.com/miraheze/mediawiki [04:55:21] ... [04:55:22] [02mediawiki] 07dependabot[bot] closed pull request 03#7868: Bump extensions/UniversalLanguageSelector from `7262dda` to `c9fef46` - 13https://github.com/miraheze/mediawiki/pull/7868 [04:55:23] [url] Page not found · GitHub · GitHub | github.com [04:55:24] [02mediawiki] 07dependabot[bot] opened pull request 03#7941: Bump extensions/CreateWiki from `e281042` to `31ac68c` - 13https://github.com/miraheze/mediawiki/pull/7941 [04:55:24] [url] Page not found · GitHub · GitHub | github.com [04:55:25] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-c9fef46 - 13https://github.com/miraheze/mediawiki [04:55:25] [url] Page not found · GitHub · GitHub | github.com [04:55:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ImportDump-44997fb [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/7bb9f594ac52 [04:55:28] [02miraheze/mediawiki] 07dependabot[bot] 037bb9f59 - Bump extensions/ImportDump from `1826c01` to `44997fb` [04:55:30] [02mediawiki] 07github-actions[bot] labeled pull request 03#7932: Bump extensions/Kartographer from `67f9065` to `bcb0b66` - 13https://github.com/miraheze/mediawiki/pull/7932 [04:55:30] ... [04:55:31] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ImportDump-44997fb - 13https://github.com/miraheze/mediawiki [04:55:31] ... [04:55:33] [02mediawiki] 07dependabot[bot] commented on pull request 03#7853: Bump extensions/CreateWiki from `d9b7b04` to `02aee6a` - 13https://github.com/miraheze/mediawiki/pull/7853#issuecomment-1171929954 [04:55:33] [url] Bump extensions/CreateWiki from `d9b7b04` to `02aee6a` by dependabot[bot] · Pull Request #7853 · miraheze/mediawiki · GitHub | github.com [04:55:34] [02mediawiki] 07dependabot[bot] opened pull request 03#7942: Bump extensions/ImportDump from `1826c01` to `44997fb` - 13https://github.com/miraheze/mediawiki/pull/7942 [04:55:35] [url] Page not found · GitHub · GitHub | github.com [04:55:36] [02mediawiki] 07github-actions[bot] labeled pull request 03#7931: Bump extensions/MirahezeMagic from `d38b6ae` to `515f04f` - 13https://github.com/miraheze/mediawiki/pull/7931 [04:55:36] [url] Page not found · GitHub · GitHub | github.com [04:55:37] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CreateWiki-02aee6a [04:55:39] [02mediawiki] 07dependabot[bot] closed pull request 03#7853: Bump extensions/CreateWiki from `d9b7b04` to `02aee6a` - 13https://github.com/miraheze/mediawiki/pull/7853 [04:55:39] ... [04:55:40] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Cosmos-579619a [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/1f33ae9c896f [04:55:42] [02miraheze/mediawiki] 07dependabot[bot] 031f33ae9 - Bump skins/Cosmos from `f2be12b` to `579619a` [04:55:43] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/CreateWiki-02aee6a - 13https://github.com/miraheze/mediawiki [04:55:43] [url] Page not found · GitHub · GitHub | github.com [04:55:45] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Cosmos-579619a - 13https://github.com/miraheze/mediawiki [04:55:45] [url] Page not found · GitHub · GitHub | github.com [04:55:46] [02mediawiki] 07github-actions[bot] labeled pull request 03#7934: Bump extensions/PortableInfobox from `6e0bc5e` to `abc29f3` - 13https://github.com/miraheze/mediawiki/pull/7934 [04:55:47] [url] Page not found · GitHub · GitHub | github.com [04:55:48] [02mediawiki] 07github-actions[bot] labeled pull request 03#7935: Bump extensions/SimpleTooltip from `2b8c3d0` to `383fa88` - 13https://github.com/miraheze/mediawiki/pull/7935 [04:55:48] ... [04:55:49] [02mediawiki] 07github-actions[bot] labeled pull request 03#7933: Bump extensions/WikimediaIncubator from `98caf22` to `1e9870d` - 13https://github.com/miraheze/mediawiki/pull/7933 [04:56:00] [02miraheze/mediawiki] 07dependabot[bot] 03e67833a - Bump extensions/AutoCreatePage from `c616215` to `9476f36` [04:56:01] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/AutoCreatePage-9476f36 - 13https://github.com/miraheze/mediawiki [04:56:03] [02mediawiki] 07dependabot[bot] opened pull request 03#7945: Bump extensions/AutoCreatePage from `c616215` to `9476f36` - 13https://github.com/miraheze/mediawiki/pull/7945 [04:56:04] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikibaseLexeme-242d4c9 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ead76845c578 [04:56:06] [02miraheze/mediawiki] 07dependabot[bot] 03ead7684 - Bump extensions/WikibaseLexeme from `3ea04a9` to `242d4c9` [04:56:07] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikibaseLexeme-242d4c9 - 13https://github.com/miraheze/mediawiki [04:56:09] [02mediawiki] 07dependabot[bot] opened pull request 03#7946: Bump extensions/WikibaseLexeme from `3ea04a9` to `242d4c9` - 13https://github.com/miraheze/mediawiki/pull/7946 [04:56:10] [02mediawiki] 07github-actions[bot] labeled pull request 03#7936: Bump extensions/Nuke from `1f07b7a` to `aeb2a1a` - 13https://github.com/miraheze/mediawiki/pull/7936 [04:56:12] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PageImages-7dccab8 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c5a0b6c5467e [04:56:13] [02miraheze/mediawiki] 07dependabot[bot] 03c5a0b6c - Bump extensions/PageImages from `4656266` to `7dccab8` [04:56:15] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PageImages-7dccab8 - 13https://github.com/miraheze/mediawiki [04:56:16] [02mediawiki] 07dependabot[bot] opened pull request 03#7947: Bump extensions/PageImages from `4656266` to `7dccab8` - 13https://github.com/miraheze/mediawiki/pull/7947 [04:56:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Gadgets-1cb0c21 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/70097dcd1b2b [04:56:19] [02miraheze/mediawiki] 07dependabot[bot] 0370097dc - Bump extensions/Gadgets from `ce7b6bd` to `1cb0c21` [04:56:21] [02mediawiki] 07github-actions[bot] labeled pull request 03#7938: Bump extensions/GrowthExperiments from `8413d8b` to `472a196` - 13https://github.com/miraheze/mediawiki/pull/7938 [04:56:22] [02mediawiki] 07github-actions[bot] labeled pull request 03#7937: Bump extensions/OAuth from `b088139` to `40a1df3` - 13https://github.com/miraheze/mediawiki/pull/7937 [04:56:24] [02mediawiki] 07dependabot[bot] opened pull request 03#7948: Bump extensions/Gadgets from `ce7b6bd` to `1cb0c21` - 13https://github.com/miraheze/mediawiki/pull/7948 [04:56:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7942: Bump extensions/ImportDump from `1826c01` to `44997fb` - 13https://github.com/miraheze/mediawiki/pull/7942 [04:56:27] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Gadgets-1cb0c21 - 13https://github.com/miraheze/mediawiki [04:56:28] [02mediawiki] 07github-actions[bot] labeled pull request 03#7937: Bump extensions/OAuth from `b088139` to `40a1df3` - 13https://github.com/miraheze/mediawiki/pull/7937 [04:56:30] [02mediawiki] 07github-actions[bot] labeled pull request 03#7942: Bump extensions/ImportDump from `1826c01` to `44997fb` - 13https://github.com/miraheze/mediawiki/pull/7942 [04:56:31] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/DiscussionTools-9a8d011 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/49b9d3bb9ff7 [04:56:33] [02miraheze/mediawiki] 07dependabot[bot] 0349b9d3b - Bump extensions/DiscussionTools from `bef5de9` to `9a8d011` [04:56:34] [02mediawiki] 07github-actions[bot] labeled pull request 03#7939: Bump extensions/UniversalLanguageSelector from `7262dda` to `6ce773e` - 13https://github.com/miraheze/mediawiki/pull/7939 [04:56:36] [02mediawiki] 07dependabot[bot] opened pull request 03#7949: Bump extensions/DiscussionTools from `bef5de9` to `9a8d011` - 13https://github.com/miraheze/mediawiki/pull/7949 [04:56:37] [02mediawiki] 07github-actions[bot] labeled pull request 03#7940: Bump extensions/PageSchemas from `589aaa2` to `1d16b69` - 13https://github.com/miraheze/mediawiki/pull/7940 [04:56:39] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/DiscussionTools-9a8d011 - 13https://github.com/miraheze/mediawiki [04:56:40] [02mediawiki] 07dependabot[bot] commented on pull request 03#7834: Bump extensions/Gadgets from `ce7b6bd` to `76c8e06` - 13https://github.com/miraheze/mediawiki/pull/7834#issuecomment-1171930107 [04:56:41] [url] Bump extensions/Gadgets from `ce7b6bd` to `76c8e06` by dependabot[bot] · Pull Request #7834 · miraheze/mediawiki · GitHub | github.com [04:56:42] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.09, 3.38, 3.40 [04:56:42] [02mediawiki] 07github-actions[bot] labeled pull request 03#7939: Bump extensions/UniversalLanguageSelector from `7262dda` to `6ce773e` - 13https://github.com/miraheze/mediawiki/pull/7939 [04:56:42] [url] Page not found · GitHub · GitHub | github.com [04:56:43] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [04:56:43] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Gadgets-76c8e06 [04:56:45] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Gadgets-76c8e06 - 13https://github.com/miraheze/mediawiki [04:56:45] [url] Page not found · GitHub · GitHub | github.com [04:56:46] [02mediawiki] 07github-actions[bot] labeled pull request 03#7941: Bump extensions/CreateWiki from `e281042` to `31ac68c` - 13https://github.com/miraheze/mediawiki/pull/7941 [04:56:47] ... [04:56:48] [02mediawiki] 07dependabot[bot] closed pull request 03#7834: Bump extensions/Gadgets from `ce7b6bd` to `76c8e06` - 13https://github.com/miraheze/mediawiki/pull/7834 [04:56:49] [url] Page not found · GitHub · GitHub | github.com [04:56:49] [02mediawiki] 07dependabot[bot] commented on pull request 03#7846: Bump extensions/DiscussionTools from `bef5de9` to `2b9999d` - 13https://github.com/miraheze/mediawiki/pull/7846#issuecomment-1171930121 [04:56:50] [url] Bump extensions/DiscussionTools from `bef5de9` to `2b9999d` by dependabot[bot] · Pull Request #7846 · miraheze/mediawiki · GitHub | github.com [04:56:51] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/WikidataPageBanner-80fb9f2 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/50e85038e976 [04:56:52] [02miraheze/mediawiki] 07dependabot[bot] 0350e8503 - Bump extensions/WikidataPageBanner from `0b82b16` to `80fb9f2` [04:56:54] [02mediawiki] 07dependabot[bot] closed pull request 03#7846: Bump extensions/DiscussionTools from `bef5de9` to `2b9999d` - 13https://github.com/miraheze/mediawiki/pull/7846 [04:56:54] [url] Page not found · GitHub · GitHub | github.com [04:56:55] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/DiscussionTools-2b9999d [04:56:57] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/WikidataPageBanner-80fb9f2 - 13https://github.com/miraheze/mediawiki [04:56:57] [url] Page not found · GitHub · GitHub | github.com [04:56:58] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/RelatedArticles-d08191d [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/e7dce79e3af1 [04:57:00] [02miraheze/mediawiki] 07dependabot[bot] 03e7dce79 - Bump extensions/RelatedArticles from `736f38a` to `d08191d` [04:57:01] [02mediawiki] 07dependabot[bot] opened pull request 03#7950: Bump extensions/WikidataPageBanner from `0b82b16` to `80fb9f2` - 13https://github.com/miraheze/mediawiki/pull/7950 [04:57:02] ... [04:57:03] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/DiscussionTools-2b9999d - 13https://github.com/miraheze/mediawiki [04:57:03] ... [04:57:04] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/RelatedArticles-d08191d - 13https://github.com/miraheze/mediawiki [04:57:04] [url] Page not found · GitHub · GitHub | github.com [04:57:06] [02mediawiki] 07dependabot[bot] opened pull request 03#7951: Bump extensions/RelatedArticles from `736f38a` to `d08191d` - 13https://github.com/miraheze/mediawiki/pull/7951 [04:57:06] [url] Page not found · GitHub · GitHub | github.com [04:57:07] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/RottenLinks-5552b0f [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/5b0eeb75a32e [04:57:09] [02miraheze/mediawiki] 07dependabot[bot] 035b0eeb7 - Bump extensions/RottenLinks from `29bb9c7` to `5552b0f` [04:57:10] [02mediawiki] 07dependabot[bot] commented on pull request 03#7829: Bump extensions/RelatedArticles from `736f38a` to `91f4177` - 13https://github.com/miraheze/mediawiki/pull/7829#issuecomment-1171930156 [04:57:11] [url] Bump extensions/RelatedArticles from `736f38a` to `91f4177` by dependabot[bot] · Pull Request #7829 · miraheze/mediawiki · GitHub | github.com [04:57:12] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/RottenLinks-5552b0f - 13https://github.com/miraheze/mediawiki [04:57:12] [url] Page not found · GitHub · GitHub | github.com [04:57:13] [02mediawiki] 07dependabot[bot] opened pull request 03#7952: Bump extensions/RottenLinks from `29bb9c7` to `5552b0f` - 13https://github.com/miraheze/mediawiki/pull/7952 [04:57:14] ... [04:57:15] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/RelatedArticles-91f4177 [04:57:16] [02mediawiki] 07dependabot[bot] closed pull request 03#7829: Bump extensions/RelatedArticles from `736f38a` to `91f4177` - 13https://github.com/miraheze/mediawiki/pull/7829 [04:57:17] [url] Page not found · GitHub · GitHub | github.com [04:57:18] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/FlaggedRevs-4bde537 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/77e5c996f626 [04:57:19] [02miraheze/mediawiki] 07dependabot[bot] 0377e5c99 - Bump extensions/FlaggedRevs from `ebcab9c` to `4bde537` [04:57:21] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/RelatedArticles-91f4177 - 13https://github.com/miraheze/mediawiki [04:57:21] [url] Page not found · GitHub · GitHub | github.com [04:57:22] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/FlaggedRevs-4bde537 - 13https://github.com/miraheze/mediawiki [04:57:23] ... [04:57:23] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 0.477 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [04:57:24] [02mediawiki] 07dependabot[bot] opened pull request 03#7953: Bump extensions/FlaggedRevs from `ebcab9c` to `4bde537` - 13https://github.com/miraheze/mediawiki/pull/7953 [04:57:24] ... [04:57:25] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/GlobalWatchlist-fdfeb34 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/864804ec515f [04:57:27] [02miraheze/mediawiki] 07dependabot[bot] 03864804e - Bump extensions/GlobalWatchlist from `c2ec938` to `fdfeb34` [04:57:28] [02mediawiki] 07dependabot[bot] opened pull request 03#7954: Bump extensions/GlobalWatchlist from `c2ec938` to `fdfeb34` - 13https://github.com/miraheze/mediawiki/pull/7954 [04:57:29] [url] Page not found · GitHub · GitHub | github.com [04:57:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/GlobalWatchlist-fdfeb34 - 13https://github.com/miraheze/mediawiki [04:57:30] [url] Page not found · GitHub · GitHub | github.com [04:57:31] [02mediawiki] 07dependabot[bot] commented on pull request 03#7865: Bump extensions/FlaggedRevs from `ebcab9c` to `d8505d0` - 13https://github.com/miraheze/mediawiki/pull/7865#issuecomment-1171930198 [04:57:32] [url] Bump extensions/FlaggedRevs from `ebcab9c` to `d8505d0` by dependabot[bot] · Pull Request #7865 · miraheze/mediawiki · GitHub | github.com [04:57:33] [02miraheze/mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/FlaggedRevs-d8505d0 [04:57:34] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Evelution-e04e412 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/094cbc0e74dd [04:57:36] [02miraheze/mediawiki] 07dependabot[bot] 03094cbc0 - Bump skins/Evelution from `b44960e` to `e04e412` [04:57:37] [02mediawiki] 07dependabot[bot] closed pull request 03#7865: Bump extensions/FlaggedRevs from `ebcab9c` to `d8505d0` - 13https://github.com/miraheze/mediawiki/pull/7865 [04:57:38] [url] Page not found · GitHub · GitHub | github.com [04:57:39] [02mediawiki] 07github-actions[bot] labeled pull request 03#7945: Bump extensions/AutoCreatePage from `c616215` to `9476f36` - 13https://github.com/miraheze/mediawiki/pull/7945 [04:57:39] ... [04:57:40] [02mediawiki] 07github-actions[bot] labeled pull request 03#7948: Bump extensions/Gadgets from `ce7b6bd` to `1cb0c21` - 13https://github.com/miraheze/mediawiki/pull/7948 [04:57:41] [url] Page not found · GitHub · GitHub | github.com [04:57:42] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Evelution-e04e412 - 13https://github.com/miraheze/mediawiki [04:57:42] [url] Page not found · GitHub · GitHub | github.com [04:57:43] [02mediawiki] 07dependabot[bot] opened pull request 03#7955: Bump skins/Evelution from `b44960e` to `e04e412` - 13https://github.com/miraheze/mediawiki/pull/7955 [04:57:44] ... [04:57:45] [02mediawiki] 07dependabot[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/FlaggedRevs-d8505d0 - 13https://github.com/miraheze/mediawiki [04:57:45] ... [04:57:46] [02mediawiki] 07github-actions[bot] labeled pull request 03#7948: Bump extensions/Gadgets from `ce7b6bd` to `1cb0c21` - 13https://github.com/miraheze/mediawiki/pull/7948 [04:57:47] [url] Page not found · GitHub · GitHub | github.com [04:57:48] [02mediawiki] 07github-actions[bot] labeled pull request 03#7944: Bump skins/Monaco from `76054b1` to `9e202c8` - 13https://github.com/miraheze/mediawiki/pull/7944 [04:57:48] [url] Page not found · GitHub · GitHub | github.com [04:57:49] [02mediawiki] 07github-actions[bot] labeled pull request 03#7943: Bump skins/Cosmos from `f2be12b` to `579619a` - 13https://github.com/miraheze/mediawiki/pull/7943 [04:57:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#7943: Bump skins/Cosmos from `f2be12b` to `579619a` - 13https://github.com/miraheze/mediawiki/pull/7943 [04:57:52] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/SimpleBlogPage-a2825df [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/d64ab1f1f36e [04:57:54] [02miraheze/mediawiki] 07dependabot[bot] 03d64ab1f - Bump extensions/SimpleBlogPage from `d850c69` to `a2825df` [04:57:55] [02mediawiki] 07dependabot[bot] opened pull request 03#7956: Bump extensions/SimpleBlogPage from `d850c69` to `a2825df` - 13https://github.com/miraheze/mediawiki/pull/7956 [04:57:57] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/SimpleBlogPage-a2825df - 13https://github.com/miraheze/mediawiki [04:57:58] [02mediawiki] 07github-actions[bot] labeled pull request 03#7947: Bump extensions/PageImages from `4656266` to `7dccab8` - 13https://github.com/miraheze/mediawiki/pull/7947 [04:58:00] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/VisualEditor-1b03980 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/adc30c2f9ceb [04:58:01] [02miraheze/mediawiki] 07dependabot[bot] 03adc30c2 - Bump extensions/VisualEditor from `fccd2a5` to `1b03980` [04:58:03] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/VisualEditor-1b03980 - 13https://github.com/miraheze/mediawiki [04:58:04] [02mediawiki] 07dependabot[bot] opened pull request 03#7957: Bump extensions/VisualEditor from `fccd2a5` to `1b03980` - 13https://github.com/miraheze/mediawiki/pull/7957 [04:58:06] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Babel-7d8a485 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/bbfef5a34659 [04:58:07] [02miraheze/mediawiki] 07dependabot[bot] 03bbfef5a - Bump extensions/Babel from `4fc4e46` to `7d8a485` [04:58:09] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Babel-7d8a485 - 13https://github.com/miraheze/mediawiki [04:58:10] [02mediawiki] 07dependabot[bot] opened pull request 03#7958: Bump extensions/Babel from `4fc4e46` to `7d8a485` - 13https://github.com/miraheze/mediawiki/pull/7958 [04:58:12] [02mediawiki] 07github-actions[bot] labeled pull request 03#7950: Bump extensions/WikidataPageBanner from `0b82b16` to `80fb9f2` - 13https://github.com/miraheze/mediawiki/pull/7950 [04:58:13] [02mediawiki] 07github-actions[bot] labeled pull request 03#7952: Bump extensions/RottenLinks from `29bb9c7` to `5552b0f` - 13https://github.com/miraheze/mediawiki/pull/7952 [04:58:15] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Preloader-58f20ef [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/1b85ed65567f [04:58:16] [02miraheze/mediawiki] 07dependabot[bot] 031b85ed6 - Bump extensions/Preloader from `0e409d4` to `58f20ef` [04:58:18] [02mediawiki] 07github-actions[bot] labeled pull request 03#7949: Bump extensions/DiscussionTools from `bef5de9` to `9a8d011` - 13https://github.com/miraheze/mediawiki/pull/7949 [04:58:19] [02mediawiki] 07github-actions[bot] labeled pull request 03#7946: Bump extensions/WikibaseLexeme from `3ea04a9` to `242d4c9` - 13https://github.com/miraheze/mediawiki/pull/7946 [04:58:21] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Preloader-58f20ef - 13https://github.com/miraheze/mediawiki [04:58:22] [02mediawiki] 07github-actions[bot] labeled pull request 03#7946: Bump extensions/WikibaseLexeme from `3ea04a9` to `242d4c9` - 13https://github.com/miraheze/mediawiki/pull/7946 [04:58:24] [02mediawiki] 07dependabot[bot] opened pull request 03#7959: Bump extensions/Preloader from `0e409d4` to `58f20ef` - 13https://github.com/miraheze/mediawiki/pull/7959 [04:58:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7946: Bump extensions/WikibaseLexeme from `3ea04a9` to `242d4c9` - 13https://github.com/miraheze/mediawiki/pull/7946 [04:58:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ExternalData-52025c9 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c092ad9e5e1f [04:58:28] [02miraheze/mediawiki] 07dependabot[bot] 03c092ad9 - Bump extensions/ExternalData from `9b5cca8` to `52025c9` [04:58:30] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ExternalData-52025c9 - 13https://github.com/miraheze/mediawiki [04:58:31] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ArticlePlaceholder-151ab21 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/365349e039dd [04:58:33] [02miraheze/mediawiki] 07dependabot[bot] 03365349e - Bump extensions/ArticlePlaceholder from `983ab1b` to `151ab21` [04:58:34] [02mediawiki] 07dependabot[bot] opened pull request 03#7960: Bump extensions/ExternalData from `9b5cca8` to `52025c9` - 13https://github.com/miraheze/mediawiki/pull/7960 [04:58:36] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ArticlePlaceholder-151ab21 - 13https://github.com/miraheze/mediawiki [04:58:37] [02mediawiki] 07dependabot[bot] opened pull request 03#7961: Bump extensions/ArticlePlaceholder from `983ab1b` to `151ab21` - 13https://github.com/miraheze/mediawiki/pull/7961 [04:58:39] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/MinervaNeue-57ca1fd [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/2b6607d64be5 [04:58:40] [02miraheze/mediawiki] 07dependabot[bot] 032b6607d - Bump skins/MinervaNeue from `c0092c5` to `57ca1fd` [04:58:42] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/MinervaNeue-57ca1fd - 13https://github.com/miraheze/mediawiki [04:58:43] [02mediawiki] 07dependabot[bot] opened pull request 03#7962: Bump skins/MinervaNeue from `c0092c5` to `57ca1fd` - 13https://github.com/miraheze/mediawiki/pull/7962 [04:58:45] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/vendor-0b71b20 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ac7ff7e2a680 [04:58:46] [02miraheze/mediawiki] 07dependabot[bot] 03ac7ff7e - Bump vendor from `d6f273d` to `0b71b20` [04:58:48] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/vendor-0b71b20 - 13https://github.com/miraheze/mediawiki [04:58:49] [02mediawiki] 07github-actions[bot] labeled pull request 03#7951: Bump extensions/RelatedArticles from `736f38a` to `d08191d` - 13https://github.com/miraheze/mediawiki/pull/7951 [04:58:51] [02mediawiki] 07github-actions[bot] labeled pull request 03#7953: Bump extensions/FlaggedRevs from `ebcab9c` to `4bde537` - 13https://github.com/miraheze/mediawiki/pull/7953 [04:58:52] [02mediawiki] 07dependabot[bot] opened pull request 03#7963: Bump vendor from `d6f273d` to `0b71b20` - 13https://github.com/miraheze/mediawiki/pull/7963 [04:58:54] [02mediawiki] 07github-actions[bot] labeled pull request 03#7955: Bump skins/Evelution from `b44960e` to `e04e412` - 13https://github.com/miraheze/mediawiki/pull/7955 [04:58:55] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ApprovedRevs-d4b29d5 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/1f3101c41ef4 [04:58:57] [02miraheze/mediawiki] 07dependabot[bot] 031f3101c - Bump extensions/ApprovedRevs from `265e4a7` to `d4b29d5` [04:58:58] [02mediawiki] 07dependabot[bot] opened pull request 03#7964: Bump extensions/ApprovedRevs from `265e4a7` to `d4b29d5` - 13https://github.com/miraheze/mediawiki/pull/7964 [04:59:00] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ApprovedRevs-d4b29d5 - 13https://github.com/miraheze/mediawiki [04:59:01] [02mediawiki] 07github-actions[bot] labeled pull request 03#7956: Bump extensions/SimpleBlogPage from `d850c69` to `a2825df` - 13https://github.com/miraheze/mediawiki/pull/7956 [04:59:03] [02mediawiki] 07github-actions[bot] labeled pull request 03#7954: Bump extensions/GlobalWatchlist from `c2ec938` to `fdfeb34` - 13https://github.com/miraheze/mediawiki/pull/7954 [04:59:04] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PageForms-f726836 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/51bfef596313 [04:59:05] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.216 second response time [04:59:06] [02miraheze/mediawiki] 07dependabot[bot] 0351bfef5 - Bump extensions/PageForms from `93457d2` to `f726836` [04:59:07] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PageForms-f726836 - 13https://github.com/miraheze/mediawiki [04:59:09] [02mediawiki] 07dependabot[bot] opened pull request 03#7965: Bump extensions/PageForms from `93457d2` to `f726836` - 13https://github.com/miraheze/mediawiki/pull/7965 [04:59:10] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/ProofreadPage-3b771f3 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/7cd1e5c3fd2c [04:59:12] [02miraheze/mediawiki] 07dependabot[bot] 037cd1e5c - Bump extensions/ProofreadPage from `00fb628` to `3b771f3` [04:59:13] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/ProofreadPage-3b771f3 - 13https://github.com/miraheze/mediawiki [04:59:15] [02mediawiki] 07dependabot[bot] opened pull request 03#7966: Bump extensions/ProofreadPage from `00fb628` to `3b771f3` - 13https://github.com/miraheze/mediawiki/pull/7966 [04:59:16] [02mediawiki] 07github-actions[bot] labeled pull request 03#7959: Bump extensions/Preloader from `0e409d4` to `58f20ef` - 13https://github.com/miraheze/mediawiki/pull/7959 [04:59:18] [02mediawiki] 07github-actions[bot] labeled pull request 03#7957: Bump extensions/VisualEditor from `fccd2a5` to `1b03980` - 13https://github.com/miraheze/mediawiki/pull/7957 [04:59:19] [02mediawiki] 07github-actions[bot] labeled pull request 03#7961: Bump extensions/ArticlePlaceholder from `983ab1b` to `151ab21` - 13https://github.com/miraheze/mediawiki/pull/7961 [04:59:20] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.055 second response time [04:59:21] [02mediawiki] 07github-actions[bot] labeled pull request 03#7960: Bump extensions/ExternalData from `9b5cca8` to `52025c9` - 13https://github.com/miraheze/mediawiki/pull/7960 [04:59:22] [02mediawiki] 07github-actions[bot] labeled pull request 03#7962: Bump skins/MinervaNeue from `c0092c5` to `57ca1fd` - 13https://github.com/miraheze/mediawiki/pull/7962 [04:59:24] [02mediawiki] 07github-actions[bot] labeled pull request 03#7958: Bump extensions/Babel from `4fc4e46` to `7d8a485` - 13https://github.com/miraheze/mediawiki/pull/7958 [04:59:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7965: Bump extensions/PageForms from `93457d2` to `f726836` - 13https://github.com/miraheze/mediawiki/pull/7965 [04:59:27] [02mediawiki] 07github-actions[bot] labeled pull request 03#7964: Bump extensions/ApprovedRevs from `265e4a7` to `d4b29d5` - 13https://github.com/miraheze/mediawiki/pull/7964 [04:59:27] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 2.989 second response time [04:59:28] [02mediawiki] 07github-actions[bot] labeled pull request 03#7964: Bump extensions/ApprovedRevs from `265e4a7` to `d4b29d5` - 13https://github.com/miraheze/mediawiki/pull/7964 [04:59:30] [02mediawiki] 07github-actions[bot] labeled pull request 03#7966: Bump extensions/ProofreadPage from `00fb628` to `3b771f3` - 13https://github.com/miraheze/mediawiki/pull/7966 [04:59:33] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 1.795 second response time [04:59:39] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.118 second response time [04:59:43] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.020 second response time [04:59:55] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.873 second response time [05:00:30] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 9.190 second response time [05:00:34] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.330 second response time [05:00:35] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures [05:00:39] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.058 second response time [05:00:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.39, 3.44, 3.28 [05:01:04] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27106 bytes in 0.559 second response time [05:02:53] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.64, 2.83, 3.08 [05:03:36] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.82, 3.46, 3.93 [05:05:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.31, 3.65, 3.94 [05:06:52] PROBLEM - mw121 MediaWiki Rendering on mw121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:01] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:07] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:17] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:19] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:40] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:46] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:07:46] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:08:02] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:08:22] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:08:50] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:09:03] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:09:04] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:09:28] PROBLEM - mwtask111 MediaWiki Rendering on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:09:41] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.609 second response time [05:09:42] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.019 second response time [05:10:18] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.057 second response time [05:11:00] RECOVERY - mw121 MediaWiki Rendering on mw121 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 5.057 second response time [05:13:54] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:13:56] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:14:24] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:14:37] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.36, 3.42, 3.11 [05:15:02] PROBLEM - mwtask111 Check Gluster Clients on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:15:23] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:15:25] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:15:28] PROBLEM - mwtask111 NTP time on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:15:49] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw111 [05:15:50] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:15:52] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 8.085 second response time [05:15:54] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:16:09] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 1.958 second response time [05:16:36] PROBLEM - m33ks.ml - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'm33ks.ml' expires in 15 day(s) (Sun 17 Jul 2022 05:09:25 GMT +0000). [05:16:37] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:16:45] PROBLEM - mwtask111 JobRunner Service on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:16:47] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:16:50] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:16:59] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:17:02] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:17:02] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:17:15] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.277 second response time [05:17:18] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.941 second response time [05:17:22] PROBLEM - mw101 MediaWiki Rendering on mw101 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:17:25] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:17:30] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.291 second response time [05:17:46] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [05:18:04] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.921 second response time [05:18:07] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 5.642 second response time [05:18:12] PROBLEM - mwtask111 Disk Space on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:18:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/585c72e95ae6...9e4dca4352b0 [05:18:24] [url] Comparing 585c72e95ae6...9e4dca4352b0 · miraheze/ssl · GitHub | github.com [05:18:25] [02miraheze/ssl] 07MirahezeSSLBot 039e4dca4 - Bot: Update SSL cert for m33ks.ml [05:18:33] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.35, 3.17, 3.11 [05:18:52] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 3.964 second response time [05:19:02] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.888 second response time [05:19:07] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 5.870 second response time [05:19:26] RECOVERY - mw101 MediaWiki Rendering on mw101 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 6.653 second response time [05:19:32] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 7.962 second response time [05:20:27] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [05:20:28] RECOVERY - mwtask111 Disk Space on mwtask111 is OK: DISK OK - free space: / 30728 MB (46% inode=89%); [05:20:33] RECOVERY - mwtask111 NTP time on mwtask111 is OK: NTP OK: Offset -0.0003970563412 secs [05:20:33] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [05:21:35] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [05:21:38] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [05:21:41] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 21 minutes ago with 0 failures [05:21:42] RECOVERY - mwtask111 JobRunner Service on mwtask111 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [05:21:45] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [05:21:54] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 3.444 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [05:22:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.84, 3.49, 3.39 [05:22:13] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [05:24:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.85, 3.30, 3.33 [05:33:34] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:33:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.05, 3.10, 3.88 [05:34:13] Ugh. [05:34:43] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:35:54] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:36:09] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:36:22] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:36:59] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.634 second response time [05:37:13] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.490 second response time [05:37:27] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 0.157 second response time [05:37:32] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.019 second response time [05:38:30] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 7.945 second response time [05:38:38] PROBLEM - db121 NTP time on db121 is WARNING: NTP WARNING: Offset -0.1182716489 secs [05:41:33] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:41:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.70, 2.59, 3.40 [05:41:36] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:41:39] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:41:39] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:42:02] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:42:07] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:42:31] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:42:36] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:42:44] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset -0.002340704203 secs [05:43:16] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:43:25] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:44:37] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.175 second response time [05:45:17] PROBLEM - cp30 Stunnel HTTP for mwtask111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:45:31] !sre db121 load just passed 260 [05:45:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.15, 3.45 [05:45:35] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.320 second response time [05:45:35] PROBLEM - cp31 Stunnel HTTP for mwtask111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:45:42] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.337 second response time [05:45:45] PROBLEM - cp21 Stunnel HTTP for mwtask111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:45:52] RECOVERY - m33ks.ml - LetsEncrypt on sslhost is OK: OK - Certificate 'm33ks.ml' will expire on Thu 29 Sep 2022 04:18:17 GMT +0000. [05:45:55] PROBLEM - cp20 Stunnel HTTP for mwtask111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:46:05] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.022 second response time [05:46:06] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.498 second response time [05:46:47] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 9.602 second response time [05:47:19] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.025 second response time [05:47:29] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 1.011 second response time [05:47:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.86, 3.76, 3.64 [05:47:46] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.312 second response time [05:47:57] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.017 second response time [05:47:58] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 8.942 second response time [05:49:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.94, 3.69, 3.62 [05:51:31] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:52:15] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [05:53:44] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:53:45] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [05:53:46] PROBLEM - mwtask111 HTTPS on mwtask111 is CRITICAL: CRITICAL - Cannot make SSL connection. [05:55:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.45, 3.94, 3.71 [05:56:19] PROBLEM - mwtask111 nutcracker process on mwtask111 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [05:56:19] PROBLEM - mwtask111 php-fpm on mwtask111 is CRITICAL: CRITICAL - Plugin timed out after 10 seconds [05:58:29] RECOVERY - mwtask111 nutcracker process on mwtask111 is OK: PROCS OK: 1 process with UID = 116 (nutcracker), command name 'nutcracker' [05:58:29] RECOVERY - mwtask111 php-fpm on mwtask111 is OK: PROCS OK: 3 processes with command name 'php-fpm7.4' [05:59:31] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [05:59:37] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 59 minutes ago with 0 failures [06:00:33] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:02:00] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [06:02:33] PROBLEM - db121 NTP time on db121 is WARNING: NTP WARNING: Offset -0.1320396662 secs [06:03:35] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [06:04:36] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset -0.003101557493 secs [06:06:13] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.191 second response time [06:06:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.38, 1.85, 1.33 [06:06:34] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.024 second response time [06:06:43] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.337 second response time [06:06:45] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.279 second response time [06:07:35] PROBLEM - cp31 Varnish Backends on cp31 is CRITICAL: 1 backends are down. mw111 [06:07:47] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.370 second response time [06:09:35] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [06:09:40] PROBLEM - cp21 Stunnel HTTP for mw112 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:10:05] PROBLEM - cp30 Stunnel HTTP for mw112 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:10:36] PROBLEM - cp20 Stunnel HTTP for mw112 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:11:02] PROBLEM - cp31 Stunnel HTTP for mw112 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:11:05] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:11:25] PROBLEM - mw112 MediaWiki Rendering on mw112 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:11:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.42, 3.78, 3.87 [06:13:17] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [06:13:31] RECOVERY - mw112 MediaWiki Rendering on mw112 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 8.297 second response time [06:13:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.46, 4.37, 4.07 [06:13:52] RECOVERY - cp21 Stunnel HTTP for mw112 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 9.231 second response time [06:14:16] RECOVERY - cp30 Stunnel HTTP for mw112 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 5.134 second response time [06:14:24] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CRITICAL - Plugin timed out while executing system call [06:14:43] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:14:43] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 328 bytes in 0.011 second response time [06:14:44] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:14:45] RECOVERY - cp20 Stunnel HTTP for mw112 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 0.316 second response time [06:15:05] RECOVERY - cp31 Stunnel HTTP for mw112 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 0.318 second response time [06:15:11] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:15:20] PROBLEM - test101 MediaWiki Rendering on test101 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 826 bytes in 1.494 second response time [06:15:56] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:16:20] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 0.331 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [06:17:19] RECOVERY - test101 MediaWiki Rendering on test101 is OK: HTTP OK: HTTP/1.1 200 OK - 32424 bytes in 4.901 second response time [06:17:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.78, 3.68, 3.87 [06:21:32] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.282 second response time [06:22:09] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 1.296 second response time [06:22:23] PROBLEM - mwtask111 Puppet on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:22:49] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 1.078 second response time [06:22:52] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 2.478 second response time [06:23:16] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.287 second response time [06:25:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.44, 2.63, 3.28 [06:25:50] PROBLEM - mwtask111 SSH on mwtask111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:26:44] PROBLEM - mwtask111 NTP time on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:26:56] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:27:05] PROBLEM - mwtask111 PowerDNS Recursor on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:10] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:27:11] PROBLEM - mwtask111 conntrack_table_size on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:43] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:27:51] PROBLEM - mwtask111 JobRunner Service on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:27:51] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:28:29] PROBLEM - mwtask111 ferm_active on mwtask111 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [06:28:32] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:28:59] RECOVERY - mwtask111 NTP time on mwtask111 is OK: NTP OK: Offset -0.001212656498 secs [06:29:08] RECOVERY - mwtask111 PowerDNS Recursor on mwtask111 is OK: DNS OK: 3.762 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [06:29:11] RECOVERY - mwtask111 conntrack_table_size on mwtask111 is OK: OK: nf_conntrack is 0 % full [06:29:43] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [06:29:49] RECOVERY - mwtask111 JobRunner Service on mwtask111 is OK: PROCS OK: 1 process with args 'redisJobRunnerService' [06:29:56] RECOVERY - mwtask111 SSH on mwtask111 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [06:30:09] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet last ran 1 hour ago [06:30:26] RECOVERY - mwtask111 ferm_active on mwtask111 is OK: OK ferm input default policy is set [06:31:51] RhinosF1, paladox, Reception123 around? I don't know what to do... [06:32:26] PROBLEM - mw102 MediaWiki Rendering on mw102 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:32:55] PROBLEM - cp30 Stunnel HTTP for mw102 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:33:11] RECOVERY - cp21 Stunnel HTTP for mwtask111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15950 bytes in 1.051 second response time [06:33:18] PROBLEM - cp20 Stunnel HTTP for mw102 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:33:20] RECOVERY - mwtask111 HTTPS on mwtask111 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 553 bytes in 0.134 second response time [06:33:58] PROBLEM - cp31 Stunnel HTTP for mw102 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:34:07] PROBLEM - cp21 Stunnel HTTP for mw102 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:34:19] hmm [06:34:27] RECOVERY - cp30 Stunnel HTTP for mwtask111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.533 second response time [06:34:39] RECOVERY - mwtask111 MediaWiki Rendering on mwtask111 is OK: HTTP OK: HTTP/1.1 200 OK - 27107 bytes in 2.533 second response time [06:34:39] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 5.545 second response time [06:34:40] Ugh. I can't even get a message to send on Discord right now... [06:34:46] RECOVERY - cp31 Stunnel HTTP for mwtask111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.735 second response time [06:34:52] Finally... [06:34:57] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 6.875 second response time [06:35:20] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 3.876 second response time [06:35:46] RECOVERY - cp20 Stunnel HTTP for mwtask111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15956 bytes in 0.229 second response time [06:35:56] RECOVERY - cp31 Stunnel HTTP for mw102 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 3.967 second response time [06:36:02] RECOVERY - cp21 Stunnel HTTP for mw102 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 1.164 second response time [06:36:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 4.04, 3.60 [06:36:29] RECOVERY - mw102 MediaWiki Rendering on mw102 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 0.581 second response time [06:37:02] RECOVERY - cp30 Stunnel HTTP for mw102 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.517 second response time [06:37:30] RECOVERY - cp20 Stunnel HTTP for mw102 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 4.794 second response time [06:38:25] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.89, 3.61, 3.49 [06:39:01] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:39:14] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:39:38] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:42:10] PROBLEM - cp31 Stunnel HTTP for mw101 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:42:21] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.43, 3.18, 3.34 [06:42:47] PROBLEM - cp21 Stunnel HTTP for mw101 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:42:52] PROBLEM - cp30 Stunnel HTTP for mw101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:42:57] PROBLEM - cp20 Stunnel HTTP for mw101 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [06:43:01] PROBLEM - db121 MariaDB on db121 is CRITICAL: Too many connections [06:43:45] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 4.770 second response time [06:44:08] RECOVERY - cp31 Stunnel HTTP for mw101 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 2.468 second response time [06:44:46] RECOVERY - cp21 Stunnel HTTP for mw101 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 3.419 second response time [06:44:55] RECOVERY - cp30 Stunnel HTTP for mw101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 4.103 second response time [06:44:56] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.035 second response time [06:44:57] RECOVERY - cp20 Stunnel HTTP for mw101 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15952 bytes in 2.079 second response time [06:45:04] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 9.084 second response time [06:45:20] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 27098 bytes in 7.238 second response time [06:45:21] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15946 bytes in 6.397 second response time [06:48:15] !log reboot db121 (too many connections) [06:48:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:49:39] PROBLEM - test101 MediaWiki Rendering on test101 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 824 bytes in 0.802 second response time [06:50:12] PROBLEM - db121 NTP time on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:50:20] PROBLEM - db121 Disk Space on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:51:15] PROBLEM - db121 conntrack_table_size on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:51:26] PROBLEM - db121 ferm_active on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:51:28] PROBLEM - db121 PowerDNS Recursor on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:51:42] PROBLEM - db121 Puppet on db121 is CRITICAL: connect to address 2a10:6740::6:302 port 5666: Connection refusedconnect to host 2a10:6740::6:302 port 5666: Connection refused [06:52:20] RECOVERY - db121 Disk Space on db121 is OK: DISK OK - free space: / 272098 MB (43% inode=98%); [06:52:53] RECOVERY - db121 MariaDB on db121 is OK: Uptime: 41 Threads: 62 Questions: 6349 Slow queries: 301 Opens: 1045 Open tables: 1039 Queries per second avg: 154.853 [06:53:01] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 3.59, 0.90, 0.30 [06:53:12] RECOVERY - db121 conntrack_table_size on db121 is OK: OK: nf_conntrack is 0 % full [06:53:23] RECOVERY - db121 PowerDNS Recursor on db121 is OK: DNS OK: 0.288 seconds response time. miraheze.org returns 198.244.148.90,2001:41d0:801:2000::1b80,2001:41d0:801:2000::4c25,51.195.220.68 [06:53:26] RECOVERY - db121 ferm_active on db121 is OK: OK ferm input default policy is set [06:53:29] RECOVERY - test101 MediaWiki Rendering on test101 is OK: HTTP OK: HTTP/1.1 200 OK - 32424 bytes in 1.390 second response time [06:53:42] RECOVERY - db121 Puppet on db121 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [06:54:18] RECOVERY - db121 NTP time on db121 is OK: NTP OK: Offset 0.0002687871456 secs [06:55:20] !log disabled cargo on anotheredenwiki for causing load issues on db121 [06:55:24] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [06:55:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:57:01] PROBLEM - db121 Current Load on db121 is CRITICAL: CRITICAL - load average: 11.16, 6.34, 2.66 [06:57:59] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.78, 1.95, 1.26 [06:59:03] [02TSPortal] 07OwenBaines commented on pull request 03#3: Add support for being able to use php 8 with composer - 13https://github.com/miraheze/TSPortal/pull/3#issuecomment-1172006109 [06:59:04] [url] Add support for being able to use php 8 with composer by paladox · Pull Request #3 · miraheze/TSPortal · GitHub | github.com [06:59:05] [02TSPortal] 07OwenBaines closed pull request 03#3: Add support for being able to use php 8 with composer - 13https://github.com/miraheze/TSPortal/pull/3 [06:59:05] [url] Page not found · GitHub · GitHub | github.com [06:59:47] [02miraheze/TSPortal] 07OwenBaines pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/TSPortal/compare/5c10745aca59...8cea5efdda33 [06:59:47] [url] Comparing 5c10745aca59...8cea5efdda33 · miraheze/TSPortal · GitHub | github.com [06:59:48] [02miraheze/TSPortal] 07paladox 038cea5ef - Regenerate composer.lock file (#4) [06:59:50] [02miraheze/TSPortal] 07OwenBaines deleted branch 03paladox-patch-2 [06:59:51] [02TSPortal] 07OwenBaines deleted branch 03paladox-patch-2 - 13https://github.com/miraheze/TSPortal [06:59:51] [url] Page not found · GitHub · GitHub | github.com [06:59:53] [02TSPortal] 07OwenBaines closed pull request 03#4: Regenerate composer.lock file - 13https://github.com/miraheze/TSPortal/pull/4 [06:59:53] [url] Page not found · GitHub · GitHub | github.com [06:59:57] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.06, 1.62, 1.22 [07:00:42] [02TSPortal] 07OwenBaines commented on pull request 03#5: Remove $id from getSubjectAttribute - 13https://github.com/miraheze/TSPortal/pull/5#issuecomment-1172007311 [07:00:43] [url] Remove $id from getSubjectAttribute by paladox · Pull Request #5 · miraheze/TSPortal · GitHub | github.com [07:00:44] [02TSPortal] 07OwenBaines closed pull request 03#5: Remove $id from getSubjectAttribute - 13https://github.com/miraheze/TSPortal/pull/5 [07:00:44] [url] Page not found · GitHub · GitHub | github.com [07:01:04] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [07:03:01] PROBLEM - db121 Current Load on db121 is WARNING: WARNING - load average: 5.11, 7.33, 4.56 [07:05:01] RECOVERY - db121 Current Load on db121 is OK: OK - load average: 4.03, 6.27, 4.51 [07:05:06] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.28, 3.19 [07:05:44] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 2.23, 2.27, 3.96 [07:07:03] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.19, 2.82, 3.03 [07:13:25] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.97, 2.23, 3.30 [07:14:47] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-8 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/c2128bff4883 [07:14:48] [02miraheze/mw-config] 07Universal-Omega 03c2128bf - Update ManageWikiExtensions.php [07:14:50] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-8 - 13https://github.com/miraheze/mw-config [07:14:50] [url] Page not found · GitHub · GitHub | github.com [07:15:00] [02mw-config] 07Universal-Omega opened pull request 03#4808: Restrict cargo - 13https://github.com/miraheze/mw-config/pull/4808 [07:15:00] ... [07:16:13] miraheze/mw-config - Universal-Omega the build passed. [07:16:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.61, 3.40, 3.14 [07:16:57] [02mw-config] 07Reception123 closed pull request 03#4808: Restrict cargo - 13https://github.com/miraheze/mw-config/pull/4808 [07:16:58] ... [07:16:59] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/36250f20e39d...03e136df55b2 [07:16:59] [url] Comparing 36250f20e39d...03e136df55b2 · miraheze/mw-config · GitHub | github.com [07:17:00] [02miraheze/mw-config] 07Universal-Omega 0303e136d - Restrict cargo (#4808) [07:17:02] [02miraheze/mw-config] 07Reception123 deleted branch 03Universal-Omega-patch-8 [07:17:03] [02mw-config] 07Reception123 deleted branch 03Universal-Omega-patch-8 - 13https://github.com/miraheze/mw-config [07:17:04] [url] Page not found · GitHub · GitHub | github.com [07:17:58] miraheze/mw-config - Reception123 the build passed. [07:18:47] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.57, 2.93, 2.99 [07:19:04] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/03e136df55b2...c49b7a874aea [07:19:04] [url] Comparing 03e136df55b2...c49b7a874aea · miraheze/mw-config · GitHub | github.com [07:19:05] [02miraheze/mw-config] 07Reception123 03c49b7a8 - wgDefaultUserOptions for finalfantasywiki per T9474 [07:20:19] miraheze/mw-config - Reception123 the build passed. [07:22:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.61, 3.70, 3.28 [07:23:30] !log [@mwtask111] starting deploy of {'config': True} to all [07:23:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:23:43] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 12s [07:23:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:24:38] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.17, 3.26, 3.17 [07:25:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.52, 3.46, 3.32 [07:26:22] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 5.83, 2.68, 1.70 [07:26:54] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.62, 4.34, 3.65 [07:28:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 9.31, 5.42, 3.90 [07:28:32] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.65, 4.45, 3.65 [07:29:40] !log [@test101] starting deploy of {'config': True} to all [07:29:42] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [07:29:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:29:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [07:30:22] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.02, 1.88, 1.59 [07:30:56] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [07:34:22] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.89, 1.38, 1.46 [07:36:32] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.55, 10.58, 9.74 [07:38:31] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.87, 9.76, 9.53 [07:40:11] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.65, 3.95 [07:42:05] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.52, 4.13, 4.10 [07:46:22] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.82, 1.46, 1.38 [07:48:22] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.47, 1.41, 1.37 [07:53:58] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.30, 3.33, 3.82 [07:54:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.72, 3.68, 3.99 [07:56:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.93, 3.55, 3.97 [07:57:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.58, 3.77, 3.88 [08:00:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.30, 4.01, 4.02 [08:00:06] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [08:00:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.23, 4.34, 4.14 [08:03:48] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.50, 3.83, 3.94 [08:04:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.45, 3.83, 3.97 [08:05:46] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.66, 4.05, 4.00 [08:06:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 4.30, 4.13 [08:07:44] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.59, 3.54, 3.82 [08:10:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.38, 3.65, 3.94 [08:13:38] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.85, 3.98, 3.88 [08:14:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.92, 4.08, 4.01 [08:15:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.16, 3.32, 3.65 [08:16:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.70, 3.88 [08:18:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.74, 4.01, 3.96 [08:20:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.61, 3.48, 3.78 [08:22:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.46, 3.64, 3.97 [08:23:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.79, 3.79, 3.68 [08:24:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.09, 4.03, 4.07 [08:24:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.40, 3.73, 3.80 [08:29:15] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [08:30:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.07, 3.63, 3.76 [08:31:31] PROBLEM - www.famedata.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'famedata.org' expires in 15 day(s) (Sun 17 Jul 2022 08:27:24 GMT +0000). [08:31:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.60, 3.79, 3.75 [08:32:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.89, 3.77, 3.99 [08:34:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.52, 3.66 [08:36:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.93, 3.14, 3.50 [08:37:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.76, 3.90, 3.78 [08:38:38] PROBLEM - famedata.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'famedata.org' expires in 15 day(s) (Sun 17 Jul 2022 08:27:24 GMT +0000). [08:39:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.62, 3.68 [08:40:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.36, 3.02, 3.39 [08:42:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.10, 2.88, 3.40 [08:42:09] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.83, 1.59, 1.42 [08:42:49] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9e4dca4352b0...235ee4e51898 [08:42:50] [url] Comparing 9e4dca4352b0...235ee4e51898 · miraheze/ssl · GitHub | github.com [08:42:51] [02miraheze/ssl] 07MirahezeSSLBot 03235ee4e - Bot: Update SSL cert for famedata.org [08:44:07] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 1.46, 1.46, 1.38 [08:45:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.66, 2.90, 3.33 [08:59:20] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.29, 3.41, 3.27 [09:00:20] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query dnd.bellinrattin.it. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:00:28] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.41, 3.29 [09:01:02] RECOVERY - www.famedata.org - LetsEncrypt on sslhost is OK: OK - Certificate 'famedata.org' will expire on Thu 29 Sep 2022 07:42:43 GMT +0000. [09:01:14] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.49, 2.67, 3.01 [09:02:26] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.80, 4.25, 3.60 [09:06:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.33, 3.72, 3.54 [09:08:19] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.40, 4.67, 3.90 [09:08:19] RECOVERY - famedata.org - LetsEncrypt on sslhost is OK: OK - Certificate 'famedata.org' will expire on Thu 29 Sep 2022 07:42:43 GMT +0000. [09:09:38] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.26, 4.25, 3.56 [09:10:43] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.90, 4.39, 3.73 [09:12:15] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.98, 3.96, 3.86 [09:12:37] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.27, 3.76, 3.58 [09:14:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.96, 4.37, 4.04 [09:14:31] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.12, 4.28, 3.81 [09:15:31] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.01, 3.52, 3.46 [09:16:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.07, 3.57, 3.78 [09:16:25] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.81, 3.40, 3.55 [09:16:57] PROBLEM - ping6 on cp20 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:17:20] PROBLEM - cp20 HTTPS on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:17:29] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.43, 3.04, 3.29 [09:17:36] PROBLEM - cp20 SSH on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:18:08] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 51.195.220.68/cpweb, 2001:41d0:801:2000::4c25/cpweb [09:18:11] PROBLEM - ns1 GDNSD Datacenters on ns1 is CRITICAL: CRITICAL - 2 datacenters are down: 51.195.220.68/cpweb, 2001:41d0:801:2000::4c25/cpweb [09:18:13] PROBLEM - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is CRITICAL: CRITICAL - NGINX Error Rate is 73% [09:18:19] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.93, 3.02, 3.39 [09:18:59] RECOVERY - ping6 on cp20 is OK: PING OK - Packet loss = 0%, RTA = 2.96 ms [09:19:00] PROBLEM - cp20 Varnish Backends on cp20 is CRITICAL: 7 backends are down. mw101 mw102 mw111 mw112 mw121 mw122 mediawiki [09:19:09] PROBLEM - cp20 NTP time on cp20 is WARNING: NTP WARNING: Offset 0.1200388968 secs [09:19:15] RECOVERY - cp20 HTTPS on cp20 is OK: HTTP OK: HTTP/1.1 301 Moved Permanently - 3402 bytes in 0.020 second response time [09:19:33] RECOVERY - cp20 SSH on cp20 is OK: SSH OK - OpenSSH_8.4p1 Debian-5 (protocol 2.0) [09:20:06] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.15, 2.62, 3.33 [09:22:08] RECOVERY - ns2 GDNSD Datacenters on ns2 is OK: OK - all datacenters are online [09:22:11] RECOVERY - ns1 GDNSD Datacenters on ns1 is OK: OK - all datacenters are online [09:22:13] RECOVERY - cp20 HTTP 4xx/5xx ERROR Rate on cp20 is OK: OK - NGINX Error Rate is 3% [09:22:59] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 14 backends are healthy [09:23:34] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.92, 1.64, 1.24 [09:25:31] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.16, 1.46, 1.22 [09:26:39] PROBLEM - wiki.eversd.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.eversd.com' expires in 15 day(s) (Sun 17 Jul 2022 09:16:49 GMT +0000). [09:26:53] PROBLEM - vault.agentisai.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'vault.agentisai.com' expires in 15 day(s) (Sun 17 Jul 2022 09:03:10 GMT +0000). [09:27:09] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 2.37, 1.38, 1.20 [09:29:06] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.98, 1.22, 1.16 [09:29:30] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [09:32:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/235ee4e51898...7c941bd24955 [09:32:34] [url] Comparing 235ee4e51898...7c941bd24955 · miraheze/ssl · GitHub | github.com [09:32:35] [02miraheze/ssl] 07MirahezeSSLBot 037c941bd - Bot: Update SSL cert for wiki.eversd.com [09:33:05] PROBLEM - ping6 on ns1 is WARNING: PING WARNING - Packet loss = 0%, RTA = 147.67 ms [09:33:09] RECOVERY - cp20 NTP time on cp20 is OK: NTP OK: Offset -0.00646802783 secs [09:33:56] PROBLEM - reviews.thejoshmeister.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'reviews.thejoshmeister.com' expires in 15 day(s) (Sun 17 Jul 2022 09:28:50 GMT +0000). [09:34:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7c941bd24955...fa78aa16bfa0 [09:34:34] [02miraheze/ssl] 07MirahezeSSLBot 03fa78aa1 - Bot: Update SSL cert for vault.agentisai.com [09:34:34] [url] Comparing 7c941bd24955...fa78aa16bfa0 · miraheze/ssl · GitHub | github.com [09:42:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/fa78aa16bfa0...8ec39ba05411 [09:42:40] [url] Comparing fa78aa16bfa0...8ec39ba05411 · miraheze/ssl · GitHub | github.com [09:42:41] [02miraheze/ssl] 07MirahezeSSLBot 038ec39ba - Bot: Update SSL cert for reviews.thejoshmeister.com [09:43:04] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.12, 1.99, 1.47 [09:43:53] PROBLEM - polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.polandballwiki.com' expires in 15 day(s) (Sun 17 Jul 2022 09:36:33 GMT +0000). [09:45:00] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.24, 1.64, 1.40 [09:47:28] RECOVERY - ping6 on ns1 is OK: PING OK - Packet loss = 0%, RTA = 102.94 ms [09:48:42] PROBLEM - www.polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.polandballwiki.com' expires in 15 day(s) (Sun 17 Jul 2022 09:36:33 GMT +0000). [09:49:39] PROBLEM - pt.polandballwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pt.polandballwiki.com' expires in 15 day(s) (Sun 17 Jul 2022 09:29:41 GMT +0000). [09:52:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8ec39ba05411...c63f76cc3b5c [09:52:59] [url] Comparing 8ec39ba05411...c63f76cc3b5c · miraheze/ssl · GitHub | github.com [09:53:00] [02miraheze/ssl] 07MirahezeSSLBot 03c63f76c - Bot: Update SSL cert for www.polandballwiki.com [09:55:48] RECOVERY - wiki.eversd.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.eversd.com' will expire on Thu 29 Sep 2022 08:32:27 GMT +0000. [09:59:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c63f76cc3b5c...add727507554 [09:59:59] [url] Comparing c63f76cc3b5c...add727507554 · miraheze/ssl · GitHub | github.com [10:00:00] [02miraheze/ssl] 07MirahezeSSLBot 03add7275 - Bot: Update SSL cert for pt.polandballwiki.com [10:00:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.47, 3.02 [10:00:36] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:02:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.89, 3.11, 2.93 [10:02:51] RECOVERY - reviews.thejoshmeister.com - LetsEncrypt on sslhost is OK: OK - Certificate 'reviews.thejoshmeister.com' will expire on Thu 29 Sep 2022 08:42:34 GMT +0000. [10:05:26] PROBLEM - db101 Current Load on db101 is CRITICAL: CRITICAL - load average: 10.12, 6.90, 5.14 [10:10:36] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.28, 4.11, 3.27 [10:11:09] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 5.45, 4.53, 2.42 [10:12:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 7.19, 4.89, 3.60 [10:12:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.18, 4.61, 3.68 [10:12:19] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 2.92, 3.62, 2.37 [10:12:33] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.94, 3.32 [10:14:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.36, 5.44, 3.96 [10:15:26] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 5.09, 7.05, 6.56 [10:17:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.14, 6.11, 6.28 [10:22:01] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 0.65, 1.62, 1.94 [10:22:54] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 0.86, 1.60, 1.93 [10:25:39] RECOVERY - vault.agentisai.com - LetsEncrypt on sslhost is OK: OK - Certificate 'vault.agentisai.com' will expire on Thu 29 Sep 2022 08:34:24 GMT +0000. [10:27:51] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 1.54, 1.36, 1.70 [10:28:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.40, 3.42, 3.88 [10:29:46] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [10:30:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.18, 3.38, 3.96 [10:30:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.71, 3.92 [10:30:44] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.93, 1.23, 1.64 [10:33:54] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.61, 3.56, 3.95 [10:34:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.69, 3.64, 3.90 [10:35:52] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.98, 3.79, 3.97 [10:36:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.54, 3.95, 3.96 [10:36:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.40, 4.37, 4.11 [10:37:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.33, 3.57, 3.87 [10:38:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.48, 3.79 [10:39:48] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.24, 3.75, 3.89 [10:40:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.26, 3.88, 3.90 [10:41:01] RECOVERY - polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.polandballwiki.com' will expire on Thu 29 Sep 2022 08:52:53 GMT +0000. [10:41:45] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.13, 3.70, 3.88 [10:43:43] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.37, 4.25, 4.07 [10:46:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.76, 3.86, 3.97 [10:46:42] RECOVERY - www.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'www.polandballwiki.com' will expire on Thu 29 Sep 2022 08:52:53 GMT +0000. [10:48:22] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.82, 1.75, 1.57 [10:48:58] RECOVERY - pt.polandballwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'pt.polandballwiki.com' will expire on Thu 29 Sep 2022 08:59:53 GMT +0000. [10:49:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.78, 3.82, 3.95 [10:50:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.73, 3.48, 3.92 [10:50:22] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.84, 1.40, 1.46 [10:55:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.13, 3.70, 3.80 [10:56:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.59, 3.72, 3.69 [10:56:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.51, 3.76 [10:58:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.23, 3.46, 3.59 [10:58:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.96, 3.31, 3.66 [10:59:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.78, 3.39, 3.67 [11:00:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.75, 3.67 [11:02:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.67, 3.66 [11:03:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.56, 3.72, 3.69 [11:04:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.96, 3.76 [11:06:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.24, 4.51, 3.96 [11:07:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.73, 3.94, 3.80 [11:08:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.93, 4.00, 3.84 [11:08:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.97, 3.82 [11:11:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.14, 3.71, 3.71 [11:12:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.73, 4.17, 3.88 [11:13:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.57, 3.35, 3.58 [11:14:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.98, 3.70, 3.75 [11:14:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.05, 2.80, 3.33 [11:15:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.13, 2.91, 3.39 [11:25:56] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c49b7a874aea...6de4e4205549 [11:25:57] [url] Comparing c49b7a874aea...6de4e4205549 · miraheze/mw-config · GitHub | github.com [11:25:58] [02miraheze/mw-config] 07Reception123 036de4e42 - end 2022 survey [11:26:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.48, 3.58, 3.54 [11:26:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.05, 3.22, 3.23 [11:27:01] miraheze/mw-config - Reception123 the build passed. [11:27:23] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.24, 3.78, 3.52 [11:28:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.35, 3.27 [11:28:35] !log [@test101] starting deploy of {'config': True} to all [11:28:38] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 3s [11:28:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:28:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:30:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.02, 3.69, 3.63 [11:30:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.12, 2.97, 3.15 [11:34:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.61, 3.53, 3.54 [11:36:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.29, 3.34, 3.47 [11:38:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.57, 3.76, 3.60 [11:39:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.86, 3.54, 3.33 [11:40:01] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.78, 3.65, 3.57 [11:41:07] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.14, 3.71, 3.41 [11:42:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.08, 4.24, 3.80 [11:43:05] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.28, 3.44, 3.34 [11:45:02] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.77, 3.40, 3.35 [11:45:04] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.10, 3.95, 3.92 [11:46:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.32, 3.54, 3.64 [11:48:29] PROBLEM - bast121 NTP time on bast121 is WARNING: NTP WARNING: Offset 0.4107119739 secs [11:48:47] PROBLEM - bast101 NTP time on bast101 is WARNING: NTP WARNING: Offset 0.4105812013 secs [11:48:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.41, 3.51, 3.41 [11:52:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.19, 4.16, 3.67 [11:53:30] !log [@mwtask111] starting deploy of {'config': True} to all [11:53:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:53:42] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 11s [11:53:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:54:49] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.39, 3.67, 3.54 [11:56:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.23, 2.83, 3.39 [11:56:52] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.68, 2.84, 3.39 [11:58:44] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.85, 3.81, 3.62 [12:00:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.65, 3.59 [12:02:29] RECOVERY - bast121 NTP time on bast121 is OK: NTP OK: Offset 0.0004046857357 secs [12:02:40] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.35, 3.83, 3.66 [12:02:47] RECOVERY - bast101 NTP time on bast101 is OK: NTP OK: Offset 0.0003220438957 secs [12:04:38] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.92, 3.37, 3.50 [12:10:31] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.60, 3.03, 3.33 [12:10:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.31, 10.67, 9.36 [12:17:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.39, 4.56, 3.73 [12:19:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.85, 3.76, 3.53 [12:20:30] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 10.32, 9.33, 8.38 [12:23:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.38, 3.87, 3.61 [12:24:30] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 9.46, 9.63, 8.73 [12:25:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.88, 3.83, 3.62 [12:27:09] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.38, 3.26 [12:29:06] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.73, 3.09, 3.16 [12:30:28] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.26, 11.36, 10.71 [12:32:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.62, 11.19, 10.74 [12:33:14] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.26, 3.46, 3.26 [12:35:08] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.77, 3.67, 3.35 [12:35:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.70, 4.07, 3.76 [12:36:23] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.89, 11.97, 11.09 [12:37:01] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.23, 3.08, 3.16 [12:37:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.01, 3.56, 3.61 [12:39:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.27, 3.62, 3.61 [12:40:20] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.82, 11.75, 11.25 [12:41:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.74, 3.66, 3.63 [12:45:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.30, 3.12, 3.40 [12:46:11] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.50, 6.84, 6.07 [12:48:07] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.26, 6.60, 6.07 [12:48:25] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.41, 3.38, 3.27 [12:49:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.60, 3.84, 3.60 [12:50:18] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.67, 4.09, 3.53 [12:52:33] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.84, 3.46 [12:54:06] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.27, 3.89, 3.59 [12:54:30] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.28, 3.66, 3.44 [12:56:09] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.60, 11.33, 11.06 [12:56:28] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.36, 3.18, 3.29 [12:57:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.74, 3.98, 3.82 [12:58:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 3.05, 3.23, 3.37 [12:58:08] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.23, 10.09, 10.64 [13:00:06] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.61, 9.02, 10.19 [13:00:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.28, 4.12, 3.62 [13:01:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.05, 3.93, 3.82 [13:02:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 3.79, 3.56 [13:02:19] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.92, 3.91, 3.60 [13:04:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 4.04, 3.68 [13:05:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.41, 3.51, 3.66 [13:06:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.12, 3.72, 3.61 [13:06:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.04, 3.64, 3.58 [13:07:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.70, 4.01, 3.81 [13:08:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.00, 3.94, 3.69 [13:08:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.57, 4.45, 3.89 [13:12:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.15, 3.98, 3.81 [13:12:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.72, 3.85, 3.81 [13:23:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.91, 3.59, 3.84 [13:24:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.93, 3.98, 3.85 [13:25:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.81, 4.36, 4.11 [13:26:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.22, 3.37, 3.65 [13:27:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.00, 3.84, 3.94 [13:28:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.86, 4.34, 3.92 [13:29:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.34, 3.99, 3.98 [13:31:50] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.69, 10.70, 9.61 [13:32:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.89, 3.98, 3.77 [13:33:45] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.30, 10.84, 9.79 [13:34:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.83, 3.96, 3.79 [13:36:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.93, 4.49, 4.00 [13:39:32] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.49, 9.85, 9.75 [13:39:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.10, 3.76, 3.91 [13:40:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.30, 3.64, 3.87 [13:40:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.29, 3.81, 3.83 [13:44:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.08, 3.58, 3.70 [13:44:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.51, 1.73, 1.98 [13:45:20] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.48, 10.99, 10.25 [13:46:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.64, 3.65, 3.72 [13:46:19] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.42, 10.54, 9.76 [13:48:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.61, 2.09, 2.06 [13:48:18] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.83, 9.78, 9.57 [13:49:11] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.92, 9.93, 10.01 [13:50:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.46, 2.93, 3.39 [13:54:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.13, 3.46, 3.58 [13:55:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.92, 4.13, 3.90 [13:56:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.69, 3.67 [13:56:48] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.92, 3.72, 3.57 [13:56:55] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.46, 10.45, 10.10 [13:58:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.96, 3.76 [13:58:51] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.28, 9.74, 9.89 [14:00:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.98, 3.89, 3.75 [14:00:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.87, 1.89, 1.98 [14:02:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.22, 1.99, 2.00 [14:03:38] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.49, 10.39, 9.78 [14:05:17] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.83, 1.58, 1.17 [14:05:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.08, 3.60, 3.79 [14:06:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.79, 1.92, 1.97 [14:07:16] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.89, 1.28, 1.11 [14:07:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.42, 3.53, 3.72 [14:08:11] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.25, 3.79, 3.62 [14:08:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.32, 3.50, 3.57 [14:10:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.68, 3.20, 3.45 [14:11:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.64, 3.90, 3.87 [14:12:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.74, 3.70, 3.62 [14:12:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.31, 2.62, 3.22 [14:12:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.10, 1.81, 1.89 [14:14:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.77, 1.87 [14:15:30] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.02, 9.90, 9.97 [14:15:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.68, 2.63, 3.35 [14:18:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.46, 1.95, 1.90 [14:19:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.87, 3.54, 3.56 [14:20:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.61, 3.54 [14:21:25] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.44, 10.77, 10.27 [14:21:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.27, 4.48, 3.90 [14:22:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.94, 3.69, 3.45 [14:23:24] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.90, 10.32, 10.17 [14:25:22] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.70, 9.53, 9.88 [14:28:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.91, 1.93, 1.96 [14:34:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.86, 3.71, 3.85 [14:34:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.20, 2.97, 3.28 [14:36:12] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.89, 10.41, 9.99 [14:37:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.71, 3.43, 3.88 [14:39:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.89, 3.91, 3.99 [14:39:42] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.51, 10.25, 9.70 [14:40:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.84, 10.01, 9.93 [14:40:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.53, 3.41 [14:41:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.88, 3.57, 3.87 [14:42:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.43, 3.96, 3.58 [14:42:33] [02puppet] 07RhinosF1 opened pull request 03#2687: rhinosf1: remove access to mw cluster - 13https://github.com/miraheze/puppet/pull/2687 [14:42:33] [url] Page not found · GitHub · GitHub | github.com [14:43:14] [02puppet] 07RhinosF1 synchronize pull request 03#2687: rhinosf1: remove access to mw cluster - 13https://github.com/miraheze/puppet/pull/2687 [14:43:14] [url] Page not found · GitHub · GitHub | github.com [14:43:33] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.25, 9.62, 9.62 [14:46:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.87, 3.61, 3.54 [14:46:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.20, 1.78, 1.78 [14:48:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.69, 3.72 [14:48:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.29, 3.73, 3.58 [14:50:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.97, 3.63, 3.71 [14:51:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.20, 4.53, 4.11 [14:52:01] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.39, 4.00, 3.84 [14:54:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.77, 3.60, 3.72 [14:54:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.46, 3.69, 3.69 [14:55:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.18, 3.95, 3.99 [14:56:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.20, 4.11, 3.82 [14:57:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 4.40, 4.15 [15:00:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.67, 4.55, 4.05 [15:00:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.93, 3.83 [15:04:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.37, 3.94, 3.84 [15:05:49] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.81, 9.10, 8.58 [15:06:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.90, 3.96, 3.86 [15:07:48] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.46, 8.37, 8.37 [15:10:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.37, 4.16, 3.95 [15:10:39] PROBLEM - cp21 Current Load on cp21 is CRITICAL: CRITICAL - load average: 7.21, 2.47, 1.42 [15:12:39] PROBLEM - cp21 Current Load on cp21 is WARNING: WARNING - load average: 1.21, 1.75, 1.28 [15:13:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.08, 3.66, 3.98 [15:13:45] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.68, 9.52, 8.69 [15:14:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.99, 3.93 [15:14:39] RECOVERY - cp21 Current Load on cp21 is OK: OK - load average: 0.62, 1.44, 1.22 [15:16:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.90, 3.89 [15:16:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.80, 1.90, 1.98 [15:18:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.22, 3.81, 3.87 [15:18:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 5.75, 2.97, 2.33 [15:19:43] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.15, 9.55, 8.95 [15:20:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.76, 4.12, 4.00 [15:23:41] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.97, 10.67, 9.53 [15:24:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.70, 3.56, 3.83 [15:24:50] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.82, 10.41, 9.62 [15:25:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.77, 2.93, 3.38 [15:25:41] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.14, 9.50, 9.23 [15:28:54] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.10, 10.17, 9.74 [15:29:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.98, 3.97, 3.73 [15:30:41] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.63, 10.04, 9.22 [15:31:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.09, 3.94, 3.74 [15:32:39] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.71, 9.90, 9.27 [15:32:50] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.96, 10.39, 9.94 [15:33:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.19, 3.78, 3.71 [15:34:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.04, 9.45, 9.65 [15:35:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.56, 3.86, 3.74 [15:36:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.67, 3.76, 3.98 [15:36:35] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.17, 10.95, 9.83 [15:38:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.91, 4.37, 4.14 [15:38:34] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.50, 10.39, 9.76 [15:40:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.85, 2.00 [15:40:32] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.69, 9.99, 9.68 [15:42:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.57, 3.98, 3.76 [15:42:33] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.18, 9.88, 9.43 [15:44:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.60, 2.06, 2.04 [15:44:28] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.78, 10.73, 10.08 [15:44:32] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.69, 8.88, 9.13 [15:45:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.49, 3.55, 3.65 [15:46:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.90, 1.96, 2.00 [15:48:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.37, 3.83, 3.80 [15:48:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.35, 2.12, 2.05 [15:48:25] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.24, 11.00, 10.29 [15:50:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.62, 3.39, 3.86 [15:50:24] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.94, 10.06, 10.05 [15:54:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.32, 3.50, 3.59 [15:55:34] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.38, 3.03, 3.39 [15:56:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.46, 3.39, 3.54 [15:58:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.33, 3.04, 3.39 [15:58:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.48, 1.87, 2.00 [16:00:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.05, 4.10, 3.88 [16:00:20] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-8 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/4dfee0287197 [16:00:21] [02miraheze/mw-config] 07Universal-Omega 034dfee02 - Remove RhinosF1 from staffwiki [16:00:23] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-8 - 13https://github.com/miraheze/mw-config [16:00:23] [url] Page not found · GitHub · GitHub | github.com [16:00:24] [02mw-config] 07Universal-Omega opened pull request 03#4809: Remove RhinosF1 from staffwiki - 13https://github.com/miraheze/mw-config/pull/4809 [16:00:25] [url] Page not found · GitHub · GitHub | github.com [16:01:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6de4e4205549...c26b66a90d29 [16:01:52] [url] Comparing 6de4e4205549...c26b66a90d29 · miraheze/mw-config · GitHub | github.com [16:01:53] [02miraheze/mw-config] 07Universal-Omega 03c26b66a - Remove RhinosF1 from staffwiki (#4809) [16:01:54] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-8 [16:01:56] [02mw-config] 07Universal-Omega closed pull request 03#4809: Remove RhinosF1 from staffwiki - 13https://github.com/miraheze/mw-config/pull/4809 [16:01:56] [url] Page not found · GitHub · GitHub | github.com [16:01:57] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-8 - 13https://github.com/miraheze/mw-config [16:01:57] ... [16:02:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.66, 3.75 [16:02:00] miraheze/mw-config - Universal-Omega the build passed. [16:02:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.28, 3.77, 3.63 [16:02:28] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.39, 4.24, 3.84 [16:02:55] miraheze/mw-config - Universal-Omega the build passed. [16:04:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 4.03, 3.88 [16:05:00] !log remove myself from github [16:05:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:06:00] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.72, 3.99, 3.89 [16:07:09] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.45, 10.50, 10.08 [16:08:00] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.39, 4.41, 4.05 [16:09:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.49, 9.75, 9.86 [16:10:16] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.56, 3.70, 3.66 [16:10:20] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.69, 3.75, 3.84 [16:11:57] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 3.83, 3.19, 1.95 [16:12:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 7.60, 5.07, 4.17 [16:12:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.04, 1.83, 1.84 [16:12:17] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 9.33, 5.52, 4.44 [16:14:00] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:14:39] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 3.16, 3.26, 1.96 [16:15:56] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15938 bytes in 1.139 second response time [16:16:17] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.77, 1.85, 1.85 [16:18:17] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.63, 2.16, 1.96 [16:18:39] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 0.85, 1.79, 1.64 [16:19:47] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.16, 1.74, 1.75 [16:20:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.71, 1.45, 1.53 [16:21:42] !log [@test101] starting deploy of {'config': True} to all [16:21:43] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [16:21:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:21:56] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.95, 11.09, 10.22 [16:22:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:25] !log [@mwtask111] starting deploy of {'config': True} to all [16:22:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:22:43] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 18s [16:22:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:23:42] PROBLEM - cp20 Current Load on cp20 is CRITICAL: CRITICAL - load average: 4.21, 2.41, 1.97 [16:23:55] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.40, 10.47, 10.10 [16:25:53] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.40, 9.84, 9.92 [16:27:37] PROBLEM - cp20 Current Load on cp20 is WARNING: WARNING - load average: 1.19, 1.81, 1.83 [16:29:35] RECOVERY - cp20 Current Load on cp20 is OK: OK - load average: 0.67, 1.47, 1.70 [16:38:50] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.25, 9.66, 9.47 [16:40:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.87, 9.19, 9.33 [16:58:01] [02puppet] 07Reception123 commented on pull request 03#2687: rhinosf1: remove access to mw cluster - 13https://github.com/miraheze/puppet/pull/2687#issuecomment-1172542106 [16:58:02] [url] rhinosf1: remove access to mw cluster by RhinosF1 · Pull Request #2687 · miraheze/puppet · GitHub | github.com [16:58:11] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/452d96a6ba00...14eee924f56b [16:58:12] [url] Comparing 452d96a6ba00...14eee924f56b · miraheze/puppet · GitHub | github.com [16:58:13] [02miraheze/puppet] 07RhinosF1 0314eee92 - rhinosf1: remove access to mw cluster (#2687) [16:58:14] [02puppet] 07Reception123 closed pull request 03#2687: rhinosf1: remove access to mw cluster - 13https://github.com/miraheze/puppet/pull/2687 [16:58:14] [url] Page not found · GitHub · GitHub | github.com [16:58:24] Reception123: flags in this channel [16:58:40] I thought I did them here [16:58:55] just forgot RVr [17:01:45] !log removed RhinosF1 from ldap groups [17:01:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:03:16] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.32, 10.03, 9.58 [17:04:36] MacFan4000: what's "fossbots channel management"? doesn't Rhinos retain access there anyway as he's with fossbots? [17:05:11] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.38, 9.78, 9.54 [17:05:19] Reception123: the file that lets me op in here [17:05:22] Channelmgnt plug-in for MirahezeBot [17:05:28] I mean I have global access anyway [17:05:32] ah ok, I'll let MacFan4000 do that then and otherwise offboarding should be complete [17:07:45] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.04, 1.93, 1.46 [17:07:53] yes and all stewards were pinged to complete remaining on wiki stuff [17:08:19] Reception123: have you locked my SRE account [17:08:47] .mhca RhinosF1 (Miraheze) [17:08:47] https://meta.miraheze.org/wiki/Special:CentralAuth/RhinosF1_(Miraheze) [17:08:49] I also asked them to remove the local meta sysadmin group [17:09:19] that's my issue that it shows up as already locked but no log [17:09:28] Reception123: reset the user cache [17:09:28] I'll try to just click save and see what happens [17:09:39] "Error: An edit conflict for this user was detected. Please verify your change and try again." [17:09:42] It'll leak from beta [17:09:45] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.92, 1.55, 1.38 [17:09:53] I locked on beta [17:10:03] meh I fixed it [17:21:22] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.69, 10.31, 9.50 [17:27:55] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.84, 9.61, 8.91 [17:31:06] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.29, 9.71, 9.10 [17:31:15] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.69, 3.41, 3.89 [17:33:58] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.71, 1.83, 1.98 [17:35:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.63, 4.00 [17:37:00] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.29, 3.99, 3.99 [17:37:46] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.72, 3.78, 4.02 [17:38:53] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.91, 10.31, 9.45 [17:38:55] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.66, 3.94, 3.97 [17:39:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.87, 3.44, 3.87 [17:40:57] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.21, 3.33, 3.94 [17:42:09] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=sctoolszhwiki --no-updates --username-prefix=starcitizen /mnt/mediawiki-static/metawiki/ImportDump/sctoolszhwiki-20220701052043.xml (START) [17:42:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:44:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.91, 3.73, 3.79 [17:45:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.93, 3.99, 3.91 [17:46:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.54, 3.63, 3.75 [17:46:42] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.97, 9.74, 9.59 [17:47:43] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.16, 1.78, 1.81 [17:48:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.45, 3.78, 3.78 [17:50:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.45, 4.43, 4.05 [17:52:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.71, 3.92, 3.90 [17:53:24] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 3.60, 3.82 [17:56:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.57, 3.92, 3.92 [17:56:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.06, 4.18, 3.98 [17:58:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.54, 3.88, 3.90 [17:59:30] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.61, 1.93, 1.96 [18:02:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.11, 3.62, 3.74 [18:03:22] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.11, 10.76, 10.00 [18:04:04] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.45, 9.70, 10.14 [18:04:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.39, 3.64 [18:05:23] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.27, 1.83, 1.88 [18:06:42] Reception123 [18:06:52] yes Naleksuh ? [18:06:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.40, 3.66, 3.69 [18:07:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.50, 3.74, 3.64 [18:07:17] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.24, 9.97, 9.85 [18:07:19] Reception123 Can you read my post on test.miraheze.org [18:07:19] Its been like 3 days [18:07:30] ok, I'll look [18:08:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.10, 3.96, 3.82 [18:08:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.14, 3.59, 3.68 [18:09:14] [02dns] 07Universal-Omega opened pull request 03#294: Add ffwiki.org zone - 13https://github.com/miraheze/dns/pull/294 [18:09:15] [url] Page not found · GitHub · GitHub | github.com [18:09:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.54, 3.53, 3.60 [18:09:32] Naleksuh: I would think it'd be more appropriate for a Steward to test out the system since they'd be doing the blocks globally? [18:10:12] What's being tested is its effectiveness, not the code, I've already made sure that the code works [18:10:22] Reception123: mind merging that DNS PR also? [18:10:24] Not all proxies are blocked on Wikimedia, and I need to come up with some backup plans [18:10:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 4.00, 3.73, 3.74 [18:10:44] Also, I was under the impression that this was what the proxy bot group was for [18:10:54] CosmicAlpha: sure [18:11:03] Thanks! [18:11:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.27, 4.00, 3.75 [18:11:20] I'd want to speak to other consuls before approving, as this is an unprecedented thing for testwiki which hasn't been used in this way before [18:12:01] That's fine, it's just nobody was talking about it at all other than one non-consul so you can check if you want [18:12:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.34, 4.05, 3.86 [18:12:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.02, 3.57, 3.62 [18:13:26] [02dns] 07Reception123 closed pull request 03#294: Add ffwiki.org zone - 13https://github.com/miraheze/dns/pull/294 [18:13:27] [url] Page not found · GitHub · GitHub | github.com [18:13:28] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/e5ec89ad039e...b26e3ca42276 [18:13:28] [url] Comparing e5ec89ad039e...b26e3ca42276 · miraheze/dns · GitHub | github.com [18:13:29] [02miraheze/dns] 07Universal-Omega 03b26e3ca - Add ffwiki.org zone (#294) [18:13:37] Thanks! [18:14:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.52, 3.90, 3.83 [18:14:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.10, 3.60, 3.65 [18:15:01] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=sctoolszhwiki --no-updates --username-prefix=starcitizen /mnt/mediawiki-static/metawiki/ImportDump/sctoolszhwiki-20220701052043.xml (END - exit=0) [18:15:02] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=sctoolszhwiki (START) [18:15:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:15:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:18:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.91, 4.24, 3.96 [18:18:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.69, 3.89, 3.72 [18:20:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.06, 3.92, 3.88 [18:20:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.61, 3.85, 3.74 [18:22:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.91, 4.52, 4.10 [18:22:56] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.22, 10.74, 10.15 [18:24:54] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.64, 11.44, 10.48 [18:24:57] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.04, 7.02, 6.53 [18:26:54] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.24, 6.19, 6.30 [18:27:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.35, 3.47, 3.76 [18:28:49] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.78, 11.26, 10.66 [18:29:16] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.22, 3.94, 3.88 [18:32:50] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.98, 11.14, 10.35 [18:33:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.59, 3.91, 3.90 [18:34:06] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+10/-10/±8] 13https://github.com/miraheze/puppet/compare/14eee924f56b...943035e0e119 [18:34:07] [url] Comparing 14eee924f56b...943035e0e119 · miraheze/puppet · GitHub | github.com [18:34:08] [02miraheze/puppet] 07Universal-Omega 03943035e - Merge letsencrypt and ssl modules; create ssl-admins group (#2657) [18:34:09] [02puppet] 07Reception123 closed pull request 03#2657: Merge letsencrypt and ssl modules; create ssl-admins group - 13https://github.com/miraheze/puppet/pull/2657 [18:34:10] [url] Page not found · GitHub · GitHub | github.com [18:34:42] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.75, 11.68, 10.98 [18:35:12] PROBLEM - mwtask111 Puppet on mwtask111 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 12 minutes ago with 0 failures [18:36:31] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.46, 3.56, 4.00 [18:36:46] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.88, 10.16, 10.10 [18:36:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.10, 3.77, 3.68 [18:37:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.12, 3.87, 3.88 [18:38:32] PROBLEM - puppet111 Puppet on puppet111 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:38:37] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.57, 11.44, 11.05 [18:38:45] PROBLEM - puppet111 MirahezeRenewSsl on puppet111 is CRITICAL: connect to address 2a10:6740::6:210 and port 5000: Connection refused [18:38:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 3.46, 3.57 [18:39:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.44, 3.61, 3.78 [18:40:47] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.50, 2.09, 1.51 [18:40:49] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 3.43, 2.91, 2.10 [18:42:31] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.32, 4.30, 4.15 [18:42:45] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.78, 1.62, 1.40 [18:42:48] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 5.09, 3.52, 2.41 [18:42:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.72, 4.08, 3.80 [18:43:11] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.14, 10.23, 9.56 [18:46:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.92, 3.63, 3.70 [18:47:03] [02puppet] 07Universal-Omega opened pull request 03#2688: absent old letsencrypt nginx site - 13https://github.com/miraheze/puppet/pull/2688 [18:47:03] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [18:47:03] [url] Page not found · GitHub · GitHub | github.com [18:47:09] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.51, 10.01, 9.62 [18:48:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.37, 3.73, 3.71 [18:50:47] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 3.79, 3.93, 3.10 [18:50:56] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.59, 3.39, 3.59 [18:51:09] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.58, 10.46, 9.90 [18:52:47] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.80, 4.27, 3.33 [18:53:09] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.13, 9.84, 9.73 [18:53:15] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.46, 3.05, 3.36 [18:54:31] RECOVERY - puppet111 Puppet on puppet111 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [18:54:56] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.20, 4.06, 3.79 [18:57:15] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.53, 4.24, 3.76 [18:58:12] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.57, 11.65, 11.34 [19:02:03] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [19:02:52] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.15, 10.63, 9.87 [19:02:58] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.08, 3.92, 3.89 [19:03:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.24, 3.61, 3.69 [19:04:51] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.21, 9.68, 9.61 [19:08:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.44, 3.85, 3.82 [19:10:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.75, 10.10, 9.55 [19:10:45] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.40, 10.19, 9.83 [19:10:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.13, 3.76, 3.80 [19:11:11] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/943035e0e119...1ef14f1f504a [19:11:12] [url] Comparing 943035e0e119...1ef14f1f504a · miraheze/puppet · GitHub | github.com [19:11:13] [02miraheze/puppet] 07Universal-Omega 031ef14f1 - absent old letsencrypt nginx site (#2688) [19:11:14] [02puppet] 07Reception123 closed pull request 03#2688: absent old letsencrypt nginx site - 13https://github.com/miraheze/puppet/pull/2688 [19:11:15] [url] Page not found · GitHub · GitHub | github.com [19:12:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.68, 10.04, 9.60 [19:12:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.95, 9.61, 9.66 [19:14:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.50, 3.40, 3.90 [19:14:46] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.15, 3.73, 3.77 [19:15:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.52, 4.25, 3.77 [19:15:43] PROBLEM - puppet111 Puppet on puppet111 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [19:16:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.99, 10.42, 9.88 [19:16:39] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 3.56, 3.87, 4.03 [19:18:37] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.44, 3.78, 3.98 [19:18:49] PROBLEM - cp20 Stunnel HTTP for puppet111 on cp20 is CRITICAL: HTTP CRITICAL - No data received from host [19:19:02] PROBLEM - cp31 Stunnel HTTP for puppet111 on cp31 is CRITICAL: HTTP CRITICAL - No data received from host [19:19:14] PROBLEM - cp30 Stunnel HTTP for puppet111 on cp30 is CRITICAL: HTTP CRITICAL - No data received from host [19:19:15] PROBLEM - cp21 Stunnel HTTP for puppet111 on cp21 is CRITICAL: HTTP CRITICAL - No data received from host [19:20:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.08, 4.15, 4.10 [19:21:28] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.12, 3.97, 3.82 [19:21:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.85, 11.51, 11.95 [19:22:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 10.16, 9.99, 9.86 [19:25:53] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 2.89, 3.18, 3.93 [19:26:45] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.75, 4.00 [19:27:46] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.14, 11.59, 11.84 [19:28:45] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.51, 4.12, 4.11 [19:29:24] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.17, 3.56, 3.60 [19:29:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.26, 11.12, 11.64 [19:30:27] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.42, 10.66, 10.06 [19:31:35] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 3.24, 2.61, 3.40 [19:31:46] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.13, 11.56, 11.74 [19:32:14] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:32:26] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.09, 10.83, 10.20 [19:32:39] [02puppet] 07Universal-Omega opened pull request 03#2689: Revert "absent old letsencrypt nginx site" - 13https://github.com/miraheze/puppet/pull/2689 [19:32:40] [url] Page not found · GitHub · GitHub | github.com [19:32:55] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1ef14f1f504a...d73de012b111 [19:32:55] [url] Comparing 1ef14f1f504a...d73de012b111 · miraheze/puppet · GitHub | github.com [19:32:56] [02miraheze/puppet] 07Universal-Omega 03d73de01 - Revert "absent old letsencrypt nginx site (#2688)" (#2689) [19:32:58] [02puppet] 07Reception123 closed pull request 03#2689: Revert "absent old letsencrypt nginx site" - 13https://github.com/miraheze/puppet/pull/2689 [19:32:58] [url] Page not found · GitHub · GitHub | github.com [19:33:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.69, 11.47, 11.68 [19:34:24] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.04, 9.80, 9.91 [19:35:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.64, 3.37, 3.56 [19:35:46] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.78, 12.16, 11.89 [19:35:56] RECOVERY - puppet111 Puppet on puppet111 is OK: OK: Puppet is currently enabled, last run 24 seconds ago with 0 failures [19:36:38] RECOVERY - cp20 Stunnel HTTP for puppet111 on cp20 is OK: HTTP OK: Status line output matched "403" - 289 bytes in 0.010 second response time [19:36:51] RECOVERY - cp21 Stunnel HTTP for puppet111 on cp21 is OK: HTTP OK: Status line output matched "403" - 289 bytes in 0.012 second response time [19:37:02] RECOVERY - cp31 Stunnel HTTP for puppet111 on cp31 is OK: HTTP OK: Status line output matched "403" - 289 bytes in 0.237 second response time [19:37:04] RECOVERY - cp30 Stunnel HTTP for puppet111 on cp30 is OK: HTTP OK: Status line output matched "403" - 289 bytes in 0.251 second response time [19:37:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.73, 11.80, 11.78 [19:39:19] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.16, 4.09, 3.80 [19:39:39] !log git clone /etc/puppetlabs/puppet/ssl-keys in /home/ssl-admins on puppet111 [19:39:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:40:30] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.65, 10.40, 9.85 [19:41:18] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.32, 3.80, 3.73 [19:42:17] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.33, 10.47, 10.07 [19:43:17] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.07, 3.95, 3.79 [19:44:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.17, 10.68, 10.20 [19:46:30] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 7.90, 9.95, 10.01 [19:46:34] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.79, 3.99 [19:48:34] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 9.27, 5.24, 4.45 [19:50:12] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.21, 9.61, 9.96 [19:51:13] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.92, 1.73, 1.48 [19:52:54] !log chowned ssl-keys from puppet-users to ssl-admins [19:53:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:53:45] !log mv /var/log/letsencrypt -> /var/log/ssl [19:53:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:55:08] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.78, 1.40, 1.42 [19:55:22] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=sctoolszhwiki (END - exit=0) [19:55:25] RECOVERY - puppet111 MirahezeRenewSsl on puppet111 is OK: TCP OK - 0.001 second response time on 2a10:6740::6:210 port 5000 [19:55:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:56:29] !log sudo service mirahezerenewssl restart (puppet111) [19:56:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:57:46] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.31, 9.27, 10.17 [20:01:02] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 14.18, 10.84, 10.08 [20:02:15] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [20:03:01] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.25, 10.53, 10.07 [20:04:07] PROBLEM - wikipedia.yogfront.ooo - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wikipedia.yogfront.ooo All nameservers failed to answer the query. [20:04:59] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.79, 9.72, 9.81 [20:09:31] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+1/-0/±0] 13https://github.com/miraheze/ssl/commit/ff1474fdc4e5 [20:09:32] [02miraheze/ssl] 07Universal-Omega 03ff1474f - Add ffwiki.org cert [20:09:34] [02ssl] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [20:09:34] [url] Page not found · GitHub · GitHub | github.com [20:09:35] [02ssl] 07Universal-Omega opened pull request 03#560: Add ffwiki.org cert - 13https://github.com/miraheze/ssl/pull/560 [20:09:36] ... [20:10:16] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ff1474fdc4e5...908cc267bd6e [20:10:17] [url] Comparing ff1474fdc4e5...908cc267bd6e · miraheze/ssl · GitHub | github.com [20:10:18] [02miraheze/ssl] 07Universal-Omega 03908cc26 - Update certs.yaml [20:10:19] [02ssl] 07Universal-Omega synchronize pull request 03#560: Add ffwiki.org cert - 13https://github.com/miraheze/ssl/pull/560 [20:10:20] [url] Page not found · GitHub · GitHub | github.com [20:10:34] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/908cc267bd6e...6bc2ef2aadba [20:10:35] [url] Comparing 908cc267bd6e...6bc2ef2aadba · miraheze/ssl · GitHub | github.com [20:10:36] [02miraheze/ssl] 07Universal-Omega 036bc2ef2 - Update certs.yaml [20:10:37] [02ssl] 07Universal-Omega synchronize pull request 03#560: Add ffwiki.org cert - 13https://github.com/miraheze/ssl/pull/560 [20:10:37] [url] Page not found · GitHub · GitHub | github.com [20:11:46] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/add727507554...3c68d72298fb [20:11:46] [url] Comparing add727507554...3c68d72298fb · miraheze/ssl · GitHub | github.com [20:11:47] [02miraheze/ssl] 07Universal-Omega 033c68d72 - Add ffwiki.org cert (#560) [20:11:49] [02miraheze/ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [20:11:50] [02ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [20:11:50] [url] Page not found · GitHub · GitHub | github.com [20:11:51] [02ssl] 07Universal-Omega closed pull request 03#560: Add ffwiki.org cert - 13https://github.com/miraheze/ssl/pull/560 [20:11:52] [url] Page not found · GitHub · GitHub | github.com [20:13:17] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.36, 10.35, 9.62 [20:15:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.24, 1.73, 1.47 [20:17:37] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.16, 1.55, 1.44 [20:18:48] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.26, 10.78, 10.25 [20:20:47] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.34, 11.20, 10.48 [20:21:32] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.49, 2.12, 1.50 [20:23:27] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.83, 1.99, 1.53 [20:24:39] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 9.24, 9.79, 9.91 [20:24:44] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.02, 10.08, 10.20 [20:28:40] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.23, 10.26, 10.30 [20:31:12] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.78, 9.91, 10.15 [20:31:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.03, 3.29, 3.97 [20:32:03] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [20:33:10] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.76, 1.32, 1.44 [20:33:15] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.42, 3.76, 4.06 [20:34:04] RECOVERY - wikipedia.yogfront.ooo - reverse DNS on sslhost is OK: SSL OK - wikipedia.yogfront.ooo reverse DNS resolves to cp21.miraheze.org - CNAME OK [20:35:01] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.17, 10.24, 10.26 [20:35:53] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.46, 3.39, 3.84 [20:38:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.38, 9.44, 9.98 [20:39:51] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.92, 4.13, 4.01 [20:41:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.55, 3.50, 3.79 [20:42:03] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [20:43:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.56, 3.62, 3.90 [20:44:53] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.05, 1.96, 1.49 [20:45:13] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.76, 3.61, 3.95 [20:46:50] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.73, 1.44, 1.35 [20:47:46] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.11, 3.66, 3.76 [20:49:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.16, 3.91, 3.82 [20:49:12] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.54, 4.11, 4.00 [20:49:45] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.57, 3.18, 3.57 [20:51:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.83, 3.31, 3.61 [20:51:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.83, 3.48, 3.78 [20:55:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.45, 2.85, 3.35 [20:55:42] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.75, 2.96, 3.34 [20:57:10] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.52, 2.64, 3.31 [20:59:38] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.96, 3.31, 3.41 [21:00:51] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [21:01:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.72, 3.70, 3.60 [21:01:16] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.24, 9.87, 9.71 [21:01:34] [02miraheze/TSPortal] 07OwenBaines pushed 033 commits to 03master [+0/-0/±6] 13https://github.com/miraheze/TSPortal/compare/8cea5efdda33...e96106cee3a7 [21:01:35] [url] Comparing 8cea5efdda33...e96106cee3a7 · miraheze/TSPortal · GitHub | github.com [21:01:36] [02miraheze/TSPortal] 07OwenBaines 03cb47d8f - Update composer.lock to correct package versions [21:01:37] [02miraheze/TSPortal] 07OwenBaines 0317eb9a1 - Prevent empty comments being submitted to user events history [21:01:37] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.17, 2.92, 3.26 [21:01:39] [02miraheze/TSPortal] 07OwenBaines 03e96106c - Correctly render HTML tags on investigation pages [21:02:41] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.04, 10.23, 9.53 [21:03:12] PROBLEM - mw102 Current Load on mw102 is CRITICAL: CRITICAL - load average: 12.71, 10.54, 9.96 [21:03:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.22, 10.62, 10.10 [21:04:36] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.51, 9.97, 9.53 [21:05:08] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.76, 10.63, 10.06 [21:05:46] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.65, 11.27, 10.40 [21:07:05] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.72, 9.70, 9.78 [21:07:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.16, 10.98, 10.41 [21:09:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.60, 3.10, 3.36 [21:13:08] [02mediawiki] 07Universal-Omega closed pull request 03#7943: Bump skins/Cosmos from `f2be12b` to `579619a` - 13https://github.com/miraheze/mediawiki/pull/7943 [21:13:08] [url] Page not found · GitHub · GitHub | github.com [21:13:09] [02miraheze/mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_38/skins/Cosmos-579619a [21:13:11] [02miraheze/mediawiki] 07Universal-Omega pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/c5c1e5da9466...61b7c599d892 [21:13:11] [url] Comparing c5c1e5da9466...61b7c599d892 · miraheze/mediawiki · GitHub | github.com [21:13:12] [02miraheze/mediawiki] 07dependabot[bot] 0361b7c59 - Bump skins/Cosmos from `f2be12b` to `579619a` (#7943) [21:13:14] [02mediawiki] 07Universal-Omega deleted branch 03dependabot/submodules/REL1_38/skins/Cosmos-579619a - 13https://github.com/miraheze/mediawiki [21:13:14] [url] Page not found · GitHub · GitHub | github.com [21:13:46] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.97, 10.05, 10.18 [21:14:02] !log [universalomega@test101] starting deploy of {'world': True, 'l10n': True, 'lang': 'en'} to all [21:14:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:14:53] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.91, 10.59, 10.13 [21:14:53] !log [universalomega@mwtask111] starting deploy of {'world': True, 'l10n': True, 'lang': 'en'} to all [21:15:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:15:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.24, 3.60, 3.44 [21:16:03] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.83, 3.50, 3.36 [21:17:29] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.80, 3.82, 3.34 [21:18:46] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.97, 9.83, 9.95 [21:19:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.42, 3.78, 3.57 [21:19:28] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.50, 3.77, 3.39 [21:20:51] !log [universalomega@test101] finished deploy of {'world': True, 'l10n': True, 'lang': 'en'} to all - SUCCESS in 409s [21:21:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:21:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.12, 3.86, 3.62 [21:21:27] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.23, 3.76, 3.42 [21:23:01] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.16, 1.76, 1.48 [21:23:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.20, 3.23, 3.42 [21:23:26] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.25, 3.54, 3.38 [21:23:39] !log [universalomega@mwtask111] finished deploy of {'world': True, 'l10n': True, 'lang': 'en'} to all - SUCCESS in 525s [21:23:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:24:41] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.61, 11.27, 10.51 [21:24:58] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.97, 1.47, 1.41 [21:26:00] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.34, 3.17, 3.35 [21:27:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.61, 3.72, 3.55 [21:27:23] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.35, 4.27, 3.69 [21:29:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.89, 3.31, 3.41 [21:29:22] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.85, 3.83, 3.59 [21:30:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.66, 11.41, 10.91 [21:31:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.01, 3.55, 3.49 [21:31:21] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.17, 4.43, 3.82 [21:31:37] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 15.08, 11.68, 9.93 [21:33:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.26, 3.40, 3.43 [21:33:37] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.69, 11.83, 10.22 [21:37:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.00, 3.87, 3.59 [21:39:53] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.14, 4.74, 3.91 [21:40:27] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.15, 8.94, 10.05 [21:41:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.04, 3.76, 3.64 [21:41:37] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.68, 9.64, 9.82 [21:43:15] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.47, 3.86, 3.85 [21:43:52] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.21, 3.91, 3.75 [21:45:52] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.19, 3.92, 3.77 [21:47:51] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.73, 3.72 [21:48:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.99, 1.65, 1.33 [21:50:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.85, 2.04, 1.51 [21:51:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.46, 4.01, 3.72 [21:51:50] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 3.76, 4.24, 3.96 [21:53:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.04, 3.45, 3.54 [21:53:49] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.72, 3.81 [21:55:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.52, 4.64, 3.96 [21:55:48] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.84, 4.06, 3.92 [21:56:40] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.65, 1.92, 1.67 [21:58:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.39, 10.20, 9.71 [21:58:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.97, 2.04, 1.75 [21:59:06] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.16, 3.72, 3.72 [21:59:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.20, 4.00, 3.97 [22:00:08] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.76, 9.81, 9.64 [22:01:05] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.98, 3.57, 3.68 [22:01:37] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.09, 10.09, 9.60 [22:03:37] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.44, 10.85, 9.93 [22:05:09] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.72, 1.40, 1.22 [22:05:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.72, 3.63, 3.91 [22:05:37] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.61, 9.90, 9.69 [22:06:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.41, 10.33, 9.80 [22:06:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.72, 1.79, 1.88 [22:07:01] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.75, 2.94, 3.40 [22:07:09] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.80, 1.14, 1.15 [22:07:59] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.31, 9.54, 9.58 [22:09:44] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 3.26, 3.53 [22:11:44] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.63, 3.15, 3.44 [22:13:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.06, 2.92, 3.40 [22:13:43] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.82, 3.12, 3.40 [22:14:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 7.49, 2.84, 2.12 [22:18:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.47, 1.97, 1.93 [22:20:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.73, 2.28, 2.04 [22:22:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.55, 1.89, 1.92 [22:23:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.85, 3.31, 3.30 [22:23:50] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.74, 3.21, 3.12 [22:25:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.01, 3.53, 3.39 [22:25:39] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.46, 3.88, 3.49 [22:25:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.12, 3.64, 3.30 [22:27:48] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.82, 3.93, 3.46 [22:29:47] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.06, 3.85, 3.48 [22:30:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.97, 1.40, 1.69 [22:31:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.11, 3.85, 3.66 [22:31:46] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.93, 3.46, 3.38 [22:35:35] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.88, 3.64, 3.71 [22:35:44] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.97, 3.61 [22:36:40] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.64, 1.84, 2.00 [22:37:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 3.10, 3.14, 3.38 [22:37:42] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.85, 3.42, 3.44 [22:38:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.71, 2.25, 1.97 [22:38:39] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 1.97, 1.94, 2.01 [22:39:34] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.84, 4.01, 3.82 [22:39:41] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.78, 4.53, 3.85 [22:41:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 7.88, 5.17, 4.10 [22:47:37] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.76, 3.73, 3.89 [22:48:39] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.58, 1.83, 1.91 [22:48:41] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.77, 1.93, 1.56 [22:49:31] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.89, 3.89, 3.98 [22:50:38] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.17, 1.67, 1.51 [22:53:34] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.88, 4.33, 4.08 [22:55:33] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.85, 3.93, 3.97 [22:59:11] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.06, 3.81, 3.96 [22:59:28] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.01, 4.04, 3.93 [22:59:31] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.86, 3.91 [23:00:40] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.27, 1.64, 1.94 [23:01:11] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.21, 4.06, 4.03 [23:02:23] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.24, 2.33, 1.88 [23:02:32] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 8.52, 5.21, 3.43 [23:03:24] PROBLEM - cp30 Stunnel HTTP for matomo101 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:03:34] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.41, 10.32, 9.53 [23:04:20] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.23, 1.91, 1.78 [23:05:24] RECOVERY - cp30 Stunnel HTTP for matomo101 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 99069 bytes in 6.695 second response time [23:05:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.47, 3.90, 3.92 [23:06:41] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.18, 1.48, 1.66 [23:07:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.28, 4.08, 3.99 [23:07:31] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 11.46, 11.10, 10.04 [23:08:14] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.28, 1.60, 1.68 [23:10:32] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.00, 5.92, 4.58 [23:10:39] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.09, 1.83, 1.76 [23:11:27] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.15, 11.40, 10.37 [23:12:39] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.29, 1.57, 1.66 [23:13:26] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.47, 10.22, 10.06 [23:13:57] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-1/±0] 13https://github.com/miraheze/ssl/commit/df451b16bc18 [23:13:59] [02miraheze/ssl] 07Universal-Omega 03df451b1 - Remove lakehub.ga cert [23:14:01] [02ssl] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [23:14:01] [url] Page not found · GitHub · GitHub | github.com [23:14:02] [02ssl] 07Universal-Omega opened pull request 03#561: Remove lakehub.ga cert - 13https://github.com/miraheze/ssl/pull/561 [23:14:03] ... [23:14:24] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/df451b16bc18...d48ef32ec11b [23:14:25] [url] Comparing df451b16bc18...d48ef32ec11b · miraheze/ssl · GitHub | github.com [23:14:26] [02miraheze/ssl] 07Universal-Omega 03d48ef32 - Update certs.yaml [23:14:27] [02ssl] 07Universal-Omega synchronize pull request 03#561: Remove lakehub.ga cert - 13https://github.com/miraheze/ssl/pull/561 [23:14:28] [url] Page not found · GitHub · GitHub | github.com [23:15:19] [02dns] 07Universal-Omega opened pull request 03#295: Delete lakehub.ga zone - 13https://github.com/miraheze/dns/pull/295 [23:15:20] [url] Page not found · GitHub · GitHub | github.com [23:15:38] [02miraheze/ssl] 07Universal-Omega pushed 031 commit to 03master [+0/-1/±1] 13https://github.com/miraheze/ssl/compare/3c68d72298fb...8b3dd030c493 [23:15:39] [url] Comparing 3c68d72298fb...8b3dd030c493 · miraheze/ssl · GitHub | github.com [23:15:40] [02miraheze/ssl] 07Universal-Omega 038b3dd03 - Remove lakehub.ga cert [23:15:41] [02ssl] 07Universal-Omega closed pull request 03#561: Remove lakehub.ga cert - 13https://github.com/miraheze/ssl/pull/561 [23:15:42] [url] Page not found · GitHub · GitHub | github.com [23:15:43] [02miraheze/ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [23:15:44] [02ssl] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://github.com/miraheze/ssl [23:15:45] ... [23:16:32] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 6.55, 5.93, 4.99 [23:17:22] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.11, 10.04, 10.01 [23:18:32] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 5.26, 5.59, 4.97 [23:18:40] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.65, 1.93, 1.82 [23:23:16] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.47, 10.03, 9.94 [23:24:13] PROBLEM - test101 Puppet on test101 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/lakehub.ga.crt] [23:24:32] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 4.12, 4.99, 4.91 [23:24:40] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.81, 2.00, 1.92 [23:25:04] PROBLEM - cp31 Puppet on cp31 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/ssl/localcerts/lakehub.ga.crt] [23:25:14] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.71, 9.34, 9.71 [23:25:20] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.21, 3.35, 3.94 [23:28:33] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.92, 5.28, 5.07 [23:31:56] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.97, 3.78, 4.00 [23:32:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.25, 1.90, 1.89 [23:33:17] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.94, 3.13, 3.85 [23:34:03] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.97, 4.84, 4.99 [23:36:06] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 6.76, 3.52, 2.25 [23:37:09] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 1.94, 2.74, 3.34 [23:38:04] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.85, 9.55, 9.32 [23:40:03] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.82, 8.96, 9.11 [23:43:13] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.99, 3.84, 3.74 [23:44:54] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.00, 3.43, 3.45 [23:44:55] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.65, 1.98, 1.56 [23:45:13] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.10, 3.60, 3.67 [23:46:50] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.37, 3.06, 3.31 [23:48:48] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.53, 2.22, 1.74 [23:50:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.48, 3.35, 3.42 [23:52:42] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.14, 1.80, 1.68 [23:53:16] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 3.37, 3.19, 3.40 [23:53:50] RECOVERY - cp31 Puppet on cp31 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:54:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.03, 3.85, 3.60 [23:54:39] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 0.73, 1.47, 1.58 [23:54:44] RECOVERY - test101 Puppet on test101 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [23:57:09] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.83, 3.57, 3.47 [23:58:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.93, 3.49, 3.52