[00:00:30] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.78, 7.94, 6.42 [00:02:35] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.63, 7.30, 6.37 [00:02:48] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:03:43] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.71, 7.67, 6.92 [00:04:48] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 14.20, 9.06, 6.80 [00:05:35] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.80, 8.00, 6.45 [00:06:25] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 16.69, 11.12, 7.73 [00:07:01] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.68, 21.75, 17.59 [00:09:03] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 22.50, 22.14, 18.27 [00:10:03] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.85, 6.75, 6.77 [00:10:53] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.07, 8.11, 7.05 [00:11:10] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.29, 23.91, 19.35 [00:12:56] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.10, 7.50, 6.94 [00:13:50] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.62, 3.22, 2.89 [00:13:56] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.96, 7.64, 7.12 [00:14:19] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.58, 6.82, 6.90 [00:14:34] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.75, 7.79, 7.52 [00:15:07] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.66, 7.70, 7.39 [00:15:58] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.79, 3.11, 2.89 [00:18:05] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 10.03, 7.69, 7.18 [00:18:34] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.97, 6.15, 6.63 [00:18:38] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/J2wkl [00:18:39] [02miraheze/CreateWiki] 07Universal-Omega 030e480b4 - Don't set wgConf from SetupAfterCache [00:18:41] [02CreateWiki] 07Universal-Omega created branch 03Universal-Omega-patch-3 - 13https://git.io/vpJTL [00:18:42] [02CreateWiki] 07Universal-Omega opened pull request 03#245: Don't set wgConf from SetupAfterCache - 13https://git.io/J2wka [00:19:23] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.21, 9.04, 7.92 [00:21:25] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 6.19, 6.54, 6.69 [00:22:21] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.64, 7.77, 7.44 [00:25:30] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.88, 7.11, 7.50 [00:25:42] miraheze/CreateWiki - Universal-Omega the build passed. [00:27:37] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.36, 22.75, 23.24 [00:28:19] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.53, 5.34, 6.49 [00:28:52] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.44, 5.57, 6.56 [00:31:37] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.46, 5.83, 6.75 [00:32:37] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:35:49] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 27.23, 22.97, 22.86 [00:37:55] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.35, 20.33, 21.91 [00:38:36] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.44, 7.43, 6.65 [00:42:43] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.29, 6.70, 6.53 [00:46:01] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.49, 17.98, 20.20 [00:48:28] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.02, 3.69, 3.38 [00:53:42] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.88, 6.87, 6.55 [00:54:10] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2rnt [00:54:12] [02miraheze/mw-config] 07Universal-Omega 03d2eb648 - Reverse wgConf and extractAllGlobals [00:54:13] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [00:54:15] [02mw-config] 07Universal-Omega opened pull request 03#4124: Reverse wgConf and extractAllGlobals - 13https://git.io/J2rnC [00:54:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.64, 3.83, 3.61 [00:55:12] miraheze/mw-config - Universal-Omega the build passed. [00:57:24] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.70, 7.43, 6.83 [00:57:49] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.91, 7.68, 6.94 [00:59:24] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.99, 7.37, 6.87 [00:59:46] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.06, 6.57, 6.23 [01:01:48] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.65, 5.85, 6.01 [01:01:53] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 5.15, 6.88, 6.83 [01:03:50] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.33, 6.02, 6.52 [01:05:25] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.87, 6.13, 6.53 [01:07:03] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.36, 3.14, 3.38 [01:15:17] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2ovJ [01:15:18] [02miraheze/mw-config] 07Universal-Omega 031a386ef - Update LocalSettings.php [01:15:20] [02mw-config] 07Universal-Omega synchronize pull request 03#4124: Reverse wgConf and extractAllGlobals - 13https://git.io/J2rnC [01:16:17] miraheze/mw-config - Universal-Omega the build passed. [01:27:24] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2o0B [01:27:25] [02miraheze/mw-config] 07Universal-Omega 03e70f85c - Update LocalSettings.php [01:27:27] [02mw-config] 07Universal-Omega synchronize pull request 03#4124: Reverse wgConf and extractAllGlobals - 13https://git.io/J2rnC [01:28:26] miraheze/mw-config - Universal-Omega the build passed. [01:31:11] [02mw-config] 07Universal-Omega closed pull request 03#4124: Reverse wgConf and extractAllGlobals - 13https://git.io/J2rnC [01:31:13] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [01:31:15] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [01:34:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2oMG [01:34:52] [02miraheze/mw-config] 07Universal-Omega 03f6aae9c - extractGlobals inside SetupAfterCache hook [01:34:54] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [01:34:55] [02mw-config] 07Universal-Omega opened pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:35:06] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [01:35:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:32] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2oDN [01:35:33] [02miraheze/mw-config] 07Universal-Omega 03c7ab293 - syntax [01:35:35] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:35:42] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [01:35:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:54] miraheze/mw-config - Universal-Omega the build has errored. [01:35:56] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2oyg [01:35:57] [02miraheze/mw-config] 07github-actions 0370e1778 - CI: lint code to MediaWiki standards [01:35:59] [02mw-config] 07github-actions[bot] synchronize pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:36:00] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.83, 6.63, 5.78 [01:36:36] miraheze/mw-config - Universal-Omega the build passed. [01:37:57] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2o7O [01:37:57] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 858 bytes in 0.080 second response time [01:37:59] [02miraheze/mw-config] 07Universal-Omega 03343b8b2 - Update LocalSettings.php [01:38:00] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.11, 6.08, 5.68 [01:38:00] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:38:06] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [01:38:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:39:06] miraheze/mw-config - Universal-Omega the build passed. [01:46:30] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [01:46:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:47:52] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2KmJ [01:47:53] [02miraheze/mw-config] 07Universal-Omega 03c694e7a - Update LocalSettings.php [01:47:55] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:48:05] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [01:48:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:48:54] miraheze/mw-config - Universal-Omega the build passed. [01:50:00] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2KsI [01:50:02] [02miraheze/mw-config] 07Universal-Omega 03aa74a29 - Update LocalSettings.php [01:50:03] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: extractGlobals inside SetupAfterCache hook - 13https://git.io/J2oM8 [01:50:58] miraheze/mw-config - Universal-Omega the build passed. [01:51:42] [02mw-config] 07Universal-Omega edited pull request 03#4125: extractAllGlobals inside MediaWikiServices hook - 13https://git.io/J2oM8 [01:51:56] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20157 bytes in 0.199 second response time [02:01:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.62, 3.47, 3.34 [02:06:42] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 11.18, 7.57, 6.45 [02:08:42] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 6.36, 6.78, 6.28 [02:11:04] [02mw-config] 07Universal-Omega edited pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [02:21:44] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.13, 4.02, 3.67 [02:23:48] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.76, 3.91, 3.68 [02:25:54] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.09, 3.89, 3.69 [02:27:53] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.89, 3.56, 3.59 [02:30:57] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-3 [+0/-0/±1] 13https://git.io/J26XR [02:30:59] [02miraheze/CreateWiki] 07Universal-Omega 03d100b5b - Use cache file instead of RemoteWiki if possible [02:31:00] [02CreateWiki] 07Universal-Omega synchronize pull request 03#245: Don't set wgConf from SetupAfterCache - 13https://git.io/J2wka [02:33:56] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.73, 3.77, 3.62 [02:37:41] miraheze/CreateWiki - Universal-Omega the build passed. [02:38:06] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.08, 3.56, 3.59 [02:46:18] [02mw-config] 07Universal-Omega edited pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [02:46:21] [02CreateWiki] 07Universal-Omega edited pull request 03#245: Don't set wgConf from SetupAfterCache - 13https://git.io/J2wka [02:46:45] [02CreateWiki] 07Universal-Omega edited pull request 03#245: Don't set `$wgConf` from SetupAfterCache - 13https://git.io/J2wka [02:50:15] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.20, 3.06, 3.39 [03:02:24] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.24, 3.59, 3.41 [03:04:22] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.89, 3.33, 3.34 [03:20:32] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.10, 3.52, 3.28 [03:22:40] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.33, 3.50, 3.31 [03:24:25] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.55, 6.61, 5.86 [03:24:46] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.31, 3.80, 3.44 [03:26:26] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.02, 5.93, 5.70 [03:26:44] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.14, 3.10, 3.23 [03:46:15] [02puppet] 07Universal-Omega opened pull request 03#2000: Enable tideways for CLI - 13https://git.io/J2XW8 [03:53:27] [02puppet] 07Universal-Omega opened pull request 03#2001: Only one file - 13https://git.io/J2Xo4 [03:55:55] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.81, 6.45, 5.73 [03:56:01] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 10.18, 8.29, 6.79 [03:56:34] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.74, 7.83, 6.30 [03:58:03] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.95, 7.93, 6.86 [03:58:36] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.47, 6.74, 6.09 [03:59:35] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 18.41, 9.05, 4.33 [04:00:01] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.14, 6.23, 5.81 [04:02:24] PROBLEM - db12 APT on db12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [04:03:04] PROBLEM - db12 Disk Space on db12 is CRITICAL: DISK CRITICAL - free space: / 23271 MB (5% inode=98%); [04:06:15] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.92, 7.94, 7.21 [04:08:19] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 4.26, 6.67, 6.85 [04:10:18] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.09, 5.85, 6.53 [04:10:30] [02puppet] 07Universal-Omega opened pull request 03#2002: MediaWiki: Install PoolCounter (T7535) - 13https://git.io/J21mZ [04:11:23] PROBLEM - db12 Disk Space on db12 is WARNING: DISK WARNING - free space: / 39413 MB (8% inode=98%); [04:15:29] PROBLEM - db12 Disk Space on db12 is CRITICAL: DISK CRITICAL - free space: / 11110 MB (2% inode=98%); [04:17:22] RECOVERY - db12 APT on db12 is OK: APT OK: 70 packages available for upgrade (0 critical updates). [04:17:30] RECOVERY - db12 Disk Space on db12 is OK: DISK OK - free space: / 62866 MB (14% inode=98%); [04:19:51] [02puppet] 07Universal-Omega opened pull request 03#2003: Remove parsoid.js from miraheze_project_sys.yaml - 13https://git.io/J212X [04:36:29] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2Mkd [04:36:30] [02miraheze/mw-config] 07Universal-Omega 03a391983 - Only do extractAllGlobals once [04:36:32] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:37:15] miraheze/mw-config - Universal-Omega the build passed. [04:37:52] PROBLEM - db12 Current Load on db12 is WARNING: WARNING - load average: 1.40, 2.08, 7.81 [04:38:06] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2MmO [04:38:07] [02miraheze/mw-config] 07Universal-Omega 034f136f3 - Update ManageWikiExtensions.php [04:38:09] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:39:01] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2MOZ [04:39:02] [02miraheze/mw-config] 07Universal-Omega 03951e993 - Update ManageWikiNamespaces.php [04:39:04] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:39:08] miraheze/mw-config - Universal-Omega the build passed. [04:39:55] miraheze/mw-config - Universal-Omega the build passed. [04:40:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2Msd [04:40:15] [02miraheze/mw-config] 07Universal-Omega 03a2f22dc - Update ManageWikiNamespaces.php [04:40:16] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:41:13] miraheze/mw-config - Universal-Omega the build passed. [04:41:56] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 0.87, 1.61, 6.30 [04:44:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2M4A [04:44:18] [02miraheze/mw-config] 07Universal-Omega 037641563 - Update ManageWikiSettings.php [04:44:19] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:45:17] miraheze/mw-config - Universal-Omega the build passed. [04:52:31] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [04:52:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:54:36] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2M9J [04:54:37] [02miraheze/mw-config] 07Universal-Omega 03225c182 - Update Database.php [04:54:39] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [04:54:42] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [04:54:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:55:34] miraheze/mw-config - Universal-Omega the build passed. [04:56:46] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.66, 6.80, 5.91 [04:56:57] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:58:46] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 4.86, 6.10, 5.77 [05:02:11] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2DfM [05:02:13] [02miraheze/mw-config] 07Universal-Omega 0357be34f - Update Wikibase.php [05:02:14] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:02:26] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:02:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:02:36] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2DJw [05:02:38] [02miraheze/mw-config] 07github-actions 034b64d55 - CI: lint code to MediaWiki standards [05:02:39] [02mw-config] 07github-actions[bot] synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:03:14] miraheze/mw-config - Universal-Omega the build passed. [05:04:49] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.72, 3.44, 3.20 [05:06:50] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.56, 3.21, 3.15 [05:26:45] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2DN2 [05:26:46] [02miraheze/mw-config] 07Universal-Omega 030f85a35 - Update Defines.php [05:26:48] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:27:40] miraheze/mw-config - Universal-Omega the build has errored. [05:30:27] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yv4 [05:30:28] [02miraheze/mw-config] 07Universal-Omega 035ce6b1f - Update LocalSettings.php [05:30:30] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:31:29] miraheze/mw-config - Universal-Omega the build has errored. [05:31:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yJx [05:31:45] [02miraheze/mw-config] 07Universal-Omega 03e106979 - Update GlobalCache.php [05:31:46] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:32:27] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yTC [05:32:29] [02miraheze/mw-config] 07Universal-Omega 031d5cb14 - Update Sitenotice.php [05:32:31] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:32:44] miraheze/mw-config - Universal-Omega the build has errored. [05:33:26] miraheze/mw-config - Universal-Omega the build has errored. [05:34:32] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2ytt [05:34:33] [02miraheze/mw-config] 07Universal-Omega 03916c2e4 - Update GlobalLogging.php [05:34:35] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:34:51] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:34:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:35:29] miraheze/mw-config - Universal-Omega the build has errored. [05:37:22] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2y3W [05:37:24] [02miraheze/mw-config] 07Universal-Omega 0384e7fb2 - Update Defines.php [05:37:25] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:37:35] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:37:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:38:22] miraheze/mw-config - Universal-Omega the build has errored. [05:38:50] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yZL [05:38:52] [02miraheze/mw-config] 07Universal-Omega 030d7a9ef - Update Defines.php [05:38:53] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:39:00] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:39:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:39:52] miraheze/mw-config - Universal-Omega the build passed. [05:41:44] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yld [05:41:46] [02miraheze/mw-config] 07Universal-Omega 0387ade80 - Update LocalSettings.php [05:41:47] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:42:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2y89 [05:42:15] [02miraheze/mw-config] 07Universal-Omega 03c0baf1d - Update LocalSettings.php [05:42:16] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:42:31] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:42:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:42:39] miraheze/mw-config - Universal-Omega the build has errored. [05:43:05] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yBc [05:43:07] [02miraheze/mw-config] 07Universal-Omega 03fcfc958 - Update LocalSettings.php [05:43:08] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:43:12] miraheze/mw-config - Universal-Omega the build has errored. [05:44:02] miraheze/mw-config - Universal-Omega the build passed. [05:45:09] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20157 bytes in 0.307 second response time [05:46:35] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2ygZ [05:46:37] [02miraheze/mw-config] 07Universal-Omega 03ff46f70 - Update LocalSettings.php [05:46:38] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:46:46] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:46:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:47:37] miraheze/mw-config - Universal-Omega the build passed. [05:48:13] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yV3 [05:48:14] [02miraheze/mw-config] 07Universal-Omega 0339b9d12 - Update LocalSettings.php [05:48:16] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:48:18] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.86, 3.63, 3.26 [05:48:30] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:48:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:49:06] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 1019 bytes in 0.085 second response time [05:49:14] miraheze/mw-config - Universal-Omega the build passed. [05:49:23] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yru [05:49:24] [02miraheze/mw-config] 07Universal-Omega 0338589d3 - Update LocalSettings.php [05:49:26] [02mw-config] 07Universal-Omega synchronize pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:49:37] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [05:49:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:50:22] miraheze/mw-config - Universal-Omega the build passed. [05:52:26] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.96, 3.35, 3.24 [05:53:14] PROBLEM - cp12 Current Load on cp12 is WARNING: WARNING - load average: 1.58, 1.89, 1.15 [05:55:18] RECOVERY - cp12 Current Load on cp12 is OK: OK - load average: 0.53, 1.36, 1.04 [05:59:05] [02mw-config] 07Universal-Omega closed pull request 03#4125: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2oM8 [05:59:08] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [05:59:09] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-1 [05:59:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yFV [05:59:19] [02miraheze/mw-config] 07Universal-Omega 0324c44c2 - do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook [05:59:21] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/vbvb3 [05:59:22] [02mw-config] 07Universal-Omega opened pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [05:59:33] [02mw-config] 07Universal-Omega edited pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [05:59:44] [02CreateWiki] 07Universal-Omega edited pull request 03#245: Don't set `$wgConf` from SetupAfterCache - 13https://git.io/J2wka [06:00:18] miraheze/mw-config - Universal-Omega the build passed. [06:00:41] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yNp [06:00:42] [02miraheze/mw-config] 07Universal-Omega 036fe9d3e - Update ManageWikiExtensions.php [06:00:43] [02mw-config] 07Universal-Omega synchronize pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [06:01:38] miraheze/mw-config - Universal-Omega the build passed. [06:03:08] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yjf [06:03:09] [02miraheze/mw-config] 07Universal-Omega 03513c159 - Update LocalSettings.php [06:03:11] [02mw-config] 07Universal-Omega synchronize pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [06:03:13] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20157 bytes in 0.232 second response time [06:03:32] [02mw-config] 07Universal-Omega synchronize pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [06:03:34] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2yj9 [06:03:35] [02miraheze/mw-config] 07Universal-Omega 03cbb4699 - Update LocalSettings.php [06:03:44] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [06:03:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:04:19] miraheze/mw-config - Universal-Omega the build passed. [06:04:26] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2SvW [06:04:28] [02miraheze/mw-config] 07Universal-Omega 03c71abca - Update LocalSettings.php [06:04:29] [02mw-config] 07Universal-Omega synchronize pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [06:04:31] miraheze/mw-config - Universal-Omega the build passed. [06:04:34] !log [universalomega@test3] DEPLOY ABORTED: Canary check failed for localhost [06:04:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:05:22] miraheze/mw-config - Universal-Omega the build passed. [06:06:09] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/J2SUP [06:06:11] [02miraheze/mw-config] 07Universal-Omega 0343cc098 - Update LocalSettings.php [06:06:12] [02mw-config] 07Universal-Omega synchronize pull request 03#4126: do `$wgConf->extractAllGlobals()` inside MediaWikiServices hook - 13https://git.io/J2yFD [06:07:14] miraheze/mw-config - Universal-Omega the build passed. [06:08:41] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.42, 3.52, 3.39 [06:10:42] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.21, 3.13, 3.27 [06:23:07] [02puppet] 07Universal-Omega closed pull request 03#2002: MediaWiki: Install PoolCounter (T7535) - 13https://git.io/J21mZ [07:23:37] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.92, 3.41, 3.08 [07:25:39] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.50, 3.08, 3.00 [07:48:18] [02puppet] 07Reception123 closed pull request 03#2001: Only one file - 13https://git.io/J2Xo4 [07:48:20] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J2QmJ [07:48:21] [02miraheze/puppet] 07Universal-Omega 0329ce190 - Only one file (#2001) [07:49:27] [02puppet] 07Reception123 closed pull request 03#1995: mwscript: drop option to use foreachwikiindblist - 13https://git.io/JzhaA [07:49:28] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/J2QOR [07:49:30] [02miraheze/puppet] 07Universal-Omega 03bc1b494 - mwscript: drop option to use foreachwikiindblist (#1995) [08:03:35] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 14.44, 8.53, 4.39 [08:07:35] PROBLEM - db12 Current Load on db12 is WARNING: WARNING - load average: 2.33, 7.08, 4.91 [08:09:35] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 1.40, 5.07, 4.44 [08:10:33] Reception123: ^ [08:10:38] Load is having spikes [08:10:58] hm, has this been happening before on db12? [08:11:04] (I mean like in the past few days) [08:12:22] Happened once earlier today. But went down again quickly [08:12:34] Well last night for you. [08:12:50] Reception123: it must be a specific query [08:13:14] Graph is obvious in last day [08:13:16] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=db12.miraheze.org&var-port=9100&from=now-12h&to=now-1m [08:13:17] [url] Grafana | grafana.miraheze.org [08:13:29] Can you match timings to data in logs? [08:13:46] oh wow that definitely is a spike [08:13:55] earlier was definitely worse than the one now [08:14:03] Yeah [08:14:22] the bad thing is the logs are flooded with the old warnings [08:14:30] like the aborted connections and duplicate keys [08:15:35] Duplicate key isn't a warning [08:20:06] yeah that's not a warning but it's been there for months [08:22:55] Can you paste that in the MWE channel [08:45:18] !log [rhinos@mw8] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/dynamicpagelist.json /srv/mediawiki/w/maintenance/test.php (END - exit=2) [08:45:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [09:26:18] !log [rhinos@mw8] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/dynamicpagelist.json /srv/mediawiki/w/maintenance/sql.php /home/rhinos/clcount.sql (END - exit=2) [09:26:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:34:50] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.98, 3.10, 2.53 [10:36:48] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.85, 3.00, 2.57 [10:57:55] PROBLEM - mw11 PowerDNS Recursor on mw11 is CRITICAL: Domain miraheze.org was not found by the server [10:59:57] RECOVERY - mw11 PowerDNS Recursor on mw11 is OK: DNS OK: 0.285 seconds response time. miraheze.org returns 2001:41d0:800:170b::5,2001:41d0:801:2000::58af,51.38.69.175,54.38.211.199 [11:34:21] !log [reception@mwtask1] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=particracyclassicwiki /home/reception/particracy_pages_full.xml --username-prefix wikia:particracywiki --report 1 (END - exit=0) [11:34:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:48:30] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.29, 8.21, 6.28 [11:50:23] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.88, 6.39, 5.47 [11:50:28] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.22, 7.59, 6.30 [11:52:19] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.88, 5.88, 5.40 [11:52:25] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.11, 6.40, 6.02 [13:20:56] PROBLEM - iesd.mobagenie.my.id - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 148, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 129, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 66, in check_records nameserversans = dns_resolver.query(root_domain, 'NS') File "/usr/lib/p [13:20:56] 3/dist-packages/dns/resolver.py", line 898, in query raise NoNameservers(request=request, errors=errors)dns.resolver.NoNameservers: All nameservers failed to answer the query mobagenie.my.id. IN NS: Server 1.1.1.1 UDP port 53 answered SERVFAIL [13:26:19] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 7.81, 6.63, 5.87 [13:28:24] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.03, 5.92, 5.70 [13:34:14] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 9.39, 7.43, 5.96 [13:35:43] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.52, 6.44, 5.68 [13:36:11] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.26, 6.94, 5.96 [13:36:12] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.87, 7.52, 6.39 [13:37:43] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.21, 6.53, 5.82 [13:38:12] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.39, 6.77, 6.25 [13:40:09] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.95, 6.09, 5.85 [13:43:37] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.44, 7.59, 6.64 [13:45:36] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.13, 6.89, 6.49 [13:47:41] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.63, 6.45, 6.38 [13:49:10] RECOVERY - iesd.mobagenie.my.id - reverse DNS on sslhost is OK: SSL OK - iesd.mobagenie.my.id reverse DNS resolves to cp14.miraheze.org - CNAME OK [14:09:34] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.57, 3.45, 3.21 [14:11:37] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.42, 3.04, 3.08 [14:22:16] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.20, 7.43, 6.31 [14:23:01] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.14, 3.63, 3.25 [14:24:20] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.14, 6.46, 6.09 [14:25:07] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.72, 3.62, 3.29 [14:27:11] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.99, 7.66, 6.30 [14:27:13] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.09, 3.34, 3.23 [14:27:15] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.59, 7.17, 6.35 [14:28:33] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.85, 7.31, 6.56 [14:29:04] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.85, 7.25, 6.27 [14:29:10] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.58, 7.54, 6.42 [14:29:18] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 6.58, 6.80, 6.31 [14:30:35] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 6.13, 6.77, 6.45 [14:31:12] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.11, 7.82, 6.66 [14:33:15] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 8.16, 7.28, 6.47 [14:34:53] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.43, 7.40, 6.49 [14:34:56] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 10.59, 8.06, 7.03 [14:35:39] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 9.50, 8.47, 7.18 [14:36:54] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.08, 7.87, 7.04 [14:37:20] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 4.87, 7.18, 6.85 [14:37:23] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.12, 7.44, 6.81 [14:37:44] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.53, 7.60, 7.02 [14:38:52] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.38, 7.75, 7.10 [14:38:57] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.02, 6.22, 6.29 [14:39:04] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 6.57, 7.36, 7.02 [14:39:16] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 5.99, 6.60, 6.66 [14:39:19] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.57, 6.32, 6.47 [14:41:07] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 11.63, 8.69, 7.53 [14:41:51] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 8.13, 7.53, 7.07 [14:43:11] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 8.04, 7.37, 6.75 [14:43:55] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 6.32, 7.23, 7.03 [14:45:03]