[00:00:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.40, 4.21, 3.61 [00:01:07] PROBLEM - mw122 Current Load on mw122 is CRITICAL: CRITICAL - load average: 10.06, 8.84, 8.32 [00:01:51] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.72, 8.49, 7.53 [00:02:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.07, 3.75, 3.41 [00:03:05] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.72, 8.33, 8.19 [00:03:07] RECOVERY - matomo101 Disk Space on matomo101 is OK: DISK OK - free space: / 1785 MB (18% inode=80%); [00:03:12] PROBLEM - mw101 Puppet on mw101 is CRITICAL: CRITICAL: Puppet has 431 failures. Last run 2 minutes ago with 431 failures. Failed resources (up to 3 shown): File[www.mcpk.wiki_private],File[wiki.warframestat.us],File[wiki.warframestat.us_private],File[wiki.shaazzz.ir] [00:03:23] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.15, 9.85, 8.87 [00:03:50] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 6.52, 7.73, 7.37 [00:03:58] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.75, 3.80, 3.47 [00:04:47] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.42, 9.04, 8.47 [00:05:20] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.22, 9.23, 8.76 [00:06:05] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.07, 3.71, 3.12 [00:06:45] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.51, 9.32, 8.65 [00:06:59] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.00, 8.61, 8.40 [00:07:50] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.97, 4.67, 3.83 [00:10:55] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 5.76, 7.66, 8.10 [00:12:01] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.18, 3.79, 3.42 [00:12:39] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.58, 9.88, 9.05 [00:14:38] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.58, 9.43, 8.99 [00:15:59] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.93, 4.22, 3.67 [00:17:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.34, 3.85, 3.60 [00:18:34] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.17, 9.85, 9.22 [00:18:54] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.64, 10.28, 9.20 [00:19:56] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 5.24, 4.05, 3.68 [00:20:23] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.11, 1.91, 1.62 [00:20:51] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.16, 9.88, 9.20 [00:22:22] RECOVERY - test101 Current Load on test101 is OK: OK - load average: 1.17, 1.60, 1.54 [00:22:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.88, 3.87, 3.99 [00:22:29] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.35, 9.08, 8.80 [00:23:36] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.14, 1.72, 1.33 [00:23:53] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.09, 3.21, 3.44 [00:24:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.33, 3.94, 3.97 [00:24:29] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.31, 8.76, 8.72 [00:25:35] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.54, 1.67, 1.36 [00:25:53] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.28, 2.97, 3.33 [00:26:27] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.30, 9.94, 9.54 [00:26:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 4.00, 3.85, 3.93 [00:28:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.63, 4.85, 4.28 [00:29:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.41, 3.58, 3.99 [00:31:03] RECOVERY - mw101 Puppet on mw101 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [00:33:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 6.91, 4.71, 4.29 [00:35:53] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.43, 3.28, 3.41 [00:36:18] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.02, 9.24, 9.22 [00:36:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 1.49, 3.16, 3.76 [00:36:29] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.25, 7.95, 8.43 [00:37:53] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 1.50, 2.72, 3.18 [00:38:16] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.08, 9.50, 9.35 [00:39:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.38, 3.43, 3.88 [00:40:13] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.39, 3.85, 2.31 [00:42:12] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.21, 9.68, 9.43 [00:42:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.84, 2.79, 3.40 [00:42:29] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.50, 9.01, 8.73 [00:44:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 6.07, 8.48, 9.03 [00:44:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.14, 7.76, 8.37 [00:45:57] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.50, 1.96, 1.90 [00:46:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:48:07] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.96, 8.90, 9.00 [00:50:05] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.97, 8.68, 8.91 [00:50:12] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03revert-4499 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/f427b2ecbf74 [00:50:13] [02miraheze/mw-config] 07Universal-Omega 03f427b2e - Revert "Adding missing Mobilefrontend configs (#4499)" [00:50:15] [02mw-config] 07Universal-Omega created branch 03revert-4499 - 13https://github.com/miraheze/mw-config [00:50:16] [02mw-config] 07Universal-Omega opened pull request 03#4516: Revert "Adding missing Mobilefrontend configs (#4499)" - 13https://github.com/miraheze/mw-config/pull/4516 [00:50:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.22, 8.89, 8.60 [00:51:23] miraheze/mw-config - Universal-Omega the build passed. [00:51:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.05, 1.43, 1.68 [00:52:03] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.33, 9.18, 9.06 [00:52:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.10, 8.49, 8.48 [00:52:29] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.03, 9.26, 8.88 [00:52:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [00:52:47] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.68, 3.41, 3.14 [00:53:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.72, 3.51, 3.54 [00:53:09] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.62, 8.02, 7.88 [00:53:14] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.22, 1.78, 1.49 [00:54:01] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.12, 8.27, 8.73 [00:54:29] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 6.55, 8.23, 8.55 [00:54:46] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.96, 3.67, 3.25 [00:55:13] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.40, 1.64, 1.47 [00:56:29] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.63, 7.88, 8.39 [00:57:57] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.31, 7.69, 8.40 [00:58:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.00, 4.18, 3.59 [00:59:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.94, 3.63, 3.61 [00:59:23] [02mw-config] 07Universal-Omega edited pull request 03#4516: Revert "Adding missing Mobilefrontend configs (#4499)" - 13https://github.com/miraheze/mw-config/pull/4516 [01:00:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.87, 3.75, 3.51 [01:00:42] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.74, 3.56, 3.39 [01:02:13] [02mw-config] 07Universal-Omega closed pull request 03#4516: Revert "Adding missing Mobilefrontend configs (#4499)" - 13https://github.com/miraheze/mw-config/pull/4516 [01:02:14] [02mw-config] 07Universal-Omega deleted branch 03revert-4499 - 13https://github.com/miraheze/mw-config [01:02:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ab883efb0614...480d32d01d02 [01:02:17] [02miraheze/mw-config] 07Universal-Omega 03480d32d - Revert "Adding missing Mobilefrontend configs (#4499)" (#4516) [01:02:19] [02miraheze/mw-config] 07Universal-Omega deleted branch 03revert-4499 [01:02:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.63, 3.28, 3.36 [01:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:02:41] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.21, 3.06, 3.22 [01:03:19] miraheze/mw-config - Universal-Omega the build passed. [01:03:50] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.22, 8.68, 8.52 [01:04:57] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 8.19, 8.37, 8.22 [01:05:48] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.31, 9.12, 8.72 [01:06:03] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 1.41, 2.79, 1.90 [01:06:27] [02mw-config] 07Universal-Omega commented on pull request 03#4510: T8932: ApprovedRevs configs - 13https://github.com/miraheze/mw-config/pull/4510#issuecomment-1072909487 [01:06:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:07:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.95, 3.14, 3.40 [01:08:34] [02mw-config] 07Universal-Omega commented on pull request 03#4507: T8925: setting Tweeki's subnav to false on obeymewiki - 13https://github.com/miraheze/mw-config/pull/4507#issuecomment-1072909842 [01:11:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.66, 3.64, 3.57 [01:11:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:11:31] [02mw-config] 07Universal-Omega commented on pull request 03#4501: ULS settings for GP - 13https://github.com/miraheze/mw-config/pull/4501#issuecomment-1072910319 [01:11:43] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.65, 9.52, 8.93 [01:11:53] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.30, 1.97, 1.80 [01:12:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.67, 3.86, 3.56 [01:13:41] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.51, 9.26, 8.92 [01:13:50] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.04, 1.59, 1.68 [01:14:21] [02mw-config] 07Universal-Omega reviewed pull request 03#4515 commit - 13https://github.com/miraheze/mw-config/pull/4515#discussion_r830420818 [01:15:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.72, 3.92, 3.68 [01:15:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [01:16:29] [02mw-config] 07Universal-Omega reviewed pull request 03#4509 commit - 13https://github.com/miraheze/mw-config/pull/4509#discussion_r830421070 [01:17:38] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.74, 9.50, 9.07 [01:18:24] [02mw-config] 07Universal-Omega reviewed pull request 03#4505 commit - 13https://github.com/miraheze/mw-config/pull/4505#discussion_r830421292 [01:19:00] !log [@test101] starting deploy of {'config': True} to all [01:19:01] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [01:19:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.54, 3.87, 3.73 [01:19:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:19:36] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.36, 9.42, 9.10 [01:19:50] [02mw-config] 07Universal-Omega reviewed pull request 03#4500 commit - 13https://github.com/miraheze/mw-config/pull/4500#discussion_r830421465 [01:19:52] [02mw-config] 07Universal-Omega reviewed pull request 03#4500 commit - 13https://github.com/miraheze/mw-config/pull/4500#discussion_r830421521 [01:20:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:20:55] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.34, 9.34, 8.44 [01:22:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 4.01, 3.68 [01:22:51] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.60, 9.16, 8.49 [01:23:03] !log [@mwtask111] starting deploy of {'config': True} to all [01:23:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.56, 4.03, 3.83 [01:23:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:17] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 13s [01:23:32] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.52, 9.86, 9.30 [01:23:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:24:24] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.74, 3.43, 3.24 [01:24:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.58, 3.42, 3.50 [01:24:40] [02mw-config] 07Universal-Omega reviewed pull request 03#4508 commit - 13https://github.com/miraheze/mw-config/pull/4508#discussion_r830422038 [01:25:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.13, 3.81, 3.78 [01:26:22] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.92, 3.32, 3.22 [01:26:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.29, 3.13, 3.39 [01:28:40] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.37, 8.28, 8.31 [01:29:54] [02miraheze/mw-config] 07Universal-Omega pushed 0353 commits to 03Universal-Omega-patch-2 [+2/-0/±68] 13https://github.com/miraheze/mw-config/compare/e60fc447c2b0...da0d5a54da49 [01:29:56] [02miraheze/mw-config] 07Universal-Omega 03da0d5a5 - Merge branch 'master' into Universal-Omega-patch-2 [01:29:57] [02mw-config] 07Universal-Omega synchronize pull request 03#4465: Rewrite SiteConfiguration setup - 13https://github.com/miraheze/mw-config/pull/4465 [01:30:11] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.90, 8.81, 8.30 [01:31:05] miraheze/mw-config - Universal-Omega the build passed. [01:31:25] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.94, 9.96, 9.70 [01:31:34] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/da0d5a54da49...8791af8d47c5 [01:31:36] [02miraheze/mw-config] 07Universal-Omega 038791af8 - Update Database.php [01:31:37] [02mw-config] 07Universal-Omega synchronize pull request 03#4465: Rewrite SiteConfiguration setup - 13https://github.com/miraheze/mw-config/pull/4465 [01:32:10] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.48, 8.31, 8.18 [01:32:37] miraheze/mw-config - Universal-Omega the build passed. [01:33:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 1.86, 2.64, 3.24 [01:33:32] [02mw-config] 07Universal-Omega commented on pull request 03#4473: Add wgPageForms{RenameEditTabs,RenameMainEditTab,LinkAllRedLinksToFor… - 13https://github.com/miraheze/mw-config/pull/4473#issuecomment-1072913676 [01:34:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.86, 3.68, 3.49 [01:35:22] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 9.33, 9.38, 8.59 [01:36:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.60, 3.65, 3.50 [01:36:28] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.03, 9.24, 8.68 [01:38:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.13, 8.49, 8.48 [01:38:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 8.03, 4.77, 3.89 [01:39:04] [02mw-config] 07Universal-Omega commented on pull request 03#4514: Fixing arrangement of this config - 13https://github.com/miraheze/mw-config/pull/4514#issuecomment-1072914433 [01:40:05] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.40, 9.10, 8.53 [01:41:16] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.16, 8.24, 8.36 [01:42:04] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.78, 9.40, 8.71 [01:43:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.21, 1.71, 1.47 [01:44:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.28, 3.90, 3.81 [01:45:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.60, 1.30, 1.35 [01:46:02] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.55, 7.97, 8.29 [01:46:57] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.65, 3.51, 3.38 [01:48:53] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.94, 3.17, 3.27 [01:51:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.72, 8.01, 8.48 [01:51:14] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.87, 8.71, 8.51 [01:51:51] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 9.13, 8.66, 7.63 [01:52:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 6.37, 4.07, 3.80 [01:53:51] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 6.67, 7.96, 7.51 [01:54:41] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 1.80, 1.85, 1.55 [01:55:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.40, 8.53, 8.62 [01:56:41] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.24, 1.65, 1.51 [01:56:55] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.67, 8.94, 8.50 [01:56:58] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 8.66, 8.26, 8.11 [01:58:56] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 7.51, 8.14, 8.10 [01:58:57] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.93, 3.67, 3.32 [01:59:00] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.03, 9.31, 8.86 [02:00:56] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.51, 9.50, 8.99 [02:01:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.17, 9.25, 8.90 [02:02:29] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.98, 3.98, 3.58 [02:02:52] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.65, 8.40, 8.45 [02:02:54] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.25, 4.52, 3.70 [02:04:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 3.97, 4.21, 3.72 [02:04:49] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.03, 9.76, 9.21 [02:06:45] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.28, 9.30, 9.11 [02:06:46] PROBLEM - mw122 Current Load on mw122 is WARNING: WARNING - load average: 6.64, 8.73, 8.46 [02:08:17] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.24, 3.53, 3.58 [02:08:50] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.69, 3.61, 3.57 [02:09:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.05, 9.48, 9.37 [02:10:13] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.53, 4.17, 3.79 [02:10:43] RECOVERY - mw122 Current Load on mw122 is OK: OK - load average: 6.74, 7.84, 8.18 [02:12:47] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.01, 9.14, 8.52 [02:14:30] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 10.49, 9.76, 9.35 [02:14:46] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.05, 3.55, 3.54 [02:15:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:16:01] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.27, 3.95, 3.85 [02:16:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.78, 3.62, 3.98 [02:16:45] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 8.57, 9.06, 8.64 [02:16:45] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.37, 3.33, 3.45 [02:18:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.24, 4.41, 4.23 [02:18:44] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.22, 2.90, 3.28 [02:19:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.01, 9.11, 9.10 [02:20:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.61, 9.88, 9.63 [02:20:43] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.87, 7.92, 8.26 [02:21:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.79, 9.42, 9.22 [02:21:49] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 5.84, 4.22, 3.94 [02:22:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:23:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.84, 9.86, 9.40 [02:24:40] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.48, 8.65, 8.47 [02:25:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.00, 9.25, 9.24 [02:25:41] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.04, 3.57, 3.73 [02:26:39] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 6.37, 7.87, 8.21 [02:28:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.08, 9.75, 9.60 [02:29:35] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.83, 3.38, 3.25 [02:30:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.23, 9.34, 9.48 [02:32:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:33:25] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.42, 3.80, 3.67 [02:35:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 13.42, 10.59, 9.70 [02:35:21] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.05, 3.49, 3.57 [02:35:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.94, 1.83, 1.36 [02:36:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [02:37:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.62, 1.92, 1.44 [02:39:31] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.42, 8.76, 8.36 [02:39:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.52, 1.85, 1.48 [02:41:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 9.02, 9.74, 9.64 [02:41:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.80, 1.49, 1.39 [02:43:25] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.88, 3.26, 3.38 [02:46:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 13.29, 9.98, 9.27 [02:47:27] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.69, 8.32, 8.35 [02:48:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.64, 3.13, 3.90 [02:49:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.10, 3.10, 3.40 [02:52:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.23, 3.49, 3.85 [02:52:49] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [02:54:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.29, 9.79, 9.60 [02:55:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.15, 3.72, 3.59 [02:57:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 4.15, 3.76 [02:57:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.13, 9.28, 9.41 [02:59:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.59, 9.57, 9.54 [02:59:12] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.65, 3.92, 3.55 [03:01:11] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.38, 3.38, 3.39 [03:05:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.04, 10.05, 9.61 [03:07:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.21, 3.79, 3.82 [03:07:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.92, 9.32, 9.41 [03:08:26] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.02, 1.68, 1.40 [03:08:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.04, 8.92, 8.95 [03:10:23] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.62, 1.63, 1.41 [03:10:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.97, 8.42, 8.76 [03:12:12] PROBLEM - mw121 Current Load on mw121 is CRITICAL: CRITICAL - load average: 10.60, 8.66, 8.18 [03:13:08] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.27, 3.44, 3.58 [03:14:11] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.78, 9.21, 8.45 [03:15:08] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.05, 3.30, 3.52 [03:16:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.24, 9.72, 9.17 [03:16:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.29, 3.43, 3.86 [03:17:08] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.47, 3.05, 3.40 [03:17:10] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 11.40, 9.75, 9.34 [03:19:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.63, 9.30, 9.23 [03:21:46] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [03:22:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.98, 10.00, 9.53 [03:23:22] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 2.35, 1.66, 1.38 [03:24:26] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 11.89, 10.56, 9.79 [03:25:21] RECOVERY - mwtask111 Current Load on mwtask111 is OK: OK - load average: 1.44, 1.58, 1.39 [03:28:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.48, 4.72, 4.15 [03:29:31] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.54, 6.47, 5.47 [03:30:03] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 7.04, 8.15, 8.46 [03:30:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 8.24, 9.87, 9.81 [03:30:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.90, 3.94, 3.93 [03:31:26] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.13, 5.60, 5.27 [03:33:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.10, 7.13, 8.23 [03:36:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:38:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.40, 7.18, 8.47 [03:40:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.20, 3.91, 3.83 [03:42:14] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/88d863f4c847...0df9756aa5db [03:42:16] [02miraheze/ssl] 07MacFan4000 030df9756 - renew www.journeytheword.wiki cert [03:42:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.17, 3.65, 3.74 [03:44:56] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0df9756aa5db...e0d8827266b8 [03:44:57] [02miraheze/ssl] 07MacFan4000 03e0d8827 - fix [03:47:42] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e0d8827266b8...31f3fb77f502 [03:47:43] [02miraheze/ssl] 07MacFan4000 0331f3fb7 - renew opendatascot.org cert [03:50:13] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/31f3fb77f502...f69ccae0e0fa [03:50:14] [02miraheze/ssl] 07MacFan4000 03f69ccae - Renew with.cpt-ra.bid cert [03:50:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.95, 3.13, 3.40 [03:50:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [03:55:52] RECOVERY - journeytheword.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.journeytheword.wiki' will expire on Fri 17 Jun 2022 02:44:07 GMT +0000. [03:56:26] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 7.39, 8.52, 8.43 [03:56:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.31, 3.85, 3.67 [03:59:11] RECOVERY - opendatascot.org - LetsEncrypt on sslhost is OK: OK - Certificate 'opendatascot.org' will expire on Fri 17 Jun 2022 02:46:18 GMT +0000. [04:00:26] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.44, 7.82, 8.21 [04:00:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.75, 4.03, 3.76 [04:02:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.73, 3.96, 3.78 [04:02:53] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.20, 3.66, 3.14 [04:03:27] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 2.92, 3.44, 3.09 [04:04:49] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.26, 3.12, 3.00 [04:05:26] RECOVERY - gluster111 Current Load on gluster111 is OK: OK - load average: 2.94, 3.25, 3.06 [04:06:27] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.27, 3.72, 3.69 [04:07:10] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 7.91, 8.80, 8.32 [04:08:43] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.50, 3.75, 3.30 [04:09:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 6.25, 7.88, 8.04 [04:09:54] RECOVERY - www.journeytheword.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'www.journeytheword.wiki' will expire on Fri 17 Jun 2022 02:44:07 GMT +0000. [04:10:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.19, 3.41, 3.56 [04:10:39] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.37, 3.22, 3.16 [04:15:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:18:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.00, 2.70, 3.23 [04:20:57] RECOVERY - www.opendatascot.org - LetsEncrypt on sslhost is OK: OK - Certificate 'opendatascot.org' will expire on Fri 17 Jun 2022 02:46:18 GMT +0000. [04:23:49] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 0.94, 1.75, 1.50 [04:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:25:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.60, 1.41, 1.41 [04:30:30] RECOVERY - with.cpt-ra.bid - LetsEncrypt on sslhost is OK: OK - Certificate 'with.cpt-ra.bid' will expire on Fri 17 Jun 2022 02:49:21 GMT +0000. [04:34:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [04:38:08] PROBLEM - cp31 Current Load on cp31 is WARNING: WARNING - load average: 1.74, 1.75, 1.24 [04:40:05] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.54, 1.59, 1.24 [04:41:47] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.29, 3.85, 3.46 [04:49:34] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.90, 3.34, 3.37 [04:57:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:02:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:07:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:07:55] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 5.38, 3.80, 3.35 [05:12:08] PROBLEM - mw121 Current Load on mw121 is WARNING: WARNING - load average: 9.21, 8.67, 7.72 [05:14:07] RECOVERY - mw121 Current Load on mw121 is OK: OK - load average: 8.17, 8.36, 7.71 [05:15:15] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 10.01, 8.75, 7.78 [05:15:41] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.52, 3.70, 3.55 [05:17:13] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.65, 8.30, 7.70 [05:19:35] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.36, 3.49, 3.47 [05:21:10] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.72, 8.36, 7.91 [05:23:29] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.38, 3.87, 3.67 [05:27:22] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.43, 3.50, 3.51 [05:27:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [05:33:12] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.42, 3.34, 3.46 [05:39:02] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.60, 3.20, 3.39 [05:50:30] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - dnd.bellinrattin.it All nameservers failed to answer the query. [06:02:27] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.03, 3.41, 3.07 [06:04:27] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.26, 3.00, 2.97 [06:04:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:09:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:18:47] [02mw-config] 07ugochimobi commented on pull request 03#4515: Update LocalSettings.php with MFE configs - 13https://github.com/miraheze/mw-config/pull/4515#issuecomment-1072951798 [06:19:27] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: Traceback (most recent call last): File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 155, in main() File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 136, in main records = check_records(args.hostname) File "/usr/lib/nagios/plugins/check_reverse_dns.py", line 73, in check_records nameserversans = dns_resolver.resolve(root_domain, 'NS') File "/usr/l [06:19:27] n3/dist-packages/dns/resolver.py", line 1040, in resolve (nameserver, port, tcp, backoff) = resolution.next_nameserver() File "/usr/lib/python3/dist-packages/dns/resolver.py", line 598, in next_nameserver raise NoNameservers(request=self.request, errors=self.errors)dns.resolver.NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:39:36] PROBLEM - mw111 NTP time on mw111 is WARNING: NTP WARNING: Offset -0.1095356047 secs [06:41:37] RECOVERY - mw111 NTP time on mw111 is OK: NTP OK: Offset -0.08575004339 secs [06:43:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [06:49:38] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hsfflwiki --username-prefix=mh:hsffl --report 1 /home/reception/hsfflwiki.xml (START) [06:49:42] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.91, 8.14, 6.95 [06:49:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:49:43] !log [reception@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/importDump.php --wiki=hsfflwiki --username-prefix=mh:hsffl --report 1 /home/reception/hsfflwiki.xml (END - exit=0) [06:49:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:51:38] PROBLEM - mw112 Current Load on mw112 is CRITICAL: CRITICAL - load average: 12.33, 9.15, 7.43 [06:53:34] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.95, 9.53, 7.79 [06:55:31] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.79, 8.50, 7.61 [07:07:49] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 1.76, 2.88, 1.74 [07:08:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [07:11:49] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.48, 1.61, 1.48 [07:16:02] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.19, 3.53, 2.93 [07:18:52] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.33, 8.05, 7.38 [07:19:54] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.68, 3.73, 3.17 [07:20:48] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 8.33, 8.23, 7.54 [07:23:46] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.47, 3.12, 3.04 [07:46:18] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp20.miraheze.org - CNAME OK [08:10:55] PROBLEM - cloud10 Puppet on cloud10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [08:24:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:29:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:34:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 13.85, 9.24, 6.22 [08:36:31] PROBLEM - db111 Current Load on db111 is WARNING: WARNING - load average: 5.98, 7.89, 6.10 [08:38:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 17.46, 11.09, 7.45 [08:38:55] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures [08:42:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:42:31] PROBLEM - db111 Current Load on db111 is WARNING: WARNING - load average: 4.46, 7.98, 7.08 [08:46:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 9.23, 7.98, 7.23 [08:47:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [08:48:31] RECOVERY - db111 Current Load on db111 is OK: OK - load average: 4.10, 6.44, 6.75 [08:59:00] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.18, 8.26, 7.40 [09:00:56] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 7.95, 7.88, 7.35 [09:10:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:15:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:24:32] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:39:32] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:40:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 32.74, 15.08, 8.59 [09:44:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [09:56:31] PROBLEM - db111 Current Load on db111 is WARNING: WARNING - load average: 3.76, 5.30, 7.79 [09:59:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:00:31] RECOVERY - db111 Current Load on db111 is OK: OK - load average: 2.72, 3.98, 6.68 [10:18:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:23:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:26:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:31:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:42:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 22.18, 14.46, 8.46 [10:51:33] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:54:31] PROBLEM - db111 Current Load on db111 is WARNING: WARNING - load average: 5.08, 7.10, 7.76 [10:56:33] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [10:58:31] PROBLEM - db111 Current Load on db111 is CRITICAL: CRITICAL - load average: 11.79, 8.25, 7.93 [11:00:31] PROBLEM - db111 Current Load on db111 is WARNING: WARNING - load average: 4.75, 6.94, 7.50 [11:02:01] PROBLEM - mw112 Current Load on mw112 is WARNING: WARNING - load average: 9.06, 8.68, 8.16 [11:03:58] RECOVERY - mw112 Current Load on mw112 is OK: OK - load average: 6.57, 7.98, 7.97 [11:04:31] RECOVERY - db111 Current Load on db111 is OK: OK - load average: 2.18, 4.75, 6.50 [11:08:31] alerting : [FIRING:1] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:13:31] ok : [RESOLVED] (PHP-FPM Worker Usage High yes mediawiki) https://grafana.miraheze.org/d/dsHv5-4nz/mediawiki [11:18:31]