[00:00:16] PROBLEM - cp21 Varnish Backends on cp21 is CRITICAL: 1 backends are down. mw112 [00:00:59] PROBLEM - cp30 Varnish Backends on cp30 is CRITICAL: 1 backends are down. mw112 [00:01:21] RECOVERY - cp31 Varnish Backends on cp31 is OK: All 14 backends are healthy [00:01:28] RECOVERY - mw112 Check Gluster Clients on mw112 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [00:01:39] works now CosmicAlpha [00:01:45] may need to sync config [00:01:47] thanks paladox [00:01:48] dunno tho [00:01:56] RECOVERY - cp20 Varnish Backends on cp20 is OK: All 14 backends are healthy [00:02:14] RECOVERY - cp21 Varnish Backends on cp21 is OK: All 14 backends are healthy [00:02:18] i guess meta should default to the new vector 2022 skin? [00:02:20] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 3.76, 2.72, 1.97 [00:02:30] !log [universalomega@mwtask111] finished deploy of {'pull': 'config', 'config': True, 'servers': 'mw112'} to mw112 - SUCCESS in 1s [00:02:37] RECOVERY - mwtask111 Check Gluster Clients on mwtask111 is OK: PROCS OK: 1 process with args '/usr/sbin/glusterfs' [00:02:38] need to sync l18n as well [00:02:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:02:46] paladox am [00:02:49] thanks [00:02:52] ok [00:02:59] RECOVERY - cp30 Varnish Backends on cp30 is OK: All 14 backends are healthy [00:03:57] [02mw-config] 07AgentIsai opened pull request 03#4761: Post-upgrade sitenotice - 13https://github.com/miraheze/mw-config/pull/4761 [00:03:57] [url] Page not found · GitHub · GitHub | github.com [00:04:22] Ty Agent [00:04:27] CosmicAlpha: ^ [00:04:30] np [00:04:45] lets go [00:04:49] [02mw-config] 07Universal-Omega reviewed pull request 03#4761 commit - 13https://github.com/miraheze/mw-config/pull/4761#discussion_r898571881 [00:04:49] [url] Post-upgrade sitenotice by AgentIsai · Pull Request #4761 · miraheze/mw-config · GitHub | github.com [00:04:50] lets freaking go [00:05:00] miraheze/mw-config - AgentIsai the build passed. [00:05:06] Bongo-Cat: we are mostly out of Read only now [00:05:17] 100% out [00:05:32] PROBLEM - cp31 Stunnel HTTP for mw111 on cp31 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:05:40] PROBLEM - cp30 Stunnel HTTP for mw111 on cp30 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:01] PROBLEM - cp20 Stunnel HTTP for mw111 on cp20 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:07] PROBLEM - mw111 MediaWiki Rendering on mw111 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:13] PROBLEM - cp21 Stunnel HTTP for mw111 on cp21 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:06:17] RECOVERY - mw122 Puppet on mw122 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:06:42] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/extensions/ManageWiki/maintenance/populateWikiSettings.php --wgsetting=wgVectorDefaultSkinVersion --sourcelist=false --remove (START) [00:06:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:07:16] !log [universalomega@mwtask111] finished deploy of {'l10n': True, 'extensionlist': True, 'force': True, 'servers': 'mw121,mw122', 'ignoretime': True} to mw121,mw122 - SUCCESS in 356s [00:07:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:08:13] i still see the old vector on https://meta.miraheze.org/wiki/Miraheze [00:08:13] [url] Meta | meta.miraheze.org [00:08:20] PROBLEM - cp30 Current Load on cp30 is WARNING: WARNING - load average: 1.41, 1.84, 1.82 [00:08:54] paladox: still running scripts for that [00:09:01] ohh [00:09:03] !log [universalomega@mwtask111] finished deploy of {'l10n': True, 'extensionlist': True, 'force': True, 'servers': 'mw112', 'ignoretime': True} to mw112 - SUCCESS in 89s [00:09:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:09:44] RECOVERY - cp31 Stunnel HTTP for mw111 on cp31 is OK: HTTP OK: HTTP/1.1 200 OK - 15887 bytes in 6.856 second response time [00:09:45] [02mw-config] 07AgentIsai synchronize pull request 03#4761: Post-upgrade sitenotice - 13https://github.com/miraheze/mw-config/pull/4761 [00:09:46] [url] Page not found · GitHub · GitHub | github.com [00:09:46] RECOVERY - cp30 Stunnel HTTP for mw111 on cp30 is OK: HTTP OK: HTTP/1.1 200 OK - 15887 bytes in 5.935 second response time [00:10:13] [02mw-config] 07Universal-Omega commented on pull request 03#4761: Post-upgrade sitenotice - 13https://github.com/miraheze/mw-config/pull/4761#issuecomment-1157089945 [00:10:13] [url] Post-upgrade sitenotice by AgentIsai · Pull Request #4761 · miraheze/mw-config · GitHub | github.com [00:10:49] miraheze/mw-config - AgentIsai the build passed. [00:12:20] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.08, 1.51, 1.68 [00:12:23] RECOVERY - mw111 MediaWiki Rendering on mw111 is OK: HTTP OK: HTTP/1.1 200 OK - 25036 bytes in 9.520 second response time [00:14:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [00:14:36] RECOVERY - cp21 Stunnel HTTP for mw111 on cp21 is OK: HTTP OK: HTTP/1.1 200 OK - 15879 bytes in 9.890 second response time [00:16:02] !log sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /home/universalomega/updateDefaultSkin.php [00:16:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:16:21] RECOVERY - cp20 Stunnel HTTP for mw111 on cp20 is OK: HTTP OK: HTTP/1.1 200 OK - 15879 bytes in 1.057 second response time [00:17:50] !log enable puppet on mwtask111 [00:18:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:18:10] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.29, 3.49, 2.55 [00:19:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [00:22:20] PROBLEM - dnd.bellinrattin.it - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query bellinrattin.it. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [00:22:21] [02mw-config] 07Universal-Omega synchronize pull request 03#4761: Post-upgrade sitenotice - 13https://github.com/miraheze/mw-config/pull/4761 [00:22:21] [url] Page not found · GitHub · GitHub | github.com [00:23:28] miraheze/mw-config - Universal-Omega the build passed. [00:23:31] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/4e4392a8c149...bf4793a55eca [00:23:32] [url] Comparing 4e4392a8c149...bf4793a55eca · miraheze/mw-config · GitHub | github.com [00:23:32] [02miraheze/mw-config] 07AgentIsai 03bf4793a - Post-upgrade sitenotice (#4761) [00:23:34] [02mw-config] 07Universal-Omega closed pull request 03#4761: Post-upgrade sitenotice - 13https://github.com/miraheze/mw-config/pull/4761 [00:23:34] [url] Page not found · GitHub · GitHub | github.com [00:24:00] !log [universalomega@mwtask111] starting deploy of {'pull': 'config', 'config': True} to all [00:24:07] RECOVERY - mwtask111 Puppet on mwtask111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:24:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:22] !log [universalomega@mwtask111] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [00:24:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:32] miraheze/mw-config - Universal-Omega the build passed. [00:27:48] Agentt: Want to do a 1.38 upgrade news item on the meta front page as well? [00:28:11] oops extra t [00:28:58] btw i tweeted the post upgrade. notice [00:29:06] [discord] Sure [00:29:44] MacFan4000: I am stepping out now, have bunch of scripts running in a screen on mwtask111, using &&, so they should be ran, but I am stepping out for a few minutes now. [00:30:20] i'll be here intermittently for awhile longer [00:37:32] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.30, 9.79, 8.86 [00:37:43] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 3.89, 4.00, 3.74 [00:39:31] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 8.97, 9.42, 8.83 [00:43:54] [02mw-config] 07AgentIsai opened pull request 03#4762: Clearer wording in sitenotice - 13https://github.com/miraheze/mw-config/pull/4762 [00:43:55] [url] Page not found · GitHub · GitHub | github.com [00:44:57] miraheze/mw-config - AgentIsai the build passed. [00:45:32] [02mw-config] 07Universal-Omega reviewed pull request 03#4762 commit - 13https://github.com/miraheze/mw-config/pull/4762#discussion_r898590213 [00:45:32] [url] Clearer wording in sitenotice by AgentIsai · Pull Request #4762 · miraheze/mw-config · GitHub | github.com [00:46:41] [02mw-config] 07AgentIsai synchronize pull request 03#4762: Clearer wording in sitenotice - 13https://github.com/miraheze/mw-config/pull/4762 [00:46:42] [url] Page not found · GitHub · GitHub | github.com [00:46:45] CosmicAlpha: oops, sorry [00:47:39] miraheze/mw-config - AgentIsai the build passed. [00:47:43] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.24, 3.96, 3.81 [00:48:43] !log [@test101] starting deploy of {'config': True} to all [00:48:44] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [00:48:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:50:55] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.37, 1.82, 1.43 [00:52:17] RECOVERY - dnd.bellinrattin.it - reverse DNS on sslhost is OK: SSL OK - dnd.bellinrattin.it reverse DNS resolves to cp21.miraheze.org - CNAME OK [00:52:38] [02mw-config] 07Universal-Omega closed pull request 03#4762: Clearer wording in sitenotice - 13https://github.com/miraheze/mw-config/pull/4762 [00:52:38] [url] Page not found · GitHub · GitHub | github.com [00:52:39] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/bf4793a55eca...6889fa98744a [00:52:40] [url] Comparing bf4793a55eca...6889fa98744a · miraheze/mw-config · GitHub | github.com [00:52:41] [02miraheze/mw-config] 07AgentIsai 036889fa9 - Clearer wording in sitenotice (#4762) [00:52:51] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 1.04, 1.54, 1.38 [00:53:43] miraheze/mw-config - Universal-Omega the build passed. [00:54:10] !log [@mwtask111] starting deploy of {'config': True} to all [00:54:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:54:22] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 12s [00:54:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:00:42] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/extensions/ManageWiki/maintenance/toggleExtension.php --wiki=obeymewiki tweeki --disable (END - exit=0) [01:00:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:02:39] PROBLEM - gluster111 Puppet on gluster111 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/etc/logrotate.d/puppet] [01:03:53] !log [universalomega@test101] sudo -u www-data php /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php --wiki=smwwikibeta (END - exit=0) [01:03:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:04:25] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (START) [01:04:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:05:55] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/semanticmediawiki.json /srv/mediawiki/w/extensions/SemanticMediaWiki/maintenance/setupStore.php (END - exit=0) [01:05:57] !log [universalomega@mwtask111] starting deploy of {'files': '../mediawiki/w/extensions/SemanticMediaWiki/.smw.json'} to all [01:06:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:06:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:06:07] !log [universalomega@mwtask111] finished deploy of {'files': '../mediawiki/w/extensions/SemanticMediaWiki/.smw.json'} to all - SUCCESS in 9s [01:06:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:07:43] PROBLEM - mwtask111 Current Load on mwtask111 is WARNING: WARNING - load average: 3.22, 3.57, 3.94 [01:15:08] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-up_complete.sql (START) [01:15:17] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-up_complete.sql (END - exit=2) [01:15:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:15:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:15:36] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/socialprofile.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-up_complete.sql (START) [01:15:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:17:43] PROBLEM - mwtask111 Current Load on mwtask111 is CRITICAL: CRITICAL - load average: 4.24, 3.70, 3.76 [01:18:35] PROBLEM - gluster101 Puppet on gluster101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [01:18:40] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 3.30, 1.96, 1.16 [01:20:42] !log [@test101] starting deploy of {'config': True} to all [01:20:43] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [01:20:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:20:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:24:38] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.95, 10.01, 8.64 [01:26:38] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.74, 9.73, 8.72 [01:28:22] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.83, 1.95, 1.62 [01:30:38] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.98, 10.08, 9.06 [01:32:38] PROBLEM - mw101 Current Load on mw101 is CRITICAL: CRITICAL - load average: 12.90, 11.30, 9.65 [01:33:58] PROBLEM - cp31 Current Load on cp31 is CRITICAL: CRITICAL - load average: 2.51, 1.75, 1.46 [01:34:03] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/socialprofile.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-up_complete.sql (END - exit=0) [01:34:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:34:11] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 1.95, 1.70 [01:34:38] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 9.82, 10.39, 9.50 [01:34:43] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/socialprofile.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-stats_overall_rank.sql (START) [01:34:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:07] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/extensions/ManageWiki/maintenance/populateWikiSettings.php --wgsetting=wgVectorDefaultSkinVersion --sourcelist=false --remove (END - exit=0) [01:35:08] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/extensions/ManageWiki/maintenance/populateWikiSettings.php --wgsetting=wgVectorDefaultSkinVersionForExistingAccounts --sourcelist=false --remove (START) [01:35:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:35:53] RECOVERY - cp31 Current Load on cp31 is OK: OK - load average: 1.48, 1.66, 1.47 [01:36:48] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.19, 10.58, 9.59 [01:38:46] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.76, 10.22, 9.55 [01:40:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.85, 1.94, 1.79 [01:40:38] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 8.28, 9.85, 9.65 [01:42:44] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 10.09, 10.10, 9.68 [01:43:04] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 10.24, 9.52, 8.79 [01:45:04] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.11, 8.84, 8.62 [01:45:20] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/GlobalBlocking/sql/mysql/patch-add-gb_by_central_id.sql (END - exit=0) [01:45:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:46:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.19, 1.86, 1.79 [01:46:21] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/extensions/CheckUser/maintenance/fixTrailingSpacesInLogs.php (START) [01:46:35] RECOVERY - gluster101 Puppet on gluster101 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [01:46:38] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.16, 10.62, 10.03 [01:46:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:47:02] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.miraheze.org/d/GtxbP1Xnk [01:47:36] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/CheckUser/archives/patch-cu_changes-actor-comment.sql (START) [01:47:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:48:38] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.23, 10.16, 9.95 [01:49:01] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/universalomega/socialprofile.json /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/extensions/SocialProfile/UserStats/sql/patches/patch-drop-column-stats_overall_rank.sql (END - exit=0) [01:49:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:57] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6889fa98744a...0760160c2ba9 [01:51:58] [url] Comparing 6889fa98744a...0760160c2ba9 · miraheze/mw-config · GitHub | github.com [01:51:59] [02miraheze/mw-config] 07Universal-Omega 030760160 - Remove wb_changes_dispatch [01:53:06] miraheze/mw-config - Universal-Omega the build passed. [01:54:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.63, 2.00, 1.94 [01:55:21] !log [@mwtask111] starting deploy of {'config': True} to all [01:55:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:55:31] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 9s [01:55:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:57:53] !log [universalomega@mwtask111] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.json /srv/mediawiki/w/maintenance/sql.php /home/universalomega/drop_wb_changes_dispatch.sql (START) [01:57:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:58:39] RECOVERY - gluster111 Puppet on gluster111 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [02:03:18] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 11.45, 10.83, 10.24 [02:04:02] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/GlobalBlocking/sql/mysql/patch-globalblocks-gb_anon_only.sql (END - exit=0) [02:04:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.12, 1.79, 1.81 [02:04:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:05:09] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/extensions/GlobalBlocking/maintenance/PopulateCentralId.php --wiki=loginwiki (END - exit=0) [02:05:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:05:24] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.14, 10.90, 10.32 [02:06:10] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-renameuser_queue-timestamp.sql (END - exit=0) [02:06:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:07:23] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.51, 10.71, 10.32 [02:07:28] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-localuser-lu_attached_timestamp.sql (END - exit=0) [02:07:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:08:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.60, 1.71, 1.78 [02:08:05] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-gu_hidden_normalize.sql (END - exit=256) [02:08:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:10:43] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-globaluser-timestamp.sql (END - exit=0) [02:11:02] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 9.05, 9.97, 10.09 [02:11:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:11:45] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-globaluser-gu_enabled.sql (END - exit=0) [02:11:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:12:11] !log [universalomega@mwtask111] sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal /srv/mediawiki/w/extensions/CentralAuth/schema/mysql/db_patches/patch-global_edit_count.sql (END - exit=0) [02:12:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:12:45] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.87, 6.89, 5.87 [02:14:39] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 6.13, 6.59, 5.88 [02:16:30] PROBLEM - mw102 Current Load on mw102 is WARNING: WARNING - load average: 11.75, 10.44, 9.71 [02:18:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.20, 1.90, 1.83 [02:18:28] RECOVERY - mw102 Current Load on mw102 is OK: OK - load average: 8.45, 9.57, 9.48 [02:18:35] PROBLEM - gluster101 Puppet on gluster101 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:21:13] !log [@test101] starting deploy of {'config': True} to all [02:21:14] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [02:21:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:21:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:22:02] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/0760160c2ba9...1f0f2d91eb3f [02:22:03] [url] Comparing 0760160c2ba9...1f0f2d91eb3f · miraheze/mw-config · GitHub | github.com [02:22:04] [02miraheze/mw-config] 07Universal-Omega 031f0f2d9 - Remove $wgVectorSkinMigrationMode [02:23:03] !log [universalomega@mwtask111] starting deploy of {'pull': 'config', 'config': True} to all [02:23:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:23:12] miraheze/mw-config - Universal-Omega the build passed. [02:23:18] !log [universalomega@mwtask111] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 14s [02:23:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:26:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.44, 1.91, 1.95 [02:27:02] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.miraheze.org/d/GtxbP1Xnk [02:28:58] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 7.94, 7.00, 6.34 [02:32:38] PROBLEM - gluster111 Puppet on gluster111 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:33:18] PROBLEM - mw101 Current Load on mw101 is WARNING: WARNING - load average: 10.23, 9.98, 9.62 [02:35:04] PROBLEM - mw111 Current Load on mw111 is CRITICAL: CRITICAL - load average: 12.65, 10.69, 10.44 [02:35:17] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1f0f2d91eb3f...c171f804f4a1 [02:35:18] [url] Comparing 1f0f2d91eb3f...c171f804f4a1 · miraheze/mw-config · GitHub | github.com [02:35:19] [02miraheze/mw-config] 07Universal-Omega 03c171f80 - centralauth-oversight -> centralauth-suppress right [02:36:25] miraheze/mw-config - Universal-Omega the build passed. [02:36:36] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 5.63, 6.69, 6.49 [02:37:02] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 8.54, 10.00, 10.24 [02:37:10] RECOVERY - mw101 Current Load on mw101 is OK: OK - load average: 7.95, 9.17, 9.40 [02:38:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.13, 1.87, 1.84 [02:40:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.94, 1.88, 1.85 [02:41:00] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 7.64, 9.39, 9.99 [02:41:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.66, 3.39, 3.92 [02:41:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.37, 3.71, 4.00 [02:42:20] !log update the default branch of miraheze/mediawiki to REL1_38 [02:42:20] PROBLEM - db101 Current Load on db101 is WARNING: WARNING - load average: 6.85, 6.60, 6.45 [02:42:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:42:51] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-7814752 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/fbd74a73bd3f [02:42:52] [02miraheze/mediawiki] 07dependabot[bot] 03fbd74a7 - Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` [02:42:54] [02mediawiki] 07dependabot[bot] opened pull request 03#7635: Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` - 13https://github.com/miraheze/mediawiki/pull/7635 [02:42:54] [url] Page not found · GitHub · GitHub | github.com [02:42:55] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-7814752 - 13https://github.com/miraheze/mediawiki [02:42:56] [url] Page not found · GitHub · GitHub | github.com [02:43:04] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/Translate-df4fc46 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/ac1d4638a4ad [02:43:05] [02miraheze/mediawiki] 07dependabot[bot] 03ac1d463 - Bump extensions/Translate from `3f1b147` to `df4fc46` [02:43:07] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/Translate-df4fc46 - 13https://github.com/miraheze/mediawiki [02:43:07] [url] Page not found · GitHub · GitHub | github.com [02:43:08] [02mediawiki] 07dependabot[bot] opened pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:43:09] [url] Page not found · GitHub · GitHub | github.com [02:43:19] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/skins/Citizen-9675aab [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/58f67f963a81 [02:43:21] [02miraheze/mediawiki] 07dependabot[bot] 0358f67f9 - Bump skins/Citizen from `eb0147e` to `9675aab` [02:43:22] [02mediawiki] 07dependabot[bot] opened pull request 03#7637: Bump skins/Citizen from `eb0147e` to `9675aab` - 13https://github.com/miraheze/mediawiki/pull/7637 [02:43:23] [url] Page not found · GitHub · GitHub | github.com [02:43:24] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/skins/Citizen-9675aab - 13https://github.com/miraheze/mediawiki [02:43:24] ... [02:43:25] [02mediawiki] 07github-actions[bot] labeled pull request 03#7635: Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` - 13https://github.com/miraheze/mediawiki/pull/7635 [02:43:26] ... [02:43:27] [02miraheze/mediawiki] 07dependabot[bot] pushed 031 commit to 03dependabot/submodules/REL1_38/extensions/PageForms-a9fbfc3 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/d9e4a7d8299c [02:43:28] [02miraheze/mediawiki] 07dependabot[bot] 03d9e4a7d - Bump extensions/PageForms from `7847a4d` to `a9fbfc3` [02:43:30] [02mediawiki] 07dependabot[bot] opened pull request 03#7638: Bump extensions/PageForms from `7847a4d` to `a9fbfc3` - 13https://github.com/miraheze/mediawiki/pull/7638 [02:43:30] ... [02:43:31] [02mediawiki] 07dependabot[bot] created branch 03dependabot/submodules/REL1_38/extensions/PageForms-a9fbfc3 - 13https://github.com/miraheze/mediawiki [02:43:41] [02mediawiki] 07github-actions[bot] labeled pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:43:42] [02mediawiki] 07github-actions[bot] labeled pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:43:44] [02mediawiki] 07github-actions[bot] labeled pull request 03#7637: Bump skins/Citizen from `eb0147e` to `9675aab` - 13https://github.com/miraheze/mediawiki/pull/7637 [02:43:52] [02mediawiki] 07github-actions[bot] labeled pull request 03#7638: Bump extensions/PageForms from `7847a4d` to `a9fbfc3` - 13https://github.com/miraheze/mediawiki/pull/7638 [02:44:09] [02mediawiki] 07Universal-Omega labeled pull request 03#7635: Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` - 13https://github.com/miraheze/mediawiki/pull/7635 [02:44:11] [02mediawiki] 07Universal-Omega labeled pull request 03#7637: Bump skins/Citizen from `eb0147e` to `9675aab` - 13https://github.com/miraheze/mediawiki/pull/7637 [02:44:14] RECOVERY - db101 Current Load on db101 is OK: OK - load average: 4.51, 5.82, 6.18 [02:44:24] [02mediawiki] 07github-actions[bot] closed pull request 03#7637: Bump skins/Citizen from `eb0147e` to `9675aab` - 13https://github.com/miraheze/mediawiki/pull/7637 [02:44:26] [02miraheze/mediawiki] 07github-actions[bot] pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/547ccd6a7dde...e4c6d9c359e7 [02:44:27] [url] Comparing 547ccd6a7dde...e4c6d9c359e7 · miraheze/mediawiki · GitHub | github.com [02:44:27] [02miraheze/mediawiki] 07dependabot[bot] 03e4c6d9c - Bump skins/Citizen from `eb0147e` to `9675aab` (#7637) [02:44:29] [02miraheze/mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/skins/Citizen-9675aab [02:44:30] [02mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/skins/Citizen-9675aab - 13https://github.com/miraheze/mediawiki [02:44:31] [url] Page not found · GitHub · GitHub | github.com [02:44:32] [02mediawiki] 07github-actions[bot] unlabeled pull request 03#7637: Bump skins/Citizen from `eb0147e` to `9675aab` - 13https://github.com/miraheze/mediawiki/pull/7637 [02:44:32] [url] Page not found · GitHub · GitHub | github.com [02:44:33] [02mediawiki] 07github-actions[bot] closed pull request 03#7635: Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` - 13https://github.com/miraheze/mediawiki/pull/7635 [02:44:34] [url] Page not found · GitHub · GitHub | github.com [02:44:35] [02miraheze/mediawiki] 07github-actions[bot] pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/e4c6d9c359e7...d96299c36dd3 [02:44:36] [url] Comparing e4c6d9c359e7...d96299c36dd3 · miraheze/mediawiki · GitHub | github.com [02:44:36] [02miraheze/mediawiki] 07dependabot[bot] 03d96299c - Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` (#7635) [02:44:38] [02mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-7814752 - 13https://github.com/miraheze/mediawiki [02:44:38] [url] Page not found · GitHub · GitHub | github.com [02:44:39] [02miraheze/mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/UniversalLanguageSelector-7814752 [02:44:41] [02mediawiki] 07github-actions[bot] unlabeled pull request 03#7635: Bump extensions/UniversalLanguageSelector from `f306092` to `7814752` - 13https://github.com/miraheze/mediawiki/pull/7635 [02:44:41] [url] Page not found · GitHub · GitHub | github.com [02:44:50] [02mediawiki] 07Universal-Omega labeled pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:44:51] ... [02:44:51] [02mediawiki] 07Universal-Omega labeled pull request 03#7638: Bump extensions/PageForms from `7847a4d` to `a9fbfc3` - 13https://github.com/miraheze/mediawiki/pull/7638 [02:44:52] ... [02:45:05] [02mediawiki] 07github-actions[bot] closed pull request 03#7638: Bump extensions/PageForms from `7847a4d` to `a9fbfc3` - 13https://github.com/miraheze/mediawiki/pull/7638 [02:45:05] ... [02:45:06] [02miraheze/mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/PageForms-a9fbfc3 [02:45:08] [02miraheze/mediawiki] 07github-actions[bot] pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/d96299c36dd3...20723c183d51 [02:45:09] [url] Comparing d96299c36dd3...20723c183d51 · miraheze/mediawiki · GitHub | github.com [02:45:09] [02miraheze/mediawiki] 07dependabot[bot] 0320723c1 - Bump extensions/PageForms from `7847a4d` to `a9fbfc3` (#7638) [02:45:11] [02mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/PageForms-a9fbfc3 - 13https://github.com/miraheze/mediawiki [02:45:11] [url] Page not found · GitHub · GitHub | github.com [02:45:12] [02mediawiki] 07github-actions[bot] unlabeled pull request 03#7638: Bump extensions/PageForms from `7847a4d` to `a9fbfc3` - 13https://github.com/miraheze/mediawiki/pull/7638 [02:45:13] [url] Page not found · GitHub · GitHub | github.com [02:45:14] [02mediawiki] 07github-actions[bot] closed pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:45:14] [url] Page not found · GitHub · GitHub | github.com [02:45:15] [02miraheze/mediawiki] 07github-actions[bot] pushed 031 commit to 03REL1_38 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/20723c183d51...7a3851567f63 [02:45:16] [url] Comparing 20723c183d51...7a3851567f63 · miraheze/mediawiki · GitHub | github.com [02:45:17] [02miraheze/mediawiki] 07dependabot[bot] 037a38515 - Bump extensions/Translate from `3f1b147` to `df4fc46` (#7636) [02:45:18] [02miraheze/mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Translate-df4fc46 [02:45:20] [02mediawiki] 07github-actions[bot] deleted branch 03dependabot/submodules/REL1_38/extensions/Translate-df4fc46 - 13https://github.com/miraheze/mediawiki [02:45:20] [url] Page not found · GitHub · GitHub | github.com [02:45:21] [02mediawiki] 07github-actions[bot] unlabeled pull request 03#7636: Bump extensions/Translate from `3f1b147` to `df4fc46` - 13https://github.com/miraheze/mediawiki/pull/7636 [02:45:22] [url] Page not found · GitHub · GitHub | github.com [02:46:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 1.75, 2.96, 3.85 [02:46:40] !log [universalomega@mwtask111] starting deploy of {'pull': 'config', 'config': True} to all [02:46:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:46:53] !log [universalomega@mwtask111] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 13s [02:47:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:47:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.46, 3.64, 3.84 [02:48:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.23, 1.83, 1.79 [02:49:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 2.52, 3.13, 3.62 [02:50:52] !log [@test101] starting deploy of {'config': True} to all [02:50:53] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 0s [02:50:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:51:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:51:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.91, 3.43, 3.65 [02:52:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.73, 3.78, 3.88 [02:53:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.52, 3.37, 3.60 [02:54:01] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c171f804f4a1...dd85fb9f3b36 [02:54:01] [url] Comparing c171f804f4a1...dd85fb9f3b36 · miraheze/mw-config · GitHub | github.com [02:54:02] [02miraheze/mw-config] 07Universal-Omega 03dd85fb9 - Re-add `$wgVectorSkinMigrationMode` [02:54:20] !log [@mwtask111] starting deploy of {'config': True} to all [02:54:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:54:32] !log [@mwtask111] finished deploy of {'config': True} to all - SUCCESS in 12s [02:54:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:55:05] miraheze/mw-config - Universal-Omega the build passed. [02:55:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.85, 3.69, 3.62 [02:56:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.23, 1.81, 1.88 [02:56:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.93, 3.83, 3.86 [02:57:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.40, 3.60, 3.60 [02:57:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.50, 3.78, 3.70 [02:59:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.13, 3.74, 3.65 [02:59:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.72, 3.76, 3.71 [03:01:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.37, 3.58, 3.60 [03:01:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.75, 4.16, 3.86 [03:02:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.06, 1.84, 1.87 [03:04:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 6.99, 4.67, 4.12 [03:05:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.11, 3.69, 3.62 [03:07:35] PROBLEM - gluster101 Current Load on gluster101 is WARNING: WARNING - load average: 3.88, 3.96, 3.86 [03:09:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.16, 3.58, 3.61 [03:11:26] PROBLEM - gluster121 Current Load on gluster121 is CRITICAL: CRITICAL - load average: 4.57, 3.76, 3.66 [03:12:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.76, 1.94, 1.95 [03:13:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 3.80, 3.72, 3.65 [03:14:21] PROBLEM - gluster111 Current Load on gluster111 is WARNING: WARNING - load average: 3.27, 3.75, 3.97 [03:14:35] RECOVERY - gluster101 Puppet on gluster101 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [03:16:03] PROBLEM - test101 Current Load on test101 is CRITICAL: CRITICAL - load average: 2.05, 1.93, 1.93 [03:17:35] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.54, 2.98, 3.35 [03:19:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.47, 3.08, 3.39 [03:20:21] PROBLEM - gluster111 Current Load on gluster111 is CRITICAL: CRITICAL - load average: 4.27, 3.71, 3.82 [03:20:36] !log [@test101] starting deploy of {'config': True} to all [03:20:37] !log [@test101] finished deploy of {'config': True} to all - SUCCESS in 1s [03:20:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:20:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:21:11] PROBLEM - cp30 Current Load on cp30 is CRITICAL: CRITICAL - load average: 2.28, 1.88, 1.43 [03:23:35] PROBLEM - gluster101 Current Load on gluster101 is CRITICAL: CRITICAL - load average: 4.68, 3.69, 3.49 [03:25:03] RECOVERY - cp30 Current Load on cp30 is OK: OK - load average: 0.87, 1.50, 1.38 [03:25:26] PROBLEM - gluster121 Current Load on gluster121 is WARNING: WARNING - load average: 2.74, 3.52, 3.52 [03:25:35] RECOVERY - gluster101 Current Load on gluster101 is OK: OK - load average: 2.54, 3.22, 3.34 [03:26:35] PROBLEM - mw111 Current Load on mw111 is WARNING: WARNING - load average: 10.29, 9.26, 8.65 [03:27:26] RECOVERY - gluster121 Current Load on gluster121 is OK: OK - load average: 2.34, 3.12, 3.37 [03:28:39] RECOVERY - gluster111 Puppet on gluster111 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:30:22] PROBLEM - gluster121 Puppet on gluster121 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [03:30:35] RECOVERY - mw111 Current Load on mw111 is OK: OK - load average: 9.14, 9.68, 8.99 [03:32:03] PROBLEM - test101 Current Load on test101 is WARNING: WARNING - load average: 1.69, 1.81, 1.99 [03:32:14] [discord] I've done something serious on the Evelution Skin Repo. I messed up the clone on my PC and as a result, after I merged the changes to the repo online, I've lost all commits and I'm stuck with two. Will this cause issues with bumping Evelution to the latest version or not in Miraheze or not? [03:32:14] [discord] Link: https://github.com/AWikia/SkinEvelution [03:32:15] [url] GitHub - AWikia/SkinEvelution: A MediaWiki Skin based on the discontinued MpistoSkin2 from Q.Qore | github.com [03:32:16] [discord] So now there's no way to revert the branch back to its proper state with a lot of commits [03:32:35] [discord]