[00:00:14] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 9.87, 9.38, 8.53 [00:04:16] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 16.34, 9.69, 4.50 [00:08:59] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.97, 10.10, 9.20 [00:10:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.90, 3.52, 3.16 [00:10:55] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.20, 10.86, 9.59 [00:12:51] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.86, 11.21, 9.87 [00:14:16] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 4.61, 7.74, 6.68 [00:16:07] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.67, 3.20, 2.28 [00:16:16] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 3.02, 6.06, 6.19 [00:16:50] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.91, 11.38, 10.19 [00:18:04] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.25, 2.86, 2.26 [00:18:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.35, 3.37, 3.25 [00:18:50] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 9.45, 10.34, 9.93 [00:26:50] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.53, 10.16, 10.16 [00:45:15] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.67, 3.24, 2.82 [00:47:14] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 3.34, 3.34, 2.91 [00:49:02] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.23, 3.49, 2.85 [00:51:03] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.60, 3.15, 2.81 [00:52:19] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.57, 3.47, 3.38 [00:56:09] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.20, 3.86, 3.55 [00:58:04] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.96, 3.50, 3.46 [00:59:59] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.80, 3.33, 3.40 [01:03:51] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.90, 3.88, 3.59 [01:05:46] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.77, 3.44, 3.46 [01:07:41] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.54, 3.20, 3.37 [01:12:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [01:16:51] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.48, 3.46, 3.04 [01:18:46] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.04, 2.60, 2.78 [01:42:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [01:43:18] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.37, 3.45, 3.10 [01:45:15] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.32, 3.10, 3.02 [01:53:23] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.45, 3.51, 3.13 [01:58:50] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.41, 9.62, 8.73 [01:59:08] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.06, 3.55, 3.31 [02:00:50] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 7.62, 9.03, 8.64 [02:01:03] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.50, 3.26, 3.23 [02:34:50] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.30, 3.80, 3.22 [02:36:45] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.65, 3.55, 3.18 [02:38:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.56, 3.82, 3.32 [02:40:44] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.80, 3.08, 3.11 [02:41:12] !log [void@mwtask141] starting deploy of {'config': True} to all [02:41:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:41:21] !log [void@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 9s [02:41:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:43:56] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.93, 6.44, 5.95 [02:45:52] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.02, 6.18, 5.91 [02:45:59] !log [void@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [02:46:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:46:05] !log [void@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 6s [02:46:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:47:17] !log [void@mwtask141] starting deploy of {'config': True} to all [02:47:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:47:24] !log [void@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [02:47:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:48:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.42, 3.46, 3.26 [02:52:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.44, 3.18, 3.21 [03:00:47] PROBLEM - mon141 Backups Grafana on mon141 is CRITICAL: FILE_AGE CRITICAL: /var/log/grafana-backup.log is 1209640 seconds old and 785 bytes [03:15:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.00, 7.00, 6.25 [03:17:15] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.63, 3.13, 2.96 [03:17:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.62, 6.37, 6.10 [03:19:10] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.51, 2.91, 2.91 [04:06:39] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.407 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [04:28:30] PROBLEM - mail121 Current Load on mail121 is CRITICAL: LOAD CRITICAL - total load average: 5.42, 3.32, 1.42 [04:32:45] PROBLEM - mail121 PowerDNS Recursor on mail121 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:33:00] PROBLEM - mail121 webmail.miraheze.org HTTPS on mail121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:34:46] RECOVERY - mail121 PowerDNS Recursor on mail121 is OK: DNS OK: 2.090 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [04:34:54] RECOVERY - mail121 webmail.miraheze.org HTTPS on mail121 is OK: HTTP OK: HTTP/1.1 200 OK - 6169 bytes in 0.175 second response time [04:35:37] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp24.miraheze.org - CNAME OK [04:52:30] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 0.25, 0.45, 1.80 [04:54:30] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.45, 0.49, 1.65 [05:27:51] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=battlecatswiki --no-updates --username-prefix=wikia:battle-cats /home/macfan/ImportDump/battlecatswiki-20231102205152.xml (END - exit=0) [05:27:52] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=battlecatswiki (START) [05:27:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:28:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:42:57] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:05:37] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [07:12:59] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [07:34:34] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp25.miraheze.org - CNAME OK [07:49:03] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.04, 2.79, 1.46 [07:53:02] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.85, 3.56, 2.14 [07:55:02] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.72, 3.35, 2.24 [09:56:07] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is CRITICAL: connect to address pyramidgames.wiki and port 443: Network is unreachableHTTP CRITICAL - Unable to open TCP socket [10:26:07] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'pyramidgames.wiki' expires in 14 day(s) (Wed 22 Nov 2023 14:28:59 GMT +0000). [10:42:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [11:06:41] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.403 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [11:12:58] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [11:35:33] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp24.miraheze.org - CNAME OK [12:37:44] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.403 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [12:46:41] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3178a6a3119d...448b11c269b3 [12:46:43] [02miraheze/mw-config] 07paladox 03448b11c - Don't use compression in $wgPoolCounterConf [12:51:04] miraheze/mw-config - paladox the build passed. [13:04:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.02, 3.39, 2.94 [13:06:40] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp24.miraheze.org - CNAME OK [13:06:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.60, 3.36, 2.98 [13:08:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.80, 3.13, 2.94 [13:15:44] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.65, 11.66, 10.14 [13:17:39] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 6.79, 10.13, 9.78 [13:31:15] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.19, 10.71, 10.25 [13:33:11] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.59, 9.88, 9.99 [13:52:50] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.82, 10.74, 10.28 [13:54:50] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.98, 10.13, 10.12 [14:15:17] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.32, 10.21, 9.79 [14:17:13] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 7.47, 9.43, 9.57 [14:22:42] paladox: when you have a chance lsbot and ircrcbot need a restart [14:24:28] done [14:29:10] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=themarksidewiki --no-updates --username-prefix=wikia:themarkside /home/macfan/ImportDump/themarksidewiki-20231102222659.xml (START) [14:29:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:34:14] [02miraheze/ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a4d7d0320d07...ec0d1b745095 [14:34:17] [02miraheze/ssl] 07MacFan4000 03ec0d1b7 - Update redirects.yaml [14:34:23] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=themarksidewiki --no-updates --username-prefix=wikia:themarkside /home/macfan/ImportDump/themarksidewiki-20231102222659.xml (END - exit=0) [14:34:24] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=themarksidewiki (START) [14:34:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:34:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:37:02] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.77, 10.10, 9.30 [14:38:19] PROBLEM - mw134 Current Load on mw134 is WARNING: WARNING - load average: 10.88, 9.73, 7.93 [14:39:02] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 10.06, 9.68, 9.23 [14:39:27] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=themarksidewiki (END - exit=0) [14:39:28] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=themarksidewiki --active --update (END - exit=0) [14:39:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:39:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:40:18] RECOVERY - mw134 Current Load on mw134 is OK: OK - load average: 9.77, 9.77, 8.17 [14:43:44] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.09, 10.38, 9.39 [14:45:46] PROBLEM - mw133 Current Load on mw133 is WARNING: WARNING - load average: 11.45, 10.67, 9.33 [14:47:36] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.75, 9.99, 9.50 [14:47:43] RECOVERY - mw133 Current Load on mw133 is OK: OK - load average: 8.06, 9.79, 9.17 [14:47:51] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 10.14, 10.83, 10.06 [14:48:13] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=tbotwiki --no-updates --username-prefix=wikia:tbot /home/macfan/ImportDump/tbotwiki-20231102225038.xml (START) [14:48:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:48:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.64, 3.33, 3.10 [14:49:06] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=tbotwiki --no-updates --username-prefix=wikia:tbot /home/macfan/ImportDump/tbotwiki-20231102225038.xml (END - exit=0) [14:49:07] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tbotwiki (START) [14:49:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:49:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:49:46] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.48, 10.11, 9.89 [14:49:54] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=tbotwiki (END - exit=0) [14:49:55] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=tbotwiki --active --update (END - exit=0) [14:50:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:50:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:50:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.95, 3.17, 3.07 [14:51:26] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=sourcedblazonswiki --no-updates --username-prefix=wikia:sourcedblazons /home/macfan/ImportDump/sourcedblazonswiki-20231103152712.xml (START) [14:51:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:53:41] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 9.38, 10.23, 9.75 [14:55:37] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 7.73, 9.29, 9.46 [14:56:41] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.77, 3.72, 3.30 [14:58:40] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.15, 3.43, 3.24 [15:00:39] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.29, 3.04, 3.12 [15:05:28] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=sourcedblazonswiki --no-updates --username-prefix=wikia:sourcedblazons /home/macfan/ImportDump/sourcedblazonswiki-20231103152712.xml (END - exit=0) [15:05:29] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=sourcedblazonswiki (START) [15:05:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:05:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:07:09] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 14.31, 10.79, 9.77 [15:09:56] PROBLEM - test2.specialchat.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'test2.specialchat.org' expires in 15 day(s) (Fri 24 Nov 2023 14:51:14 GMT +0000). [15:10:12] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ec0d1b745095...1fccc6c08d45 [15:10:13] [02miraheze/ssl] 07MirahezeSSLBot 031fccc6c - Bot: Update SSL cert for test2.specialchat.org [15:11:08] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 10.14, 10.62, 9.92 [15:11:09] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.70, 10.76, 9.38 [15:11:33] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.87, 3.74, 3.35 [15:13:05] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.31, 10.01, 9.28 [15:13:08] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 9.42, 10.05, 9.80 [15:13:32] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.74, 3.32, 3.25 [15:16:54] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=sourcedblazonswiki (END - exit=0) [15:16:55] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=sourcedblazonswiki --active --update (END - exit=0) [15:17:00] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 16.03, 11.72, 10.02 [15:17:00] RECOVERY - jobchron121 Puppet on jobchron121 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [15:17:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:17:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:18:56] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.62, 11.75, 10.25 [15:19:06] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.13, 10.15, 9.83 [15:21:06] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.16, 9.51, 9.65 [15:24:45] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 7.77, 9.73, 9.84 [15:33:08] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 11.86, 7.89, 6.54 [15:34:20] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.03, 4.00, 3.37 [15:34:28] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.13, 10.20, 9.79 [15:35:06] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.14, 7.66, 6.64 [15:36:19] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.60, 3.47, 3.25 [15:36:24] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.04, 10.33, 9.90 [15:38:18] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.94, 3.10, 3.15 [15:38:20] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 9.26, 9.94, 9.80 [15:38:30] RECOVERY - test2.specialchat.org - LetsEncrypt on sslhost is OK: OK - Certificate 'test2.specialchat.org' will expire on Tue 06 Feb 2024 14:10:04 GMT +0000. [15:39:01] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.93, 6.76, 6.50 [15:42:15] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 13.67, 11.43, 10.44 [15:46:00] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.57, 11.78, 10.23 [15:48:00] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 6.94, 10.09, 9.81 [15:48:08] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.08, 11.95, 11.07 [15:54:08] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 7.83, 9.35, 10.20 [15:54:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.31, 6.79, 6.64 [15:56:35] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.67, 6.32, 6.49 [16:18:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.96, 6.88, 6.67 [16:20:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.52, 7.31, 6.84 [16:20:55] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 8.83, 10.45, 10.04 [16:22:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.04, 7.18, 6.85 [16:22:52] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 9.36, 10.04, 9.94 [16:24:15] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=goblinsdendndwiki --no-updates --username-prefix=wikia:goblins-den-dnd /home/macfan/ImportDump/goblinsdendndwiki-20231104020816.xml (START) [16:24:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:26:04] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=goblinsdendndwiki --no-updates --username-prefix=wikia:goblins-den-dnd /home/macfan/ImportDump/goblinsdendndwiki-20231104020816.xml (END - exit=0) [16:26:05] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=goblinsdendndwiki (START) [16:26:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:26:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:26:46] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.40, 10.97, 10.30 [16:27:57] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=goblinsdendndwiki (END - exit=0) [16:27:58] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=goblinsdendndwiki --active --update (END - exit=0) [16:28:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:28:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:28:50] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.96, 3.35, 3.07 [16:30:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.74, 6.27, 6.71 [16:30:39] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.31, 11.22, 10.58 [16:31:48] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 7.89, 5.19, 3.91 [16:31:49] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.19, 10.37, 9.64 [16:32:35] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 13.13, 12.14, 11.01 [16:32:48] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.48, 3.85, 3.29 [16:33:48] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 6.24, 5.17, 4.04 [16:33:49] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 12.69, 11.37, 10.10 [16:33:55] PROBLEM - mw134 Current Load on mw134 is WARNING: WARNING - load average: 10.48, 9.51, 8.58 [16:34:31] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 10.58, 11.47, 10.90 [16:35:48] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 9.38, 11.19, 10.22 [16:36:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.86, 6.76, 6.78 [16:36:46] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.65, 3.94, 3.49 [16:37:55] RECOVERY - mw134 Current Load on mw134 is OK: OK - load average: 8.50, 9.45, 8.82 [16:40:20] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 15.63, 12.30, 11.31 [16:40:46] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.66, 3.39, 3.37 [16:41:05] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=n01rfixationwiki --no-updates --username-prefix=wikia:n01r-fiction /home/macfan/ImportDump/n01rfixationwiki-20231104042555.xml (START) [16:41:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:42:55] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=n01rfixationwiki --no-updates --username-prefix=wikia:n01r-fiction /home/macfan/ImportDump/n01rfixationwiki-20231104042555.xml (END - exit=0) [16:42:56] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=n01rfixationwiki (START) [16:43:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:43:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:43:47] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 17.08, 12.92, 11.17 [16:44:25] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=n01rfixationwiki (END - exit=0) [16:44:26] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=n01rfixationwiki --active --update (END - exit=0) [16:44:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:45:17] PROBLEM - mw133 Current Load on mw133 is CRITICAL: CRITICAL - load average: 14.73, 11.85, 10.01 [16:46:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.79, 7.30, 7.00 [16:46:42] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.31, 3.63, 3.46 [16:47:17] PROBLEM - mw133 Current Load on mw133 is WARNING: WARNING - load average: 11.89, 11.66, 10.17 [16:48:41] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.62, 3.51, 3.43 [16:50:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.08, 7.34, 7.07 [16:50:52] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.24, 7.08, 6.41 [16:51:17] PROBLEM - mw133 Current Load on mw133 is CRITICAL: CRITICAL - load average: 14.65, 13.16, 11.08 [16:52:48] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.11, 6.18, 6.16 [16:55:07] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.59, 3.18, 2.90 [16:57:06] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.07, 2.69, 2.75 [16:57:17] PROBLEM - mw133 Current Load on mw133 is WARNING: WARNING - load average: 9.58, 11.61, 11.14 [16:58:08] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 8.75, 11.43, 11.93 [17:00:36] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.32, 3.10, 3.32 [17:01:43] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 8.24, 10.48, 11.75 [17:04:07] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.406 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [17:04:08] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 13.35, 11.22, 11.52 [17:05:17] RECOVERY - mw133 Current Load on mw133 is OK: OK - load average: 9.47, 9.13, 10.05 [17:05:42] PROBLEM - mw132 Current Load on mw132 is CRITICAL: CRITICAL - load average: 13.56, 11.68, 11.90 [17:07:42] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.71, 11.37, 11.74 [17:10:31] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 4.00, 3.56, 3.41 [17:14:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.36, 6.46, 6.76 [17:14:08] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 9.58, 11.48, 11.81 [17:20:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.46, 7.47, 7.06 [17:22:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.38, 7.63, 7.18 [17:22:25] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.30, 3.09, 3.31 [17:23:39] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.59, 9.04, 10.06 [17:23:56] PROBLEM - wiki.orvyn.ca - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.orvyn.ca' expires in 15 day(s) (Fri 24 Nov 2023 17:01:57 GMT +0000). [17:24:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.98, 7.92, 7.32 [17:24:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1fccc6c08d45...a7374d19d6ed [17:24:11] [02miraheze/ssl] 07MirahezeSSLBot 03a7374d1 - Bot: Update SSL cert for wiki.orvyn.ca [17:26:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.10, 7.48, 7.22 [17:30:30] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.10, 7.36, 6.79 [17:32:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.14, 7.20, 7.13 [17:32:26] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.29, 7.34, 6.85 [17:34:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.95, 6.73, 6.97 [17:36:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.58, 7.26, 7.13 [17:36:08] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.92, 9.25, 9.99 [17:38:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.89, 6.88, 7.00 [17:38:13] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.44, 6.71, 6.77 [17:42:08] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.43, 10.45, 10.25 [17:44:08] PROBLEM - mw131 Current Load on mw131 is CRITICAL: CRITICAL - load average: 12.07, 10.96, 10.46 [17:48:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.22, 6.36, 6.77 [17:48:08] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 6.75, 9.69, 10.15 [17:52:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.99, 6.70, 6.82 [17:53:52] RECOVERY - wiki.orvyn.ca - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.orvyn.ca' will expire on Tue 06 Feb 2024 16:24:03 GMT +0000. [17:54:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.96, 6.45, 6.72 [17:57:05] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.26, 3.62, 3.29 [17:59:05] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.66, 3.38, 3.25 [18:01:52] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp25.miraheze.org - CNAME OK [18:14:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.70, 6.91, 6.74 [18:16:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.48, 6.72, 6.69 [18:20:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.17, 7.17, 6.84 [18:22:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.15, 6.90, 6.79 [18:23:13] PROBLEM - decyclopedia.tk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'decyclopedia.tk' expires in 15 day(s) (Fri 24 Nov 2023 18:08:29 GMT +0000). [18:24:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.10, 6.58, 6.69 [18:27:29] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.50, 7.06, 6.35 [18:28:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.91, 6.80, 6.82 [18:29:25] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.05, 6.64, 6.28 [18:36:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.79, 6.57, 6.77 [18:51:39] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.45, 6.69, 6.26 [18:53:36] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.81, 6.21, 6.13 [18:54:39] PROBLEM - comprehensibleinputwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'comprehensibleinputwiki.org' expires in 15 day(s) (Fri 24 Nov 2023 18:27:01 GMT +0000). [18:54:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a7374d19d6ed...99bb95142631 [18:54:57] [02miraheze/ssl] 07MirahezeSSLBot 0399bb951 - Bot: Update SSL cert for comprehensibleinputwiki.org [19:08:32] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.20, 3.56, 3.18 [19:09:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.00, 6.77, 6.52 [19:10:31] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.76, 3.27, 3.12 [19:15:23] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.94, 6.35, 6.53 [19:23:20] RECOVERY - comprehensibleinputwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'comprehensibleinputwiki.org' will expire on Tue 06 Feb 2024 17:54:46 GMT +0000. [19:31:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.84, 7.08, 6.62 [19:33:01] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.58, 6.73, 6.54 [19:51:24] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=ramdaramwiki --no-updates --username-prefix=wikia:ramdaram /home/macfan/ImportDump/ramdaramwiki-20231104183151.xml (START) [19:51:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:59:37] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=ramdaramwiki --no-updates --username-prefix=wikia:ramdaram /home/macfan/ImportDump/ramdaramwiki-20231104183151.xml (END - exit=0) [19:59:38] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ramdaramwiki (START) [19:59:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:59:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:01:12] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.47, 6.62, 6.07 [20:02:03] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.17, 3.93, 3.29 [20:03:07] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.17, 6.40, 6.05 [20:04:02] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.33, 3.76, 3.32 [20:04:06] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [20:04:56] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=ramdaramwiki (END - exit=0) [20:04:57] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=ramdaramwiki --active --update (END - exit=0) [20:05:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:05:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:06:01] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.25, 3.17, 3.15 [20:14:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.13, 6.83, 6.54 [20:16:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.73, 6.44, 6.43 [20:22:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.37, 8.01, 7.05 [20:22:16] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=epicsfanonlorewiki --no-updates --username-prefix=wikia:epics-fanon-lore /home/macfan/ImportDump/epicsfanonlorewiki-20231104211214.xml (START) [20:22:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:23:29] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/importDump.php --wiki=epicsfanonlorewiki --no-updates --username-prefix=wikia:epics-fanon-lore /home/macfan/ImportDump/epicsfanonlorewiki-20231104211214.xml (END - exit=0) [20:23:30] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=epicsfanonlorewiki (START) [20:23:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:23:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:23:50] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.95, 3.52, 3.21 [20:24:04] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.09, 7.59, 7.01 [20:24:44] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/rebuildall.php --wiki=epicsfanonlorewiki (END - exit=0) [20:24:45] !log [macfan@mwtask141] sudo -u www-data php /srv/mediawiki/w/maintenance/run.php /srv/mediawiki/w/maintenance/initSiteStats.php --wiki=epicsfanonlorewiki --active --update (END - exit=0) [20:24:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:24:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:25:49] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.09, 3.27, 3.15 [20:30:04] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.15, 6.64, 6.74 [20:39:41] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.88, 3.08, 2.90 [20:41:40] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.09, 3.02, 2.90 [20:56:04] PROBLEM - cp34 Varnish Backends on cp34 is CRITICAL: 3 backends are down. mw141 mw142 mw134 [20:56:17] PROBLEM - ping6 on cp24 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:56:18] PROBLEM - cp24 Varnish Backends on cp24 is CRITICAL: 3 backends are down. mw131 mw142 mw133 [20:56:23] PROBLEM - cp35 Varnish Backends on cp35 is CRITICAL: 2 backends are down. mw131 mw132 [20:56:50] PROBLEM - cp25 Varnish Backends on cp25 is CRITICAL: 5 backends are down. mw131 mw132 mw141 mw142 mw143 [20:56:53] PROBLEM - Host cp24 is DOWN: PING CRITICAL - Packet loss = 100% [20:58:35] PROBLEM - cp34 Puppet on cp34 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:01:21] RECOVERY - Host cp24 is UP: PING OK - Packet loss = 0%, RTA = 17.61 ms [21:01:35] PROBLEM - ping6 on cp35 is CRITICAL: PING CRITICAL - Packet loss = 100% [21:02:35] RECOVERY - ping6 on cp24 is OK: PING OK - Packet loss = 0%, RTA = 16.38 ms [21:03:55] PROBLEM - cp25 HTTPS on cp25 is CRITICAL: HTTP CRITICAL: HTTP/2 503 - 3249 bytes in 0.070 second response time [21:04:03] PROBLEM - ping6 on cp25 is CRITICAL: PING CRITICAL - Packet loss = 100% [21:05:37] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.48, 6.84, 6.45 [21:05:44] RECOVERY - ping6 on cp35 is OK: PING OK - Packet loss = 0%, RTA = 89.31 ms [21:05:52] hey, I was just creating a bunch of pages (manually) and I got a bad gateway from nginx, then I got a 503 backend fetch failed [21:05:56] RECOVERY - cp25 HTTPS on cp25 is OK: HTTP OK: HTTP/2 301 - 3556 bytes in 1.114 second response time [21:06:04] did I overload the server or smth? [21:06:14] PROBLEM - Host cp25 is DOWN: PING CRITICAL - Packet loss = 100% [21:06:16] nvm wrong irc channel [21:10:16] PROBLEM - cp24 Puppet on cp24 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:11:30] PROBLEM - cp35 Puppet on cp35 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [21:11:30] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.22, 7.57, 6.87 [21:11:44] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.27, 7.90, 6.79 [21:13:39] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.44, 7.27, 6.69 [21:15:35] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 10.61, 8.09, 7.02 [21:15:38] RECOVERY - Host cp25 is UP: PING OK - Packet loss = 0%, RTA = 17.51 ms [21:16:01] RECOVERY - cp24 Varnish Backends on cp24 is OK: All 15 backends are healthy [21:16:03] RECOVERY - cp34 Varnish Backends on cp34 is OK: All 15 backends are healthy [21:16:27] RECOVERY - cp35 Varnish Backends on cp35 is OK: All 15 backends are healthy [21:16:43] RECOVERY - ping6 on cp25 is OK: PING OK - Packet loss = 0%, RTA = 16.48 ms [21:16:50] RECOVERY - cp25 Varnish Backends on cp25 is OK: All 15 backends are healthy [21:17:54] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is CRITICAL: CRITICAL - load average: 10.22, 5.28, 2.93 [21:19:54] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is WARNING: WARNING - load average: 7.84, 5.72, 3.37 [21:20:18] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.93, 3.29, 2.56 [21:20:47] PROBLEM - swiftobject121 Current Load on swiftobject121 is CRITICAL: CRITICAL - load average: 8.63, 6.94, 5.77 [21:21:54] PROBLEM - swiftproxy131 Current Load on swiftproxy131 is CRITICAL: CRITICAL - load average: 8.70, 6.34, 3.87 [21:22:17] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 3.33, 3.36, 2.68 [21:22:46] RECOVERY - swiftobject121 Current Load on swiftobject121 is OK: OK - load average: 5.29, 6.23, 5.65 [21:23:22] PROBLEM - swiftobject113 Current Load on swiftobject113 is WARNING: WARNING - load average: 7.18, 6.87, 6.11 [21:23:54] RECOVERY - swiftproxy131 Current Load on swiftproxy131 is OK: OK - load average: 4.36, 5.57, 3.89 [21:24:31] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/commit/c5e28545645a [21:24:32] [02miraheze/mediawiki] 07paladox 03c5e2854 - Experiment [21:24:34] [02mediawiki] 07paladox created branch 03paladox-patch-2 - 13https://github.com/miraheze/mediawiki [21:24:35] [02mediawiki] 07paladox opened pull request 03#17289: Experiment - 13https://github.com/miraheze/mediawiki/pull/17289 [21:25:14] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 5.99, 7.73, 7.47 [21:25:23] RECOVERY - swiftobject113 Current Load on swiftobject113 is OK: OK - load average: 6.47, 6.44, 6.03 [21:26:14] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.59, 3.98, 3.08 [21:26:27] RECOVERY - cp34 Puppet on cp34 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:28:13] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.69, 3.61, 3.07 [21:28:19] !log [paladox@mwtask141] starting deploy of {'world': True} to all [21:28:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:30:05] !log [paladox@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 105s [21:30:12] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.71, 3.89, 3.22 [21:30:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:33:12] [02mediawiki] 07paladox synchronize pull request 03#17289: Experiment - 13https://github.com/miraheze/mediawiki/pull/17289 [21:33:14] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/c5e28545645a...03dc53f2e257 [21:33:15] [02miraheze/mediawiki] 07paladox 0303dc53f - Update JobQueueRedis.php [21:33:27] !log [paladox@mwtask141] starting deploy of {'world': True} to all [21:33:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:34:10] !log [paladox@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 43s [21:34:10] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.34, 3.75, 3.33 [21:34:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:34:52]