[00:00:16] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:02:16] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:04:04] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21175MiB (23% inode=98%); [00:04:06] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21162MiB (23% inode=98%); [00:05:23] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 19226MiB (20% inode=98%); [00:08:50] PROBLEM - coffeewiki.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'coffeewiki.net' expires in 15 day(s) (Tue 16 Apr 2024 11:41:24 PM GMT +0000). [00:09:01] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5bb9902f7576...379b05092414 [00:09:03] [02miraheze/ssl] 07WikiTideSSLBot 03379b050 - Bot: Update SSL cert for coffeewiki.net [00:10:57] [02miraheze/ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/379b05092414...73244fd54077 [00:11:00] [02miraheze/ssl] 07AgentIsai 0373244fd - Temporarily disable redirect [00:15:55] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/refreshLinks.php --wiki=sagan4alphawiki (END - exit=0) [00:16:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:24:20] RECOVERY - richterian.com - LetsEncrypt on sslhost is OK: OK - Certificate 'richterian.com' will expire on Sat 29 Jun 2024 10:54:46 PM GMT +0000. [00:24:43] [02miraheze/ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/73244fd54077...f333be03a46b [00:24:46] [02miraheze/ssl] 07AgentIsai 03f333be0 - Add ssl [00:26:47] RECOVERY - cp41 Puppet on cp41 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:38:10] RECOVERY - coffeewiki.net - LetsEncrypt on sslhost is OK: OK - Certificate 'coffeewiki.net' will expire on Sat 29 Jun 2024 11:08:56 PM GMT +0000. [00:46:40] RECOVERY - www.clinitheque.fr - LetsEncrypt on sslhost is OK: OK - Certificate 'www.clinitheque.fr' will expire on Sat 29 Jun 2024 10:46:52 PM GMT +0000. [00:49:11] PROBLEM - *.orain.org - LetsEncrypt on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [00:53:58] [02miraheze/ManageWiki] 07The-Voidwalker pushed 031 commit to 03The-Voidwalker-patch-2 [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/commit/0f552935ce3b [00:54:01] [02miraheze/ManageWiki] 07The-Voidwalker 030f55293 - Fix populateNamespaces.php [00:54:02] [02ManageWiki] 07The-Voidwalker created branch 03The-Voidwalker-patch-2 - 13https://github.com/miraheze/ManageWiki [00:54:07] [02ManageWiki] 07The-Voidwalker opened pull request 03#458: Fix populateNamespaces.php - 13https://github.com/miraheze/ManageWiki/pull/458 [00:58:56] miraheze/ManageWiki - The-Voidwalker the build passed. [01:02:03] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/227853d0ca16...e537655031c2 [01:02:06] [02miraheze/mw-config] 07AgentIsai 03e537655 - new badge [01:02:06] !log [@mwtask181] starting deploy of {'config': True} to all [01:02:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:02:16] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [01:02:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:03:03] miraheze/mw-config - AgentIsai the build passed. [01:03:17] [02miraheze/landing] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/0a228d0ab883...753ed3ac865c [01:03:18] [02miraheze/landing] 07AgentIsai 03753ed3a - Modify [01:04:08] miraheze/landing - AgentIsai the build passed. [01:05:54] [02ManageWiki] 07The-Voidwalker commented on pull request 03#457: Improve handling of overridedefault for MWN - 13https://github.com/miraheze/ManageWiki/pull/457#issuecomment-2028986743 [01:07:05] !log [@mwtask171] starting deploy of {'config': True} to all [01:07:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:07:13] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [01:07:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:54] !log [@test151] starting deploy of {'config': True} to test151 [01:23:55] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [01:24:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:24:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:57:40] [02miraheze/RequestSSL] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 [+0/-0/±2] 13https://github.com/miraheze/RequestSSL/commit/37e6d6cd0e42 [01:57:41] [02miraheze/RequestSSL] 07dependabot[bot] 0337e6d6c - Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 [01:57:43] [02RequestSSL] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 - 13https://github.com/miraheze/RequestSSL [01:57:45] [02RequestSSL] 07dependabot[bot] labeled pull request 03#43: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/RequestSSL/pull/43 [01:57:46] [02RequestSSL] 07dependabot[bot] labeled pull request 03#43: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/RequestSSL/pull/43 [01:57:49] [02RequestSSL] 07dependabot[bot] opened pull request 03#43: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/RequestSSL/pull/43 [02:01:32] [02landing] 07colleirose commented on commit 03753ed3ac865c318902a0b5d90531a83533afaa4c - 13https://github.com/miraheze/landing/commit/753ed3ac865c318902a0b5d90531a83533afaa4c#commitcomment-140425497 [02:05:38] miraheze/RequestSSL - dependabot[bot] the build passed. [02:18:29] !log upgrade grafana and fix /logout endpoint [02:18:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:45:04] [02landing] 07colleirose opened pull request 03#142: Improve code formatting, use tinypng.com on images, and fix contrast issues on donation page - 13https://github.com/miraheze/landing/pull/142 [02:45:26] [02landing] 07colleirose edited pull request 03#142: Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page - 13https://github.com/miraheze/landing/pull/142 [02:45:54] miraheze/landing - colleirose the build passed. [02:45:57] [02landing] 07Universal-Omega closed pull request 03#142: Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page - 13https://github.com/miraheze/landing/pull/142 [02:46:00] [02miraheze/landing] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±18] 13https://github.com/miraheze/landing/compare/753ed3ac865c...ac601bd0d115 [02:46:03] [02miraheze/landing] 07colleirose 03ac601bd - Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page (#142) [02:46:47] miraheze/landing - Universal-Omega the build passed. [03:03:34] PROBLEM - db162 Backups SQL on db162 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864200 seconds old and 1218 bytes [07:29:41] WD Green my beloved [08:19:45] <.labster> The best color of WD. [10:05:02] PROBLEM - db171 Backups SQL on db171 is CRITICAL: FILE_AGE CRITICAL: /var/log/sql-backup.log is 1209686 seconds old and 116092 bytes [10:48:23] [02miraheze/landing] 07redbluegreenhat pushed 031 commit to 03revert-142-patch-1 [+0/-0/±18] 13https://github.com/miraheze/landing/commit/027ffccc97af [10:48:26] [02miraheze/landing] 07redbluegreenhat 03027ffcc - Revert "Improve code formatting, replace images with compressed versions from…" [10:48:27] [02landing] 07redbluegreenhat created branch 03revert-142-patch-1 - 13https://github.com/miraheze/landing [10:48:41] [02landing] 07redbluegreenhat opened pull request 03#143: DNM: Revert "Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page" - 13https://github.com/miraheze/landing/pull/143 [10:49:25] miraheze/landing - redbluegreenhat the build passed. [11:10:59] [02landing] 07redbluegreenhat commented on commit 03ac601bd0d11582875df876b980734658e6f246a3 - 13https://github.com/miraheze/landing/commit/ac601bd0d11582875df876b980734658e6f246a3#r140439635 [11:11:54] [02landing] 07redbluegreenhat closed pull request 03#143: DNM: Revert "Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page" - 13https://github.com/miraheze/landing/pull/143 [11:18:35] [02landing] 07redbluegreenhat commented on pull request 03#142: Improve code formatting, replace images with compressed versions from tinypng.com, and fix contrast issues on donation page - 13https://github.com/miraheze/landing/pull/142#issuecomment-2029605776 [12:14:54] [02miraheze/SpriteSheet] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/SpriteSheet/compare/bf8dc0b5f17d...2e57f6279119 [12:14:56] [02miraheze/SpriteSheet] 07translatewiki 032e57f62 - Localisation updates from https://translatewiki.net. [12:14:59] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/landing/compare/ac601bd0d115...51184825127f [12:15:01] [02miraheze/landing] 07translatewiki 035118482 - Localisation updates from https://translatewiki.net. [12:15:02] [02miraheze/YouTube] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/YouTube/compare/144ec8ab46e1...118939def53b [12:15:03] [02miraheze/YouTube] 07translatewiki 03118939d - Localisation updates from https://translatewiki.net. [12:15:06] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/MirahezeMagic/compare/9a9dd7e7b848...198e71cea4cc [12:15:08] [02miraheze/MirahezeMagic] 07translatewiki 03198e71c - Localisation updates from https://translatewiki.net. [12:15:51] miraheze/landing - translatewiki the build passed. [12:18:22] miraheze/SpriteSheet - translatewiki the build has errored. [12:18:57] miraheze/YouTube - translatewiki the build passed. [12:19:14] miraheze/MirahezeMagic - translatewiki the build has errored. [12:23:52] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [12:23:53] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:24:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:02] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [12:24:03] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:24:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:32:30] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [12:32:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:32:39] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 8s [12:32:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:02] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 205.82 ms [12:37:01] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.59 ms [13:07:05] [02miraheze/ImportDump] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 [+0/-0/±2] 13https://github.com/miraheze/ImportDump/commit/62eb3acee49f [13:07:07] [02miraheze/ImportDump] 07dependabot[bot] 0362eb3ac - Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 [13:07:08] [02ImportDump] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 - 13https://github.com/miraheze/ImportDump [13:07:10] [02ImportDump] 07dependabot[bot] opened pull request 03#92: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ImportDump/pull/92 [13:07:13] [02ImportDump] 07dependabot[bot] labeled pull request 03#92: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ImportDump/pull/92 [13:07:15] [02ImportDump] 07dependabot[bot] labeled pull request 03#92: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ImportDump/pull/92 [13:15:03] [02ManageWiki] 07redbluegreenhat closed pull request 03#458: Fix populateNamespaces.php - 13https://github.com/miraheze/ManageWiki/pull/458 [13:15:04] [02miraheze/ManageWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/7a9745456906...e6b172c3b6cc [13:15:07] [02miraheze/ManageWiki] 07The-Voidwalker 03e6b172c - Fix populateNamespaces.php (#458) [13:15:10] miraheze/ImportDump - dependabot[bot] the build passed. [13:19:54] miraheze/ManageWiki - redbluegreenhat the build passed. [13:56:28] PROBLEM - wiki.overwood.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.overwood.xyz' expires in 15 day(s) (Wed 17 Apr 2024 01:45:02 PM GMT +0000). [13:56:41] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f333be03a46b...f17c98410b19 [13:56:42] [02miraheze/ssl] 07WikiTideSSLBot 03f17c984 - Bot: Update SSL cert for wiki.overwood.xyz [14:00:52] PROBLEM - cp36 Current Load on cp36 is CRITICAL: LOAD CRITICAL - total load average: 28.01, 20.33, 8.91 [14:06:34] PROBLEM - cp36 Current Load on cp36 is WARNING: LOAD WARNING - total load average: 0.87, 7.03, 6.42 [14:08:28] RECOVERY - cp36 Current Load on cp36 is OK: LOAD OK - total load average: 1.57, 5.31, 5.86 [14:12:41] PROBLEM - wiki.tmyt105.leyhp.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.tmyt105.leyhp.com' expires in 15 day(s) (Wed 17 Apr 2024 02:04:14 PM GMT +0000). [14:12:47] PROBLEM - wiki.jsal.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.jsal.org' expires in 15 day(s) (Wed 17 Apr 2024 01:52:08 PM GMT +0000). [14:12:53] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f17c98410b19...3cfb29aae8bf [14:12:54] [02miraheze/ssl] 07WikiTideSSLBot 033cfb29a - Bot: Update SSL cert for wiki.tmyt105.leyhp.com [14:13:03] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3cfb29aae8bf...9a63175caf8f [14:13:05] [02miraheze/ssl] 07WikiTideSSLBot 039a63175 - Bot: Update SSL cert for wiki.jsal.org [14:15:51] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - franchise.franchising.org.ua All nameservers failed to answer the query. [14:25:27] RECOVERY - wiki.overwood.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.overwood.xyz' will expire on Sun 30 Jun 2024 12:56:35 PM GMT +0000. [14:45:27] RECOVERY - franchise.franchising.org.ua - reverse DNS on sslhost is OK: SSL OK - franchise.franchising.org.ua reverse DNS resolves to cp36.wikitide.net - CNAME OK [15:06:33] PROBLEM - rodzinka.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rodzinka.wiki' expires in 15 day(s) (Wed 17 Apr 2024 02:49:03 PM GMT +0000). [15:06:45] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9a63175caf8f...471f49a82358 [15:06:48] [02miraheze/ssl] 07WikiTideSSLBot 03471f49a - Bot: Update SSL cert for rodzinka.wiki [15:08:29] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset -0.103305012 secs [15:10:27] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset -0.05809605122 secs [15:10:42] RECOVERY - wiki.tmyt105.leyhp.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tmyt105.leyhp.com' will expire on Sun 30 Jun 2024 01:12:47 PM GMT +0000. [15:10:47] RECOVERY - wiki.jsal.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jsal.org' will expire on Sun 30 Jun 2024 01:12:57 PM GMT +0000. [15:13:07] PROBLEM - ns2 APT on ns2 is WARNING: APT WARNING: 0 packages available for upgrade (0 critical updates). warnings detected, errors detected. [15:15:04] [1/2] Gonna love some [15:15:04] [2/2] https://cdn.discordapp.com/attachments/808001911868489748/1224376490754576415/62bb8707addbb_wd-green-ssd-240gb-frontpng.png?ex=661d4477&is=660acf77&hm=aa56331b3715e2c60c58df38ed75a4a456f51ca99985d3c78f808fa20b67a76c& [15:15:06] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (13 critical updates). [15:35:06] RECOVERY - rodzinka.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'rodzinka.wiki' will expire on Sun 30 Jun 2024 02:06:39 PM GMT +0000. [15:44:22] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/alex/themanaworldwiki/imageDump --search-recursively --comment=[[phorge:T11978]] (START) [15:44:22] https://meta.miraheze.org/wiki/phorge:T11978 [15:44:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:45:58] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/alex/themanaworldwiki/imageDump --search-recursively --comment=[[phorge:T11978]] (END - exit=0) [15:45:58] https://meta.miraheze.org/wiki/phorge:T11978 [15:46:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:28:00] PROBLEM - removededm.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'removededm.com' expires in 15 day(s) (Wed 17 Apr 2024 04:03:46 PM GMT +0000). [16:28:13] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/471f49a82358...f2ee74059a59 [16:28:14] [02miraheze/ssl] 07WikiTideSSLBot 03f2ee740 - Bot: Update SSL cert for removededm.com [16:28:15] PROBLEM - tno.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tno.wiki' expires in 15 day(s) (Wed 17 Apr 2024 03:59:57 PM GMT +0000). [16:28:29] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f2ee74059a59...f9318683b281 [16:28:30] [02miraheze/ssl] 07WikiTideSSLBot 03f931868 - Bot: Update SSL cert for tno.wiki [16:28:41] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 170.49 ms [16:30:40] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 172.31 ms [16:37:45] [02miraheze/landing] 07Universal-Omega deleted branch 03revert-142-patch-1 [16:37:46] [02landing] 07Universal-Omega deleted branch 03revert-142-patch-1 - 13https://github.com/miraheze/landing [16:50:19] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:51:53] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 10224MiB (10% inode=98%); [16:54:47] [02miraheze/MatomoAnalytics] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 [+0/-0/±2] 13https://github.com/miraheze/MatomoAnalytics/commit/6897d621e58f [16:54:49] [02miraheze/MatomoAnalytics] 07dependabot[bot] 036897d62 - Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 [16:54:51] [02MatomoAnalytics] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 - 13https://github.com/miraheze/MatomoAnalytics [16:54:52] [02MatomoAnalytics] 07dependabot[bot] opened pull request 03#123: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/MatomoAnalytics/pull/123 [16:54:54] [02MatomoAnalytics] 07dependabot[bot] labeled pull request 03#123: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/MatomoAnalytics/pull/123 [16:54:56] [02MatomoAnalytics] 07dependabot[bot] labeled pull request 03#123: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/MatomoAnalytics/pull/123 [16:57:00] RECOVERY - removededm.com - LetsEncrypt on sslhost is OK: OK - Certificate 'removededm.com' will expire on Sun 30 Jun 2024 03:28:06 PM GMT +0000. [16:57:44] RECOVERY - tno.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'tno.wiki' will expire on Sun 30 Jun 2024 03:28:22 PM GMT +0000. [16:59:15] miraheze/MatomoAnalytics - dependabot[bot] the build passed. [17:19:58] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [17:24:27] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/alex/themanaworldwiki/imageDump --search-recursively --comment=[[phorge:T11978]] (START) [17:24:27] https://meta.miraheze.org/wiki/phorge:T11978 [17:24:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:34:01] PROBLEM - wiki.gab.pt.eu.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.gab.pt.eu.org' expires in 7 day(s) (Tue 09 Apr 2024 05:31:08 PM GMT +0000). [17:35:10] [02puppet] 07RhinosF1 opened pull request 03#3817: rename-wiki: various - 13https://github.com/miraheze/puppet/pull/3817 [17:36:29] miraheze/puppet - RhinosF1 the build passed. [17:57:18] [02miraheze/ManageWiki] 07dependabot[bot] pushed 031 commit to 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 [+0/-0/±2] 13https://github.com/miraheze/ManageWiki/commit/cc39ab1a4c37 [17:57:21] [02miraheze/ManageWiki] 07dependabot[bot] 03cc39ab1 - Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 [17:57:23] [02ManageWiki] 07dependabot[bot] created branch 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.27.0 - 13https://github.com/miraheze/ManageWiki [17:57:26] [02ManageWiki] 07dependabot[bot] labeled pull request 03#459: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ManageWiki/pull/459 [17:57:29] [02ManageWiki] 07dependabot[bot] labeled pull request 03#459: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ManageWiki/pull/459 [17:57:32] [02ManageWiki] 07dependabot[bot] opened pull request 03#459: Bump eslint-config-wikimedia from 0.26.0 to 0.27.0 - 13https://github.com/miraheze/ManageWiki/pull/459 [18:01:53] miraheze/ManageWiki - dependabot[bot] the build passed. [19:01:14] [02puppet] 07Universal-Omega reviewed pull request 03#3817 commit - 13https://github.com/miraheze/puppet/pull/3817#discussion_r1546723945 [19:03:54] [02puppet] 07RhinosF1 reviewed pull request 03#3817 commit - 13https://github.com/miraheze/puppet/pull/3817#discussion_r1546727016 [19:22:31] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [19:32:23] [02CreateWiki] 07redbluegreenhat closed pull request 03#500: add more details needed status - 13https://github.com/miraheze/CreateWiki/pull/500 [19:32:24] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+1/-0/±7] 13https://github.com/miraheze/CreateWiki/compare/09af630a0441...36c74842cbf2 [19:32:26] [02miraheze/CreateWiki] 07Reception123 0336c7484 - add more details needed status (#500) [19:32:29] [02miraheze/CreateWiki] 07redbluegreenhat deleted branch 03Reception123-patch-1 [19:32:31] [02CreateWiki] 07redbluegreenhat deleted branch 03Reception123-patch-1 - 13https://github.com/miraheze/CreateWiki [19:32:51] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [19:32:52] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [19:32:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:33:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:35:29] miraheze/CreateWiki - redbluegreenhat the build has errored. [19:38:43] !log [alex@test151] starting deploy of {'l10n': True, 'versions': '1.41'} to test151 [19:38:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:43:02] !log [alex@test151] finished deploy of {'l10n': True, 'versions': '1.41'} to test151 - SUCCESS in 259s [19:43:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:52:05] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [19:54:51] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03redbluegreenhat-patch-1 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/commit/e891bc3fbcfd [19:54:53] [02miraheze/CreateWiki] 07redbluegreenhat 03e891bc3 - Add missing i18n [19:54:55] [02CreateWiki] 07redbluegreenhat created branch 03redbluegreenhat-patch-1 - 13https://github.com/miraheze/CreateWiki [19:54:57] [02CreateWiki] 07redbluegreenhat opened pull request 03#501: Add missing i18n - 13https://github.com/miraheze/CreateWiki/pull/501 [19:57:12] [02miraheze/CreateWiki] 07github-actions[bot] pushed 031 commit to 03redbluegreenhat-patch-1 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/e891bc3fbcfd...b76fe1f4ea9c [19:57:13] [02miraheze/CreateWiki] 07github-actions 03b76fe1f - CI: lint code to MediaWiki standards [19:57:16] [02CreateWiki] 07github-actions[bot] synchronize pull request 03#501: Add missing i18n - 13https://github.com/miraheze/CreateWiki/pull/501 [19:58:18] [02CreateWiki] 07redbluegreenhat edited pull request 03#501: Add missing i18n for the new "requestmoredetails" log - 13https://github.com/miraheze/CreateWiki/pull/501 [19:58:40] miraheze/CreateWiki - redbluegreenhat the build has errored. [19:59:52] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03redbluegreenhat-patch-1 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/b76fe1f4ea9c...1ec1e1977de5 [19:59:55] [02miraheze/CreateWiki] 07redbluegreenhat 031ec1e19 - Add to ActionFilteredLogs [19:59:58] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#501: Add missing i18n for the new "requestmoredetails" log - 13https://github.com/miraheze/CreateWiki/pull/501 [20:00:31] [02CreateWiki] 07redbluegreenhat edited pull request 03#501: Add "requestmoredetails" to ActionFilteredLogs - 13https://github.com/miraheze/CreateWiki/pull/501 [20:02:24] [02CreateWiki] 07redbluegreenhat closed pull request 03#501: Add "requestmoredetails" to ActionFilteredLogs - 13https://github.com/miraheze/CreateWiki/pull/501 [20:02:26] [02miraheze/CreateWiki] 07redbluegreenhat deleted branch 03redbluegreenhat-patch-1 [20:02:29] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/CreateWiki/compare/36c74842cbf2...e6fe952056f4 [20:02:31] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [20:02:32] [02miraheze/CreateWiki] 07redbluegreenhat 03e6fe952 - Add "requestmoredetails" to ActionFilteredLogs (#501) [20:02:32] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 1s [20:02:33] [02CreateWiki] 07redbluegreenhat deleted branch 03redbluegreenhat-patch-1 - 13https://github.com/miraheze/CreateWiki [20:02:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:02:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:02:53] !log [alex@test151] starting deploy of {'l10n': True, 'versions': '1.41'} to test151 [20:03:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:04:09] miraheze/CreateWiki - redbluegreenhat the build has errored. [20:06:10] miraheze/CreateWiki - redbluegreenhat the build has errored. [20:07:04] !log [alex@test151] finished deploy of {'l10n': True, 'versions': '1.41'} to test151 - SUCCESS in 251s [20:07:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:14:46] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03workers [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/352b01d5a07a [20:14:49] [02miraheze/puppet] 07Universal-Omega 03352b01d - changeprop: make num_workers configurable [20:14:51] [02puppet] 07Universal-Omega created branch 03workers - 13https://github.com/miraheze/puppet [20:14:54] [02puppet] 07Universal-Omega opened pull request 03#3818: changeprop: make num_workers configurable - 13https://github.com/miraheze/puppet/pull/3818 [20:15:01] [02CreateWiki] 07redbluegreenhat opened pull request 03#502: fix CI - 13https://github.com/miraheze/CreateWiki/pull/502 [20:16:56] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03workers [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/352b01d5a07a...b70e3a558330 [20:16:58] [02miraheze/puppet] 07Universal-Omega 03b70e3a5 - Add [20:17:01] [02puppet] 07Universal-Omega synchronize pull request 03#3818: changeprop: make num_workers configurable - 13https://github.com/miraheze/puppet/pull/3818 [20:17:04] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#502: fix CI - 13https://github.com/miraheze/CreateWiki/pull/502 [20:17:53] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03workers [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b70e3a558330...5920a32e7862 [20:17:54] [02miraheze/puppet] 07Universal-Omega 035920a32 - Set to 12 [20:17:55] [02puppet] 07Universal-Omega synchronize pull request 03#3818: changeprop: make num_workers configurable - 13https://github.com/miraheze/puppet/pull/3818 [20:19:53] [02puppet] 07Universal-Omega synchronize pull request 03#3818: changeprop: make num_workers configurable - 13https://github.com/miraheze/puppet/pull/3818 [20:20:02] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03workers [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5920a32e7862...e17cecd4c0f2 [20:20:03] [02miraheze/puppet] 07Universal-Omega 03e17cecd - Default 1 [20:20:33] [02puppet] 07Universal-Omega closed pull request 03#3818: changeprop: make num_workers configurable - 13https://github.com/miraheze/puppet/pull/3818 [20:20:34] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/85bfcff623c7...74d32a298fc4 [20:20:35] [02miraheze/puppet] 07Universal-Omega 0374d32a2 - changeprop: make num_workers configurable (#3818) [20:20:38] [02miraheze/puppet] 07Universal-Omega deleted branch 03workers [20:20:41] [02puppet] 07Universal-Omega deleted branch 03workers - 13https://github.com/miraheze/puppet [20:22:50] miraheze/CreateWiki - redbluegreenhat the build passed. [20:24:48] miraheze/CreateWiki - redbluegreenhat the build passed. [20:27:06] [02CreateWiki] 07redbluegreenhat closed pull request 03#502: fix CI - 13https://github.com/miraheze/CreateWiki/pull/502 [20:27:09] [02miraheze/CreateWiki] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/e6fe952056f4...18f9455b313a [20:27:10] [02miraheze/CreateWiki] 07redbluegreenhat 0318f9455 - fix CI (#502) [20:31:51] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/d8ade73d1e9a [20:31:53] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=othertowerwiki --no-updates --username-prefix=wikia:other-tower /home/alex/othertowerwiki-20240401192631.xml (START) [20:31:54] [02miraheze/puppet] 07Universal-Omega 03d8ade73 - changeprop: add new import and videoscaler job queues [20:31:57] [02puppet] 07Universal-Omega created branch 03queues - 13https://github.com/miraheze/puppet [20:31:59] [02puppet] 07Universal-Omega opened pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:32:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:33:15] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=othertowerwiki --no-updates --username-prefix=wikia:other-tower /home/alex/othertowerwiki-20240401192631.xml (END - exit=0) [20:33:16] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=othertowerwiki (START) [20:33:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:33:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:33:31] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=othertowerwiki (END - exit=0) [20:33:33] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/initSiteStats.php --wiki=othertowerwiki --active --update (END - exit=0) [20:33:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:33:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:34:58] miraheze/CreateWiki - redbluegreenhat the build passed. [20:36:00] [02puppet] 07Universal-Omega synchronize pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:36:14] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d8ade73d1e9a...06e943d92207 [20:36:15] [02miraheze/puppet] 07Universal-Omega 0306e943d - Add [20:41:20] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/06e943d92207...8b079348c0ef [20:41:23] [02miraheze/puppet] 07Universal-Omega 038b07934 - Add [20:41:25] [02puppet] 07Universal-Omega synchronize pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:44:52] [02puppet] 07Universal-Omega synchronize pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:45:40] [02puppet] 07Universal-Omega synchronize pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:47:04] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a19d63944ed4...84bd833231b6 [20:47:07] [02miraheze/puppet] 07Universal-Omega 0384bd833 - Add [20:47:09] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8b079348c0ef...a19d63944ed4 [20:47:11] [02miraheze/puppet] 07Universal-Omega 03a19d639 - Use separate import_host [20:47:35] [02puppet] 07Universal-Omega synchronize pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [20:47:38] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/84bd833231b6...cc201ed8ba44 [20:47:39] [02miraheze/puppet] 07Universal-Omega 03cc201ed - Add [20:56:10] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/74d32a298fc4...d6d7150959c6 [20:56:13] [02miraheze/puppet] 07Universal-Omega 03d6d7150 - use cpjobqueue again for now [21:05:51] PROBLEM - mwtask181 JobRunner Service on mwtask181 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobRunnerService' [21:19:58] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/alex/themanaworldwiki/imageDump --search-recursively --comment=[[phorge:T11978]] (END - exit=0) [21:19:58] https://meta.miraheze.org/wiki/phorge:T11978 [21:20:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:25:43] [02puppet] 07Universal-Omega closed pull request 03#3819: changeprop: add new import and videoscaler job queues - 13https://github.com/miraheze/puppet/pull/3819 [21:25:44] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/d6d7150959c6...5ccc54e16698 [21:25:46] [02miraheze/puppet] 07Universal-Omega 035ccc54e - changeprop: add new import and videoscaler job queues (#3819) [21:25:47] [02miraheze/puppet] 07Universal-Omega deleted branch 03queues [21:25:50] [02puppet] 07Universal-Omega deleted branch 03queues - 13https://github.com/miraheze/puppet [21:28:20] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9704MiB (10% inode=98%); [21:43:33] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9697MiB (10% inode=98%); [22:53:31] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [22:53:33] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [22:53:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:53:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:28:49] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [23:58:19] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK