[00:00:24] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:24] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 50% [00:03:48] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21336MiB (24% inode=98%); [00:03:50] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21363MiB (24% inode=98%); [00:04:24] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [00:04:32] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 19409MiB (20% inode=98%); [00:07:51] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/50c1c020ee52...f8ec89c02790 [00:07:54] [02miraheze/ssl] 07WikiTideSSLBot 03f8ec89c - Bot: Add SSL cert for wiki.orivium.io [00:08:34] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/f8ec89c02790...5324370e3d63 [00:08:35] [02miraheze/ssl] 07WikiTideSSLBot 035324370 - Bot: Add SSL cert for en.noblework.org [00:09:04] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/5324370e3d63...bcef397befd0 [00:09:06] [02miraheze/ssl] 07WikiTideSSLBot 03bcef397 - Bot: Add SSL cert for wiki.kscucf.org [00:14:05] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03more-domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/c43bb477dc78 [00:14:08] [02miraheze/mw-config] 07Universal-Omega 03c43bb47 - Allow using orain.org and wiki.surf also [00:14:09] [02mw-config] 07Universal-Omega created branch 03more-domains - 13https://github.com/miraheze/mw-config [00:14:10] [02mw-config] 07Universal-Omega opened pull request 03#5529: Allow using orain.org and wiki.surf also - 13https://github.com/miraheze/mw-config/pull/5529 [00:14:53] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CreateWiki/maintenance/deleteWiki.php --wiki=metawiki --deletewiki elysiumwiki --delete (END - exit=0) [00:15:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:15:08] miraheze/mw-config - Universal-Omega the build passed. [00:17:24] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03more-domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c43bb477dc78...a625c1eff9c5 [00:17:27] [02miraheze/mw-config] 07Universal-Omega 03a625c1e - Add [00:17:29] [02mw-config] 07Universal-Omega synchronize pull request 03#5529: Allow using orain.org and wiki.surf also - 13https://github.com/miraheze/mw-config/pull/5529 [00:18:18] miraheze/mw-config - Universal-Omega the build passed. [00:20:41] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [00:26:19] [02miraheze/dns] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/59592b6f65fb...9922adca2638 [00:26:21] [02miraheze/dns] 07Universal-Omega 039922adc - wikitide.org: +mwcosmos [00:35:14] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=elysiumwiki elysiumwiki_xml_7998eb1b0013a9d5377b.xml --no-updates (START) [00:35:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:36:44] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=elysiumwiki elysiumwiki_xml_7998eb1b0013a9d5377b.xml --no-updates (END - exit=0) [00:36:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:41:29] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=elysiumwiki . --search-recursively --summary Importing from WikiTide version (START) [00:41:30] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=elysiumwiki . --search-recursively --summary Importing from WikiTide version (END - exit=2) [00:41:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:41:45] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=elysiumwiki . --search-recursively --summary Importing from WikiTide version (START) [00:41:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:41:48] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=elysiumwiki . --search-recursively --summary Importing from WikiTide version (END - exit=0) [00:41:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:42:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:53:59] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=elysiumwiki (START) [00:54:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:54:12] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=elysiumwiki (END - exit=0) [00:54:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:00:03] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=chronicleofastriawiki chronicleofastriawiki_xml_b79dee7ae49aa95802b5.xml --no-updates (START) [01:00:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:01:37] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=chronicleofastriawiki chronicleofastriawiki_xml_b79dee7ae49aa95802b5.xml --no-updates (END - exit=0) [01:01:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:02:18] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=chronicleofastriawiki . --search-recursively --summary Importing from WikiTide version (START) [01:02:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:05:38] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=vvobestuurwiki vvobestuurwiki_xml_e7c9dd3efa3d35c4f82d.xml.gz --no-updates (START) [01:05:39] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=vvobestuurwiki vvobestuurwiki_xml_e7c9dd3efa3d35c4f82d.xml.gz --no-updates (END - exit=256) [01:05:44] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=vvobestuurwiki vvobestuurwiki_xml_e7c9dd3efa3d35c4f82d.xml --no-updates (START) [01:05:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:05:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:06:01] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=vvobestuurwiki vvobestuurwiki_xml_e7c9dd3efa3d35c4f82d.xml --no-updates (END - exit=0) [01:06:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:06:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:08:01] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=vvobestuurwiki . --search-recursively (START) [01:08:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:08:34] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=vvobestuurwiki . --search-recursively (END - exit=0) [01:08:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:09:30] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/initSiteStats.php --wiki=elysiumwiki --update (END - exit=0) [01:09:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:09:47] RECOVERY - ru-teirailway.f5.si - LetsEncrypt on sslhost is OK: OK - Certificate 'ru-teirailway.f5.si' will expire on Sun 16 Jun 2024 04:11:56 PM GMT +0000. [01:10:07] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=vvobestuurwiki (START) [01:10:11] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=vvobestuurwiki (END - exit=0) [01:10:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:10:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:14:10] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=chronicleofastriawiki . --search-recursively --summary Importing from WikiTide version (END - exit=0) [01:14:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:17:13] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=dharmapearlswiki dharmapearlswiki_xml_15f51d19236d5ee90959.xml --no-updates (START) [01:17:17] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=dharmapearlswiki dharmapearlswiki_xml_15f51d19236d5ee90959.xml --no-updates (END - exit=0) [01:17:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:17:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:17:43] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=dharmapearlswiki (START) [01:17:46] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=dharmapearlswiki (END - exit=0) [01:17:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:17:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:29:12] [02miraheze/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/9922adca2638...b8b4487fe345 [01:29:15] [02miraheze/dns] 07AgentIsai 03b8b4487 - Add redemptionwiki redirect [01:30:04] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/restoreManageWikiBackup.php --wiki=redemptionwiki --filename https://static.wikitide.net/redemptionwiki/dumps/redemptionwiki_managewiki_backup_1a322d3f3341a96c57ca.json (END - exit=256) [01:30:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:36:10] RECOVERY - ru-teirailway.f5.si - reverse DNS on sslhost is OK: SSL OK - ru-teirailway.f5.si reverse DNS resolves to cp36.wikitide.net - CNAME OK [01:49:48] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/restoreManageWikiBackup.php --wiki=redemptionwiki --filename redemptionwiki_managewiki_backup_1a322d3f3341a96c57ca.json (END - exit=256) [01:49:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:00] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=redemptionwiki redemptionwiki_xml_436fc1578dd1297d0380.xml --no-updates (START) [01:51:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:30] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=redemptionwiki redemptionwiki_xml_436fc1578dd1297d0380.xml --no-updates (END - exit=0) [01:51:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:51:56] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:52:58] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=redemptionwiki . --search-recursively (START) [01:53:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:00:46] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=redemptionwiki . --search-recursively (END - exit=0) [02:00:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:01:47] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=redemptionwiki (START) [02:01:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:01:55] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=redemptionwiki (END - exit=0) [02:02:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:02:34] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/restoreManageWikiBackup.php --wiki=pocketdragonswiki --filename pocketdragonswiki_managewiki_backup_58b6ef8c12d18d99e5df.json (END - exit=0) [02:02:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:03:11] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=pocketdragonswiki pocketdragonswiki_xml_525d303449de55cc8bd1.xml --no-updates (START) [02:03:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:03:37] [02miraheze/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/b8b4487fe345...5d959e26aadd [02:03:38] [02miraheze/dns] 07AgentIsai 035d959e2 - Add pocketdragons dyna [02:10:35] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=pocketdragonswiki pocketdragonswiki_xml_525d303449de55cc8bd1.xml --no-updates (END - exit=0) [02:10:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:15] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=chronicleofastriawiki (START) [02:14:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:14:29] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=chronicleofastriawiki (END - exit=0) [02:14:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:15:52] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/43112f375609...88407609635c [02:15:55] [02miraheze/mw-config] 07Universal-Omega 038840760 - Switch mwtask timeout to 1 day [02:16:26] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03jobs [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/013daae2568a...5253cecc17b9 [02:16:27] [02miraheze/mw-config] 07Universal-Omega 035253cec - Switch some job types back to redis temporarily [02:16:29] [02mw-config] 07Universal-Omega synchronize pull request 03#5523: Switch some job types back to redis temporarily - 13https://github.com/miraheze/mw-config/pull/5523 [02:16:45] miraheze/mw-config - Universal-Omega the build passed. [02:17:19] miraheze/mw-config - Universal-Omega the build passed. [02:27:15] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=pocketdragonswiki . --search-recursively (START) [02:27:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:32:09] !log [@mwtask181] starting deploy of {'config': True} to all [02:32:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:32:19] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [02:32:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:36:15] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/88407609635c...6546e9acf33a [02:36:16] [02miraheze/mw-config] 07Universal-Omega 036546e9a - Use default server instead of hard-coded key [02:36:49] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [02:36:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:36:59] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [02:37:06] miraheze/mw-config - Universal-Omega the build passed. [02:37:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:17] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [02:37:18] !log [universalomega@test151] DEPLOY ABORTED: Canary check failed for meta.mirabeta.org@localhost [02:37:30] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True, 'force': True} to test151 [02:37:31] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True, 'force': True} to test151 - SUCCESS in 0s [02:37:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:43] !log [@mwtask171] starting deploy of {'config': True} to all [02:37:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:51] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [02:37:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:38:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:38:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:41:14] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [02:46:04] [02miraheze/mw-config] 07Universal-Omega pushed 032 commits to 03more-domains [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/a625c1eff9c5...e19a29c83190 [02:46:06] [02miraheze/mw-config] 07Universal-Omega 03666d519 - Allow using orain.org and wiki.surf also [02:46:07] [02miraheze/mw-config] 07Universal-Omega 03e19a29c - Add [02:46:09] [02mw-config] 07Universal-Omega synchronize pull request 03#5529: Allow using orain.org and wiki.surf also - 13https://github.com/miraheze/mw-config/pull/5529 [02:46:59] miraheze/mw-config - Universal-Omega the build passed. [02:50:07] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [03:00:09] RECOVERY - db181 Backups SQL on db181 is OK: FILE_AGE OK: /var/log/sql-backup.log is 8 seconds old and 0 bytes [03:00:46] PROBLEM - db162 Backups SQL on db162 is CRITICAL: FILE_AGE CRITICAL: /var/log/sql-backup.log is 1209632 seconds old and 1218 bytes [03:01:26] RECOVERY - db161 Backups SQL on db161 is OK: FILE_AGE OK: /var/log/sql-backup.log is 85 seconds old and 0 bytes [03:08:31] !log [agent@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=pocketdragonswiki . --search-recursively (END - exit=0) [03:08:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:09:45] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [03:48:25] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [04:46:40] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [05:15:50] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:43:12] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [07:02:58] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 169.66 ms [07:04:59] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.23 ms [07:53:15] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [08:03:28] PROBLEM - db181 Current Load on db181 is CRITICAL: LOAD CRITICAL - total load average: 38.86, 30.22, 13.54 [08:05:11] PROBLEM - db181 PowerDNS Recursor on db181 is CRITICAL: CRITICAL - Plugin timed out while executing system call [08:06:52] PROBLEM - db181 Puppet on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:07:10] PROBLEM - db181 SSH on db181 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:10:39] PROBLEM - db181 Backups SQL on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:10:44] PROBLEM - db181 conntrack_table_size on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:12:29] PROBLEM - db181 Backups SQL mhglobal on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:12:31] PROBLEM - mwtask171 MediaWiki Rendering on mwtask171 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [08:12:41] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:13:05] PROBLEM - db181 ferm_active on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [08:13:56] PROBLEM - mwtask171 HTTPS on mwtask171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10001 milliseconds with 0 bytes received [08:22:00] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [08:30:36] PROBLEM - db181 Puppet on db181 is WARNING: WARNING: Puppet last ran 1 hour ago [08:30:47] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 4 backends are down. mw151 mw161 mw172 mw182 [08:31:37] RECOVERY - mwtask171 MediaWiki Rendering on mwtask171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.294 second response time [08:31:55] RECOVERY - db181 PowerDNS Recursor on db181 is OK: DNS OK: 0.063 seconds response time. wikitide.net returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [08:31:57] RECOVERY - db181 ferm_active on db181 is OK: OK ferm input default policy is set [08:32:01] RECOVERY - db181 Backups SQL mhglobal on db181 is OK: FILE_AGE OK: /var/log/sql-mhglobal-backup-weekly.log is 444713 seconds old and 208 bytes [08:32:02] RECOVERY - db181 conntrack_table_size on db181 is OK: OK: nf_conntrack is 0 % full [08:32:03] RECOVERY - db181 SSH on db181 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [08:32:03] RECOVERY - db181 Backups SQL on db181 is OK: FILE_AGE OK: /var/log/sql-backup.log is 1875 seconds old and 107143 bytes [08:32:27] RECOVERY - mwtask171 HTTPS on mwtask171 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 285 bytes in 0.065 second response time [08:32:36] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [08:32:47] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 19 backends are healthy [08:41:48] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [08:55:23] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [09:10:03] PROBLEM - db181 Current Load on db181 is WARNING: LOAD WARNING - total load average: 0.06, 0.18, 10.97 [09:10:54] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [09:12:02] RECOVERY - db181 Current Load on db181 is OK: LOAD OK - total load average: 0.08, 0.17, 9.65 [09:23:58] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [09:25:06] PROBLEM - ns1 Puppet on ns1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [09:25:59] PROBLEM - wiki.lesunk.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.lesunk.com' expires in 7 day(s) (Sat 13 Apr 2024 08:58:25 AM GMT +0000). [09:31:03] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 440.80 ms [09:33:04] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 178.54 ms [09:33:20] PROBLEM - db151 Backups SQL on db151 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864211 seconds old and 117176 bytes [09:40:00] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [09:53:04] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [11:40:12] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:58:22] PROBLEM - franchise.franchising.org.ua - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - franchise.franchising.org.ua All nameservers failed to answer the query. [12:09:20] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [12:24:35] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [12:38:26] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:45:59] PROBLEM - 100acg.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate '100acg.wiki' expires in 7 day(s) (Sat 13 Apr 2024 12:37:36 PM GMT +0000). [12:53:06] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [13:24:35] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [13:25:19] RECOVERY - franchise.franchising.org.ua - reverse DNS on sslhost is OK: SSL OK - franchise.franchising.org.ua reverse DNS resolves to cp37.wikitide.net - CNAME OK [13:44:15] PROBLEM - pyramidgames.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'pyramidgames.wiki' expires in 7 day(s) (Sat 13 Apr 2024 01:26:13 PM GMT +0000). [13:52:13] PROBLEM - wiki.fediverse.kr - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.fediverse.kr' expires in 7 day(s) (Sat 13 Apr 2024 01:31:16 PM GMT +0000). [13:52:42] PROBLEM - www.pyramidgames.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'pyramidgames.wiki' expires in 7 day(s) (Sat 13 Apr 2024 01:26:13 PM GMT +0000). [13:53:08] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [14:08:06] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [14:39:51] PROBLEM - talumn.neptune.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'talumn.neptune.wiki' expires in 7 day(s) (Sat 13 Apr 2024 02:31:46 PM GMT +0000). [14:40:16] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:40:45] PROBLEM - wiki.esnmilanostatale.it - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.esnmilanostatale.it' expires in 7 day(s) (Sat 13 Apr 2024 02:26:38 PM GMT +0000). [14:51:39] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [14:55:55] miraheze/CreateWiki - redbluegreenhat the build has errored. [15:00:08] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [15:09:01] miraheze/CreateWiki - redbluegreenhat the build passed. [15:09:22] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [15:28:49] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 169.53 ms [15:32:53] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.45 ms [15:34:06] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [15:42:46] miraheze/CreateWiki - redbluegreenhat the build passed. [15:52:20] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 10217MiB (10% inode=98%); [15:56:14] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/createCargoDb.php --wiki=evosystemwiki (END - exit=256) [15:56:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:56:37] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/MirahezeMagic/maintenance/createCargoDB.php --wiki=evosystemwiki (END - exit=0) [15:56:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:41:54] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fd3aa5839a7c...527b10c10370 [16:41:55] [02miraheze/puppet] 07Universal-Omega 03527b10c - cvtbot: update description to match hostname [16:50:11] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/527b10c10370...5f722d2c2614 [16:50:14] [02miraheze/puppet] 07Universal-Omega 035f722d2 - Remove unnecessary .git [16:50:45] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/5f722d2c2614...49d7f9f3bb1e [16:50:48] [02miraheze/puppet] 07Universal-Omega 0349d7f9f - mwdeploy: rename miraheze extension pack and add RequestSSL [16:52:04] miraheze/puppet - Universal-Omega the build has errored. [16:53:42] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/49d7f9f3bb1e...473a110710cf [16:53:43] [02miraheze/puppet] 07Universal-Omega 03473a110 - mwdeploy: fix test [16:55:13] miraheze/puppet - Universal-Omega the build passed. [16:56:48] [02miraheze/puppet] 07Universal-Omega created branch 03debug 13https://github.com/miraheze/puppet/commit/473a110710cf2d1db6619eb9262407abe6c160df [16:56:51] [02puppet] 07Universal-Omega created branch 03debug - 13https://github.com/miraheze/puppet [17:04:30] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03debug [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/473a110710cf...d2a2071b140c [17:04:31] [02miraheze/puppet] 07Universal-Omega 03d2a2071 - varnish: rename X-Miraheze-Debug to match hostname [17:04:45] [02puppet] 07Universal-Omega opened pull request 03#3825: varnish: rename X-Miraheze-Debug to match hostname - 13https://github.com/miraheze/puppet/pull/3825 [17:06:01] miraheze/puppet - Universal-Omega the build has errored. [17:33:55] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03params [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/commit/21df970033b7 [17:33:57] [02miraheze/MatomoAnalytics] 07Universal-Omega 0321df970 - MatomoAnalyticsWiki: add option to pass date and support flat param [17:34:00] [02MatomoAnalytics] 07Universal-Omega created branch 03params - 13https://github.com/miraheze/MatomoAnalytics [17:34:01] [02MatomoAnalytics] 07Universal-Omega opened pull request 03#124: MatomoAnalyticsWiki: add option to pass date and support flat param - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:35:34] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#124: MatomoAnalyticsWiki: add option to pass date and support flat param - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:35:36] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03params [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/21df970033b7...2a28b20dff62 [17:35:39] [02miraheze/MatomoAnalytics] 07Universal-Omega 032a28b20 - Fix [17:40:52] miraheze/MatomoAnalytics - Universal-Omega the build passed. [17:46:17] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#124: MatomoAnalyticsWiki: add option to pass date and support flat param - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:46:20] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03params [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/2a28b20dff62...fcc3f8ff1c38 [17:46:22] [02miraheze/MatomoAnalytics] 07Universal-Omega 03fcc3f8f - Add getMostVisistedPages() [17:46:48] [02MatomoAnalytics] 07Universal-Omega edited pull request 03#124: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:49:25] miraheze/MatomoAnalytics - Universal-Omega the build has errored. [17:51:06] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#124: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:51:07] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03params [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/fcc3f8ff1c38...0a5864f934b3 [17:51:09] [02miraheze/MatomoAnalytics] 07Universal-Omega 030a5864f - Fix [17:53:11] [02miraheze/MatomoAnalytics] 07Universal-Omega created branch 03mostvisistedpages 13https://github.com/miraheze/MatomoAnalytics/commit/0a5864f934b3907c0c6690431711d854d26da945 [17:53:12] [02MatomoAnalytics] 07Universal-Omega created branch 03mostvisistedpages - 13https://github.com/miraheze/MatomoAnalytics [17:53:15] [02miraheze/MatomoAnalytics] 07Universal-Omega deleted branch 03params [17:53:18] [02MatomoAnalytics] 07Universal-Omega deleted branch 03params - 13https://github.com/miraheze/MatomoAnalytics [17:53:21] [02MatomoAnalytics] 07Universal-Omega closed pull request 03#124: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/124 [17:53:33] [02MatomoAnalytics] 07Universal-Omega opened pull request 03#125: Mostvisistedpages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [17:55:30] miraheze/MatomoAnalytics - Universal-Omega the build passed. [17:56:24] [02MatomoAnalytics] 07Universal-Omega edited pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [17:58:03] miraheze/MatomoAnalytics - Universal-Omega the build passed. [18:01:43] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/0a5864f934b3...dfc91e72fcd7 [18:01:45] [02miraheze/MatomoAnalytics] 07Universal-Omega 03dfc91e7 - Add [18:01:47] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:05:00] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:05:02] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/dfc91e72fcd7...713d8699b9e7 [18:05:03] [02miraheze/MatomoAnalytics] 07Universal-Omega 03713d869 - Fix [18:08:38] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:08:41] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/713d8699b9e7...c80153d6b61a [18:08:43] [02miraheze/MatomoAnalytics] 07Universal-Omega 03c80153d - Fix [18:11:02] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:11:04] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/c80153d6b61a...3f8164c81e8f [18:11:06] [02miraheze/MatomoAnalytics] 07Universal-Omega 033f8164c - Fix [18:12:25] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:12:28] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/3f8164c81e8f...b87a4a194cea [18:12:30] [02miraheze/MatomoAnalytics] 07Universal-Omega 03b87a4a1 - Fix [18:14:09] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:14:11] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/b87a4a194cea...d6d4e5849867 [18:14:13] [02miraheze/MatomoAnalytics] 07Universal-Omega 03d6d4e58 - - [18:15:14] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:15:15] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/d6d4e5849867...3a6fabbee7b3 [18:15:16] [02miraheze/MatomoAnalytics] 07Universal-Omega 033a6fabb - no flat [18:18:29] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:18:32] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/3a6fabbee7b3...23ea50ef90ab [18:18:33] [02miraheze/MatomoAnalytics] 07Universal-Omega 0323ea50e - - [18:19:33] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:19:35] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/23ea50ef90ab...32246596117f [18:19:36] [02miraheze/MatomoAnalytics] 07Universal-Omega 033224659 - Readd [18:21:10] [02MatomoAnalytics] 07Universal-Omega synchronize pull request 03#125: MatomoAnalyticsWiki: support for getting most visited pages - 13https://github.com/miraheze/MatomoAnalytics/pull/125 [18:21:11] [02miraheze/MatomoAnalytics] 07Universal-Omega pushed 031 commit to 03mostvisistedpages [+0/-0/±1] 13https://github.com/miraheze/MatomoAnalytics/compare/32246596117f...97f58153cfe7 [18:21:12] [02miraheze/MatomoAnalytics] 07Universal-Omega 0397f5815 - + [18:25:43] miraheze/MatomoAnalytics - Universal-Omega the build passed. [18:28:00] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6546e9acf33a...d328c5839f25 [18:28:03] [02miraheze/mw-config] 07Universal-Omega 03d328c58 - Use suffix key again [18:28:08] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [18:28:09] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [18:28:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:28:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:29:03] miraheze/mw-config - Universal-Omega the build passed. [18:30:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/d328c5839f25...ec01e7109317 [18:30:45] [02miraheze/mw-config] 07Universal-Omega 03ec01e71 - Revert [18:31:51] miraheze/mw-config - Universal-Omega the build passed. [18:51:21] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03suffix [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/1c15fe13eaed [18:51:23] [02miraheze/mw-config] 07Universal-Omega 031c15fe1 - Fix getting default server from suffix [18:51:26] [02mw-config] 07Universal-Omega created branch 03suffix - 13https://github.com/miraheze/mw-config [18:51:29] [02mw-config] 07Universal-Omega opened pull request 03#5530: Fix getting default server from suffix - 13https://github.com/miraheze/mw-config/pull/5530 [18:52:05] [02mw-config] 07Universal-Omega synchronize pull request 03#5530: Fix getting default server from suffix - 13https://github.com/miraheze/mw-config/pull/5530 [18:52:06] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03suffix [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1c15fe13eaed...f09c912fc829 [18:52:08] [02miraheze/mw-config] 07Universal-Omega 03f09c912 - Fix [18:52:38] [02mw-config] 07Universal-Omega closed pull request 03#5530: Fix getting default server from suffix - 13https://github.com/miraheze/mw-config/pull/5530 [18:52:40] [02miraheze/mw-config] 07Universal-Omega deleted branch 03suffix [18:52:41] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ec01e7109317...e5e5481d6f6f [18:52:43] [02miraheze/mw-config] 07Universal-Omega 03e5e5481 - Fix getting default server from suffix (#5530) [18:52:45] [02mw-config] 07Universal-Omega deleted branch 03suffix - 13https://github.com/miraheze/mw-config [18:52:58] miraheze/mw-config - Universal-Omega the build passed. [18:53:15] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [18:53:16] !log [universalomega@test151] DEPLOY ABORTED: Canary check failed for meta.mirabeta.org@localhost [18:53:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:53:26] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True, 'force': True} to test151 [18:53:27] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True, 'force': True} to test151 - SUCCESS in 0s [18:53:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:53:33] miraheze/mw-config - Universal-Omega the build passed. [18:53:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:53:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:56:37] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [18:58:01] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [18:58:33] PROBLEM - test151 MediaWiki Rendering on test151 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 5856 bytes in 0.076 second response time [19:00:03] miraheze/CreateWiki - redbluegreenhat the build has errored. [19:00:32] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.376 second response time [19:02:08] !log [@mwtask181] starting deploy of {'config': True} to all [19:02:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:02:17] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [19:02:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:06:53] miraheze/CreateWiki - redbluegreenhat the build passed. [19:07:15] !log [@mwtask171] starting deploy of {'config': True} to all [19:07:23] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [19:07:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:07:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:50:37] [02puppet] 07Universal-Omega edited pull request 03#3825: varnish: rename X-Miraheze-Debug to match hostnames - 13https://github.com/miraheze/puppet/pull/3825 [19:56:45] [02miraheze/MirahezeDebug] 07Universal-Omega created branch 03rebrand 13https://github.com/miraheze/MirahezeDebug/commit/98b027e4b8bcddcdcf1c61b7cd426a56a4c13353 [19:56:47] [02MirahezeDebug] 07Universal-Omega created branch 03rebrand - 13https://github.com/miraheze/MirahezeDebug [19:59:40] [02miraheze/MirahezeDebug] 07Universal-Omega pushed 031 commit to 03rebrand [+0/-0/±7] 13https://github.com/miraheze/MirahezeDebug/compare/98b027e4b8bc...db7956de2795 [19:59:43] [02miraheze/MirahezeDebug] 07Universal-Omega 03db7956d - Rebrand as WikiTideDebug to match hostnames [19:59:58] [02MirahezeDebug] 07Universal-Omega opened pull request 03#13: Rebrand as WikiTideDebug to match hostnames - 13https://github.com/miraheze/MirahezeDebug/pull/13 [20:03:50] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e5e5481d6f6f...b0c56c1dab1d [20:03:53] [02miraheze/mw-config] 07Universal-Omega 03b0c56c1 - Use X-WikiTide-Debug in $wgAllowedCorsHeaders [20:04:10] [02MirahezeDebug] 07Universal-Omega closed pull request 03#13: Rebrand as WikiTideDebug to match hostnames - 13https://github.com/miraheze/MirahezeDebug/pull/13 [20:04:11] [02miraheze/MirahezeDebug] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±7] 13https://github.com/miraheze/MirahezeDebug/compare/98b027e4b8bc...61b05cc61124 [20:04:14] [02miraheze/MirahezeDebug] 07Universal-Omega 0361b05cc - Rebrand as WikiTideDebug to match hostnames [20:04:17] [02miraheze/MirahezeDebug] 07Universal-Omega deleted branch 03rebrand [20:04:19] [02MirahezeDebug] 07Universal-Omega deleted branch 03rebrand - 13https://github.com/miraheze/MirahezeDebug [20:04:43] miraheze/mw-config - Universal-Omega the build passed. [20:04:55] [02puppet] 07Universal-Omega closed pull request 03#3825: varnish: rename X-Miraheze-Debug to match hostnames - 13https://github.com/miraheze/puppet/pull/3825 [20:04:57] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/473a110710cf...70c652ae4910 [20:04:58] [02miraheze/puppet] 07Universal-Omega 0370c652a - varnish: rename X-Miraheze-Debug to match hostnames (#3825) [20:05:01] [02miraheze/puppet] 07Universal-Omega deleted branch 03debug [20:05:03] [02puppet] 07Universal-Omega deleted branch 03debug - 13https://github.com/miraheze/puppet [20:06:12] miraheze/puppet - Universal-Omega the build has errored. [20:07:34] !log [@mwtask171] starting deploy of {'config': True} to all [20:07:42] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [20:07:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:07:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:09:35] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/70c652ae4910...b52fa6c5f946 [20:09:37] [02miraheze/puppet] 07Universal-Omega 03b52fa6c - mariadb: move grants folder to match hostnames [20:17:58] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b52fa6c5f946...da6eff47ce0d [20:18:01] [02miraheze/puppet] 07Universal-Omega 03da6eff4 - fail2ban: rename [20:20:48] [02miraheze/puppet] 07Universal-Omega created branch 03swift 13https://github.com/miraheze/puppet/commit/da6eff47ce0d410cf9ffd3abd53ca4bed1f422d9 [20:20:51] [02puppet] 07Universal-Omega created branch 03swift - 13https://github.com/miraheze/puppet [20:23:30] !log [@test151] starting deploy of {'config': True} to test151 [20:23:31] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [20:23:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:23:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:28:37] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03swift [+4/-4/±2] 13https://github.com/miraheze/puppet/compare/da6eff47ce0d...ced15708ba6e [20:28:39] [02miraheze/puppet] 07Universal-Omega 03ced1570 - swift: rename package [20:28:52] [02puppet] 07Universal-Omega opened pull request 03#3826: swift: rename package - 13https://github.com/miraheze/puppet/pull/3826 [20:30:42] miraheze/puppet - Universal-Omega the build passed. [20:32:18] !log [@mwtask181] starting deploy of {'config': True} to all [20:32:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:32:28] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [20:32:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:52:24] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03thumb_handler-redirect [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/002162eff452 [20:52:26] [02miraheze/mw-config] 07Universal-Omega 03002162e - thumb_handler: redirect default server to primary domain [20:52:27] [02mw-config] 07Universal-Omega created branch 03thumb_handler-redirect - 13https://github.com/miraheze/mw-config [20:52:29] [02mw-config] 07Universal-Omega opened pull request 03#5531: thumb_handler: redirect default server to primary domain - 13https://github.com/miraheze/mw-config/pull/5531 [20:53:15] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [20:53:19] miraheze/mw-config - Universal-Omega the build passed. [21:10:16] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:13:56] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CreateWiki/maintenance/setContainersAccess.php --wiki=solittletimewikiwiki (END - exit=0) [21:14:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:22:00] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [21:39:22] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [22:08:03] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9692MiB (10% inode=98%); [22:15:40] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9715MiB (10% inode=98%); [22:33:43] [02puppet] 07Universal-Omega closed pull request 03#3826: swift: rename package - 13https://github.com/miraheze/puppet/pull/3826 [22:33:44] [02miraheze/puppet] 07Universal-Omega deleted branch 03swift [22:33:45] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+4/-4/±2] 13https://github.com/miraheze/puppet/compare/da6eff47ce0d...4876bf357daf [22:33:48] [02miraheze/puppet] 07Universal-Omega 034876bf3 - swift: rename package (#3826) [22:33:49] [02puppet] 07Universal-Omega deleted branch 03swift - 13https://github.com/miraheze/puppet [22:35:04] miraheze/puppet - Universal-Omega the build passed. [22:41:12] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/4876bf357daf...45225ffd8ae8 [22:41:13] [02miraheze/puppet] 07Universal-Omega 0345225ff - adminlog: update user agent [22:42:28] miraheze/puppet - Universal-Omega the build passed. [22:45:34] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-1/±1] 13https://github.com/miraheze/puppet/compare/45225ffd8ae8...e7e6014795a4 [22:45:35] [02miraheze/puppet] 07Universal-Omega 03e7e6014 - pywikibot: update family [22:46:18] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e7e6014795a4...00e255e901ee [22:46:20] [02miraheze/puppet] 07Universal-Omega 0300e255e - Fix [22:46:55] miraheze/puppet - Universal-Omega the build passed. [22:47:36] miraheze/puppet - Universal-Omega the build passed. [22:50:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/00e255e901ee...a7558570ed7e [22:50:01] [02miraheze/puppet] 07Universal-Omega 03a755857 - iaupload: update default subject [22:51:23] miraheze/puppet - Universal-Omega the build passed. [22:52:31] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a7558570ed7e...49fb451b8ba9 [22:52:34] [02miraheze/puppet] 07Universal-Omega 0349fb451 - mwdeploy: fix choice [22:53:53] miraheze/puppet - Universal-Omega the build passed. [22:55:12] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/49fb451b8ba9...274bb01f3ccc [22:55:15] [02miraheze/puppet] 07Universal-Omega 03274bb01 - Update comment [23:00:29] [02miraheze/puppet] 07Universal-Omega created branch 03mcrouter 13https://github.com/miraheze/puppet/commit/274bb01f3cccc129b48504067f00441ef73b1e93 [23:00:31] [02puppet] 07Universal-Omega created branch 03mcrouter - 13https://github.com/miraheze/puppet [23:01:21] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03mcrouter [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/274bb01f3ccc...10d2640e2586 [23:01:23] [02miraheze/puppet] 07Universal-Omega 0310d2640 - mcrouter: rename route [23:01:35] [02puppet] 07Universal-Omega opened pull request 03#3827: mcrouter: rename route - 13https://github.com/miraheze/puppet/pull/3827 [23:05:27] [02puppet] 07Universal-Omega closed pull request 03#3827: mcrouter: rename route - 13https://github.com/miraheze/puppet/pull/3827 [23:05:30] [02miraheze/puppet] 07Universal-Omega deleted branch 03mcrouter [23:05:33] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/274bb01f3ccc...c49f19681e7b [23:05:35] [02miraheze/puppet] 07Universal-Omega 03c49f196 - mcrouter: rename route (#3827) [23:05:37] [02puppet] 07Universal-Omega deleted branch 03mcrouter - 13https://github.com/miraheze/puppet [23:10:24] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:39:27] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK