[00:00:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:01:19] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.44, 19.18, 17.60 [00:01:20] RECOVERY - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is OK: OK - NGINX Error Rate is 0% [00:01:23] PROBLEM - mw171 Current Load on mw171 is CRITICAL: LOAD CRITICAL - total load average: 30.00, 22.98, 21.47 [00:02:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 50% [00:04:01] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 9655MiB (10% inode=98%); [00:04:06] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21525MiB (24% inode=98%); [00:04:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [00:05:12] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.69, 22.78, 19.52 [00:08:01] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 22920MiB (24% inode=98%); [00:12:58] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.83, 23.25, 20.74 [00:14:15] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 24.41, 19.49, 16.96 [00:16:46] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 23.13, 20.00, 16.62 [00:18:41] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 20.11, 20.20, 17.11 [00:19:31] PROBLEM - mw161 Current Load on mw161 is CRITICAL: LOAD CRITICAL - total load average: 25.22, 22.85, 18.41 [00:21:25] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 21.64, 22.22, 18.68 [00:22:35] PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 24.76, 23.96, 19.42 [00:24:31] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 21.69, 23.69, 19.89 [00:25:56] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 20.10, 22.53, 20.48 [00:27:24] PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 25.15, 20.50, 17.04 [00:27:24] PROBLEM - mw161 Current Load on mw161 is CRITICAL: LOAD CRITICAL - total load average: 26.95, 22.93, 19.91 [00:27:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [00:27:56] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 26.82, 24.46, 21.45 [00:28:23] PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 26.00, 24.20, 20.84 [00:29:04] PROBLEM - eventgate181 Puppet on eventgate181 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 29 minutes ago with 0 failures [00:29:22] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 12.14, 17.22, 16.26 [00:29:24] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 15.84, 20.86, 19.57 [00:29:56] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 18.73, 22.40, 21.08 [00:30:19] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 18.43, 21.44, 20.21 [00:31:24] PROBLEM - mw161 Current Load on mw161 is CRITICAL: LOAD CRITICAL - total load average: 25.04, 21.80, 20.01 [00:31:56] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 27.57, 24.13, 21.84 [00:33:24] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 23.99, 22.70, 20.57 [00:33:56] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 18.60, 22.51, 21.56 [00:35:24] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 15.34, 19.64, 19.69 [00:36:08] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 16.00, 19.01, 19.75 [00:37:56] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 14.71, 18.79, 20.29 [00:38:22] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [00:40:45] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 8.35, 16.77, 22.13 [00:41:22] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 9.65, 17.28, 23.45 [00:42:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [00:44:11] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 10.16, 15.87, 22.70 [00:44:45] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 8.42, 12.18, 19.00 [00:45:22] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 9.06, 12.74, 20.16 [00:48:11] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 9.25, 12.04, 19.57 [01:02:02] PROBLEM - eventgate181 APT on eventgate181 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (1 critical updates). [01:04:00] RECOVERY - eventgate181 APT on eventgate181 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [01:07:04] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [01:18:27] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [01:29:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 40% [01:36:13] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [01:47:12] PROBLEM - kafka181 ferm_active on kafka181 is CRITICAL: ERROR ferm input drop default policy not set, ferm might not have been started correctly [01:48:11] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [01:54:00] [02miraheze/eventgate-deploy] 07Universal-Omega pushed 032 commits to 03master [+5/-7/±272] 13https://github.com/miraheze/eventgate-deploy/compare/a8c8f213cd19...ff21de4f5023 [01:54:02] [02miraheze/eventgate-deploy] 07Universal-Omega 037fd2ade - Fix againsudo -u eventgate git config --global user.email cosmic.alpha24@gmail.com! [01:54:04] [02miraheze/eventgate-deploy] 07Universal-Omega 03ff21de4 - Fix againsudo -u eventgate git add .! [02:05:06] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e766897afd66...7bcc60dc7ea3 [02:05:07] [02miraheze/puppet] 07Universal-Omega 037bcc60d - Kafka: add firewall rule for eventgate [02:06:36] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7bcc60dc7ea3...fcf00f8fa017 [02:06:37] [02miraheze/puppet] 07Universal-Omega 03fcf00f8 - eventgate: fix Kafka broker connection [02:09:04] RECOVERY - eventgate181 Puppet on eventgate181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:09:12] RECOVERY - kafka181 ferm_active on kafka181 is OK: OK ferm input default policy is set [02:10:31] [02miraheze/mediawiki-repos] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mediawiki-repos/compare/5d7390219b5d...5c178c1de3f5 [02:10:33] [02miraheze/mediawiki-repos] 07Universal-Omega 035c178c1 - install EventBus [02:18:20] !log [universalomega@mwtask181] starting deploy of {'world': True, 'extension_list': True, 'versions': '1.41'} to all [02:18:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:19:44] !log [universalomega@mwtask181] finished deploy of {'world': True, 'extension_list': True, 'versions': '1.41'} to all - SUCCESS in 83s [02:19:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:19:55] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:21:19] !log [universalomega@mwtask181] starting deploy of {'world': True, 'extension_list': True, 'versions': '1.41'} to all [02:21:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:21:27] !log [universalomega@test151] starting deploy of {'world': True, 'extension_list': True, 'versions': ['1.41', '1.42']} to test151 [02:21:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:21:38] !log [universalomega@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [02:21:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:22:18] !log [universalomega@mwtask181] finished deploy of {'world': True, 'extension_list': True, 'versions': '1.41'} to all - SUCCESS in 59s [02:22:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:24:11] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fcf00f8fa017...f2644a348260 [02:24:12] [02miraheze/puppet] 07Universal-Omega 03f2644a3 - eventgate: add firewall rule for icinga [02:25:26] !log [universalomega@test151] starting deploy of {'world': True, 'extension_list': True, 'versions': ['1.41', '1.42']} to test151 [02:25:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:26:04] !log [universalomega@test151] finished deploy of {'world': True, 'extension_list': True, 'versions': ['1.41', '1.42']} to test151 - SUCCESS in 38s [02:26:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:27:02] RECOVERY - eventgate181 eventgate on eventgate181 is OK: TCP OK - 0.000 second response time on 10.0.18.147 port 8192 [02:28:03] !log [universalomega@test151] starting deploy of {'l10n': True, 'versions': ['1.41', '1.42']} to test151 [02:28:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:28:24] !log [universalomega@mwtask181] starting deploy of {'l10n': True, 'versions': '1.41'} to all [02:28:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:35:39] !log [universalomega@mwtask181] finished deploy of {'l10n': True, 'versions': '1.41'} to all - SUCCESS in 434s [02:35:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:37:41] !log [universalomega@test151] finished deploy of {'l10n': True, 'versions': ['1.41', '1.42']} to test151 - SUCCESS in 577s [02:37:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:42:23] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03eventbus-beta [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/e2f40c68305d [02:42:24] [02miraheze/mw-config] 07Universal-Omega 03e2f40c6 - Use EventBus for jobs on beta [02:42:25] [02mw-config] 07Universal-Omega created branch 03eventbus-beta - 13https://github.com/miraheze/mw-config [02:42:27] [02mw-config] 07Universal-Omega opened pull request 03#5506: Use EventBus for jobs on beta - 13https://github.com/miraheze/mw-config/pull/5506 [02:42:52] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03eventbus-beta [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e2f40c68305d...c49d409ad308 [02:42:54] [02miraheze/mw-config] 07github-actions 03c49d409 - CI: lint code to MediaWiki standards [02:42:56] [02mw-config] 07github-actions[bot] synchronize pull request 03#5506: Use EventBus for jobs on beta - 13https://github.com/miraheze/mw-config/pull/5506 [02:43:24] miraheze/mw-config - Universal-Omega the build passed. [02:43:28] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f2644a348260...1f6f49d89af1 [02:43:32] [02miraheze/puppet] 07Universal-Omega 031f6f49d - eventgate: add MediaWiki to firewall [02:46:57] [02mw-config] 07Universal-Omega synchronize pull request 03#5506: Use EventBus for jobs on beta - 13https://github.com/miraheze/mw-config/pull/5506 [02:46:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03eventbus-beta [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c49d409ad308...126bc397c91c [02:46:59] [02miraheze/mw-config] 07Universal-Omega 03126bc39 - Reduce timeout to 5 [02:47:42] [02mw-config] 07Universal-Omega synchronize pull request 03#5506: Use EventBus for jobs on beta - 13https://github.com/miraheze/mw-config/pull/5506 [02:47:43] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03eventbus-beta [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/126bc397c91c...7261f065fc37 [02:47:44] [02miraheze/mw-config] 07Universal-Omega 037261f06 - Also enable on testwiki [02:47:55] miraheze/mw-config - Universal-Omega the build passed. [02:47:55] [02mw-config] 07Universal-Omega edited pull request 03#5506: Use EventBus for jobs on beta and testwiki - 13https://github.com/miraheze/mw-config/pull/5506 [02:48:23] [02mw-config] 07Universal-Omega synchronize pull request 03#5506: Use EventBus for jobs on beta and testwiki - 13https://github.com/miraheze/mw-config/pull/5506 [02:48:25] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03eventbus-beta [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/7261f065fc37...80701daac3c0 [02:48:27] [02miraheze/mw-config] 07Universal-Omega 0380701da - + [02:48:38] miraheze/mw-config - Universal-Omega the build passed. [02:49:18] miraheze/mw-config - Universal-Omega the build passed. [02:49:37] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [02:49:48] [02mw-config] 07Universal-Omega closed pull request 03#5506: Use EventBus for jobs on beta and testwiki - 13https://github.com/miraheze/mw-config/pull/5506 [02:49:49] [02miraheze/mw-config] 07Universal-Omega deleted branch 03eventbus-beta [02:49:50] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/e11655d347d0...bcee9bbf2746 [02:49:51] [02miraheze/mw-config] 07Universal-Omega 03bcee9bb - Use EventBus for jobs on beta and testwiki (#5506) [02:49:54] [02mw-config] 07Universal-Omega deleted branch 03eventbus-beta - 13https://github.com/miraheze/mw-config [02:50:08] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [02:50:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:19] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [02:50:19] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [02:50:20] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [02:50:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [02:50:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:50:43] miraheze/mw-config - Universal-Omega the build passed. [02:58:16] PROBLEM - wiki.buryland.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.buryland.net' expires in 15 day(s) (Sun 07 Apr 2024 02:39:34 AM GMT +0000). [02:58:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/811fc269a7dd...1b887404743d [02:58:30] [02miraheze/ssl] 07MirahezeSSLBot 031b88740 - Bot: Update SSL cert for wiki.buryland.net [02:59:19] miraheze/ssl - MirahezeSSLBot the build has errored. [03:01:11] RECOVERY - db162 Backups SQL on db162 is OK: FILE_AGE OK: /var/log/sql-backup.log is 57 seconds old and 1218 bytes [03:01:19] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03fix-eventgate [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/407cc46695b0 [03:01:21] [02miraheze/mw-config] 07Universal-Omega 03407cc46 - Fix EventGate stream configuration [03:01:22] [02mw-config] 07Universal-Omega created branch 03fix-eventgate - 13https://github.com/miraheze/mw-config [03:01:25] [02mw-config] 07Universal-Omega opened pull request 03#5507: Fix EventGate stream configuration - 13https://github.com/miraheze/mw-config/pull/5507 [03:02:16] miraheze/mw-config - Universal-Omega the build passed. [03:03:28] [02mw-config] 07Universal-Omega closed pull request 03#5507: Fix EventGate stream configuration - 13https://github.com/miraheze/mw-config/pull/5507 [03:03:30] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/bcee9bbf2746...5eb1e5802c64 [03:03:31] [02miraheze/mw-config] 07Universal-Omega 035eb1e58 - Fix EventGate stream configuration (#5507) [03:03:34] [02miraheze/mw-config] 07Universal-Omega deleted branch 03fix-eventgate [03:03:36] [02mw-config] 07Universal-Omega deleted branch 03fix-eventgate - 13https://github.com/miraheze/mw-config [03:03:52] !log [universalomega@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [03:03:53] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s [03:03:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:00] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [03:04:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:10] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [03:04:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:04:26] miraheze/mw-config - Universal-Omega the build passed. [03:06:20] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5eb1e5802c64...08bf2669b165 [03:06:21] [02miraheze/mw-config] 07Universal-Omega 0308bf266 - - [03:07:12] miraheze/mw-config - Universal-Omega the build passed. [03:07:19] !log [@mwtask171] starting deploy of {'config': True} to all [03:07:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:07:27] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [03:07:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:10:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [03:18:21] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/08bf2669b165...39ee87acebf0 [03:18:24] [02miraheze/mw-config] 07Universal-Omega 0339ee87a - Add EventBus logging [03:19:19] miraheze/mw-config - Universal-Omega the build passed. [03:19:21] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:23:39] !log [@test151] starting deploy of {'config': True} to test151 [03:23:40] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [03:23:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:23:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:28:13] RECOVERY - wiki.buryland.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.buryland.net' will expire on Thu 20 Jun 2024 01:58:22 AM GMT +0000. [03:28:35] [02mw-config] 07Universal-Omega closed pull request 03#5505: Disable Matomo for genshinimpactwiki - 13https://github.com/miraheze/mw-config/pull/5505 [03:28:36] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/39ee87acebf0...6584120d3d29 [03:28:39] [02miraheze/mw-config] 07colleirose 036584120 - Disable Matomo for genshinimpactwiki (#5505) [03:29:30] miraheze/mw-config - Universal-Omega the build passed. [03:33:44] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6584120d3d29...f59628a252c2 [03:33:47] [02miraheze/mw-config] 07Universal-Omega 03f59628a - Fix classes [03:34:43] miraheze/mw-config - Universal-Omega the build passed. [03:37:42] !log [@mwtask171] starting deploy of {'config': True} to all [03:37:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:37:51] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [03:37:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:40:11] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [03:53:10] !log [@test151] starting deploy of {'config': True} to test151 [03:53:11] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [03:53:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:53:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [04:08:49] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [04:13:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [04:15:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [05:56:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/1b887404743d...924e8818cbee [05:56:35] [02miraheze/ssl] 07MirahezeSSLBot 03924e881 - Bot: Add SSL cert for wiki.blackorchestra.net [05:57:24] miraheze/ssl - MirahezeSSLBot the build has errored. [06:02:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/924e8818cbee...3ddf9be65e2a [06:02:23] [02miraheze/ssl] 07MirahezeSSLBot 033ddf9be - Bot: Add SSL cert for aryavratpedia.co [06:03:20] miraheze/ssl - MirahezeSSLBot the build has errored. [06:49:24] PROBLEM - db151 Backups SQL on db151 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864122 seconds old and 68505 bytes [12:09:51] [02CreateWiki] 07redbluegreenhat opened pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:10:37] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:13:33] miraheze/CreateWiki - redbluegreenhat the build has errored. [12:14:26] miraheze/CreateWiki - redbluegreenhat the build has errored. [12:14:51] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:18:43] miraheze/CreateWiki - redbluegreenhat the build has errored. [12:21:07] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:24:42] miraheze/CreateWiki - redbluegreenhat the build has errored. [12:54:08] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:55:06] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [12:57:53] miraheze/CreateWiki - redbluegreenhat the build has errored. [12:58:37] miraheze/CreateWiki - redbluegreenhat the build has errored. [13:06:59] [02CreateWiki] 07redbluegreenhat synchronize pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487 [13:14:01] [02CreateWiki] 07redbluegreenhat commented on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015077546 [13:14:51] [02CreateWiki] 07redbluegreenhat edited a comment on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015077546 [13:14:57] miraheze/CreateWiki - redbluegreenhat the build passed. [13:17:40] [02CreateWiki] 07redbluegreenhat commented on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015083482 [13:19:10] [02CreateWiki] 07redbluegreenhat edited a comment on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015083482 [13:22:51] [02CreateWiki] 07redbluegreenhat deleted a comment on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015083482 [13:37:44] [02CreateWiki] 07redbluegreenhat commented on pull request 03#487: T10683: Expose the canned responses to JavaScript - 13https://github.com/miraheze/CreateWiki/pull/487#issuecomment-2015124129 [14:21:39] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [14:21:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:25:59] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 169.57 ms [14:27:59] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.68 ms [15:36:27] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [16:05:47] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [16:13:40] PROBLEM - wiki.elysiamc.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.elysiamc.xyz' expires in 15 day(s) (Sun 07 Apr 2024 03:46:22 PM GMT +0000). [16:13:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3ddf9be65e2a...75ab6181c9f0 [16:13:57] [02miraheze/ssl] 07MirahezeSSLBot 0375ab618 - Bot: Update SSL cert for wiki.elysiamc.xyz [16:14:51] miraheze/ssl - MirahezeSSLBot the build has errored. [16:17:44] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=thenamaworld /home/reception/themanaworld.xml (START) [16:17:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:18:45] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=thenamaworld /home/reception/themanaworld.xml (END - exit=2) [16:18:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:19:24] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=thenamaworld /home/reception/themanaworld.xml --report 1 (START) [16:19:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:10] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=thenamaworld /home/reception/themanaworld.xml --report 1 (END - exit=2) [16:20:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:22] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [16:20:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:20:42] restarted again because of wrong --username-prefix [16:28:15] PROBLEM - wiki.worldx.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.worldx.org' expires in 15 day(s) (Sun 07 Apr 2024 04:16:04 PM GMT +0000). [16:28:27] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/75ab6181c9f0...db0b361a8daa [16:28:28] [02miraheze/ssl] 07MirahezeSSLBot 03db0b361 - Bot: Update SSL cert for wiki.worldx.org [16:29:23] miraheze/ssl - MirahezeSSLBot the build has errored. [16:31:59] [02miraheze/mw-config] 07OAuthority pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/f59628a252c2...42252c11c7a5 [16:32:00] [02miraheze/mw-config] 07OAuthority 0342252c1 - don't use jobqueue on beta [16:32:47] !log [@mwtask181] starting deploy of {'config': True} to all [16:32:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:57] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 10s [16:32:58] miraheze/mw-config - OAuthority the build passed. [16:33:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:05] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [16:34:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:41] lock wait timeout exceeded [16:34:46] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [16:34:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:04] [02miraheze/mw-config] 07OAuthority pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/42252c11c7a5...dc42a0f5f01a [16:36:06] [02miraheze/mw-config] 07OAuthority 03dc42a0f - maybe this is correct [16:36:55] miraheze/mw-config - OAuthority the build passed. [16:37:05] !log [@mwtask171] starting deploy of {'config': True} to all [16:37:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:37:13] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [16:37:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:44:43] PROBLEM - www.moiswiki.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.moiswiki.net' expires in 15 day(s) (Sun 07 Apr 2024 04:24:02 PM GMT +0000). [16:48:20] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [16:48:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:48:41] this is going to take a while... [16:48:41] PROBLEM - themagical.world - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'themagical.world' expires in 15 day(s) (Sun 07 Apr 2024 04:38:48 PM GMT +0000). [16:48:51] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [16:48:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:53:27] !log [@test151] starting deploy of {'config': True} to test151 [16:53:28] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [16:53:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:53:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:58:00] RECOVERY - wiki.worldx.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.worldx.org' will expire on Thu 20 Jun 2024 03:28:21 PM GMT +0000. [16:58:43] PROBLEM - www.themagical.world - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'themagical.world' expires in 15 day(s) (Sun 07 Apr 2024 04:38:48 PM GMT +0000). [17:02:22] !log [@mwtask181] starting deploy of {'config': True} to all [17:02:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:02:32] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [17:02:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:03:10] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [17:03:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:03:51] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [17:03:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:06:31] maybe it's just a bad time to stress the database like this, maybe a lot of wikis on that db just happen to be editing at the same time as the import? [17:07:41] Why does it keep dying? [17:07:52] hitting the lock wait timeout [17:07:57] hmm [17:08:01] what db is it on? [17:08:20] @bluemoon0332 that's not normal [17:08:35] it's on cluster c3... [17:09:38] so db171 @agentisai [17:12:13] RECOVERY - wiki.elysiamc.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.elysiamc.xyz' will expire on Thu 20 Jun 2024 03:13:46 PM GMT +0000. [17:18:00] i really hate that they're called clusters [17:19:41] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [17:19:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:20:17] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [17:20:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:23:19] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/26f3e0fc053a [17:23:22] [02miraheze/mw-config] 07Universal-Omega 0326f3e0f - Remove use of $wi->hostname [17:23:24] [02mw-config] 07Universal-Omega created branch 03no-hostname-var - 13https://github.com/miraheze/mw-config [17:23:25] [02mw-config] 07Universal-Omega opened pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:23:49] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/26f3e0fc053a...d826e7321174 [17:23:51] [02miraheze/mw-config] 07github-actions 03d826e73 - CI: lint code to MediaWiki standards [17:23:54] [02mw-config] 07github-actions[bot] synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:24:20] miraheze/mw-config - Universal-Omega the build passed. [17:24:44] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/d826e7321174...cf6b8ce0c670 [17:24:45] [02miraheze/mw-config] 07Universal-Omega 03cf6b8ce - Remove [17:24:46] [02mw-config] 07Universal-Omega synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:25:42] miraheze/mw-config - Universal-Omega the build has errored. [17:26:08] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/cf6b8ce0c670...06cb368d18c2 [17:26:10] [02miraheze/mw-config] 07Universal-Omega 0306cb368 - Use server [17:26:12] [02mw-config] 07Universal-Omega synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:26:59] miraheze/mw-config - Universal-Omega the build passed. [17:28:41] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/06cb368d18c2...21e1578d1982 [17:28:42] [02miraheze/mw-config] 07Universal-Omega 0321e1578 - Remove [17:28:45] [02mw-config] 07Universal-Omega synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:29:37] miraheze/mw-config - Universal-Omega the build passed. [17:33:08] [02mw-config] 07Universal-Omega synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:33:10] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/21e1578d1982...3ccc3176e218 [17:33:11] [02miraheze/mw-config] 07Universal-Omega 033ccc317 - Fix [17:34:00] miraheze/mw-config - Universal-Omega the build passed. [17:34:55] [02mw-config] 07Universal-Omega synchronize pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:34:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03no-hostname-var [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3ccc3176e218...e28c45d930a3 [17:35:01] [02miraheze/mw-config] 07Universal-Omega 03e28c45d - Only set $wgMFStopRedirectCookieHost if MobileFrontend is enabled [17:36:00] miraheze/mw-config - Universal-Omega the build passed. [17:36:39] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [17:36:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:37:42] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [17:37:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:46:44] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/b60dca5a34f0...78a450f31f30 [17:46:46] [02miraheze/ErrorPages] 07Universal-Omega 0378a450f - DeletedWiki: set no-cache [17:47:24] miraheze/ErrorPages - Universal-Omega the build passed. [17:48:45] [02miraheze/mw-config] 07Universal-Omega deleted branch 03no-hostname-var [17:48:46] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±4] 13https://github.com/miraheze/mw-config/compare/dc42a0f5f01a...bc36adf7317b [17:48:49] [02miraheze/mw-config] 07Universal-Omega 03bc36adf - Remove use of $wi->hostname (#5508) [17:48:52] [02mw-config] 07Universal-Omega deleted branch 03no-hostname-var - 13https://github.com/miraheze/mw-config [17:48:55] [02mw-config] 07Universal-Omega closed pull request 03#5508: Remove use of $wi->hostname - 13https://github.com/miraheze/mw-config/pull/5508 [17:49:41] miraheze/mw-config - Universal-Omega the build passed. [17:53:25] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [17:53:27] !log [@test151] starting deploy of {'config': True} to test151 [17:53:28] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [17:53:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:53:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:53:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:54:18] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [17:54:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:07:15] !log [@mwtask171] starting deploy of {'config': True} to all [18:07:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:07:24] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [18:07:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:32] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [18:10:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:54] didn't even last half an hour... [18:11:42] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [18:11:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:27:47] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [18:27:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:28:35] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [18:28:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:30:28] @Site Reliability Engineers, @MediaWiki Engineers: Someone else wants to take over baby sitting this import after it inevitably times out again? [18:44:39] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=256) [18:44:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:46:24] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [18:46:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:01:14] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1f6f49d89af1...3b5e11014d25 [19:01:16] [02miraheze/puppet] 07Universal-Omega 033b5e110 - eventgate: update config [19:07:14] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3b5e11014d25...11ef1084b33e [19:07:16] [02miraheze/puppet] 07Universal-Omega 0311ef108 - eventgate: set schema_base_uris to schema.wikimedia.org temporarily [19:07:31] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:14:36] I can take over now 🙂 [19:14:53] well now it doesn't want to stop smg [19:22:25] [02MirahezeMagic] 07amire80 opened pull request 03#483: Improve the message managewiki-setting-wgAllowSlowParserFunctions-help - 13https://github.com/miraheze/MirahezeMagic/pull/483 [19:24:32] [02MirahezeMagic] 07Universal-Omega closed pull request 03#483: Improve the message managewiki-setting-wgAllowSlowParserFunctions-help - 13https://github.com/miraheze/MirahezeMagic/pull/483 [19:24:34] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/8c82427d1f2b...0372a365454a [19:24:36] [02miraheze/MirahezeMagic] 07amire80 030372a36 - Improve the message managewiki-setting-wgAllowSlowParserFunctions-help (#483) [19:26:02] miraheze/MirahezeMagic - amire80 the build has errored. [19:28:04] miraheze/MirahezeMagic - Universal-Omega the build has errored. [19:32:27] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [19:32:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:32:36] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 9s [19:32:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:35:51] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (END - exit=2) [19:35:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:36:48] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [19:37:11]