[00:00:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:00:46] [02ImportDump] 07Universal-Omega closed pull request 03#78: Add ImportDumpHookRunnerTest - 13https://github.com/miraheze/ImportDump/pull/78 [00:00:48] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03ImportDumpHookRunnerTest [00:00:50] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ImportDump/compare/22702b2a01c2...a26b3f9c1889 [00:00:52] [02miraheze/ImportDump] 07Universal-Omega 03a26b3f9 - Add ImportDumpHookRunnerTest (#78) [00:00:55] [02ImportDump] 07Universal-Omega deleted branch 03ImportDumpHookRunnerTest - 13https://github.com/miraheze/ImportDump [00:02:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:03:52] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21296MiB (24% inode=98%); [00:04:12] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21350MiB (24% inode=98%); [00:06:20] RECOVERY - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is OK: OK - NGINX Error Rate is 18% [00:06:26] miraheze/ImportDump - Universal-Omega the build passed. [00:06:56] [02ImportDump] 07Universal-Omega closed pull request 03#79: Migrate to use InsertQueryBuilder - 13https://github.com/miraheze/ImportDump/pull/79 [00:06:58] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03InsertQueryBuilder [00:07:00] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/ImportDump/compare/a26b3f9c1889...5e24a11759a9 [00:07:03] [02miraheze/ImportDump] 07Universal-Omega 035e24a11 - Migrate to use InsertQueryBuilder (#79) [00:07:04] [02ImportDump] 07Universal-Omega deleted branch 03InsertQueryBuilder - 13https://github.com/miraheze/ImportDump [00:14:51] miraheze/ImportDump - Universal-Omega the build passed. [00:19:40] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03UpdateQueryBuilder [+0/-0/±1] 13https://github.com/miraheze/ImportDump/commit/497e6b8db96f [00:19:43] [02miraheze/ImportDump] 07Universal-Omega 03497e6b8 - Migrate to use UpdateQueryBuilder [00:19:45] [02ImportDump] 07Universal-Omega created branch 03UpdateQueryBuilder - 13https://github.com/miraheze/ImportDump [00:19:46] [02ImportDump] 07Universal-Omega opened pull request 03#80: Migrate to use UpdateQueryBuilder - 13https://github.com/miraheze/ImportDump/pull/80 [00:27:19] miraheze/ImportDump - Universal-Omega the build passed. [00:27:31] [02ImportDump] 07Universal-Omega closed pull request 03#80: Migrate to use UpdateQueryBuilder - 13https://github.com/miraheze/ImportDump/pull/80 [00:27:32] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/5e24a11759a9...0b21ca5c7f26 [00:27:35] [02miraheze/ImportDump] 07Universal-Omega 030b21ca5 - Migrate to use UpdateQueryBuilder (#80) [00:27:38] [02ImportDump] 07Universal-Omega deleted branch 03UpdateQueryBuilder - 13https://github.com/miraheze/ImportDump [00:27:39] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03UpdateQueryBuilder [00:28:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 43% [00:32:53] [02miraheze/ImportDump] 07Universal-Omega pushed 034 commits to 03rename-tables [+2/-0/±7] 13https://github.com/miraheze/ImportDump/compare/5c9eab3318ed...df1f53cfcfb4 [00:32:56] [02miraheze/ImportDump] 07Universal-Omega 03df1f53c - Merge branch 'master' into rename-tables [00:32:59] [02ImportDump] 07Universal-Omega synchronize pull request 03#77: Rename importdump_request* tables - 13https://github.com/miraheze/ImportDump/pull/77 [00:36:09] miraheze/ImportDump - Universal-Omega the build passed. [00:38:31] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/0b21ca5c7f26...0bde5abc56e3 [00:38:32] [02miraheze/ImportDump] 07Universal-Omega 030bde5ab - Add Unit to TestAutoloadNamespaces [00:41:32] miraheze/ImportDump - Universal-Omega the build passed. [00:47:08] miraheze/ImportDump - Universal-Omega the build passed. [01:04:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [01:15:17] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [01:27:29] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/commit/1e5447ab3ca8 [01:27:31] [02miraheze/CreateWiki] 07Universal-Omega 031e5447a - Fix CI for 1.42 [01:27:33] [02CreateWiki] 07Universal-Omega created branch 03fix-ci-1.42 - 13https://github.com/miraheze/CreateWiki [01:27:36] [02CreateWiki] 07Universal-Omega opened pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [01:36:11] miraheze/CreateWiki - Universal-Omega the build passed. [01:36:42] PROBLEM - phorge171 Current Load on phorge171 is WARNING: LOAD WARNING - total load average: 1.98, 1.45, 0.85 [01:38:42] PROBLEM - phorge171 Current Load on phorge171 is CRITICAL: LOAD CRITICAL - total load average: 2.09, 1.63, 0.99 [01:40:42] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 0.66, 1.22, 0.92 [01:49:53] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [01:49:54] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/1e5447ab3ca8...7a05aeada921 [01:49:57] [02miraheze/CreateWiki] 07Universal-Omega 037a05aea - Fix [01:57:48] miraheze/CreateWiki - Universal-Omega the build passed. [01:58:47] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [01:58:50] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/7a05aeada921...7018f1ba516d [01:58:52] [02miraheze/CreateWiki] 07Universal-Omega 037018f1b - Fix [02:01:14] PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[libboost-regex1.74.0-icu72] [02:07:30] miraheze/CreateWiki - Universal-Omega the build passed. [02:09:44] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:09:47] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/7018f1ba516d...dcb7c02a8db9 [02:09:49] [02miraheze/CreateWiki] 07Universal-Omega 03dcb7c02 - Try using addDBDataOnce [02:17:27] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:17:28] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/dcb7c02a8db9...a2c2be7e7a90 [02:17:29] [02miraheze/CreateWiki] 07Universal-Omega 03a2c2be7 - Reduce duplication [02:18:08] miraheze/CreateWiki - Universal-Omega the build passed. [02:18:23] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/a2c2be7e7a90...88598c4f346f [02:18:25] [02miraheze/CreateWiki] 07Universal-Omega 0388598c4 - Ensure wikidb is added [02:18:28] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:26:44] miraheze/CreateWiki - Universal-Omega the build passed. [02:29:14] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:31:47] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03fix-ci-1.42 [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/88598c4f346f...694850d6c2cb [02:31:50] [02miraheze/CreateWiki] 07Universal-Omega 03694850d - Remove duplicate entry in @dataProvider [02:31:51] [02CreateWiki] 07Universal-Omega synchronize pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:34:38] [02CreateWiki] 07Universal-Omega edited pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:35:52] [02CreateWiki] 07Universal-Omega edited pull request 03#484: Fix CI for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:37:07] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 332.33 ms [02:38:14] [02CreateWiki] 07Universal-Omega edited pull request 03#484: Fix tests for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:39:22] PROBLEM - ping6 on cp28 is CRITICAL: PING CRITICAL - Packet loss = 0%, RTA = 318.21 ms [02:39:26] PROBLEM - ping6 on cp41 is WARNING: PING WARNING - Packet loss = 0%, RTA = 329.32 ms [02:39:36] miraheze/CreateWiki - Universal-Omega the build passed. [02:39:53] PROBLEM - ping6 on ns2 is CRITICAL: PING CRITICAL - Packet loss = 0%, RTA = 313.78 ms [02:40:15] [02CreateWiki] 07Universal-Omega closed pull request 03#484: Fix tests for 1.42 - 13https://github.com/miraheze/CreateWiki/pull/484 [02:40:17] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/CreateWiki/compare/b438b6891807...087b35765732 [02:40:19] [02miraheze/CreateWiki] 07Universal-Omega 03087b357 - Fix tests for 1.42 (#484) [02:40:20] [02miraheze/CreateWiki] 07Universal-Omega deleted branch 03fix-ci-1.42 [02:40:21] [02CreateWiki] 07Universal-Omega deleted branch 03fix-ci-1.42 - 13https://github.com/miraheze/CreateWiki [02:41:16] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 162.61 ms [02:41:25] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 151.71 ms [02:41:56] RECOVERY - ping6 on ns2 is OK: PING OK - Packet loss = 0%, RTA = 145.41 ms [02:42:43] [02puppet] 07Universal-Omega closed pull request 03#3769: build(deps-dev): bump yard from 0.9.35 to 0.9.36 in /modules/graylog - 13https://github.com/miraheze/puppet/pull/3769 [02:42:44] [02miraheze/puppet] 07Universal-Omega deleted branch 03dependabot/bundler/modules/graylog/yard-0.9.36 [02:42:46] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3b5e436c070d...c032ad64fd67 [02:42:48] [02miraheze/puppet] 07dependabot[bot] 03c032ad6 - build(deps-dev): bump yard from 0.9.35 to 0.9.36 in /modules/graylog (#3769) [02:42:50] [02puppet] 07Universal-Omega deleted branch 03dependabot/bundler/modules/graylog/yard-0.9.36 - 13https://github.com/miraheze/puppet [02:43:26] RECOVERY - ping6 on cp28 is OK: PING OK - Packet loss = 0%, RTA = 155.66 ms [02:43:52] [02CreateWiki] 07Universal-Omega closed pull request 03#482: deleteWiki: give fatalError if WikiManager::delete has output - 13https://github.com/miraheze/CreateWiki/pull/482 [02:43:53] [02miraheze/CreateWiki] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/087b35765732...c5be15fa7eec [02:43:54] [02miraheze/CreateWiki] 07Universal-Omega 03c5be15f - deleteWiki: give fatalError if WikiManager::delete has output (#482) [02:43:56] [02miraheze/CreateWiki] 07Universal-Omega deleted branch 03deletewikis-delete-output [02:43:58] [02CreateWiki] 07Universal-Omega deleted branch 03deletewikis-delete-output - 13https://github.com/miraheze/CreateWiki [02:46:05] [02miraheze/CreateWiki] 07Universal-Omega pushed 0346 commits to 03subdomains [+4/-0/±180] 13https://github.com/miraheze/CreateWiki/compare/2538e4811213...af2a1f2c2af6 [02:46:07] [02miraheze/CreateWiki] 07Universal-Omega 03af2a1f2 - Merge branch 'master' into subdomains [02:46:08] [02CreateWiki] 07Universal-Omega synchronize pull request 03#463: Support additional subdomains - 13https://github.com/miraheze/CreateWiki/pull/463 [02:48:51] [02.github] 07Universal-Omega closed pull request 03#46: Bump actions/cache from 4.0.0 to 4.0.1 - 13https://github.com/miraheze/.github/pull/46 [02:48:52] [02miraheze/.github] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/.github/compare/64df55693f4f...8a00ac8eb122 [02:48:55] [02miraheze/.github] 07dependabot[bot] 038a00ac8 - Bump actions/cache from 4.0.0 to 4.0.1 (#46) [02:48:57] [02miraheze/.github] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-4.0.1 [02:48:58] [02.github] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-4.0.1 - 13https://github.com/miraheze/.github [02:51:36] miraheze/CreateWiki - Universal-Omega the build passed. [02:51:43] miraheze/CreateWiki - Universal-Omega the build has errored. [03:07:49] PROBLEM - wiki.pulsus.cc - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.pulsus.cc' expires in 15 day(s) (Mon 18 Mar 2024 02:42:15 AM GMT +0000). [03:08:03] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/42375ed0d8cd...78b51732074a [03:08:04] [02miraheze/ssl] 07MirahezeSSLBot 0378b5173 - Bot: Update SSL cert for wiki.pulsus.cc [03:37:42] RECOVERY - wiki.pulsus.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.pulsus.cc' will expire on Fri 31 May 2024 02:07:57 AM GMT +0000. [04:08:43] PROBLEM - phorge171 Current Load on phorge171 is CRITICAL: LOAD CRITICAL - total load average: 5.83, 2.81, 1.46 [04:12:42] PROBLEM - phorge171 Current Load on phorge171 is WARNING: LOAD WARNING - total load average: 0.51, 1.77, 1.38 [04:14:42] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 0.32, 1.30, 1.25 [04:43:47] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 179.73 ms [04:49:44] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 179.12 ms [04:57:47] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 180.61 ms [05:01:47] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 178.33 ms [05:05:52] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 178.41 ms [05:11:50] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 167.48 ms [05:15:54] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 176.61 ms [05:17:53] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 182.82 ms [05:48:33] I am Beast Gohan [05:48:48] me too [05:49:00] Yeah [05:54:08] !log inserted slot_roles and content_models for simpleelectronicswiki [05:54:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:58:42] PROBLEM - phorge171 Current Load on phorge171 is CRITICAL: LOAD CRITICAL - total load average: 4.31, 2.58, 1.37 [06:00:42] PROBLEM - phorge171 Current Load on phorge171 is WARNING: LOAD WARNING - total load average: 1.05, 1.89, 1.26 [06:02:42] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 0.48, 1.39, 1.15 [06:34:52] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 171.36 ms [06:36:51] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 181.10 ms [06:48:48] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 183.92 ms [06:50:46] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 180.66 ms [07:16:55] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 193.25 ms [07:20:56] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 193.71 ms [07:25:05] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 194.00 ms [07:37:14] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 168.52 ms [07:41:18] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 205.81 ms [07:45:19] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 174.34 ms [08:39:30] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 173.09 ms [08:41:28] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 179.39 ms [09:27:35] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 177.21 ms [09:29:34] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 172.85 ms [09:59:19] [02mw-config] 07redbluegreenhat opened pull request 03#5494: Fix managewiki-core perm for Stewards on metawikibeta - 13https://github.com/miraheze/mw-config/pull/5494 [10:00:17] miraheze/mw-config - redbluegreenhat the build passed. [10:01:49] [02mw-config] 07Universal-Omega closed pull request 03#5494: Fix managewiki-core perm for Stewards on metawikibeta - 13https://github.com/miraheze/mw-config/pull/5494 [10:01:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/df251d25b861...4595901edcdc [10:01:53] [02miraheze/mw-config] 07redbluegreenhat 034595901 - Fix managewiki-core perm for Stewards on metawikibeta (#5494) [10:02:06] !log [@mwtask181] starting deploy of {'config': True} to all [10:02:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:02:16] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [10:02:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:02:41] miraheze/mw-config - Universal-Omega the build passed. [10:04:33] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 182.19 ms [10:07:33] !log [@mwtask171] starting deploy of {'config': True} to all [10:07:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:07:41] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [10:07:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:08:32] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 182.04 ms [10:13:50] !log [alex@test151] starting deploy of {'config': True} to test151 [10:13:51] !log [alex@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [10:13:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:13:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:17:57] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [10:19:34] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 174.48 ms [10:21:32] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 178.73 ms [10:47:35] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [10:56:21] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 171.18 ms [10:58:20] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 182.74 ms [11:15:19] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 176.47 ms [11:17:18] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 180.95 ms [11:21:24] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 194.79 ms [11:23:22] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 185.40 ms [11:26:21] PROBLEM - lebork.info.pl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lebork.info.pl' expires in 15 day(s) (Mon 18 Mar 2024 11:11:52 AM GMT +0000). [11:26:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/78b51732074a...da36c41981f8 [11:26:34] [02miraheze/ssl] 07MirahezeSSLBot 03da36c41 - Bot: Update SSL cert for lebork.info.pl [11:29:26] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 188.34 ms [11:33:24] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 180.42 ms [11:55:11] RECOVERY - lebork.info.pl - LetsEncrypt on sslhost is OK: OK - Certificate 'lebork.info.pl' will expire on Fri 31 May 2024 10:26:26 AM GMT +0000. [12:27:01] RECOVERY - wiki.ooer.ooo - reverse DNS on sslhost is OK: SSL OK - wiki.ooer.ooo reverse DNS resolves to cp37.wikitide.net - CNAME OK [12:27:08] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 188.85 ms [12:29:44] RECOVERY - wiki.ooer.ooo - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ooer.ooo' will expire on Sun 24 Mar 2024 03:34:42 PM GMT +0000. [12:33:05] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 181.84 ms [13:12:11] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 37%, RTA = 186.62 ms [13:16:08] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 183.97 ms [13:24:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [13:34:20] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 184.02 ms [13:42:16] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 185.37 ms [13:54:36] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [14:05:34] PROBLEM - wiki.sukotuze.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.sukotuze.com' expires in 15 day(s) (Mon 18 Mar 2024 01:47:16 PM GMT +0000). [14:05:46] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/da36c41981f8...b383e0aacb6a [14:05:49] [02miraheze/ssl] 07MirahezeSSLBot 03b383e0a - Bot: Update SSL cert for wiki.sukotuze.com [14:15:30] [Grafana] !sre FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:26:24] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 188.60 ms [14:28:55] PROBLEM - dlfm-wiki.top - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'dlfm-wiki.top' expires in 15 day(s) (Mon 18 Mar 2024 01:57:56 PM GMT +0000). [14:29:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b383e0aacb6a...e100cb59d03d [14:29:08] [02miraheze/ssl] 07MirahezeSSLBot 03e100cb5 - Bot: Update SSL cert for dlfm-wiki.top [14:31:44] PROBLEM - o.kaoyu666.cn - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'o.kaoyu666.cn' expires in 15 day(s) (Mon 18 Mar 2024 02:26:14 PM GMT +0000). [14:32:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e100cb59d03d...ebf31d762442 [14:32:20] [02miraheze/ssl] 07MirahezeSSLBot 03ebf31d7 - Bot: Update SSL cert for o.kaoyu666.cn [14:32:21] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 186.93 ms [14:34:38] RECOVERY - wiki.sukotuze.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.sukotuze.com' will expire on Fri 31 May 2024 01:05:39 PM GMT +0000. [14:35:30] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [14:40:24] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 188.62 ms [14:42:22] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 187.15 ms [14:58:49] RECOVERY - dlfm-wiki.top - LetsEncrypt on sslhost is OK: OK - Certificate 'dlfm-wiki.top' will expire on Fri 31 May 2024 01:29:00 PM GMT +0000. [15:01:30] RECOVERY - o.kaoyu666.cn - LetsEncrypt on sslhost is OK: OK - Certificate 'o.kaoyu666.cn' will expire on Fri 31 May 2024 01:32:11 PM GMT +0000. [15:04:35] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 193.34 ms [15:12:31] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 192.44 ms [15:18:35] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 191.12 ms [15:20:33] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 191.57 ms [15:30:42] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 191.96 ms [15:38:39] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 123.70 ms [15:39:29] !log [salt-user@mwtask181] Renamed czgdtwiki to csgamedevtycoonwiki using renamewiki.py [15:39:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:48:31] [02RequestSSL] 07redbluegreenhat opened pull request 03#40: T11699 Create a job to automate CNAME checks - 13https://github.com/miraheze/RequestSSL/pull/40 [15:48:38] [02RequestSSL] 07redbluegreenhat edited pull request 03#40: T11699: Create a job to automate CNAME checks - 13https://github.com/miraheze/RequestSSL/pull/40 [15:51:43] [02RequestSSL] 07redbluegreenhat synchronize pull request 03#40: T11699: Create a job to automate CNAME checks - 13https://github.com/miraheze/RequestSSL/pull/40 [15:52:15] miraheze/RequestSSL - redbluegreenhat the build has errored. [15:54:50] miraheze/RequestSSL - redbluegreenhat the build has errored. [16:27:45] @reception123 can you update RequestSSL on production? [16:27:50] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 187.95 ms [16:29:31] it seems to be broken, can't reproduce on Mirabeta which is running the latest master [16:29:48] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 188.81 ms [16:32:16] let me check [16:32:20] you mean just on mwtask? [16:32:36] Without looking at the stacktrace, I think it is because of CreateWiki now requiring the hook runner on the constructor [16:32:42] @reception123 yes, on mwtask [16:33:46] !log [reception@mwtask181] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [16:33:47] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [16:33:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:33:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:17] Failed to upgrade RequestSSL (exit code: 128). [16:35:10] hmm [16:35:11] fatal: detected dubious ownership in repository at '/srv/mediawiki-staging/1.41/extensions/RequestSSL' [16:35:40] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [16:35:41] !log [alex@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [16:35:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:35:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:35:52] huh, getting the same thing here [16:36:09] !log [reception@mwtask181] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [16:36:10] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [16:36:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:57] @bluemoon0332 I even tried removing the extension and manual git clone and same thing [16:37:17] not being able to upgrade is pretty bad [16:37:36] @cosmicalpha ^ [16:41:13] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'WikiDiscover'} to test151 [16:41:14] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'WikiDiscover'} to test151 - SUCCESS in 1s [16:41:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:41:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:41:30] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [16:41:31] !log [alex@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [16:41:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:41:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:41:39] some work, at least [16:41:48] on test151, that is [16:43:51] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 187.14 ms [16:44:36] Got a new error of the week [16:47:49] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 182.28 ms [16:48:43] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:52:44] @reception123 I see what the problem is on mwtask [16:53:16] :squint: [16:53:18] !log [reception@mwtask181] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [16:53:19] !log [reception@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [16:53:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:53:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:53:28] now exit code 1! [16:54:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [16:55:01] whoooo [16:55:07] /srv/mediawiki-staging/1.41/extensions/RequestSSL is owned by root:root where as any other extension is owned by www-data:www-data [16:55:23] let me fix that [16:55:30] Mwdeploy runs as www-data thus the ownership error [16:56:23] yeah that's my fault when I was testing it but the error was there before [16:56:26] I chown'd it back and same thing [16:57:41] @reception123 not quite [16:57:58] You changed the root directory of tve extension [16:58:09] You need to use -R to do recursively [17:02:48] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 179.16 ms [17:04:37] Everything in the RequestSSL directory is still owned by root [17:09:20] !log [macfan@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [17:09:20] !log [macfan@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [17:09:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:09:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:10:44] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 177.98 ms [17:10:59] !log [macfan@test151] starting deploy of {'force_upgrade': True, 'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [17:11:00] !log [macfan@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [17:11:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:11:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:14:50] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 177.81 ms [17:17:08] On test151 createwiki seems to have merge conflicts [17:17:38] [02ImportDump] 07Universal-Omega closed pull request 03#76: Use request-import for RateLimits - 13https://github.com/miraheze/ImportDump/pull/76 [17:17:41] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ImportDump/compare/0bde5abc56e3...50078613e84c [17:17:42] [02miraheze/ImportDump] 07Universal-Omega 035007861 - Use request-import for RateLimits (#76) [17:17:43] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03ratelimits [17:17:44] [02ImportDump] 07Universal-Omega deleted branch 03ratelimits - 13https://github.com/miraheze/ImportDump [17:18:23] [02ImportDump] 07Universal-Omega closed pull request 03#75: Add SpecialPageAliases for RequestImport/RequestImportQueue - 13https://github.com/miraheze/ImportDump/pull/75 [17:18:26] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03alias [17:18:29] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/50078613e84c...67c215536a53 [17:18:30] [02miraheze/ImportDump] 07Universal-Omega 0367c2155 - Add SpecialPageAliases for RequestImport/RequestImportQueue (#75) [17:18:32] [02ImportDump] 07Universal-Omega deleted branch 03alias - 13https://github.com/miraheze/ImportDump [17:18:47] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 177.76 ms [17:22:02] !log [macfan@test151] starting deploy of {'force_upgrade': True, 'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 [17:22:03] !log [macfan@test151] finished deploy of {'force_upgrade': True, 'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to test151 - SUCCESS in 0s [17:22:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:22:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:22:27] With the conflicts fixed now that works fine [17:23:54] [02miraheze/ImportDump] 07Universal-Omega created branch 03rename-rights 13https://github.com/miraheze/ImportDump/commit/67c215536a53c95467cb1c82cabbcd5756147fee [17:23:56] [02ImportDump] 07Universal-Omega created branch 03rename-rights - 13https://github.com/miraheze/ImportDump [17:24:36] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [17:24:57] For prod it is still the permissions problem I mentioned earlier [17:26:11] miraheze/ImportDump - Universal-Omega the build passed. [17:29:06] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03rename-rights [+0/-0/±27] 13https://github.com/miraheze/ImportDump/compare/67c215536a53...a5a93525bfb6 [17:29:07] [02miraheze/ImportDump] 07Universal-Omega 03a5a9352 - Rename import-dump rights [17:29:20] [02ImportDump] 07Universal-Omega opened pull request 03#81: Rename import-dump rights - 13https://github.com/miraheze/ImportDump/pull/81 [17:32:51] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 173.20 ms [17:33:26] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03systemusers [+0/-0/±1] 13https://github.com/miraheze/ImportDump/commit/2ef2c8df75cc [17:33:27] [02miraheze/ImportDump] 07Universal-Omega 032ef2c8d - Add RequestImport system users [17:33:29] [02ImportDump] 07Universal-Omega created branch 03systemusers - 13https://github.com/miraheze/ImportDump [17:33:31] [02ImportDump] 07Universal-Omega opened pull request 03#82: Add RequestImport system users - 13https://github.com/miraheze/ImportDump/pull/82 [17:34:32] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03systemusers [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/2ef2c8df75cc...44cd948bb35f [17:34:34] [02miraheze/ImportDump] 07Universal-Omega 0344cd948 - Add [17:34:35] [02ImportDump] 07Universal-Omega synchronize pull request 03#82: Add RequestImport system users - 13https://github.com/miraheze/ImportDump/pull/82 [17:34:50] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 179.68 ms [17:37:46] miraheze/ImportDump - Universal-Omega the build passed. [17:42:12] miraheze/ImportDump - Universal-Omega the build passed. [17:42:29] [02ImportDump] 07Universal-Omega closed pull request 03#82: Add RequestImport system users - 13https://github.com/miraheze/ImportDump/pull/82 [17:42:31] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ImportDump/compare/67c215536a53...f35279625cce [17:42:34] [02miraheze/ImportDump] 07Universal-Omega 03f352796 - Add RequestImport system users (#82) [17:42:38] [02miraheze/ImportDump] 07Universal-Omega deleted branch 03systemusers [17:42:41] [02ImportDump] 07Universal-Omega deleted branch 03systemusers - 13https://github.com/miraheze/ImportDump [17:44:56] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 176.35 ms [17:45:48] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03rename-rights [+0/-0/±21] 13https://github.com/miraheze/ImportDump/compare/a5a93525bfb6...8e7af4a003d9 [17:45:49] [02miraheze/ImportDump] 07Universal-Omega 038e7af4a - Use handle-import-request-interwiki [17:45:52] [02ImportDump] 07Universal-Omega synchronize pull request 03#81: Rename import-dump rights - 13https://github.com/miraheze/ImportDump/pull/81 [17:45:55] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03rename-rights [+0/-0/±0] 13https://github.com/miraheze/ImportDump/compare/8e7af4a003d9...259f459314fb [17:45:58] [02miraheze/ImportDump] 07Universal-Omega 03259f459 - Use handle-import-request-interwiki [17:45:59] [02ImportDump] 07Universal-Omega synchronize pull request 03#81: Rename import-dump rights - 13https://github.com/miraheze/ImportDump/pull/81 [17:46:15] [02miraheze/ImportDump] 07Universal-Omega pushed 032 commits to 03rename-rights [+0/-0/±48] 13https://github.com/miraheze/ImportDump/compare/259f459314fb...882e47053051 [17:46:17] [02miraheze/ImportDump] 07Universal-Omega 035da4ab6 - Rename import-dump rights [17:46:18] [02miraheze/ImportDump] 07Universal-Omega 03882e470 - Use handle-import-request-interwiki [17:46:21] [02ImportDump] 07Universal-Omega synchronize pull request 03#81: Rename import-dump rights - 13https://github.com/miraheze/ImportDump/pull/81 [17:47:56] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [17:48:54] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 173.68 ms [17:50:22] miraheze/ImportDump - Universal-Omega the build passed. [17:54:37] miraheze/ImportDump - Universal-Omega the build passed. [18:02:15] PROBLEM - ping6 on cp28 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 156.72 ms [18:04:17] RECOVERY - ping6 on cp28 is OK: PING OK - Packet loss = 0%, RTA = 149.71 ms [20:00:52] PROBLEM - cp28 Disk Space on cp28 is CRITICAL: connect to address 2001:470:25:715::2 port 5666: No route to hostconnect to host 2001:470:25:715::2 port 5666: No route to host [20:02:47] PROBLEM - cp28 Disk Space on cp28 is WARNING: DISK WARNING - free space: / 30232MiB (6% inode=99%); [20:24:35] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [20:54:36] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['NS.ANKH.FR.eu.org.', 'NS1.eu.org.', 'NS1.ERIOMEM.NET.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [21:12:31] turns out the problem that prompted me to request upgrading RequestSSL on prod is cause by RequestSSL [21:12:58] how fun [21:15:54] Special:Version on Mirabeta says it is running a commit from January 31 for some reason [21:16:28] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to test151 [21:16:29] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to test151 - SUCCESS in 0s [21:16:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:16:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:18:13] [1/18] ``` [21:18:14] [2/18] [eaeabf0c4360adb392f50c73] /wiki/Special:RequestSSLQueue/8 TypeError: Miraheze\CreateWiki\RemoteWiki::__construct(): Argument #2 ($hookRunner) must be of type Miraheze\CreateWiki\Hooks\CreateWikiHookRunner, null given, called in /srv/mediawiki/1.41/extensions/RequestSSL/includes/RequestSSLManager.php on line 365 [21:18:14] [3/18] Backtrace: [21:18:14] [4/18] from /srv/mediawiki/1.41/extensions/CreateWiki/includes/RemoteWiki.php(35) [21:18:14] [5/18] #0 /srv/mediawiki/1.41/extensions/RequestSSL/includes/RequestSSLManager.php(365): Miraheze\CreateWiki\RemoteWiki->__construct(string, NULL) [21:18:15] [6/18] #1 /srv/mediawiki/1.41/extensions/RequestSSL/includes/RequestSSLViewer.php(56): Miraheze\RequestSSL\RequestSSLManager->isPrivate() [21:18:15] [7/18] #2 /srv/mediawiki/1.41/extensions/RequestSSL/includes/RequestSSLViewer.php(403): Miraheze\RequestSSL\RequestSSLViewer->getFormDescriptor() [21:18:15] [8/18] #3 /srv/mediawiki/1.41/extensions/RequestSSL/includes/Specials/SpecialRequestSSLQueue.php(137): Miraheze\RequestSSL\RequestSSLViewer->getForm(integer) [21:18:16] [9/18] #4 /srv/mediawiki/1.41/extensions/RequestSSL/includes/Specials/SpecialRequestSSLQueue.php(55): Miraheze\RequestSSL\Specials\SpecialRequestSSLQueue->lookupRequest(string) [21:18:16] [10/18] #5 /srv/mediawiki/1.41/includes/specialpage/SpecialPage.php(727): Miraheze\RequestSSL\Specials\SpecialRequestSSLQueue->execute(string) [21:18:16] [11/18] #6 /srv/mediawiki/1.41/includes/specialpage/SpecialPageFactory.php(1621): MediaWiki\SpecialPage\SpecialPage->run(string) [21:18:17] [12/18] #7 /srv/mediawiki/1.41/includes/MediaWiki.php(357): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext) [21:18:17] [13/18] #8 /srv/mediawiki/1.41/includes/MediaWiki.php(960): MediaWiki->performRequest() [21:18:18] [14/18] #9 /srv/mediawiki/1.41/includes/MediaWiki.php(613): MediaWiki->main() [21:18:18] [15/18] #10 /srv/mediawiki/config/initialise/entrypoints/index.php(100): MediaWiki->run() [21:18:19] [16/18] #11 /srv/mediawiki/config/initialise/entrypoints/index.php(95): wfIndexMain() [21:18:19] [17/18] #12 {main} [21:18:20] [18/18] ``` [21:18:20] that's some bizarre stuff [21:19:27] because the constructor for RequestSSLManager requires an instance of CreateWikiHookRunner, if that wasn't being passed this would fail before RemoteWiki [21:19:49] Oh [21:19:50] I see it [21:20:10] [1/25] ``` [21:20:10] [2/25] public function __construct( [21:20:10] [3/25] Config $config, [21:20:11] [4/25] ActorStoreFactory $actorStoreFactory, [21:20:11] [5/25] CreateWikiHookRunner $createWikiHookRunner, [21:20:11] [6/25] ILBFactory $dbLoadBalancerFactory, [21:20:11] [7/25] LinkRenderer $linkRenderer, [21:20:12] [8/25] RepoGroup $repoGroup, [21:20:12] [9/25] MessageLocalizer $messageLocalizer, [21:20:12] [10/25] ServiceOptions $options, [21:20:13] [11/25] UserFactory $userFactory, [21:20:13] [12/25] UserGroupManagerFactory $userGroupManagerFactory [21:20:14] [13/25] ) { [21:20:14] [14/25] $options->assertRequiredOptions( self::CONSTRUCTOR_OPTIONS ); [21:20:15] [15/25] $this->config = $config; [21:20:15] [16/25] $this->actorStoreFactory = $actorStoreFactory; [21:20:16] [17/25] $this->dbLoadBalancerFactory = $dbLoadBalancerFactory; [21:20:16] [18/25] $this->linkRenderer = $linkRenderer; [21:20:17] [19/25] $this->messageLocalizer = $messageLocalizer; [21:20:17] [20/25] $this->options = $options; [21:20:18] [21/25] $this->repoGroup = $repoGroup; [21:20:18] [22/25] $this->userFactory = $userFactory; [21:20:19] [23/25] $this->userGroupManagerFactory = $userGroupManagerFactory; [21:20:19] [24/25] } [21:20:20] [25/25] ``` [21:20:23] Forgot to store the instance of CreateWikiHookRunner in the class [21:27:14] [02RequestSSL] 07redbluegreenhat opened pull request 03#41: Fix CreateWikiHookRunner on RequestSSLManager - 13https://github.com/miraheze/RequestSSL/pull/41 [21:27:19] [02RequestSSL] 07redbluegreenhat closed pull request 03#41: Fix CreateWikiHookRunner on RequestSSLManager - 13https://github.com/miraheze/RequestSSL/pull/41 [21:27:22] [02miraheze/RequestSSL] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RequestSSL/compare/c4c115ce7ce8...157cec6e4781 [21:27:24] !log [alex@test151] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to test151 [21:27:24] [02miraheze/RequestSSL] 07redbluegreenhat 03157cec6 - Fix CreateWikiHookRunner on RequestSSLManager (#41) [21:27:25] !log [alex@test151] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to test151 - SUCCESS in 1s [21:27:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:27:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:27:56] @Site Reliability Engineers Just fixed RequestSSL, please upgrade the extension on prod as soon as possible [21:30:23] miraheze/RequestSSL - redbluegreenhat the build has errored. [21:30:29] miraheze/RequestSSL - redbluegreenhat the build has errored. [22:55:52] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9700MiB (10% inode=98%); [22:56:12] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9705MiB (10% inode=98%);