[00:01:41] [02python-functions] 07dependabot[bot] created branch 03dependabot/pip/dot-github/setuptools-75.6.0 - 13https://github.com/miraheze/python-functions [00:01:44] [02python-functions] 07dependabot[bot] pushed 031 commit to 03dependabot/pip/dot-github/setuptools-75.6.0 [+0/-0/±1] 13https://github.com/miraheze/python-functions/commit/8efc4cd25e95 [00:01:47] [02python-functions] 07dependabot[bot] 038efc4cd - Bump setuptools from 75.5.0 to 75.6.0 in /.github [00:01:49] [02python-functions] 07dependabot[bot] labeled pull request 03#103: Bump setuptools from 75.5.0 to 75.6.0 in /.github - 13https://github.com/miraheze/python-functions/pull/103 [00:01:52] [02python-functions] 07dependabot[bot] opened pull request 03#103: Bump setuptools from 75.5.0 to 75.6.0 in /.github - 13https://github.com/miraheze/python-functions/pull/103 [00:01:55] [02python-functions] 07dependabot[bot] labeled pull request 03#103: Bump setuptools from 75.5.0 to 75.6.0 in /.github - 13https://github.com/miraheze/python-functions/pull/103 [00:01:58] [02python-functions] 07coderabbitai[bot] commented on pull request 03#103: Bump setuptools from 75.5.0 to 75.6.0 in /.github - 13https://github.com/miraheze/python-functions/pull/103#issuecomment-2489785978 [00:05:32] miraheze/python-functions - dependabot[bot] the build passed. [00:07:02] [02python-functions] 07github-merge-queue[bot] created branch 03gh-readonly-queue/master/pr-103-157dce94eea0128b96b60d5d444fa9ad694724f1 - 13https://github.com/miraheze/python-functions [00:07:03] [02python-functions] 07github-merge-queue[bot] pushed 031 commit to 03gh-readonly-queue/master/pr-103-157dce94eea0128b96b60d5d444fa9ad694724f1 [+0/-0/±1] 13https://github.com/miraheze/python-functions/commit/6fa9544111e6 [00:07:04] [02python-functions] 07dependabot[bot] 036fa9544 - Bump setuptools from 75.5.0 to 75.6.0 in /.github (#103) [00:10:32] [02python-functions] 07github-merge-queue[bot] pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/python-functions/compare/157dce94eea0...6fa9544111e6 [00:10:33] [02python-functions] 07github-merge-queue[bot] deleted branch 03gh-readonly-queue/master/pr-103-157dce94eea0128b96b60d5d444fa9ad694724f1 - 13https://github.com/miraheze/python-functions [00:10:35] [02python-functions] 07github-actions[bot] closed pull request 03#103: Bump setuptools from 75.5.0 to 75.6.0 in /.github - 13https://github.com/miraheze/python-functions/pull/103 [00:10:37] [02python-functions] 07github-merge-queue[bot] deleted branch 03gh-readonly-queue/master/pr-103-157dce94eea0128b96b60d5d444fa9ad694724f1 [00:10:40] [02python-functions] 07github-actions[bot] deleted branch 03dependabot/pip/dot-github/setuptools-75.6.0 - 13https://github.com/miraheze/python-functions [00:10:41] [02python-functions] 07github-actions[bot] deleted branch 03dependabot/pip/dot-github/setuptools-75.6.0 [00:10:52] [02python-functions] 07dependabot[bot] edited pull request 03#102: Bump langcodes from 3.4.1 to 3.5.0 in /.github - 13https://github.com/miraheze/python-functions/pull/102 [00:10:55] [02python-functions] 07dependabot[bot] edited pull request 03#100: Bump langcodes from 3.4.1 to 3.5.0 - 13https://github.com/miraheze/python-functions/pull/100 [00:11:34] [02python-functions] 07dependabot[bot] pushed 031 commit to 03dependabot/pip/dot-github/langcodes-3.5.0 [+0/-0/±1] 13https://github.com/miraheze/python-functions/compare/b7a8bff2a735...a3a862ec2886 [00:11:35] [02python-functions] 07dependabot[bot] 03a3a862e - Bump langcodes from 3.4.1 to 3.5.0 in /.github [00:11:37] [02python-functions] 07dependabot[bot] edited pull request 03#102: Bump langcodes from 3.4.1 to 3.5.0 in /.github - 13https://github.com/miraheze/python-functions/pull/102 [00:11:38] [02python-functions] 07dependabot[bot] synchronize pull request 03#102: Bump langcodes from 3.4.1 to 3.5.0 in /.github - 13https://github.com/miraheze/python-functions/pull/102 [00:11:41] [02python-functions] 07dependabot[bot] edited pull request 03#100: Bump langcodes from 3.4.1 to 3.5.0 - 13https://github.com/miraheze/python-functions/pull/100 [00:11:43] [02python-functions] 07dependabot[bot] pushed 031 commit to 03dependabot/pip/langcodes-3.5.0 [+0/-0/±1] 13https://github.com/miraheze/python-functions/compare/6db6ac3ddc0f...fbb1ec4d0d60 [00:11:45] [02python-functions] 07dependabot[bot] 03fbb1ec4 - Bump langcodes from 3.4.1 to 3.5.0 [00:11:46] [02python-functions] 07dependabot[bot] synchronize pull request 03#100: Bump langcodes from 3.4.1 to 3.5.0 - 13https://github.com/miraheze/python-functions/pull/100 [00:14:51] miraheze/python-functions - dependabot[bot] the build has errored. [00:14:59] miraheze/python-functions - github-merge-queue[bot] the build passed. [00:15:08] miraheze/python-functions - dependabot[bot] the build has errored. [00:22:16] [02DataDump] 07Universal-Omega created branch 03failed-comments - 13https://github.com/miraheze/DataDump [00:22:19] [02DataDump] 07Universal-Omega pushed 031 commit to 03failed-comments [+0/-0/±1] 13https://github.com/miraheze/DataDump/commit/aa9609981508 [00:22:20] [Grafana] FIRING: The mediawiki job queue has more than 2000 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [00:22:22] [02DataDump] 07Universal-Omega 03aa96099 - Allow custom handling of failed status comments [00:22:25] [02DataDump] 07Universal-Omega opened pull request 03#92: Allow custom handling of failed status comments - 13https://github.com/miraheze/DataDump/pull/92 [00:22:27] [02DataDump] 07coderabbitai[bot] commented on pull request 03#92: Allow custom handling of failed status comments - 13https://github.com/miraheze/DataDump/pull/92#issuecomment-2489812007 [00:24:44] [02MirahezeMagic] 07Universal-Omega pushed 031 commit to 03swiftdump-storage [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/commit/3de258964fcb [00:24:45] [02MirahezeMagic] 07Universal-Omega 033de2589 - SwiftDump: use more reliable storage check [00:24:47] [02MirahezeMagic] 07Universal-Omega created branch 03swiftdump-storage - 13https://github.com/miraheze/MirahezeMagic [00:24:49] [02MirahezeMagic] 07Universal-Omega opened pull request 03#538: SwiftDump: use more reliable storage check - 13https://github.com/miraheze/MirahezeMagic/pull/538 [00:24:56] [02MirahezeMagic] 07coderabbitai[bot] commented on pull request 03#538: SwiftDump: use more reliable storage check - 13https://github.com/miraheze/MirahezeMagic/pull/538#issuecomment-2489814825 [00:26:09] [02MirahezeMagic] 07Universal-Omega edited pull request 03#538: SwiftDump: use more reliable storage check - 13https://github.com/miraheze/MirahezeMagic/pull/538 [00:27:10] miraheze/DataDump - Universal-Omega the build passed. [00:28:09] miraheze/MirahezeMagic - Universal-Omega the build has errored. [00:30:51] [02MirahezeMagic] 07Universal-Omega synchronize pull request 03#538: SwiftDump: use more reliable storage check - 13https://github.com/miraheze/MirahezeMagic/pull/538 [00:30:51] [02MirahezeMagic] 07Universal-Omega pushed 031 commit to 03swiftdump-storage [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/3de258964fcb...ec1d824c8550 [00:30:52] [02MirahezeMagic] 07Universal-Omega 03ec1d824 - Use int [00:39:27] miraheze/MirahezeMagic - Universal-Omega the build passed. [00:57:20] [Grafana] RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [01:10:07] [02DataDump] 07Universal-Omega created branch 03service - 13https://github.com/miraheze/DataDump [01:10:10] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+1/-1/±0] 13https://github.com/miraheze/DataDump/commit/7ef22243eff9 [01:10:13] [02DataDump] 07Universal-Omega 037ef2224 - Convert DataDump class into a DataDumpFileBackend service [01:10:15] [02DataDump] 07Universal-Omega opened pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:10:19] [02DataDump] 07coderabbitai[bot] commented on pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93#issuecomment-2489859912 [01:10:46] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:10:49] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/7ef22243eff9...54f90108ea89 [01:10:50] [02DataDump] 07Universal-Omega 0354f9010 - Fix [01:11:02] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 1 backends are down. mw151 [01:13:02] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 29 backends are healthy [01:13:30] miraheze/DataDump - Universal-Omega the build has errored. [01:14:10] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+1/-0/±0] 13https://github.com/miraheze/DataDump/compare/54f90108ea89...c9d68a9c6f57 [01:14:13] [02DataDump] 07Universal-Omega 03c9d68a9 - Add [01:14:14] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:14:56] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/c9d68a9c6f57...b3db5c696afa [01:14:57] [02DataDump] 07Universal-Omega 03b3db5c6 - Add [01:14:59] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:17:38] miraheze/DataDump - Universal-Omega the build has errored. [01:18:14] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/b3db5c696afa...a5c5a46b14e0 [01:18:15] [02DataDump] 07Universal-Omega 03a5c5a46 - Add [01:18:17] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:19:39] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/a5c5a46b14e0...2175c92e8f47 [01:19:41] [02DataDump] 07Universal-Omega 032175c92 - Rename method [01:19:42] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:21:43] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/2175c92e8f47...a22821bdf59f [01:21:46] [02DataDump] 07Universal-Omega 03a22821b - Update [01:21:48] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:24:25] miraheze/DataDump - Universal-Omega the build has errored. [01:25:05] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/a22821bdf59f...5b2f1b6e2164 [01:25:06] [02DataDump] 07Universal-Omega 035b2f1b6 - Update [01:25:08] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:27:38] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/5b2f1b6e2164...b85ea8713aae [01:27:39] [02DataDump] 07Universal-Omega 03b85ea87 - Update [01:27:41] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:27:45] miraheze/DataDump - Universal-Omega the build has errored. [01:29:10] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/b85ea8713aae...b935c694d530 [01:29:13] [02DataDump] 07Universal-Omega 03b935c69 - Update [01:29:14] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:29:50] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/b935c694d530...f810a2cac7e6 [01:29:53] [02DataDump] 07Universal-Omega 03f810a2c - Update [01:29:55] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:30:38] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/f810a2cac7e6...4c20db7bd56f [01:30:41] [02DataDump] 07Universal-Omega 034c20db7 - Update [01:30:44] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:32:51] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±7] 13https://github.com/miraheze/DataDump/compare/4c20db7bd56f...06e4ab2f5878 [01:32:53] [02DataDump] 07Universal-Omega 0306e4ab2 - Rename method [01:32:54] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:33:19] miraheze/DataDump - Universal-Omega the build has errored. [01:35:19] [02DataDump] 07Universal-Omega pushed 031 commit to 03service [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/06e4ab2f5878...14c2b5d52a4d [01:35:20] [02DataDump] 07Universal-Omega 0314c2b5d - Fix [01:35:21] [02DataDump] 07Universal-Omega synchronize pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:35:59] miraheze/DataDump - Universal-Omega the build has errored. [01:40:04] miraheze/DataDump - Universal-Omega the build passed. [01:40:31] PROBLEM - weavefarers.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'weavefarers.wiki' expires in 14 day(s) (Fri 06 Dec 2024 01:24:18 AM GMT +0000). [01:40:51] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7c89bedf3cf9...9b00649fcc16 [01:40:54] [02ssl] 07WikiTideSSLBot 039b00649 - Bot: Update SSL cert for weavefarers.wiki [01:41:53] [02DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/2af27a14edf5...47853967b0fe [01:41:54] [02DataDump] 07Universal-Omega 034785396 - Allow custom handling of failed status comments (#92) [01:41:55] [02DataDump] 07Universal-Omega closed pull request 03#92: Allow custom handling of failed status comments - 13https://github.com/miraheze/DataDump/pull/92 [01:41:57] [02DataDump] 07Universal-Omega deleted branch 03failed-comments [01:41:59] [02DataDump] 07Universal-Omega deleted branch 03failed-comments - 13https://github.com/miraheze/DataDump [01:46:32] miraheze/DataDump - Universal-Omega the build passed. [01:47:09] [02MirahezeMagic] 07Universal-Omega created branch 03datadump-service - 13https://github.com/miraheze/MirahezeMagic [01:47:11] [02MirahezeMagic] 07Universal-Omega pushed 031 commit to 03datadump-service [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/commit/08903218d117 [01:47:12] [02MirahezeMagic] 07Universal-Omega 030890321 - Use DataDumpFileBackend service [01:47:13] [02MirahezeMagic] 07Universal-Omega opened pull request 03#539: Use DataDumpFileBackend service - 13https://github.com/miraheze/MirahezeMagic/pull/539 [01:47:20] [02MirahezeMagic] 07coderabbitai[bot] commented on pull request 03#539: Use DataDumpFileBackend service - 13https://github.com/miraheze/MirahezeMagic/pull/539#issuecomment-2489897046 [01:49:27] [02MirahezeMagic] 07Universal-Omega closed pull request 03#538: SwiftDump: use more reliable storage check - 13https://github.com/miraheze/MirahezeMagic/pull/538 [01:49:28] [02MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/cab266147f95...a58cfabe99ed [01:49:30] [02MirahezeMagic] 07Universal-Omega 03a58cfab - SwiftDump: use more reliable storage check (#538) [01:49:31] [02MirahezeMagic] 07Universal-Omega deleted branch 03swiftdump-storage - 13https://github.com/miraheze/MirahezeMagic [01:49:33] [02MirahezeMagic] 07Universal-Omega deleted branch 03swiftdump-storage [01:51:32] [02DataDump] 07Universal-Omega closed pull request 03#93: Convert DataDump class into a DataDumpFileBackend service - 13https://github.com/miraheze/DataDump/pull/93 [01:51:33] [02DataDump] 07Universal-Omega pushed 031 commit to 03master [+2/-1/±7] 13https://github.com/miraheze/DataDump/compare/47853967b0fe...2e70d5c40629 [01:51:35] [02DataDump] 07Universal-Omega 032e70d5c - Convert DataDump class into a DataDumpFileBackend service (#93) [01:51:37] [02DataDump] 07Universal-Omega deleted branch 03service - 13https://github.com/miraheze/DataDump [01:51:40] [02DataDump] 07Universal-Omega deleted branch 03service [01:51:46] [02MirahezeMagic] 07Universal-Omega closed pull request 03#539: Use DataDumpFileBackend service - 13https://github.com/miraheze/MirahezeMagic/pull/539 [01:51:49] [02MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/a58cfabe99ed...16bfde069022 [01:51:51] [02MirahezeMagic] 07Universal-Omega 0316bfde0 - Use DataDumpFileBackend service (#539) [01:51:53] [02MirahezeMagic] 07Universal-Omega deleted branch 03datadump-service - 13https://github.com/miraheze/MirahezeMagic [01:51:54] [02MirahezeMagic] 07Universal-Omega deleted branch 03datadump-service [01:55:26] miraheze/MirahezeMagic - Universal-Omega the build passed. [01:55:55] miraheze/DataDump - Universal-Omega the build passed. [01:59:00] miraheze/MirahezeMagic - Universal-Omega the build passed. [01:59:35] [02DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/2e70d5c40629...37084b271c32 [01:59:38] [02DataDump] 07Universal-Omega 0337084b2 - Fix [02:04:23] [02mw-config] 07Universal-Omega created branch 0375 - 13https://github.com/miraheze/mw-config [02:04:24] miraheze/DataDump - Universal-Omega the build passed. [02:04:26] [02mw-config] 07Universal-Omega pushed 031 commit to 0375 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/32cdfc4fb7b0 [02:04:28] [02mw-config] 07Universal-Omega 0332cdfc4 - Add to large handling for SwiftDump [02:04:31] [02mw-config] 07Universal-Omega opened pull request 03#5749: Add to large handling for SwiftDump - 13https://github.com/miraheze/mw-config/pull/5749 [02:05:29] miraheze/mw-config - Universal-Omega the build passed. [02:13:58] PROBLEM - cp37 Varnish Backends on cp37 is CRITICAL: 1 backends are down. mw153 [02:15:56] RECOVERY - cp37 Varnish Backends on cp37 is OK: All 29 backends are healthy [02:40:07] RECOVERY - weavefarers.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'weavefarers.wiki' will expire on Wed 19 Feb 2025 12:42:15 AM GMT +0000. [03:21:08] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.84, 15.84, 9.52 [03:21:15] [02ssl] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9b00649fcc16...c5f9a010f92f [03:21:17] [02ssl] 07MacFan4000 03c5f9a01 - add lagrancaidawiki.arsengonx.xyz [03:23:08] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 18.88, 16.87, 10.68 [03:30:02] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.65, 20.14, 14.45 [03:35:50] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 12.46, 18.06, 15.57 [03:47:07] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.07, 17.40, 15.35 [03:53:02] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 9.93, 16.44, 16.00 [03:55:57] !log [agent@mwtask151] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/sql.php --wiki=nationswiki /srv/mediawiki/1.42/extensions/LoginNotify/sql/mysql/tables-generated.sql (END - exit=0) [03:56:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:56:14] !log [agent@mwtask151] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/deleted.php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/sql.php /srv/mediawiki/1.42/extensions/LoginNotify/sql/mysql/tables-generated.sql (START) [03:56:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:14] !log [agent@mwtask151] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/deleted.php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/sql.php /srv/mediawiki/1.42/extensions/LoginNotify/sql/mysql/tables-generated.sql (END - exit=256) [03:57:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:57:42] !log [agent@mwtask151] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/sql.php /srv/mediawiki/1.42/extensions/LoginNotify/sql/mysql/tables-generated.sql (START) [03:57:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:58:58] PROBLEM - mwtask181 Current Load on mwtask181 is WARNING: LOAD WARNING - total load average: 21.85, 20.01, 17.75 [04:00:14] [02puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d72683f68cd1...6f679fda5c17 [04:00:16] [02puppet] 07AgentIsai 036f679fd - T12840: Install REL1_43 on all servers [04:02:58] RECOVERY - mwtask181 Current Load on mwtask181 is OK: LOAD OK - total load average: 1.11, 0.25, 0.08 [04:04:42] PROBLEM - mwtask181 Puppet on mwtask181 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [04:05:48] PROBLEM - mw154 Puppet on mw154 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:07:51] PROBLEM - mw151 Puppet on mw151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:08:48] PROBLEM - mw164 Puppet on mw164 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:09:59] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:10:34] PROBLEM - mw162 Puppet on mw162 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:10:58] PROBLEM - mw184 Puppet on mw184 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:11:33] PROBLEM - mw161 Puppet on mw161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:13:09] PROBLEM - mw173 Puppet on mw173 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:15:33] PROBLEM - mw174 Puppet on mw174 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:18:27] PROBLEM - mw171 Puppet on mw171 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:18:37] PROBLEM - mw153 Puppet on mw153 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:19:03] PROBLEM - mw172 Puppet on mw172 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:19:21] PROBLEM - mwtask161 Puppet on mwtask161 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:20:07] PROBLEM - mw183 Puppet on mw183 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:20:56] PROBLEM - mwtask151 Puppet on mwtask151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:21:29] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/extensions/MirahezeMagic/maintenance/resetWikiCaches.php --wiki=lagrancaidawiki (END - exit=0) [04:21:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:22:58] [02mediawiki-repos] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mediawiki-repos/compare/ee6ad38edacf...5b65169a370e [04:22:59] [02mediawiki-repos] 07AgentIsai 035b65169 - ShortDescription: Use main branch [04:26:21] PROBLEM - mw152 Puppet on mw152 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:27:14] !log [agent@mwtask181] starting deploy of {'force': True, 'folders': '1.43'} to all [04:27:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:27:31] PROBLEM - mw181 Puppet on mw181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:30:41] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importDump.php --wiki=animalroyalewiki animalroyale_gamepedia_en_pages_full_21.11.xml --username-prefix=fandom:animalroyale --no-updates (START) [04:30:45] PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/srv/mediawiki/1.43/skins/Femiwiki/node_modules] [04:30:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:31:05] !log [agent@mwtask181] finished deploy of {'force': True, 'folders': '1.43'} to all - SUCCESS in 230s [04:31:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:31:21] !log [agent@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.42'} to all [04:31:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:33:28] RECOVERY - mw154 Puppet on mw154 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [04:35:06] !log [agent@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [04:35:11] !log [agent@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.43'} to all [04:35:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:35:12] !log [agent@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [04:35:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:35:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:35:34] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:36:15] RECOVERY - mw164 Puppet on mw164 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures [04:37:06] RECOVERY - mw184 Puppet on mw184 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [04:37:44] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 13 seconds ago with 0 failures [04:37:53] RECOVERY - mw162 Puppet on mw162 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [04:38:37] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importDump.php --wiki=animalroyalewiki animalroyale_gamepedia_en_pages_full_21.11.xml --username-prefix=fandom:animalroyale --no-updates (END - exit=0) [04:38:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:39:10] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [04:39:56] !log [agent@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.43'} to all [04:40:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:42:08] !log [agent@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [04:42:11] !log [agent@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.43'} to all [04:42:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:42:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:42:49] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importImages.php --wiki=animalroyalewiki --search-recursively . --comment (START) [04:42:50] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importImages.php --wiki=animalroyalewiki --search-recursively . --comment (END - exit=256) [04:42:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:42:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:43:09] RECOVERY - mw173 Puppet on mw173 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:43:17] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importImages.php --wiki=animalroyalewiki --search-recursively . --comment Re-importing images from Fandom wiki (START) [04:43:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:43:30] RECOVERY - mw174 Puppet on mw174 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:46:31] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:46:44] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:46:48] !log [agent@mwtask181] finished deploy of {'l10n': True, 'force': True, 'versions': '1.43'} to all - SUCCESS in 277s [04:46:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:47:03] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [04:48:06] RECOVERY - mw183 Puppet on mw183 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures [04:48:56] RECOVERY - mwtask151 Puppet on mwtask151 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [04:49:16] RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:55:09] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [04:55:39] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:56:27] PROBLEM - test151 Puppet on test151 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 2 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_checkout_MediaWiki-REL1_42 ShortDescription],Exec[git_checkout_MediaWiki-REL1_43 ShortDescription],Exec[git_checkout_MediaWiki-master ShortDescription] [04:57:01] @agentisai I already had a PR to ad 1.43 to all servers but doing it now creates a ton of extea work so we should wait until we are closer to ready to upgrade otherwise now we have 2 versions to always have to keep up to date in extensions etc... in production now [04:58:45] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:58:49] in the future we will not install new versions on production servers until it becomes stable as we can't even start planning a time till then. [04:59:11] Otherwise we just have to keep updating it. [04:59:21] Which creates a ton of extra work for us. [04:59:48] are we still going forward with the nexttide 2: electric boogaloo idea? [05:00:49] Oh yeah I forgot to finish adding support to beta for nexttide lol [05:00:59] I got distracted rewriting extensions [05:01:19] is this your equivalent of a fun night out [05:01:25] not that im complainin [05:01:46] also: did we decide if we're gonna stick with LTS or keep clawing to latest? [05:01:46] ohhhh [05:01:47] I did CreateWiki, SubTranslate (now MachineTranslation), and most recently (like yesterday) DataDump lol [05:01:58] I was planning to upgrade beta wikis already [05:01:59] that's why [05:02:05] it comes out this month so I see no real harm [05:02:18] We likely wont upgrade to January though [05:02:26] *till [05:02:35] At least in full [05:02:37] i'd say i should see about working on that adding image dump and cloud drive support to import dump but [05:02:38] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! [05:02:41] what! [05:02:45] i have too much on my plate [05:03:01] I was already excited to upgrade the day of release [05:03:05] I should go through and fill out merch cord wiki [05:03:17] I need to find more options to look into [05:03:24] Either January or very early December or this month. But I'd prefer not in December to much [05:03:45] If we can make it the day it releases then I suppose we can [05:03:50] and here i was hoping id work on my other little non wiki FOSS project. delusion [05:03:52] s [05:04:13] That's what we did for 1.42. Not like that had any adverse side effects [05:04:17] But lets not rush it. Stability is more important than upgrading immediately, if we aren't prepared, we will have to postpone it. [05:04:25] yep [05:04:40] stability? on my miraheze? insane [05:04:44] preposterous [05:04:48] ludicrous [05:04:54] well, actually, our first post-merger upgrade went very well [05:05:02] abosuteltly unheard of [05:05:06] well [05:05:06] apart from that one job bug [05:05:07] yea [05:05:19] we don't talk about kafka [05:05:35] oh its thursday now [05:05:37] our second post-merger upgrade (read: last one) went poorly because of various factors, including having the majority of the team out [05:06:03] see we're learning so many good lessons this yea [05:06:05] Yep. Also 1.42 was the first upgrade since 1.34 (maybe 1.35) I had absolutely no part in as I was away lol. [05:06:09] s/yea/year [05:06:29] OS carried iirc [05:07:14] wow, so many migrations [05:07:15] Yep OS did it. I can't blame OS at all for anything that went wrong in it though. I wasn't around and I also didn't right docs for multiversion deployments after I wrote the system also. [05:07:23] Linter migration is crazy as well [05:08:32] 1.43 has more than last 4 upgrades (since 1.39, maybe even including 1.39) combined I think [05:08:38] It is a super massive upgrade [05:08:58] Also PortableInfobox is completely broke on it IIRC I still need to fix both it and DPL3 lol [05:09:08] oh yea that [05:09:28] I honestly have no idea how to make PI work with Parsoid... [05:09:41] bullshit bullshit more bullshit [05:09:45] I tried for probably 6 hours 1 day and made 0% progress [05:09:49] PROBLEM - mwtask161 Current Load on mwtask161 is WARNING: LOAD WARNING - total load average: 23.70, 18.08, 10.78 [05:09:50] that should work right [05:10:06] if not i have some dice [05:10:24] not sure how to use them to figure out how to make PI work with parsoid [05:10:36] but im sure they'll help someho [05:10:38] Lol yeah I don't understand Parsoid hardly at all right now.. [05:11:09] see [05:11:14] imagine if the parser [05:11:15] I could rewrite all of CW in a week and DataDump in a few hours but I can't figure out Parsoid lol [05:11:17] was javascript [05:11:19] and sucked [05:11:26] just a liiiiitle bit les [05:11:32] i think thats the gist [05:11:49] RECOVERY - mwtask161 Current Load on mwtask161 is OK: LOAD OK - total load average: 17.04, 18.13, 11.73 [05:12:00] its still wikitext though so filed under "Horror, Hopeless" [05:12:23] oh my goooooood its raining the jersey drought its over [05:12:35] ..very slightly! [05:13:36] I just realized DPL3 is broke on 1.42 also probably at least parts of it [05:13:51] I never even updated it for the externallinks migration or some others [05:13:56] when I go to wiki game night on friday ill ask some of the tech people to explain parsoid comediclly maybe [05:14:29] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/importImages.php --wiki=animalroyalewiki --search-recursively . --comment Re-importing images from Fandom wiki (END - exit=0) [05:14:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:15:08] Wait so now I understand what some people were talking about in my DMs lol I was confused... but yeah that is why... [05:15:27] I was about to say "weeeeelll, no one noticed?" [05:16:09] If no one complained is it reaaaaally an issue? [05:16:22] Nope just my brain not putting 1 and 1 together, kinda like PHP sometimes thinking 1 != 1 [05:16:40] thak you php very cool [05:17:46] !log [agent@mwtask161] sudo -u www-data php /srv/mediawiki/1.42/maintenance/run.php /srv/mediawiki/1.42/maintenance/rebuildall.php --wiki=animalroyalewiki (START) [05:17:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:17:59] ```PHP Fatal error: Uncaught ArgumentCountError: Too few arguments to function, 1 passed and exactly 1 expected``` [05:19:24] which one is this thats complaining [05:19:39] [02mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c92e930683a4...a0dcfacc46d9 [05:19:42] [02mw-config] 07AgentIsai 03a0dcfac - Add Linter migration configuration [05:19:52] I know that is probably just some poisoned opcache but the way it says that error is just bizarre to me lol [05:20:13] a MediaWiki core class sometimes does that randomly [05:20:21] i feel ive seen that in python [05:20:24] huh [05:20:32] is this a known bug [05:20:32] miraheze/mw-config - AgentIsai the build passed. [05:21:08] Not really a bug it is a PHP thing not a problem with MediaWiki afaik [05:21:18] opcache most likely [05:21:21] I love how miraheze-github walks in just to say that and bolts before anyone can talk to him [05:21:34] lol [05:21:37] Don't think ive read into what opcache is huh [05:22:37] https://www.php.net/manual/en/intro.opcache.php [05:22:56] why do i wann implement passkeys/passwordless login to a project that's prob just gonna use discord OAuth [05:22:58] who knows [05:23:01] anyways [05:23:31] how will nexttide(or nexthive?) work for testing prods wikis on 42 [05:23:32] !log [@test151] starting deploy of {'config': True} to test151 [05:23:33] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [05:23:34] 43* [05:23:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:23:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:24:39] I have to go for now [05:25:17] (not to sleep, yet) will likely be back later tonight. [05:25:59]