[01:46:35] [02WikiForge/puppet] 07Universal-Omega created branch 03mariadb-remove-ssl 13https://github.com/WikiForge/puppet/commit/c3a8b09f4a73e95603c259b951a3b9903ed574c4 [01:46:36] [02puppet] 07Universal-Omega created branch 03mariadb-remove-ssl - 13https://github.com/WikiForge/puppet [01:46:58] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03mariadb-remove-ssl [+0/-0/±12] 13https://github.com/WikiForge/puppet/compare/c3a8b09f4a73...015010a32f00 [01:47:00] [02WikiForge/puppet] 07Universal-Omega 03015010a - mariadb: remove SSL [01:47:09] [02puppet] 07Universal-Omega opened pull request 03#210: mariadb: remove SSL - 13https://github.com/WikiForge/puppet/pull/210 [01:47:49] [02WikiForge/puppet] 07github-actions[bot] pushed 031 commit to 03mariadb-remove-ssl [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/015010a32f00...85320708cde9 [01:47:51] [02WikiForge/puppet] 07github-actions 038532070 - CI: lint puppet code to standards Check commit and GitHub actions for more details [01:47:54] [02puppet] 07github-actions[bot] synchronize pull request 03#210: mariadb: remove SSL - 13https://github.com/WikiForge/puppet/pull/210 [01:51:05] PROBLEM - mon1 Current Load on mon1 is WARNING: LOAD WARNING - total load average: 3.39, 5.24, 2.44 [01:53:02] RECOVERY - mon1 Current Load on mon1 is OK: LOAD OK - total load average: 0.54, 3.58, 2.16 [01:55:17] [02puppet] 07Universal-Omega closed pull request 03#210: mariadb: remove SSL - 13https://github.com/WikiForge/puppet/pull/210 [01:55:20] [02WikiForge/puppet] 07Universal-Omega deleted branch 03mariadb-remove-ssl [01:55:23] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±12] 13https://github.com/WikiForge/puppet/compare/c3a8b09f4a73...474a162804e0 [01:55:25] [02WikiForge/puppet] 07Universal-Omega 03474a162 - mariadb: remove SSL [01:55:27] [02puppet] 07Universal-Omega deleted branch 03mariadb-remove-ssl - 13https://github.com/WikiForge/puppet [01:57:24] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03db-remove-ssl [+0/-0/±1] 13https://github.com/WikiForge/mw-config/commit/d750012ec2a1 [01:57:25] [02WikiForge/mw-config] 07Universal-Omega 03d750012 - Remove SSL from DB [01:57:27] [02mw-config] 07Universal-Omega created branch 03db-remove-ssl - 13https://github.com/WikiForge/mw-config [01:57:28] PROBLEM - db1 Puppet on db1 is WARNING: WARNING: Puppet is currently disabled, message: reason not specified, last run 9 minutes ago with 0 failures [01:57:29] [02mw-config] 07Universal-Omega opened pull request 03#139: Remove SSL from DB - 13https://github.com/WikiForge/mw-config/pull/139 [01:57:57] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03db-remove-ssl [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/d750012ec2a1...160a7c6b6842 [01:58:00] [02WikiForge/mw-config] 07Universal-Omega 03160a7c6 - Remove SSL from DB [01:58:01] [02mw-config] 07Universal-Omega synchronize pull request 03#139: Remove SSL from DB - 13https://github.com/WikiForge/mw-config/pull/139 [01:58:20] WikiForge/mw-config - Universal-Omega the build passed. [01:58:56] WikiForge/mw-config - Universal-Omega the build passed. [01:59:47] [02mw-config] 07Universal-Omega closed pull request 03#139: Remove SSL from DB - 13https://github.com/WikiForge/mw-config/pull/139 [01:59:48] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/WikiForge/mw-config/compare/d303d7e72daf...b5462c630d92 [01:59:51] [02WikiForge/mw-config] 07Universal-Omega 03b5462c6 - Remove SSL from DB [01:59:54] [02WikiForge/mw-config] 07Universal-Omega deleted branch 03db-remove-ssl [01:59:56] [02mw-config] 07Universal-Omega deleted branch 03db-remove-ssl - 13https://github.com/WikiForge/mw-config [02:00:09] !log [universalomega@jobrunner1] starting deploy of {'pull': 'config', 'config': True, 'force': True, 'versions': ['1.40']} to [mw1, mw2, mw3, jobrunner1] [02:00:13] !log [universalomega@jobrunner1] finished deploy of {'pull': 'config', 'config': True, 'force': True, 'versions': ['1.40']} to [mw1, mw2, mw3, jobrunner1] - SUCCESS in 3s [02:00:40] WikiForge/mw-config - Universal-Omega the build passed. [02:01:27] RECOVERY - db1 Puppet on db1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:06:53] RECOVERY - db1 MariaDB Connections on db1 is OK: Usage: php check_mysql_connections.php --host=HOST --user=USER --password=PASSWORD --max-connections=MAX_CONNECTIONS --critical-threshold=CRITICAL_THRESHOLD --warning-threshold=WARNING_THRESHOLD [02:10:10] RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct [02:10:20] RECOVERY - db1 MariaDB on db1 is OK: Uptime: 1282265 Threads: 9 Questions: 359192575 Slow queries: 81543392 Opens: 9766 Open tables: 9223 Queries per second avg: 280.123 [02:11:09] PROBLEM - www.farthestfrontier.wiki - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:11] PROBLEM - rippaverse.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:12] PROBLEM - www.dcmultiversewiki.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:13] PROBLEM - www.rosettacode.org - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:13] PROBLEM - www.wikitide.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:14] PROBLEM - www.mwcosmos.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:14] PROBLEM - dc.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:16] PROBLEM - www.avid.wiki - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:17] PROBLEM - mw2 PowerDNS Recursor on mw2 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:19] PROBLEM - jobrunner1 PowerDNS Recursor on jobrunner1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:21] PROBLEM - www.director.wiki - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:23] PROBLEM - beaconspace.unrestrictedlorefare.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:23] PROBLEM - greatamerica.wiki - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:25] PROBLEM - ldap1 PowerDNS Recursor on ldap1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:27] PROBLEM - mw1 PowerDNS Recursor on mw1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:33] PROBLEM - mem1 PowerDNS Recursor on mem1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:39] PROBLEM - dcmultiverse.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:39] PROBLEM - dcmultiversewiki.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:41] PROBLEM - db1 PowerDNS Recursor on db1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:41] PROBLEM - wikiforge.work - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:46] PROBLEM - polandballwiki.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:47] PROBLEM - perilous-times.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:52] PROBLEM - graylog1 PowerDNS Recursor on graylog1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:52] PROBLEM - www.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:52] PROBLEM - director.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:53] PROBLEM - meta.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:53] PROBLEM - farthestfrontier.wikitide.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:54] PROBLEM - avid.wiki - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:54] PROBLEM - bast1 PowerDNS Recursor on bast1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:11:56] PROBLEM - rippaversewiki.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:11:59] PROBLEM - os1 PowerDNS Recursor on os1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:12:04] PROBLEM - www.rippaversewiki.com - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:12:05] PROBLEM - avid.wikiforge.net - pointing to WikiForge on sslhost is CRITICAL: DNS CRITICAL - expected 'mw-lb.wikiforge.net.' but got '104.21.36.88,172.67.190.178,2606:4700:3036::6815:2458,2606:4700:3036::ac43:beb2' [02:12:06] PROBLEM - bast1 APT on bast1 is CRITICAL: APT CRITICAL: 4 packages available for upgrade (3 critical updates). [02:12:07] PROBLEM - puppet1 PowerDNS Recursor on puppet1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:12:09] PROBLEM - jobchron1 PowerDNS Recursor on jobchron1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:12:10] PROBLEM - mon1 PowerDNS Recursor on mon1 is CRITICAL: Domain 'wikiforge.net' was not found by the server [02:12:15] Lol I fixed monitoring and I guess we know a lot of alerts [02:13:09] @agentisai ^ [02:14:12] The rDNS warnings are to be expected [02:14:20] I tried to fix it to accommodate our CF usage to no avail [02:14:29] oh lol I just realized icinga does not relay here lol [02:14:44] I will look later [02:15:00] thanks [02:15:02] PROBLEM - www.rosettacode.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Cannot make SSL connection.40E7CE4058140000:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1586:SSL alert number 40 [02:15:25] No problem [02:44:24] RECOVERY - www.rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Sat 20 Jan 2024 06:52:33 AM GMT +0000. [02:51:11] [02WikiForge/puppet] 07Universal-Omega pushed 0340 commits to 03cleanup-phorge [+32/-8/±383] 13https://github.com/WikiForge/puppet/compare/0bf21ac841e7...fb805cfff491 [02:51:14] [02WikiForge/puppet] 07Universal-Omega 03fb805cf - Merge branch 'master' into cleanup-phorge [02:51:17] [02puppet] 07Universal-Omega synchronize pull request 03#205: Cleanup phorge - 13https://github.com/WikiForge/puppet/pull/205 [03:28:58] [02WikiForge/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/MediaWikiDebugJS/compare/e2917c05a28e...14cb5b1b3d61 [03:29:01] [02WikiForge/MediaWikiDebugJS] 07Universal-Omega 0314cb5b1 - Support static.your.wf [03:31:59] [02WikiForge/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/MediaWikiDebugJS/compare/14cb5b1b3d61...74d469a5b048 [03:32:02] [02WikiForge/MediaWikiDebugJS] 07Universal-Omega 0374d469a - Revert We don't need to support static.your.wf we can't support both as one overrides the other [18:05:51] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/b5462c630d92...61763d08e2c4 [18:05:54] [02WikiForge/mw-config] 07Universal-Omega 0361763d0 - Fix PoolCounter config [18:06:53] WikiForge/mw-config - Universal-Omega the build passed. [18:07:46] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/61763d08e2c4...b6d0c4ad1f34 [18:07:47] [02WikiForge/mw-config] 07Universal-Omega 03b6d0c4a - Update comment [18:07:54] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, mw3, jobrunner1] [18:07:55] !log [@jobrunner1] DEPLOY ABORTED: Canary check failed for hub.wikiforge.net@mw1.inside.wf [18:09:45] WikiForge/mw-config - Universal-Omega the build passed. [18:14:49] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03move-wgHooks [+0/-0/±1] 13https://github.com/WikiForge/mw-config/commit/2f95358f4c26 [18:14:52] [02WikiForge/mw-config] 07Universal-Omega 032f95358 - Move wgHooks to GlobalSettings [18:14:55] [02mw-config] 07Universal-Omega created branch 03move-wgHooks - 13https://github.com/WikiForge/mw-config [18:14:56] [02mw-config] 07Universal-Omega opened pull request 03#140: Move wgHooks to GlobalSettings - 13https://github.com/WikiForge/mw-config/pull/140 [18:15:37] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03move-wgHooks [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/2f95358f4c26...d33f2c5ab75b [18:15:40] [02WikiForge/mw-config] 07Universal-Omega 03d33f2c5 - Add [18:15:43] [02mw-config] 07Universal-Omega synchronize pull request 03#140: Move wgHooks to GlobalSettings - 13https://github.com/WikiForge/mw-config/pull/140 [18:15:49] WikiForge/mw-config - Universal-Omega the build passed. [18:16:19] [02mw-config] 07Universal-Omega edited pull request 03#140: Move wgHooks to GlobalSettings - 13https://github.com/WikiForge/mw-config/pull/140 [18:16:33] WikiForge/mw-config - Universal-Omega the build passed. [18:16:43] [02mw-config] 07Universal-Omega edited pull request 03#140: Move wgHooks to GlobalSettings - 13https://github.com/WikiForge/mw-config/pull/140 [18:20:46] !log [agent@mw1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/namespaceDupes.php --wiki=harrypotterwiki (END - exit=0) [18:22:56] [02mw-config] 07Universal-Omega closed pull request 03#140: Move wgHooks to GlobalSettings - 13https://github.com/WikiForge/mw-config/pull/140 [18:22:57] [02WikiForge/mw-config] 07Universal-Omega deleted branch 03move-wgHooks [18:23:00] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/WikiForge/mw-config/compare/b6d0c4ad1f34...8a0cba9d6e57 [18:23:02] [02WikiForge/mw-config] 07Universal-Omega 038a0cba9 - Move wgHooks to GlobalSettings [18:23:04] [02mw-config] 07Universal-Omega deleted branch 03move-wgHooks - 13https://github.com/WikiForge/mw-config [18:23:16] [02mw-config] 07Universal-Omega commented on pull request 03#110: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/mw-config/pull/110#issuecomment-1837221739 [18:23:19] [02mw-config] 07dependabot[bot] edited pull request 03#110: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/mw-config/pull/110 [18:23:33] [02mw-config] 07dependabot[bot] edited pull request 03#110: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/mw-config/pull/110 [18:23:34] [02WikiForge/mw-config] 07dependabot[bot] pushed 031 commit to 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/bb03e0b94b86...6eaf633a3c43 [18:23:35] [02WikiForge/mw-config] 07dependabot[bot] 036eaf633 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tools-codes [18:23:36] [02mw-config] 07dependabot[bot] synchronize pull request 03#110: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/mw-config/pull/110 [18:23:47] WikiForge/mw-config - Universal-Omega the build passed. [18:24:38] WikiForge/mw-config - dependabot[bot] the build passed. [18:24:49] [02mw-config] 07Universal-Omega closed pull request 03#110: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/mw-config/pull/110 [18:24:50] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/8a0cba9d6e57...76a8d00a8b91 [18:24:53] [02WikiForge/mw-config] 07dependabot[bot] 0376a8d00 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 (#110) Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tool [18:24:56] [02WikiForge/mw-config] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 [18:24:59] [02mw-config] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 - 13https://github.com/WikiForge/mw-config [18:25:45] WikiForge/mw-config - Universal-Omega the build passed. [18:31:58] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildall.php --wiki=harrypotterwiki (START) [18:31:59] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildall.php --wiki=harrypotterwiki (END - exit=256) [18:35:59] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/sql.php --wiki=harrypotterwiki (END - exit=2) [18:36:40] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildtextindex.php --wiki=harrypotterwiki (END - exit=256) [18:38:39] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, mw3, jobrunner1] [18:38:41] !log [@jobrunner1] DEPLOY ABORTED: Canary check failed for hub.wikiforge.net@mw2.inside.wf [18:40:11] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/sql.php --wiki=harrypotterwiki (END - exit=2) [18:40:15] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/rebuildall.php --wiki=harrypotterwiki (START) [21:14:21] PROBLEM - www.rosettacode.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Cannot make SSL connection.40374394F7140000:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1586:SSL alert number 40 [21:43:45] RECOVERY - www.rosettacode.org - LetsEncrypt on sslhost is OK: OK - Certificate 'rosettacode.org' will expire on Sat 20 Jan 2024 06:52:33 AM GMT +0000. [22:58:21] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/474a162804e0...a3b4efa3aaa5 [22:58:23] [02WikiForge/puppet] 07Universal-Omega 03a3b4efa - grafana: disable SSL for database [23:08:22] RECOVERY - mon1 grafana.inside.wf HTTPS on mon1 is OK: HTTP OK: HTTP/1.1 302 Found - 429 bytes in 0.015 second response time [23:22:14] [02puppet] 07Universal-Omega closed pull request 03#160: Revert "mediawiki: stream sitemap files directly" - 13https://github.com/WikiForge/puppet/pull/160 [23:22:16] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/WikiForge/puppet/compare/a3b4efa3aaa5...fbb18051ebdf [23:22:17] [02WikiForge/puppet] 07Universal-Omega 03fbb1805 - Revert "mediawiki: stream sitemap files directly" This reverts commit bc531e6fc3b437d038c8e50451e9189d79ac1fab. [23:22:20] [02WikiForge/puppet] 07Universal-Omega deleted branch 03revert-152-mediawiki-sitemap-streaming [23:22:20] [02puppet] 07Universal-Omega deleted branch 03revert-152-mediawiki-sitemap-streaming - 13https://github.com/WikiForge/puppet [23:32:20] [02puppet] 07Universal-Omega synchronize pull request 03#205: Cleanup phorge - 13https://github.com/WikiForge/puppet/pull/205 [23:32:23] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03cleanup-phorge [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/fb805cfff491...a7df5343f59a [23:32:26] [02WikiForge/puppet] 07Universal-Omega 03a7df534 - Re-add variables [23:32:35] [02WikiForge/puppet] 07Universal-Omega pushed 034 commits to 03cleanup-phorge [+1/-1/±3] 13https://github.com/WikiForge/puppet/compare/a7df5343f59a...13d72b83606e [23:32:38] [02WikiForge/puppet] 07Universal-Omega 03e3229af - Cleanup phorge [23:32:41] [02WikiForge/puppet] 07Universal-Omega 03be11b3d - Rename [23:32:44] [02WikiForge/puppet] 07Universal-Omega 0315f7e1e - Add monitoring [23:32:46] [02WikiForge/puppet] 07Universal-Omega 0313d72b8 - Re-add variables [23:32:48] [02puppet] 07Universal-Omega synchronize pull request 03#205: Cleanup phorge - 13https://github.com/WikiForge/puppet/pull/205 [23:34:41] [02puppet] 07Universal-Omega closed pull request 03#205: Cleanup phorge - 13https://github.com/WikiForge/puppet/pull/205 [23:34:43] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-1/±1] 13https://github.com/WikiForge/puppet/compare/fbb18051ebdf...57ca5ac7840b [23:34:45] [02WikiForge/puppet] 07Universal-Omega 0357ca5ac - Cleanup phorge [23:34:47] [02WikiForge/puppet] 07Universal-Omega deleted branch 03cleanup-phorge [23:34:49] [02puppet] 07Universal-Omega deleted branch 03cleanup-phorge - 13https://github.com/WikiForge/puppet [23:54:31] [02WikiForge/puppet] 07Universal-Omega created branch 03remove-support-archive 13https://github.com/WikiForge/puppet/commit/57ca5ac7840b8171e2cf233ce3011ba8bc9585a5 [23:54:33] [02puppet] 07Universal-Omega created branch 03remove-support-archive - 13https://github.com/WikiForge/puppet [23:55:14] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03remove-support-archive [+0/-3/±2] 13https://github.com/WikiForge/puppet/compare/57ca5ac7840b...9423b665894b [23:55:16] [02WikiForge/puppet] 07Universal-Omega 039423b66 - phorge: remove support-archive [23:55:29] [02puppet] 07Universal-Omega opened pull request 03#211: phorge: remove support-archive - 13https://github.com/WikiForge/puppet/pull/211 [23:56:26] WikiForge/puppet - Universal-Omega the build has errored. [23:58:28] [02puppet] 07Universal-Omega synchronize pull request 03#211: phorge: remove support-archive - 13https://github.com/WikiForge/puppet/pull/211 [23:58:29] [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03remove-support-archive [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/9423b665894b...44ab32275073 [23:58:31] [02WikiForge/puppet] 07Universal-Omega 0344ab322 - Fix