[00:00:25] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:25] PROBLEM - cp1 HTTP 4xx/5xx ERROR Rate on cp1 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [03:02:21] PROBLEM - cp1 PowerDNS Recursor on cp1 is CRITICAL: Domain 'wikitide.org' was not found by the server [03:06:30] RECOVERY - cp1 PowerDNS Recursor on cp1 is OK: DNS OK: 4.929 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66 [03:57:48] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega pushed 031 commit to 03miraheze-CreateWiki [+0/-0/±1] 13https://github.com/WikiTideOrg/mediawiki-repos/commit/0136958bde64 [03:57:50] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega 030136958 - Switch to miraheze/CreateWiki [03:57:53] [02mediawiki-repos] 07Universal-Omega created branch 03miraheze-CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos [03:57:55] [02mediawiki-repos] 07Universal-Omega opened pull request 03#2: Switch to miraheze/CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos/pull/2 [04:04:42] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega created branch 03miraheze 13https://github.com/WikiTideOrg/mediawiki-repos/commit/0136958bde6402b316b5bace03b9902a24630a7e [04:04:44] [02mediawiki-repos] 07Universal-Omega created branch 03miraheze - 13https://github.com/WikiTideOrg/mediawiki-repos [04:04:46] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega deleted branch 03miraheze-CreateWiki [04:04:49] [02mediawiki-repos] 07Universal-Omega deleted branch 03miraheze-CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos [04:04:52] [02mediawiki-repos] 07Universal-Omega closed pull request 03#2: Switch to miraheze/CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos/pull/2 [04:05:24] [02mediawiki-repos] 07Universal-Omega opened pull request 03#3: Switch to miraheze/CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos/pull/3 [13:32:51] PROBLEM - swiftproxy1 SSH on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:33:44] PROBLEM - swiftproxy1 Current Load on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:35:22] PROBLEM - swiftproxy1 Puppet on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:35:53] PROBLEM - swiftproxy1 PowerDNS Recursor on swiftproxy1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [13:36:02] PROBLEM - swiftproxy1 ferm_active on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:36:03] PROBLEM - swiftproxy1 conntrack_table_size on swiftproxy1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [13:36:51] PROBLEM - swiftproxy1 NTP time on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:39:46] RECOVERY - swiftproxy1 conntrack_table_size on swiftproxy1 is OK: OK: nf_conntrack is 0 % full [13:39:46] RECOVERY - swiftproxy1 ferm_active on swiftproxy1 is OK: OK ferm input default policy is set [13:39:51] RECOVERY - swiftproxy1 NTP time on swiftproxy1 is OK: NTP OK: Offset 0.0119023025 secs [13:41:38] RECOVERY - swiftproxy1 PowerDNS Recursor on swiftproxy1 is OK: DNS OK: 3.605 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66 [13:50:07] PROBLEM - swiftproxy1 PowerDNS Recursor on swiftproxy1 is CRITICAL: CRITICAL - Plugin timed out while executing system call [13:50:33] PROBLEM - swiftproxy1 NTP time on swiftproxy1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [13:51:28] RECOVERY - swiftproxy1 SSH on swiftproxy1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [13:51:33] RECOVERY - swiftproxy1 Puppet on swiftproxy1 is OK: OK: Puppet is currently enabled, last run 52 minutes ago with 0 failures [13:52:02] RECOVERY - swiftproxy1 PowerDNS Recursor on swiftproxy1 is OK: DNS OK: 0.328 seconds response time. wikitide.org returns 2001:41d0:801:2000::4089,51.75.170.66 [13:52:36] RECOVERY - swiftproxy1 NTP time on swiftproxy1 is OK: NTP OK: Offset 0.01629820466 secs [14:16:39] PROBLEM - swiftproxy1 Current Load on swiftproxy1 is WARNING: LOAD WARNING - total load average: 0.07, 0.42, 7.39 [14:18:32] RECOVERY - swiftproxy1 Current Load on swiftproxy1 is OK: LOAD OK - total load average: 0.13, 0.33, 6.54 [17:03:02] [02mediawiki-repos] 07Universal-Omega closed pull request 03#3: Switch to miraheze/CreateWiki - 13https://github.com/WikiTideOrg/mediawiki-repos/pull/3 [17:03:05] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/mediawiki-repos/compare/196e4af39a52...656cbe117b8f [17:03:07] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega 03656cbe1 - Switch to miraheze/CreateWiki [17:03:09] [02WikiTideOrg/mediawiki-repos] 07Universal-Omega deleted branch 03miraheze [17:03:12] [02mediawiki-repos] 07Universal-Omega deleted branch 03miraheze - 13https://github.com/WikiTideOrg/mediawiki-repos [17:06:29] [02WikiTideOrg/WikiTideMagic] 07Universal-Omega created branch 03miraheze-CreateWiki 13https://github.com/WikiTideOrg/WikiTideMagic/commit/89b3619d5db08b6cc1c946d6f05ce184419a0f4e [17:06:30] [02WikiTideMagic] 07Universal-Omega created branch 03miraheze-CreateWiki - 13https://github.com/WikiTideOrg/WikiTideMagic [17:07:45] [02WikiTideOrg/WikiTideMagic] 07Universal-Omega pushed 031 commit to 03miraheze-CreateWiki [+0/-0/±6] 13https://github.com/WikiTideOrg/WikiTideMagic/compare/89b3619d5db0...20d6ae91444f [17:07:46] [02WikiTideOrg/WikiTideMagic] 07Universal-Omega 0320d6ae9 - Use miraheze/CreateWiki [17:08:27] [02WikiTideMagic] 07Universal-Omega opened pull request 03#8: Use miraheze/CreateWiki - 13https://github.com/WikiTideOrg/WikiTideMagic/pull/8 [17:10:27] [02WikiTideOrg/mw-config] 07Universal-Omega pushed 031 commit to 03main [+0/-0/±1] 13https://github.com/WikiTideOrg/mw-config/compare/ad60be87a991...5cfed2f9721d [17:10:30] [02WikiTideOrg/mw-config] 07Universal-Omega 035cfed2f - Update namespace to use Miraheze\CreateWiki [17:10:33] [02WikiTideOrg/WikiTideMagic] 07github-actions[bot] pushed 031 commit to 03miraheze-CreateWiki [+0/-0/±2] 13https://github.com/WikiTideOrg/WikiTideMagic/compare/20d6ae91444f...d8ceb78a0ef0 [17:10:34] [02WikiTideOrg/WikiTideMagic] 07github-actions 03d8ceb78 - CI: lint code to MediaWiki standards Check commit and GitHub actions for more details [17:10:37] [02WikiTideMagic] 07github-actions[bot] synchronize pull request 03#8: Use miraheze/CreateWiki - 13https://github.com/WikiTideOrg/WikiTideMagic/pull/8 [17:11:21] WikiTideOrg/mw-config - Universal-Omega the build passed. [17:17:14] WikiTideOrg/WikiTideMagic - Universal-Omega the build passed. [17:18:13] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, jobrunner1] [17:18:16] !log [@jobrunner1] finished deploy of {'config': True, 'versions': ['']} to [mw1, mw2, jobrunner1] - SUCCESS in 2s [17:18:20] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [17:18:26] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [17:18:55] [02.github] 07Universal-Omega closed pull request 03#1: Bump actions/cache from 3.3.2 to 3.3.3 - 13https://github.com/WikiTideOrg/.github/pull/1 [17:18:57] [02WikiTideOrg/.github] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-3.3.3 [17:18:58] [02.github] 07dependabot[bot] commented on pull request 03#1: Bump actions/cache from 3.3.2 to 3.3.3 - 13https://github.com/WikiTideOrg/.github/pull/1#issuecomment-1890598709 [17:19:00] [02.github] 07Universal-Omega deleted branch 03dependabot/github_actions/actions/cache-3.3.3 - 13https://github.com/WikiTideOrg/.github [17:21:59] [02WikiTideOrg/.github] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/.github/compare/ee914ccbca7f...5cbc4a617d3f [17:22:00] [02WikiTideOrg/.github] 07Universal-Omega 035cbc4a6 - ARCHIVE [17:31:08] !log [@test1] starting deploy of {'config': True, 'versions': ['']} to [test1] [17:31:09] !log [@test1] finished deploy of {'config': True, 'versions': ['']} to [test1] - SUCCESS in 0s [17:31:20] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [17:31:27] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [18:42:50] [02WikiTideOrg/dns] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/09b7e07d6420...1268fc51b00e [18:42:52] [02WikiTideOrg/dns] 07Universal-Omega 031268fc5 - Add acme challenges