[01:58:50] [02WikiForge/puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/WikiForge/puppet/compare/e59006d660de...ca06f1c3d291 [01:58:52] [02WikiForge/puppet] 07AgentIsai 03ca06f1c - Update mail address [05:04:31] [02WikiForge/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/e3429221e99e...a1ef2bfc89bb [05:04:32] [02WikiForge/mw-config] 07AgentIsai 03a1ef2bf - Finish file maintenance [05:05:43] WikiForge/mw-config - AgentIsai the build passed. [05:07:51] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, mwdedi1, jobrunner1] [05:07:56] !log [@jobrunner1] DEPLOY ABORTED: Canary check failed for hub.wikiforge.net@mwdedi1.inside.wf [05:08:22] [02WikiForge/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/dns/compare/a7d7f6688526...0beec0c23166 [05:08:25] [02WikiForge/dns] 07AgentIsai 030beec0c - Add cloud1 to inside.wf [05:15:35] [02WikiForge/puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/WikiForge/puppet/compare/ca06f1c3d291...2f5242d54bd7 [05:15:37] [02WikiForge/puppet] 07AgentIsai 032f5242d - Add mwdedi2 specifics [05:16:55] WikiForge/puppet - AgentIsai the build has errored. [05:21:22] PROBLEM - swiftobject1 Current Load on swiftobject1 is CRITICAL: LOAD CRITICAL - total load average: 8.68, 5.53, 2.67 [05:33:22] RECOVERY - swiftobject1 Current Load on swiftobject1 is OK: LOAD OK - total load average: 1.30, 6.67, 6.09 [05:47:53] [02WikiForge/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/a1ef2bfc89bb...f269418f4f16 [05:47:56] [02WikiForge/mw-config] 07AgentIsai 03f269418 - Update mw* server IPs [05:48:56] WikiForge/mw-config - AgentIsai the build passed. [05:51:28] PROBLEM - Host mwdedi1 is DOWN: PING CRITICAL - Packet loss = 100% [05:51:41] PROBLEM - Host mwdedi2 is DOWN: PING CRITICAL - Packet loss = 100% [05:58:53] [02WikiForge/puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/2f5242d54bd7...1a62cb2e5b7c [05:58:55] [02WikiForge/puppet] 07AgentIsai 031a62cb2 - Fix hosts [06:03:28] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/runJobs.php --wiki=avidwiki --procs 50 (START) [06:03:29] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/runJobs.php --wiki=avidwiki --procs 50 (END - exit=0) [06:04:59] !log [agent@jobrunner1] sudo -u www-data php /srv/mediawiki/1.40/maintenance/run.php /srv/mediawiki/1.40/maintenance/manageJobs.php --wiki=avidwiki --repush-abandoned (END - exit=256) [06:08:13] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, mwdedi1, mwdedi2, jobrunner1] [06:08:19] !log [@jobrunner1] finished deploy of {'config': True, 'versions': ['']} to [mw1, mw2, mwdedi1, mwdedi2, jobrunner1] - SUCCESS in 5s [06:08:58] [02WikiForge/puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/1a62cb2e5b7c...f41d6d9a70b6 [06:09:01] [02WikiForge/puppet] 07AgentIsai 03f41d6d9 - Return hash for custom backends [06:50:20] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [06:50:44] PROBLEM - phorge1 Puppet on phorge1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 19 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[phd] [08:01:10] PROBLEM - swiftac1 NTP time on swiftac1 is WARNING: NTP WARNING: Offset -0.1079190969 secs [08:09:20] RECOVERY - swiftac1 NTP time on swiftac1 is OK: NTP OK: Offset 0.04308572412 secs [12:31:12] RECOVERY - phorge1 phd on phorge1 is OK: PROCS OK: 1 process with args 'phd' [12:35:12] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [14:07:09] [02WikiForge/.github] 07dependabot[bot] pushed 031 commit to 03dependabot/github_actions/actions/cache-3.3.3 [+0/-0/±1] 13https://github.com/WikiForge/.github/commit/a102ad696ae2 [14:07:10] [02WikiForge/.github] 07dependabot[bot] 03a102ad6 - Bump actions/cache from 3.3.2 to 3.3.3 Bumps [actions/cache](https://github.com/actions/cache) from 3.3.2 to 3.3.3. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](https://github.com/actions/cache/compare/v3.3.2...v3.3.3) --- updated-dependencies: - dependency-name: actions/cache dependency-typ [14:07:13] [02.github] 07dependabot[bot] created branch 03dependabot/github_actions/actions/cache-3.3.3 - 13https://github.com/WikiForge/.github [14:07:15] [02.github] 07dependabot[bot] labeled pull request 03#6: Bump actions/cache from 3.3.2 to 3.3.3 - 13https://github.com/WikiForge/.github/pull/6 [14:07:17] [02.github] 07dependabot[bot] opened pull request 03#6: Bump actions/cache from 3.3.2 to 3.3.3 - 13https://github.com/WikiForge/.github/pull/6 [18:30:28] [02WikiForge/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/f269418f4f16...9765108ff4f1 [18:30:30] [02WikiForge/mw-config] 07AgentIsai 039765108 - Add 127.0.0.1 to wgCdnServersNoPurge [18:31:30] WikiForge/mw-config - AgentIsai the build passed. [18:35:28] RECOVERY - mail1 APT on mail1 is OK: APT OK: 0 packages available for upgrade (0 critical updates). [18:52:05] [02WikiForge/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/dns/compare/0beec0c23166...eb2899b2969f [18:52:07] [02WikiForge/dns] 07AgentIsai 03eb2899b - Add mail/phorge records [19:04:09] PROBLEM - phorge1 SSH on phorge1 is CRITICAL: connect to address 10.0.0.107 and port 22: No route to host [19:04:09] PROBLEM - Host phorge1 is DOWN: CRITICAL - Host Unreachable (10.0.0.107) [19:07:52] [02WikiForge/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/dns/compare/eb2899b2969f...b00a22ed8b12 [19:07:54] [02WikiForge/dns] 07AgentIsai 03b00a22e - Update inside.wf [19:10:20] [02WikiForge/puppet] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/f41d6d9a70b6...747aeffd6188 [19:10:21] [02WikiForge/puppet] 07AgentIsai 03747aeff - Update common.yaml [19:19:28] RECOVERY - Host phorge1 is UP: PING OK - Packet loss = 0%, RTA = 0.32 ms [19:19:41] RECOVERY - phorge1 SSH on phorge1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [20:11:29] RECOVERY - phorge1 phd on phorge1 is OK: PROCS OK: 1 process with args 'phd' [20:15:28] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [20:31:28] RECOVERY - phorge1 phd on phorge1 is OK: PROCS OK: 1 process with args 'phd' [20:35:28] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [21:25:28] RECOVERY - phorge1 phd on phorge1 is OK: PROCS OK: 1 process with args 'phd' [21:29:29] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [21:59:28] RECOVERY - phorge1 phd on phorge1 is OK: PROCS OK: 1 process with args 'phd' [22:03:28] PROBLEM - phorge1 phd on phorge1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'phd' [23:24:02] PROBLEM - swiftac1 NTP time on swiftac1 is WARNING: NTP WARNING: Offset 0.1062971056 secs [23:51:11] [02WikiForge/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/mw-config/compare/9765108ff4f1...5a831664c912 [23:51:13] [02WikiForge/mw-config] 07Universal-Omega 035a83166 - Disable $wgCdnMatchParameterOrder [23:52:12] WikiForge/mw-config - Universal-Omega the build passed. [23:56:36] RECOVERY - swiftac1 NTP time on swiftac1 is OK: NTP OK: Offset 0.05483528972 secs