[01:56:53] <botifico-49a823b>	 [02FileStorageMonitor] 07Universal-Omega closed pull request 03#2: Update mediawiki/mediawiki-phan-config requirement from 0.12.1 to 0.13.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/2
[01:56:56] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/FileStorageMonitor/compare/e22041317587...fdf4818efef5
[01:56:58] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07dependabot[bot] 03fdf4818 - Update mediawiki/mediawiki-phan-config requirement from 0.12.1 to 0.13.0 (#2) Updates the requirements on [mediawiki/mediawiki-phan-config](https://github.com/wikimedia/mediawiki-tools-phan) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-phan/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tools-phan/
[01:57:01] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.13.0
[01:57:03] <botifico-49a823b>	 [02FileStorageMonitor] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.13.0 - 13https://github.com/WikiForge/FileStorageMonitor
[01:57:04] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:05] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#2: Update mediawiki/mediawiki-phan-config requirement from 0.12.1 to 0.13.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/2
[01:57:11] <botifico-49a823b>	 [02FileStorageMonitor] 07Universal-Omega commented on pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3#issuecomment-1854988121
[01:57:13] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:29] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:30] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07dependabot[bot] pushed 031 commit to 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 [+0/-0/±1] 13https://github.com/WikiForge/FileStorageMonitor/compare/0a86cef22a79...1189d86a328d
[01:57:32] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07dependabot[bot] 031189d86 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-to
[01:57:34] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] synchronize pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:36] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:42] <botifico-49a823b>	 [02FileStorageMonitor] 07Universal-Omega closed pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[01:57:43] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0
[01:57:46] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/FileStorageMonitor/compare/fdf4818efef5...5c0edc2e383b
[01:57:47] <botifico-49a823b>	 [02WikiForge/FileStorageMonitor] 07dependabot[bot] 035c0edc2 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 (#3) Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawi
[01:57:49] <botifico-49a823b>	 [02FileStorageMonitor] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor
[01:57:50] <botifico-49a823b>	 [02FileStorageMonitor] 07dependabot[bot] edited pull request 03#3: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/FileStorageMonitor/pull/3
[02:03:22] <wikiforge-github>	 WikiForge/FileStorageMonitor - dependabot[bot] the build passed.
[02:03:31] <wikiforge-github>	 WikiForge/FileStorageMonitor - Universal-Omega the build passed.
[10:17:27] <icinga-wikiforge>	 PROBLEM - mon1 SSH on mon1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[10:18:06] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is CRITICAL: LOAD CRITICAL - total load average: 29.81, 29.64, 26.14
[10:18:37] <icinga-wikiforge>	 PROBLEM - mon1 grafana.inside.wf HTTPS on mon1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - db1 MariaDB Connections on db1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).> PROBLEM - mon1 Puppet on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. PROBLEM - mon1 Backups Grafana on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. PROBLEM 
[10:18:38] <icinga-wikiforge>	 ps Grafana on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[11:06:04] <icinga-wikiforge>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. PROBLEM - puppet1 Disk Space on puppet1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).> PROBLEM - phorge1 APT on phorge1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[11:27:22] <icinga-wikiforge>	 PROBLEM - mon1 APT on mon1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[11:28:58] <icinga-wikiforge>	 PROBLEM - rippaversewiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - puppet1 Disk Space on puppet1 is CRITICAL: DISK CRITICAL - free space: / 205MiB (2% inode=69%);
[11:29:07] <icinga-wikiforge>	 PROBLEM - phorge1 APT on phorge1 is CRITICAL: APT CRITICAL: 45 packages available for upgrade (4 critical updates).
[11:30:05] <icinga-wikiforge>	 RECOVERY - mon1 SSH on mon1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u1 (protocol 2.0) RECOVERY - mon1 Backups Grafana on mon1 is OK: FILE_AGE OK: /var/log/grafana-backup.log is 635402 seconds old and 349 bytes
[11:30:07] <icinga-wikiforge>	 RECOVERY - db1 MariaDB Connections on db1 is OK: OK connection usage: 1.4%Current connections: 7
[11:30:17] <icinga-wikiforge>	 PROBLEM - mon1 APT on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[11:31:13] <icinga-wikiforge>	 RECOVERY - mon1 grafana.inside.wf HTTPS on mon1 is OK: HTTP OK: HTTP/1.1 302 Found - 429 bytes in 0.084 second response time
[11:31:54] <icinga-wikiforge>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[11:32:12] <icinga-wikiforge>	 RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[11:57:26] <icinga-wikiforge>	 RECOVERY - rippaversewiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'rippaversewiki.com' will expire on Mon 12 Feb 2024 03:53:15 AM GMT +0000.
[11:58:53] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is WARNING: LOAD WARNING - total load average: 0.05, 0.17, 5.63
[12:00:52] <icinga-wikiforge>	 RECOVERY - mon1 Current Load on mon1 is OK: LOAD OK - total load average: 0.03, 0.12, 4.94
[17:17:01] <botifico-49a823b>	 [02DataDump] 07Universal-Omega closed pull request 03#1: Update mediawiki/mediawiki-phan-config requirement from 0.12.1 to 0.13.0 - 13https://github.com/WikiForge/DataDump/pull/1
[17:17:02] <botifico-49a823b>	 [02WikiForge/DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/DataDump/compare/d4a3f6f29d96...7781ddf3aed6
[17:17:04] <botifico-49a823b>	 [02WikiForge/DataDump] 07dependabot[bot] 037781ddf - Update mediawiki/mediawiki-phan-config requirement from 0.12.1 to 0.13.0 (#1) Updates the requirements on [mediawiki/mediawiki-phan-config](https://github.com/wikimedia/mediawiki-tools-phan) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-phan/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tools-phan/compare/0.
[17:17:05] <botifico-49a823b>	 [02WikiForge/DataDump] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.13.0
[17:17:06] <botifico-49a823b>	 [02DataDump] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-phan-config-0.13.0 - 13https://github.com/WikiForge/DataDump
[17:17:07] <botifico-49a823b>	 [02DataDump] 07dependabot[bot] edited pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2
[17:17:17] <botifico-49a823b>	 [02DataDump] 07Universal-Omega commented on pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2#issuecomment-1856256391
[17:17:21] <botifico-49a823b>	 [02DataDump] 07dependabot[bot] commented on pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2#issuecomment-1856256470
[17:17:36] <botifico-49a823b>	 [02DataDump] 07Universal-Omega commented on pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2#issuecomment-1856256841
[17:17:39] <botifico-49a823b>	 [02DataDump] 07dependabot[bot] edited pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2
[17:17:54] <botifico-49a823b>	 [02DataDump] 07dependabot[bot] edited pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2
[17:17:56] <botifico-49a823b>	 [02WikiForge/DataDump] 07dependabot[bot] pushed 031 commit to 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 [+0/-0/±1] 13https://github.com/WikiForge/DataDump/compare/150dc26f262d...450e700e9bfb
[17:17:59] <botifico-49a823b>	 [02WikiForge/DataDump] 07dependabot[bot] 03450e700 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tools-codesn
[17:18:01] <botifico-49a823b>	 [02DataDump] 07dependabot[bot] synchronize pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2
[17:18:03] <botifico-49a823b>	 [02DataDump] 07Universal-Omega closed pull request 03#2: Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 - 13https://github.com/WikiForge/DataDump/pull/2
[17:18:06] <botifico-49a823b>	 [02WikiForge/DataDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/DataDump/compare/7781ddf3aed6...4d25df5975d1
[17:18:08] <botifico-49a823b>	 [02WikiForge/DataDump] 07dependabot[bot] 034d25df5 - Update mediawiki/mediawiki-codesniffer requirement from 41.0.0 to 42.0.0 (#2) Updates the requirements on [mediawiki/mediawiki-codesniffer](https://github.com/wikimedia/mediawiki-tools-codesniffer) to permit the latest version. - [Changelog](https://github.com/wikimedia/mediawiki-tools-codesniffer/blob/master/HISTORY.md) - [Commits](https://github.com/wikimedia/mediawiki-tools-c
[17:18:11] <botifico-49a823b>	 [02WikiForge/DataDump] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0
[17:18:13] <botifico-49a823b>	 [02DataDump] 07Universal-Omega deleted branch 03dependabot/composer/mediawiki/mediawiki-codesniffer-42.0.0 - 13https://github.com/WikiForge/DataDump
[17:23:04] <wikiforge-github>	 WikiForge/DataDump - dependabot[bot] the build passed.
[17:23:13] <wikiforge-github>	 WikiForge/DataDump - Universal-Omega the build passed.
[17:24:58] <botifico-49a823b>	 [02WikiForge/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiForge/puppet/compare/a8580fa9bd7b...0736b4840ce3
[17:25:01] <botifico-49a823b>	 [02WikiForge/puppet] 07Universal-Omega 030736b48 - prometheus-jobqueue-stats: add RottenLinksJob
[17:30:28] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/ea14d7a3ba20...7a09e0e6ef74
[17:30:29] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega 037a09e0e - Update qqq
[17:30:32] <botifico-49a823b>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[17:35:55] <wikiforge-github>	 WikiForge/RottenLinks - Universal-Omega the build passed.
[17:48:06] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/7a09e0e6ef74...9ca275aa71e2
[17:48:07] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega 039ca275a - Remove script stats
[17:48:09] <botifico-49a823b>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[17:49:04] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/9ca275aa71e2...dd2e8bf8f900
[17:49:06] <botifico-49a823b>	 [02WikiForge/RottenLinks] 07Universal-Omega 03dd2e8bf - Remove script stats messages
[17:49:09] <botifico-49a823b>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[17:51:30] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/dd2e8bf8f900...53e0bbe4415c
[17:51:31] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 0353e0bbe - Add messages to qqq
[17:51:33] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[17:56:24] <wikiforge-github>	 WikiForge/RottenLinks - Universal-Omega the build passed.
[19:46:16] <icinga-wikiforge>	 PROBLEM - mw2 APT on mw2 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[19:47:12] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is CRITICAL: LOAD CRITICAL - total load average: 24.79, 24.37, 21.35
[19:47:19] <icinga-wikiforge>	 PROBLEM - mw2 APT on mw2 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (8 critical updates).
[19:48:18] <icinga-wikiforge>	 PROBLEM - mon1 Puppet on mon1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[20:02:27] <icinga-wikiforge>	 PROBLEM - db1 MariaDB Connections on db1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[20:19:33] <icinga-wikiforge>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[20:22:02] <icinga-wikiforge>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[20:22:14] <icinga-wikiforge>	 RECOVERY - db1 MariaDB Connections on db1 is OK: OK connection usage: 2.6%Current connections: 13
[20:24:11] <icinga-wikiforge>	 RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures
[20:45:31] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is WARNING: LOAD WARNING - total load average: 0.09, 0.28, 5.71
[20:47:30] <icinga-wikiforge>	 RECOVERY - mon1 Current Load on mon1 is OK: LOAD OK - total load average: 0.53, 0.37, 5.08
[21:47:22] <icinga-wikiforge>	 PROBLEM - mon1 grafana.inside.wf HTTPS on mon1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[21:48:04] <icinga-wikiforge>	 PROBLEM - ldap1 LDAP on ldap1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - mon1 SSH on mon1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - mon1 monitoring.inside.wf HTTPS on mon1 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[21:48:26] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is CRITICAL: LOAD CRITICAL - total load average: 32.64, 32.65, 26.25 PROBLEM - mon1 Current Load on mon1 is CRITICAL: LOAD CRITICAL - total load average: 32.64, 32.65, 26.25
[21:49:20] <icinga-wikiforge>	 PROBLEM - mon1 Puppet on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. PROBLEM - db1 MariaDB Connections on db1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[21:52:24] <icinga-wikiforge>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[23:17:50] <icinga-wikiforge>	 PROBLEM - mon1 Current Load on mon1 is CRITICAL: LOAD CRITICAL - total load average: 30.83, 41.37, 39.96 PROBLEM - mw2 APT on mw2 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (8 critical updates). PROBLEM - mon1 PowerDNS Recursor on mon1 is CRITICAL: CRITICAL - Plugin timed out while executing system call PROBLEM - wc.wikiforge.net on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - dcmultivers
[23:17:50] <icinga-wikiforge>	 LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds PROBLEM - cloud1 APT on cloud1 is CRITICAL: APT CRITICAL: 36 packages available for upgrade (2 critical updates). PROBLEM - graylog1 PowerDNS Recursor on graylog1 is CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - mw2 PowerDNS Recursor on mw2 is CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - os1 PowerDNS Recurs
[23:17:50] <icinga-wikiforge>	  CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - db1 Puppet on db1 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. PROBLEM - cloud1 PowerDNS Recursor on cloud1 is CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - jobchron1 PowerDNS Recursor on jobchron1 is CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - 
[23:17:50] <icinga-wikiforge>	  Recursor on mw1 is CRITICAL: Domain 'wikiforge.net' was not found by the server PROBLEM - meta.wikiforge.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[23:17:54] <icinga-wikiforge>	 PROBLEM - mon1 APT on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. PROBLEM - mon1 Puppet on mon1 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds.
[23:17:55] <icinga-wikiforge>	 RECOVERY - mon1 SSH on mon1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u1 (protocol 2.0)
[23:28:58] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/53e0bbe4415c...d213b06cf8d8
[23:29:00] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 03d213b06 - Use selectQueryBuilder
[23:29:02] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:34:08] <wikiforge-github>	 WikiForge/RottenLinks - Universal-Omega the build passed.
[23:40:39] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/d213b06cf8d8...90e0478de8f8
[23:40:40] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 0390e0478 - Use selectquerybuilder
[23:40:41] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:44:32] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/90e0478de8f8...4f631b562c4c
[23:44:35] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 034f631b5 - use selectQueryBuilder
[23:44:38] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:48:55] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/4f631b562c4c...7d0f1c71cccf
[23:48:58] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 037d0f1c7 - use SelectQueryBuilder
[23:49:01] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:52:42] <icinga-wikiforge>	 PROBLEM - mon1 Check correctness of the icinga configuration on mon1 is UNKNOWN: <Timeout exceeded.><Terminated by signal 15 (Terminated).>
[23:52:47] <icinga-wikiforge>	 RECOVERY - wc.wikiforge.net on sslhost is OK: OK - Certificate 'wikiforge.net' will expire on Sun 04 Feb 2024 02:51:55 AM GMT +0000.
[23:52:53] <icinga-wikiforge>	 RECOVERY - mon1 monitoring.inside.wf HTTPS on mon1 is OK: HTTP OK: HTTP/1.1 302 Found - 298 bytes in 6.896 second response time
[23:53:10] <icinga-wikiforge>	 RECOVERY - dcmultiversewiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'dcmultiversewiki.com' will expire on Mon 12 Feb 2024 07:02:02 PM GMT +0000. RECOVERY - meta.wikiforge.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wikiforge.net' will expire on Sun 04 Feb 2024 02:51:55 AM GMT +0000.
[23:53:12] <icinga-wikiforge>	 RECOVERY - db1 MariaDB Connections on db1 is OK: OK connection usage: 1.4%Current connections: 7
[23:54:06] <botifico>	 [02RottenLinks] 07Universal-Omega edited pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:54:10] <wikiforge-github>	 WikiForge/RottenLinks - Universal-Omega the build passed.
[23:55:57] <botifico>	 [02RottenLinks] 07Universal-Omega synchronize pull request 03#3: Redesign RottenLinks to not depend on a maintenance script - 13https://github.com/WikiForge/RottenLinks/pull/3
[23:56:00] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega pushed 031 commit to 03redesign [+0/-0/±1] 13https://github.com/WikiForge/RottenLinks/compare/7d0f1c71cccf...a29f8b1e5e93
[23:56:01] <botifico>	 [02WikiForge/RottenLinks] 07Universal-Omega 03a29f8b1 - Update CHANGELOG
[23:56:23] <icinga-wikiforge>	 RECOVERY - mon1 grafana.inside.wf HTTPS on mon1 is OK: HTTP OK: HTTP/1.1 302 Found - 429 bytes in 0.040 second response time
[23:56:33] <icinga-wikiforge>	 RECOVERY - mon1 Puppet on mon1 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures
[23:56:51] <icinga-wikiforge>	 RECOVERY - Host mail1 is UP: PING OK - Packet loss = 0%, RTA = 0.31 ms
[23:57:04] <icinga-wikiforge>	 RECOVERY - mail1 php-fpm on mail1 is OK: PROCS OK: 9 processes with command name 'php-fpm8.2'
[23:57:05] <icinga-wikiforge>	 RECOVERY - mon1 Check correctness of the icinga configuration on mon1 is OK: Icinga configuration is correct
[23:58:14] <icinga-wikiforge>	 RECOVERY - mail1 Puppet on mail1 is OK: OK: Puppet is currently enabled, last run 5 minutes ago with 0 failures
[23:58:19] <icinga-wikiforge>	 RECOVERY - mail1 NTP time on mail1 is OK: NTP OK: Offset -0.000163435936 secs
[23:58:24] <icinga-wikiforge>	 RECOVERY - mail1 Current Load on mail1 is OK: LOAD OK - total load average: 0.03, 0.03, 0.00
[23:58:39] <icinga-wikiforge>	 RECOVERY - mail1 Disk Space on mail1 is OK: DISK OK - free space: / 25125MiB (86% inode=93%);
[23:58:39] <icinga-wikiforge>	 RECOVERY - mail1 SSH on mail1 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u1 (protocol 2.0)