[00:08:51] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[00:10:51] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [Inlet Temp = Critical, 140 system event log (SEL) entries present]
[00:59:09] <icinga-miraheze>	 PROBLEM - wikislamica.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikislamica.org' expires in 15 day(s) (Thu 01 Aug 2024 12:57:05 AM GMT +0000).
[00:59:23] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5f6ff08c57ee...8af24b0979e1
[00:59:26] <botifico>	 [02ssl] 07WikiTideSSLBot 038af24b0 - Bot: Update SSL cert for wikislamica.org
[01:06:46] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[01:08:46] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [Inlet Temp = Critical, 142 system event log (SEL) entries present]
[01:21:21] <icinga-miraheze>	 PROBLEM - wiki.ivy.cm - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ivy.cm' expires in 15 day(s) (Thu 01 Aug 2024 01:00:58 AM GMT +0000).
[01:21:33] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8af24b0979e1...e135d46cef0f
[01:21:36] <botifico>	 [02ssl] 07WikiTideSSLBot 03e135d46 - Bot: Update SSL cert for wiki.ivy.cm
[01:28:30] <icinga-miraheze>	 RECOVERY - wikislamica.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wikislamica.org' will expire on Sun 13 Oct 2024 11:59:16 PM GMT +0000.
[01:36:11] <icinga-miraheze>	 PROBLEM - wiki.mcjones.gay - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mcjones.gay' expires in 15 day(s) (Thu 01 Aug 2024 01:14:15 AM GMT +0000).
[01:36:22] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e135d46cef0f...c47b0a32042b
[01:36:25] <botifico>	 [02ssl] 07WikiTideSSLBot 03c47b0a3 - Bot: Update SSL cert for wiki.mcjones.gay
[01:42:25] <icinga-miraheze>	 [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[01:47:42] <PixDeVl>	 another day another complaint from Icinga
[01:50:34] <icinga-miraheze>	 RECOVERY - wiki.ivy.cm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ivy.cm' will expire on Mon 14 Oct 2024 12:21:27 AM GMT +0000.
[01:52:41] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[01:54:41] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [143 system event log (SEL) entries present]
[02:05:52] <icinga-miraheze>	 RECOVERY - wiki.mcjones.gay - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mcjones.gay' will expire on Mon 14 Oct 2024 12:36:17 AM GMT +0000.
[03:28:32] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[03:30:32] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [145 system event log (SEL) entries present]
[03:51:03] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 28.30, 23.27, 18.51
[03:56:57] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 20.66, 23.38, 20.29
[04:02:50] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 18.15, 20.22, 19.86
[04:06:43] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 19.30, 20.53, 20.11
[04:10:39] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 24.59, 21.78, 20.62
[04:12:37] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 19.82, 21.04, 20.50
[04:14:27] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[04:14:34] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 9.75, 16.82, 19.02
[04:16:27] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [146 system event log (SEL) entries present]
[04:21:12] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 27.51, 20.51, 18.03
[04:23:10] <icinga-miraheze>	 RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 16.96, 18.99, 17.80
[04:36:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.63, 20.22, 18.06
[04:38:32] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.67, 18.52, 17.68
[05:16:18] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[05:22:25] <icinga-miraheze>	 [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[05:41:16] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=baldisbasicswiki /home/reception/baldisbasicsineducationandlearning_pages_full.xml --username-prefix=wikia:baldis-basics-in-education-and-learning (START)
[05:41:22] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[05:44:18] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[05:50:17] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN:> Your server seems to be powered off. (Execution of FreeIPMI returned an empty output or only 1 header row!)-> /usr/sbin/ipmi-sensors was executed with the following parameters: [HIDDEN]sudo /usr/sbin/ipmi-sensors --exclude-sensor-types Drive_Slot,Entity_Presence --quiet-cache --sdr-cache-recreate --interpret-oem-data --output-sensor-state --ignore-not-available-sensors --output-sens
[05:50:17] <icinga-miraheze>	 lds
[05:52:18] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [148 system event log (SEL) entries present]
[06:04:48] <botifico>	 [02ManageWiki] 07BlankEclair opened pull request 03#474: T12340: Replace : in namespaces with _ - 13https://github.com/miraheze/ManageWiki/pull/474
[06:04:51] <botifico>	 [02mw-config] 07BlankEclair opened pull request 03#5608: T12340: Replace : in namespaces with _ - 13https://github.com/miraheze/mw-config/pull/5608
[06:04:53] <botifico>	 [02ManageWiki] 07coderabbitai[bot] commented on pull request 03#474: T12340: Replace : in namespaces with _ - 13https://github.com/miraheze/ManageWiki/pull/474#issuecomment-2230092571
[06:05:45] <miraheze-github>	 miraheze/mw-config - BlankEclair the build passed.
[06:09:32] <miraheze-github>	 miraheze/ManageWiki - BlankEclair the build passed.
[06:34:14] <icinga-miraheze>	 PROBLEM - legacygt.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - legacygt.wiki All nameservers failed to answer the query.
[06:55:25] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.58, 20.70, 18.28
[07:01:13] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.81, 19.73, 18.67
[07:04:08] <icinga-miraheze>	 PROBLEM - sonicpedia.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - sonicpedia.org All nameservers failed to answer the query.
[07:04:10] <icinga-miraheze>	 RECOVERY - legacygt.wiki - reverse DNS on sslhost is OK: SSL OK - legacygt.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK
[07:16:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[07:17:01] <icinga-miraheze>	 PROBLEM - wiki.ff6worldscollide.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ff6worldscollide.com All nameservers failed to answer the query.
[07:17:19] <icinga-miraheze>	 PROBLEM - wiki.leapit.no - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.leapit.no All nameservers failed to answer the query.
[07:18:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [150 system event log (SEL) entries present]
[07:18:55] <icinga-miraheze>	 PROBLEM - wiki.wikimedia.cat - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.wikimedia.cat All nameservers failed to answer the query.
[07:31:32] <icinga-miraheze>	 PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query.
[07:34:04] <icinga-miraheze>	 RECOVERY - sonicpedia.org - reverse DNS on sslhost is OK: SSL OK - sonicpedia.org reverse DNS resolves to cp36.wikitide.net - CNAME FLAT
[07:39:53] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.82, 20.84, 19.32
[07:41:32] <icinga-miraheze>	 PROBLEM - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.tmyt105.leyhp.com All nameservers failed to answer the query.
[07:45:32] <icinga-miraheze>	 RECOVERY - wiki.ff6worldscollide.com - reverse DNS on sslhost is OK: SSL OK - wiki.ff6worldscollide.com reverse DNS resolves to cp36.wikitide.net - CNAME OK
[07:45:40] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.53, 18.44, 18.86
[07:46:51] <icinga-miraheze>	 RECOVERY - wiki.leapit.no - reverse DNS on sslhost is OK: SSL OK - wiki.leapit.no reverse DNS resolves to cp36.wikitide.net - CNAME OK
[07:48:17] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_dns]
[07:48:30] <icinga-miraheze>	 RECOVERY - wiki.wikimedia.cat - reverse DNS on sslhost is OK: SSL OK - wiki.wikimedia.cat reverse DNS resolves to cp36.wikitide.net - CNAME OK
[07:50:17] <icinga-miraheze>	 PROBLEM - with.cpt-ra.bid - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket
[08:01:05] <icinga-miraheze>	 RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK
[08:04:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[08:06:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [151 system event log (SEL) entries present]
[08:06:53] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=baldisbasicswiki /home/reception/baldisbasicsineducationandlearning_pages_full.xml --username-prefix=wikia:baldis-basics-in-education-and-learning (START)
[08:06:59] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[08:11:10] <icinga-miraheze>	 RECOVERY - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is OK: SSL OK - wiki.tmyt105.leyhp.com reverse DNS resolves to cp36.wikitide.net - CNAME OK
[08:14:23] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[08:18:56] <icinga-miraheze>	 RECOVERY - with.cpt-ra.bid - LetsEncrypt on sslhost is OK: OK - Certificate 'with.cpt-ra.bid' will expire on Tue 06 Aug 2024 02:45:02 PM GMT +0000.
[08:29:07] <icinga-miraheze>	 PROBLEM - wiki.digitalcandela.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.digitalcandela.com All nameservers failed to answer the query.
[08:31:15] <icinga-miraheze>	 PROBLEM - minescape.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - minescape.wiki All nameservers failed to answer the query.
[08:31:22] <icinga-miraheze>	 PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query.
[08:32:28] <icinga-miraheze>	 PROBLEM - wiki.mobilityengineer.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mobilityengineer.com All nameservers failed to answer the query.
[08:36:50] <icinga-miraheze>	 PROBLEM - tno.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - tno.wiki All nameservers failed to answer the query.
[08:48:33] <icinga-miraheze>	 PROBLEM - wiki.solarconquest.net - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.solarconquest.net All nameservers failed to answer the query.
[08:50:02] <icinga-miraheze>	 PROBLEM - wiki.whentheycry.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.whentheycry.org All nameservers failed to answer the query.
[08:50:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[08:52:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [151 system event log (SEL) entries present]
[08:57:44] <icinga-miraheze>	 RECOVERY - wiki.digitalcandela.com - reverse DNS on sslhost is OK: SSL OK - wiki.digitalcandela.com reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:00:31] <icinga-miraheze>	 RECOVERY - minescape.wiki - reverse DNS on sslhost is OK: SSL OK - minescape.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT
[09:01:27] <icinga-miraheze>	 RECOVERY - wiki.mobilityengineer.com - reverse DNS on sslhost is OK: SSL OK - wiki.mobilityengineer.com reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:05:41] <icinga-miraheze>	 RECOVERY - tno.wiki - reverse DNS on sslhost is OK: SSL OK - tno.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:17:08] <icinga-miraheze>	 RECOVERY - wiki.solarconquest.net - reverse DNS on sslhost is OK: SSL OK - wiki.solarconquest.net reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:18:34] <icinga-miraheze>	 RECOVERY - wiki.whentheycry.org - reverse DNS on sslhost is OK: SSL OK - wiki.whentheycry.org reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:30:44] <icinga-miraheze>	 RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK
[09:46:32] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[09:56:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 24.00, 22.24, 20.15
[09:58:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.26, 23.01, 20.67
[10:00:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.67, 21.81, 20.49
[10:02:25] <icinga-miraheze>	 [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[10:04:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.30, 22.26, 20.89
[10:06:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 18.32, 20.69, 20.49
[10:08:32] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.92, 19.66, 20.10
[10:14:16] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 43 seconds ago with 0 failures
[10:22:25] <icinga-miraheze>	 [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1
[10:26:16] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.89, 21.31, 20.55
[10:33:59] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.17, 22.25, 21.05
[10:43:39] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.60, 23.23, 22.52
[11:02:57] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.40, 17.76, 19.90
[11:25:47] <icinga-miraheze>	 PROBLEM - opendatascot.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'opendatascot.org' expires in 15 day(s) (Thu 01 Aug 2024 11:13:15 AM GMT +0000).
[11:26:02] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c47b0a32042b...5d082576f0fc
[11:26:05] <botifico>	 [02ssl] 07WikiTideSSLBot 035d08257 - Bot: Update SSL cert for opendatascot.org
[11:26:40] <icinga-miraheze>	 PROBLEM - mwtask181 Disk Space on mwtask181 is WARNING: DISK WARNING - free space: / 20256MiB (9% inode=94%);
[11:34:40] <icinga-miraheze>	 PROBLEM - mwtask181 Disk Space on mwtask181 is CRITICAL: DISK CRITICAL - free space: / 11679MiB (5% inode=94%);
[11:40:56] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[11:40:56] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [153 system event log (SEL) entries present]
[11:46:22] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_dns]
[11:55:31] <icinga-miraheze>	 RECOVERY - opendatascot.org - LetsEncrypt on sslhost is OK: OK - Certificate 'opendatascot.org' will expire on Mon 14 Oct 2024 10:25:53 AM GMT +0000.
[12:12:19] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[12:14:18] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures
[12:14:19] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [154 system event log (SEL) entries present]
[13:08:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 26.95, 19.97, 17.10
[13:12:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.69, 21.50, 18.50
[13:14:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.04, 22.15, 19.06
[13:16:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.64, 21.57, 19.18
[13:18:32] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.85, 18.50, 18.34
[13:26:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[13:26:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.79, 21.66, 19.77
[13:28:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [156 system event log (SEL) entries present]
[13:28:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.94, 20.48, 19.59
[13:30:32] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.43, 20.27, 19.61
[14:38:14] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[14:40:14] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [157 system event log (SEL) entries present]
[15:40:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[15:42:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [158 system event log (SEL) entries present]
[15:44:32] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.12, 18.66, 17.05
[15:46:32] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.74, 16.61, 16.50
[16:16:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 23.97, 20.44, 16.24
[16:18:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 25.06, 21.17, 16.96
[16:26:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 18.01, 22.20, 19.80
[16:31:19] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 24.25, 22.41, 18.49
[16:33:17] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 23.68, 22.22, 18.84
[16:38:28] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 15.79, 19.82, 20.15
[16:39:09] <icinga-miraheze>	 RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 11.70, 18.89, 18.82
[16:44:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 24.00, 23.25, 21.37
[16:48:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 24.59, 23.62, 21.87
[16:51:50] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 23.12, 21.39, 19.90
[16:52:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 17.23, 21.46, 21.48
[16:55:45] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 24.03, 21.39, 20.11
[16:59:40] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 18.78, 21.11, 20.39
[17:00:28] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 17.25, 18.79, 20.21
[17:01:38] <icinga-miraheze>	 RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 15.45, 19.71, 20.00
[17:07:28] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 25.92, 21.97, 20.68
[17:08:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 22.36, 19.52, 19.55
[17:09:26] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 20.09, 21.45, 20.68
[17:10:28] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 17.49, 18.33, 19.10
[17:11:24] <icinga-miraheze>	 RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 20.23, 20.38, 20.34
[17:15:16] <icinga-miraheze>	 PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 21.18, 20.98, 20.61
[17:17:14] <icinga-miraheze>	 RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 17.20, 19.86, 20.27
[17:23:20] <icinga-miraheze>	 PROBLEM - wiki.case-clicker.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.case-clicker.com' expires in 15 day(s) (Thu 01 Aug 2024 05:09:54 PM GMT +0000).
[17:23:31] <botifico>	 [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5d082576f0fc...7c4287ec4a63
[17:23:33] <botifico>	 [02ssl] 07WikiTideSSLBot 037c4287e - Bot: Update SSL cert for wiki.case-clicker.com
[17:42:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[17:43:44] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.03, 20.27, 18.92
[17:44:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [160 system event log (SEL) entries present]
[17:45:40] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.83, 19.34, 18.72
[17:52:10] <icinga-miraheze>	 RECOVERY - wiki.case-clicker.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.case-clicker.com' will expire on Mon 14 Oct 2024 04:23:25 PM GMT +0000.
[17:54:28] <icinga-miraheze>	 PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 24.85, 20.84, 18.66
[17:55:23] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.33, 21.08, 19.73
[17:56:28] <icinga-miraheze>	 RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 15.34, 18.77, 18.18
[17:57:19] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.02, 21.40, 19.95
[17:59:14] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.50, 21.72, 20.22
[18:06:58] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.78, 19.78, 20.08
[19:33:28] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H <host> [-4|-6] [-w <warn>] [-c <crit>] [-v verbose] [-o <time offset>]
[19:37:44] <icinga-miraheze>	 PROBLEM - ns2 NTP time on ns2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds
[19:39:43] <icinga-miraheze>	 RECOVERY - ns2 NTP time on ns2 is OK: NTP OK: Offset -0.0008193552494 secs
[19:52:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[19:54:00] <MirahezeLSBot>	 !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151
[19:54:02] <MirahezeLSBot>	 !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s
[19:54:08] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[19:54:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [162 system event log (SEL) entries present]
[19:54:14] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[20:17:50] <icinga-miraheze>	 PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle.
[20:51:23] <icinga-miraheze>	 PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 59 packages available for upgrade (1 critical updates).
[20:51:53] <icinga-miraheze>	 PROBLEM - cp41 APT on cp41 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:52:12] <icinga-miraheze>	 PROBLEM - bast161 APT on bast161 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:52:24] <icinga-miraheze>	 PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 56 packages available for upgrade (1 critical updates).
[20:52:47] <icinga-miraheze>	 PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[20:52:48] <icinga-miraheze>	 PROBLEM - cp27 APT on cp27 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:53:04] <icinga-miraheze>	 PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:53:16] <icinga-miraheze>	 PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[20:53:41] <icinga-miraheze>	 PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (1 critical updates).
[20:53:56] <icinga-miraheze>	 PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates).
[20:54:03] <icinga-miraheze>	 PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates).
[20:54:38] <icinga-miraheze>	 PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 52 packages available for upgrade (1 critical updates).
[20:54:40] <icinga-miraheze>	 PROBLEM - cp26 APT on cp26 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:55:15] <icinga-miraheze>	 PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 76 packages available for upgrade (1 critical updates).
[20:55:29] <icinga-miraheze>	 PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates).
[20:55:54] <icinga-miraheze>	 PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:55:54] <icinga-miraheze>	 PROBLEM - mem161 APT on mem161 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:56:12] <icinga-miraheze>	 PROBLEM - cp36 APT on cp36 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:56:40] <icinga-miraheze>	 PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates).
[20:56:47] <icinga-miraheze>	 PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 109 packages available for upgrade (1 critical updates).
[20:56:58] <icinga-miraheze>	 PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:57:11] <icinga-miraheze>	 PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (1 critical updates).
[20:57:28] <icinga-miraheze>	 PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 54 packages available for upgrade (1 critical updates).
[20:57:45] <icinga-miraheze>	 PROBLEM - swiftac171 APT on swiftac171 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (1 critical updates).
[20:58:00] <icinga-miraheze>	 PROBLEM - os161 APT on os161 is CRITICAL: APT CRITICAL: 54 packages available for upgrade (1 critical updates).
[20:58:13] <icinga-miraheze>	 PROBLEM - graphite151 APT on graphite151 is CRITICAL: APT CRITICAL: 41 packages available for upgrade (1 critical updates).
[20:58:24] <icinga-miraheze>	 PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (1 critical updates).
[20:58:39] <icinga-miraheze>	 PROBLEM - db171 APT on db171 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (1 critical updates).
[20:59:00] <icinga-miraheze>	 PROBLEM - db161 APT on db161 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (1 critical updates).
[20:59:07] <icinga-miraheze>	 PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[20:59:10] <icinga-miraheze>	 PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (1 critical updates).
[20:59:25] <icinga-miraheze>	 PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (1 critical updates).
[20:59:35] <icinga-miraheze>	 PROBLEM - eventgate181 APT on eventgate181 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (1 critical updates).
[20:59:43] <icinga-miraheze>	 PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 54 packages available for upgrade (1 critical updates).
[20:59:47] <icinga-miraheze>	 PROBLEM - rdb151 APT on rdb151 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[21:00:22] <icinga-miraheze>	 PROBLEM - graylog161 APT on graylog161 is CRITICAL: APT CRITICAL: 57 packages available for upgrade (1 critical updates).
[21:00:45] <icinga-miraheze>	 PROBLEM - cloud18 APT on cloud18 is CRITICAL: APT CRITICAL: 102 packages available for upgrade (1 critical updates).
[21:00:46] <icinga-miraheze>	 PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[21:01:56] <icinga-miraheze>	 PROBLEM - mem151 APT on mem151 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[21:02:19] <icinga-miraheze>	 PROBLEM - cp37 APT on cp37 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[21:02:24] <icinga-miraheze>	 PROBLEM - db181 APT on db181 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (1 critical updates).
[21:02:25] <icinga-miraheze>	 PROBLEM - jobchron171 APT on jobchron171 is CRITICAL: APT CRITICAL: 63 packages available for upgrade (1 critical updates).
[21:02:43] <icinga-miraheze>	 PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 42 packages available for upgrade (1 critical updates).
[21:03:01] <icinga-miraheze>	 PROBLEM - cp51 APT on cp51 is CRITICAL: APT CRITICAL: 53 packages available for upgrade (1 critical updates).
[21:03:11] <icinga-miraheze>	 PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates).
[21:03:48] <icinga-miraheze>	 PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[21:03:56] <icinga-miraheze>	 PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[21:03:58] <icinga-miraheze>	 PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates).
[21:04:05] <icinga-miraheze>	 PROBLEM - db182 APT on db182 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (1 critical updates).
[21:04:06] <icinga-miraheze>	 PROBLEM - os162 APT on os162 is CRITICAL: APT CRITICAL: 55 packages available for upgrade (1 critical updates).
[21:04:20] <icinga-miraheze>	 PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 101 packages available for upgrade (1 critical updates).
[21:04:49] <icinga-miraheze>	 PROBLEM - kafka181 APT on kafka181 is CRITICAL: APT CRITICAL: 39 packages available for upgrade (1 critical updates).
[21:05:03] <icinga-miraheze>	 PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[21:06:05] <icinga-miraheze>	 PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (1 critical updates).
[21:06:48] <icinga-miraheze>	 PROBLEM - db151 APT on db151 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (1 critical updates).
[21:14:13] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[21:15:04] <icinga-miraheze>	 RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures
[21:16:13] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [Inlet Temp = Critical, 165 system event log (SEL) entries present]
[22:16:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[22:18:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [Inlet Temp = Critical, 167 system event log (SEL) entries present]
[22:48:56] <MirahezeLSBot>	 !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=baldisbasicswiki /home/reception/baldisbasicsineducationandlearning_pages_full.xml --username-prefix=wikia:baldis-basics-in-education-and-learning (END - exit=0)
[22:49:01] <MirahezeLogbot>	 Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log
[23:06:11] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is UNKNOWN: ipmi_sdr_cache_open: /root/.freeipmi/sdr-cache/sdr-cache-cloud15.localhost: internal IPMI error-> Execution of /usr/sbin/ipmi-sel failed with return code 1.-> /usr/sbin/ipmi-sel was executed with the following parameters: sudo /usr/sbin/ipmi-sel --output-event-state --interpret-oem-data --entity-sensor-names --sensor-types=all
[23:08:12] <icinga-miraheze>	 PROBLEM - cloud15 IPMI Sensors on cloud15 is CRITICAL: IPMI Status: Critical [168 system event log (SEL) entries present]
[23:44:52] <icinga-miraheze>	 PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 25.51, 20.15, 16.87
[23:46:48] <icinga-miraheze>	 RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.51, 18.99, 16.83