[00:02:32] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.59, 7.01, 5.64 [00:03:15] PROBLEM - db112 Current Load on db112 is CRITICAL: CRITICAL - load average: 14.83, 7.17, 3.40 [00:08:24] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.02, 7.11, 6.20 [00:10:21] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 5.40, 6.71, 6.17 [00:11:01] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 6.96, 5.79, 4.27 [00:11:27] PROBLEM - os131 Current Load on os131 is CRITICAL: LOAD CRITICAL - total load average: 4.16, 3.37, 2.15 [00:11:49] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.84, 3.50, 3.07 [00:13:01] RECOVERY - db131 Current Load on db131 is OK: OK - load average: 5.09, 5.86, 4.50 [00:13:25] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 3.27, 3.17, 2.22 [00:15:02] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp25.miraheze.org - CNAME OK [00:15:48] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.96, 3.39, 3.15 [00:17:15] PROBLEM - db112 Current Load on db112 is WARNING: WARNING - load average: 6.75, 7.86, 7.54 [00:17:19] PROBLEM - matomo121 Current Load on matomo121 is WARNING: LOAD WARNING - total load average: 3.61, 3.19, 2.34 [00:18:07] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.87, 6.71, 6.35 [00:19:19] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.69, 3.01, 2.38 [00:21:15] RECOVERY - db112 Current Load on db112 is OK: OK - load average: 2.90, 4.99, 6.41 [00:22:03] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 6.04, 6.63, 6.42 [00:30:03] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.22, 7.22, 6.64 [00:35:36] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [00:41:11] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/edfe100cc7c5...145d8e9dcba1 [00:41:14] [02miraheze/mw-config] 07paladox 03145d8e9 - redis: Experiment setting readTimeout to 2 [00:41:43] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.54, 3.10, 2.56 [00:42:06] !log [@mwtask141] starting deploy of {'config': True} to all [00:42:15] miraheze/mw-config - paladox the build passed. [00:42:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:42:30] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 25s [00:42:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:43:38] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.70, 2.93, 2.56 [00:43:51] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.65, 3.49, 2.87 [00:47:48] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.10, 3.50, 2.99 [00:48:30] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/145d8e9dcba1...df93414b52ef [00:48:33] [02miraheze/mw-config] 07paladox 03df93414 - Unset claimTTL for JobTypeConf [00:48:52] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [00:48:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:04] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 12s [00:49:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:13] !log [paladox@mwtask141] starting deploy of {'config': True} to all [00:49:20] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [00:49:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:26] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [00:49:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:28] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 1s [00:49:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:35] !log [paladox@test131] starting deploy of {'config': True} to all [00:49:36] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [00:49:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:38] miraheze/mw-config - paladox the build passed. [00:49:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:49:46] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.69, 3.19, 2.94 [00:49:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:52:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.24, 7.34, 7.81 [01:00:03] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.62, 7.87, 7.83 [01:08:31] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.09, 3.40, 2.97 [01:10:29] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.77, 2.98, 2.86 [01:24:01] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/b5742f5fa41b...573456c57b90 [01:24:02] [02miraheze/ssl] 07MirahezeSSLBot 03573456c - Bot: Add SSL cert for www.clinitheque.fr [01:24:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/573456c57b90...ec615857dbe4 [01:24:31] [02miraheze/ssl] 07MirahezeSSLBot 03ec61585 - Bot: Add SSL cert for richterian.com [01:25:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/ec615857dbe4...95a3bf8cd066 [01:25:37] [02miraheze/ssl] 07MirahezeSSLBot 0395a3bf8 - Bot: Add SSL cert for coffeewiki.net [01:38:03] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.29, 6.61, 7.98 [01:40:03] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.97, 7.42, 8.10 [01:57:36] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.59, 7.42, 6.40 [01:59:32] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 4.82, 6.44, 6.16 [02:05:29] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.18, 7.08, 7.85 [02:13:07] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.00, 7.27, 7.55 [02:15:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.94, 7.49, 7.61 [02:19:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.00, 7.85, 7.69 [02:34:06] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.98, 3.80, 3.27 [02:35:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.20, 7.26, 7.80 [02:36:01] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.59, 3.29, 3.14 [02:37:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.28, 7.64, 7.86 [03:01:23] RECOVERY - db121 Backups SQL on db121 is OK: FILE_AGE OK: /var/log/sql-backup.log is 81 seconds old and 0 bytes [03:31:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.49, 7.16, 7.91 [03:35:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.12, 7.41, 7.82 [03:37:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.08, 7.31, 7.73 [03:45:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.68, 7.60, 7.57 [03:51:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.60, 7.97, 7.98 [03:57:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.58, 3.43, 2.90 [03:59:38] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.19, 2.94, 2.78 [04:01:01] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.41, 7.71, 7.74 [04:05:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.86, 7.90, 7.82 [04:11:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.24, 7.73, 7.70 [04:13:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.97, 7.46, 7.61 [04:14:10] PROBLEM - wiki.albrusmusic.ru - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.402 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [04:14:48] [02miraheze/mw-config] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/df93414b52ef...5a5466d4bb13 [04:14:51] [02miraheze/mw-config] 07The-Voidwalker 035a5466d - disable parsoid cache warmup for now [04:15:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.78, 8.03, 7.80 [04:15:02] !log [void@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [04:15:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:15:13] !log [void@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [04:15:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:15:55] miraheze/mw-config - The-Voidwalker the build passed. [04:26:09] !log [@test131] starting deploy of {'config': True} to all [04:26:10] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [04:26:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:26:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:29:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 3.16, 2.63 [04:31:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.90, 3.14, 2.70 [04:36:35] PROBLEM - mail121 Current Load on mail121 is CRITICAL: LOAD CRITICAL - total load average: 16.99, 6.54, 2.52 [04:37:33] PROBLEM - mail121 PowerDNS Recursor on mail121 is CRITICAL: CRITICAL - Plugin timed out while executing system call [04:37:57] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.05, 7.23, 6.39 [04:39:00] PROBLEM - mail121 webmail.miraheze.org HTTPS on mail121 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [04:40:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [04:41:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.91, 7.90, 6.86 [04:42:51] PROBLEM - mail121 Puppet on mail121 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [04:43:01] RECOVERY - wiki.albrusmusic.ru - reverse DNS on sslhost is OK: SSL OK - wiki.albrusmusic.ru reverse DNS resolves to cp24.miraheze.org - CNAME OK [04:43:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.15, 8.20, 7.10 [04:43:57] RECOVERY - mail121 PowerDNS Recursor on mail121 is OK: DNS OK: 3.090 seconds response time. miraheze.org returns 2001:41d0:801:2000::3a18,2001:41d0:801:2000::5d68,51.195.201.140,51.89.139.24 [04:44:50] RECOVERY - mail121 Puppet on mail121 is OK: OK: Puppet is currently enabled, last run 31 minutes ago with 0 failures [04:45:03] RECOVERY - mail121 webmail.miraheze.org HTTPS on mail121 is OK: HTTP OK: HTTP/1.1 200 OK - 6169 bytes in 1.787 second response time [04:45:40] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.61, 7.58, 7.00 [04:51:28] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.60, 6.76 [05:10:30] PROBLEM - mail121 Current Load on mail121 is WARNING: LOAD WARNING - total load average: 0.16, 0.24, 1.95 [05:14:30] RECOVERY - mail121 Current Load on mail121 is OK: LOAD OK - total load average: 0.62, 0.49, 1.66 [05:43:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.89, 6.43, 6.15 [05:45:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.03, 6.13, 6.06 [05:49:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.59, 7.76, 7.99 [05:53:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.03, 7.94, 8.01 [05:57:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.60, 7.92, 8.00 [05:58:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.69, 3.53, 3.18 [06:02:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.73, 4.20, 3.52 [06:04:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.15, 3.49, 3.35 [06:07:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.77, 8.11, 8.01 [06:08:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.33, 3.03, 3.20 [06:12:08] PROBLEM - cloud10 Puppet on cloud10 is UNKNOWN: NRPE: Unable to read output [06:16:49] PROBLEM - swiftac111 Current Load on swiftac111 is WARNING: WARNING - load average: 7.39, 6.84, 5.79 [06:20:48] RECOVERY - swiftac111 Current Load on swiftac111 is OK: OK - load average: 5.57, 6.41, 5.88 [06:40:08] RECOVERY - cloud10 Puppet on cloud10 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [06:55:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.33, 6.91, 6.26 [06:57:49] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.52, 6.35, 6.12 [07:01:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.82, 7.45, 7.88 [07:13:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 9.09, 7.98, 7.82 [07:20:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [07:29:21] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [07:45:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 5.14, 7.03, 7.78 [07:55:00] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 5.45, 5.76, 6.68 [07:58:49] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns.ankh.fr.eu.org.', 'ns1.eu.org.', 'ns1.eriomem.net.'], 'CNAME': 'bouncingwiki.miraheze.org.'} [08:11:49] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.03, 6.85, 6.58 [08:13:44] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 5.94, 6.57, 6.52 [08:19:27] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.36, 7.45, 6.89 [08:21:22] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.09, 7.35, 6.92 [08:25:10] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 6.32, 6.76, 6.78 [08:29:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.11, 7.16, 6.93 [08:31:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.80, 6.97, 6.88 [08:31:42] PROBLEM - db131 Disk Space on db131 is WARNING: DISK WARNING - free space: / 26170 MB (10% inode=96%); [08:45:00] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 4.26, 5.75, 6.47 [09:03:03] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.75, 3.34, 2.93 [09:04:58] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.32, 2.95, 2.84 [09:13:52] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.56, 6.69, 6.21 [09:15:48] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.60, 6.26, 6.11 [09:19:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.81, 7.08, 6.45 [09:21:38] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.19, 6.78, 6.42 [10:57:53] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [11:26:41] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:33:28] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.48, 3.27, 2.91 [11:34:42] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/5a6896b55cc2 [11:34:44] [02miraheze/mw-config] 07paladox 035a6896b - Enable warmup for wikis with 0-9 or starting with a and also beta [11:34:47] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [11:34:49] [02mw-config] 07paladox opened pull request 03#5360: Enable warmup for wikis with 0-9 or starting with a and also beta - 13https://github.com/miraheze/mw-config/pull/5360 [11:35:10] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5a6896b55cc2...ec9b8533baad [11:35:11] [02miraheze/mw-config] 07github-actions 03ec9b853 - CI: lint code to MediaWiki standards [11:35:13] [02mw-config] 07github-actions[bot] synchronize pull request 03#5360: Enable warmup for wikis with 0-9 or starting with a and also beta - 13https://github.com/miraheze/mw-config/pull/5360 [11:35:22] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.52, 3.01, 2.86 [11:35:49] [02mw-config] 07paladox synchronize pull request 03#5360: Enable warmup for wikis with 0-9 or starting with a and also beta - 13https://github.com/miraheze/mw-config/pull/5360 [11:35:50] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ec9b8533baad...5137ffb2ea0a [11:35:51] miraheze/mw-config - paladox the build passed. [11:35:52] [02miraheze/mw-config] 07paladox 035137ffb - Update GlobalCache.php [11:37:03] miraheze/mw-config - paladox the build passed. [11:42:39] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d84f7343ac17...b2fbb5f81250 [11:42:40] [02miraheze/puppet] 07paladox 03b2fbb5f - mediawiki: lower parser cache to 7 days from 10 [12:05:42] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/575d4449e360 [12:05:45] [02miraheze/mw-config] 07paladox 03575d444 - Add db-mainstash [12:05:46] [02mw-config] 07paladox created branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [12:05:48] [02mw-config] 07paladox opened pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:06:20] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/575d4449e360...3be8471abce2 [12:06:21] [02miraheze/mw-config] 07github-actions 033be8471 - CI: lint code to MediaWiki standards [12:06:24] [02mw-config] 07github-actions[bot] synchronize pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:08:16] miraheze/mw-config - paladox the build passed. [12:08:53] [02mw-config] 07paladox synchronize pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:08:56] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3be8471abce2...2d7a5913370f [12:08:57] [02miraheze/mw-config] 07paladox 032d7a591 - Update GlobalCache.php [12:09:33] [02mw-config] 07paladox synchronize pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:09:34] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2d7a5913370f...1134f304341a [12:09:37] [02miraheze/mw-config] 07paladox 031134f30 - Update GlobalCache.php [12:09:58] miraheze/mw-config - paladox the build passed. [12:11:49] [02mw-config] 07paladox synchronize pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:11:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/1134f304341a...fd22d1ca7e51 [12:11:55] [02miraheze/mw-config] 07paladox 03fd22d1c - Update GlobalCache.php [12:12:52] miraheze/mw-config - paladox the build passed. [12:16:11] [02mw-config] 07paladox synchronize pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:16:13] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-5 [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/fd22d1ca7e51...e8dc0295266c [12:16:14] [02miraheze/mw-config] 07paladox 03e8dc029 - Update GlobalCache.php [12:17:09] miraheze/mw-config - paladox the build passed. [12:20:06] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b2fbb5f81250...377bfa924696 [12:20:09] [02miraheze/puppet] 07paladox 03377bfa9 - Add grants for mainstash and testmainstash [12:21:51] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/377bfa924696...f59f084d8126 [12:21:54] [02miraheze/puppet] 07paladox 03f59f084 - Add missing grant [12:23:10] [02mw-config] 07paladox closed pull request 03#5361: Add db-mainstash - 13https://github.com/miraheze/mw-config/pull/5361 [12:23:12] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5a5466d4bb13...8cdc606d7c4b [12:23:15] [02miraheze/mw-config] 07paladox 038cdc606 - Add db-mainstash (#5361) [12:23:17] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-5 [12:23:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.50, 7.21, 6.45 [12:23:20] [02mw-config] 07paladox deleted branch 03paladox-patch-5 - 13https://github.com/miraheze/mw-config [12:23:42] [02miraheze/mw-config] 07paladox pushed 032 commits to 03paladox-patch-4 [+0/-0/±2] 13https://github.com/miraheze/mw-config/compare/5137ffb2ea0a...547473f35c35 [12:23:43] [02miraheze/mw-config] 07paladox 03547473f - Merge branch 'master' into paladox-patch-4 [12:23:45] [02mw-config] 07paladox synchronize pull request 03#5360: Enable warmup for wikis with 0-9 or starting with a and also beta - 13https://github.com/miraheze/mw-config/pull/5360 [12:24:14] miraheze/mw-config - paladox the build passed. [12:24:17] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:24:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:28] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [12:24:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:35] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:24:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:43] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [12:24:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:50] miraheze/mw-config - paladox the build passed. [12:25:18] [02mw-config] 07paladox closed pull request 03#5360: Enable warmup for wikis with 0-9 or starting with a and also beta - 13https://github.com/miraheze/mw-config/pull/5360 [12:25:18] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:25:19] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8cdc606d7c4b...8af3d2052c9c [12:25:22] [02miraheze/mw-config] 07paladox 038af3d20 - Enable warmup for wikis with 0-9 or starting with a and also beta (#5360) [12:25:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:25] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [12:25:27] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [12:25:28] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [12:25:33] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:25:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:40] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 7s [12:25:41] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [12:25:43] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 2s [12:25:44] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:25:45] !log [paladox@test131] starting deploy of {'config': True} to all [12:25:46] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:25:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:54] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [12:25:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:26:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:26:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:26:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:26:18] miraheze/mw-config - paladox the build passed. [12:26:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:27:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.96, 6.57, 6.36 [12:39:32] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8af3d2052c9c...3f0592f4d67f [12:39:33] [02miraheze/mw-config] 07paladox 033f0592f - Unset wgMWOAuthSessionCacheType for beta [12:40:33] miraheze/mw-config - paladox the build passed. [12:40:54] !log [paladox@test131] starting deploy of {'pull': 'config', 'config': True} to all [12:40:55] !log [paladox@test131] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 1s [12:41:02] !log [paladox@test131] starting deploy of {'config': True} to all [12:41:03] !log [paladox@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:41:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:41:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:41:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:41:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:43:57] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.19, 3.61, 3.14 [12:45:51] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.16, 3.53, 3.18 [12:51:36] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.76, 3.35, 3.23 [12:51:45] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 8.00, 7.56, 6.80 [12:53:07] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp24.miraheze.org - CNAME OK [12:54:08] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3f0592f4d67f...6d235ebfd6b8 [12:54:09] [02miraheze/mw-config] 07paladox 036d235eb - Enable warmup on wikis beginning with b [12:54:26] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [12:54:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:54:40] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 13s [12:54:42] !log [paladox@mwtask141] starting deploy of {'config': True} to all [12:54:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:54:49] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [12:54:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:55:05] miraheze/mw-config - paladox the build passed. [12:56:44] !log [@test131] starting deploy of {'config': True} to all [12:56:45] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [12:56:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:56:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:59:29] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.14, 6.46, 6.62 [13:04:02] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.53, 4.58, 2.29 [13:05:57] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 2.48, 3.80, 2.27 [13:07:53] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 1.86, 3.20, 2.23 [13:08:51] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.69, 3.98, 3.45 [13:10:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.06, 7.02, 6.72 [13:10:18] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6d235ebfd6b8...cb013faac26a [13:10:19] [02miraheze/mw-config] 07paladox 03cb013fa - Enable warmup on wikis beginning with c [13:10:33] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [13:10:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:10:44] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [13:10:47] !log [paladox@mwtask141] starting deploy of {'config': True} to all [13:10:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:10:54] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [13:10:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:11:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:11:22] miraheze/mw-config - paladox the build passed. [13:12:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.17, 6.75, 6.67 [13:12:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.24, 3.92, 3.57 [13:14:44] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.34, 4.22, 3.71 [13:16:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.37, 3.91, 3.66 [13:20:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.85, 7.03, 6.80 [13:23:53] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f59f084d8126...d5cd09984ad0 [13:23:56] [02miraheze/puppet] 07paladox 03d5cd099 - Fix swift firewall [13:24:43] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.56, 6.40, 6.61 [13:24:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.88, 2.62, 3.17 [13:26:34] !log [@test131] starting deploy of {'config': True} to all [13:26:35] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [13:26:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:26:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:39:02] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.85, 2.47, 1.54 [13:39:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.32, 7.04, 6.76 [13:41:02] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 3.10, 2.70, 1.74 [13:44:59] PROBLEM - db101 Backups SQL on db101 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864191 seconds old and 121568 bytes [13:45:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.42, 6.60 [13:46:16] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.81, 3.76, 3.37 [13:48:11] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.47, 3.66, 3.38 [13:50:06] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.69, 3.95, 3.51 [13:54:09] PROBLEM - os131 Current Load on os131 is WARNING: LOAD WARNING - total load average: 3.73, 3.30, 2.56 [13:55:51] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.00, 3.94, 3.69 [13:56:06] RECOVERY - os131 Current Load on os131 is OK: LOAD OK - total load average: 2.92, 3.20, 2.62 [14:01:35] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.08, 3.53, 3.54 [14:03:30] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.75, 3.62, 3.58 [14:05:35] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.85, 3.14, 2.49 [14:07:20] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.20, 2.97, 3.32 [14:07:31] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.57, 3.52, 2.70 [14:11:11] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.83, 3.66, 3.49 [14:13:06] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.56, 3.46, 3.43 [14:13:20] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 3.15, 3.81, 3.17 [14:15:01] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.91, 3.22, 3.35 [14:16:51] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.47, 9.40, 8.31 [14:18:50] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 9.63, 8.90, 8.22 [14:19:09] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 2.22, 3.10, 3.09 [14:21:11] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/a710e84a317f [14:21:14] [02miraheze/mw-config] 07paladox 03a710e84 - Enable warmup for wikis beginning with d [14:21:15] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [14:22:23] [02mw-config] 07paladox opened pull request 03#5362: Enable warmup for wikis beginning with d - 13https://github.com/miraheze/mw-config/pull/5362 [14:23:19] [02mw-config] 07paladox closed pull request 03#5362: Enable warmup for wikis beginning with d - 13https://github.com/miraheze/mw-config/pull/5362 [14:23:23] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/cb013faac26a...8e77a7f5a45b [14:23:23] [02miraheze/mw-config] 07paladox 038e77a7f - Enable warmup for wikis beginning with d (#5362) [14:23:24] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [14:23:28] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [14:23:28] miraheze/mw-config - paladox the build passed. [14:24:21] miraheze/mw-config - paladox the build passed. [14:26:18] !log [@test131] starting deploy of {'config': True} to all [14:26:19] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [14:26:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:26:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:28:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.69, 3.15, 3.06 [14:34:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.16, 3.79, 3.33 [14:36:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.29, 3.19, 3.16 [14:39:52] !log [@mwtask141] starting deploy of {'config': True} to all [14:39:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:40:14] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 23s [14:40:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:00:51] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.66, 6.45, 5.77 [15:02:47] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.42, 6.01, 5.69 [15:22:35] PROBLEM - wiki.overwood.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.overwood.xyz' expires in 15 day(s) (Tue 21 Nov 2023 15:11:54 GMT +0000). [15:22:53] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/95a3bf8cd066...1fbe1e975cda [15:22:56] [02miraheze/ssl] 07MirahezeSSLBot 031fbe1e9 - Bot: Update SSL cert for wiki.overwood.xyz [15:33:29] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.86, 3.81, 3.31 [15:35:21] PROBLEM - wiki.tmyt105.leyhp.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.tmyt105.leyhp.com' expires in 15 day(s) (Tue 21 Nov 2023 15:22:27 GMT +0000). [15:35:38] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1fbe1e975cda...47463e0d7433 [15:35:40] [02miraheze/ssl] 07MirahezeSSLBot 0347463e0 - Bot: Update SSL cert for wiki.tmyt105.leyhp.com [15:37:27] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.29, 6.75, 6.48 [15:39:13] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.98, 3.07, 3.16 [15:39:23] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.34, 6.51, 6.44 [15:39:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.14, 6.82, 6.25 [15:39:53] PROBLEM - wiki.jsal.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.jsal.org' expires in 15 day(s) (Tue 21 Nov 2023 15:22:50 GMT +0000). [15:40:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/47463e0d7433...40bacdab4c91 [15:40:08] [02miraheze/ssl] 07MirahezeSSLBot 0340bacda - Bot: Update SSL cert for wiki.jsal.org [15:43:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.24, 6.74, 6.35 [15:44:44] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/02f630cc93ba [15:44:46] [02miraheze/mw-config] 07paladox 0302f630c - Enable warmup for wikis beginning with e [15:44:49] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [15:44:52] [02mw-config] 07paladox opened pull request 03#5363: Enable warmup for wikis beginning with e - 13https://github.com/miraheze/mw-config/pull/5363 [15:44:59] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.35, 3.44, 3.32 [15:45:14] [02mw-config] 07paladox closed pull request 03#5363: Enable warmup for wikis beginning with e - 13https://github.com/miraheze/mw-config/pull/5363 [15:45:16] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8e77a7f5a45b...5bb51ea8db98 [15:45:18] [02miraheze/mw-config] 07paladox 035bb51ea - Enable warmup for wikis beginning with e (#5363) [15:45:19] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [15:45:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.59, 7.28, 6.77 [15:45:22] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [15:45:43] miraheze/mw-config - paladox the build passed. [15:46:08] miraheze/mw-config - paladox the build passed. [15:47:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.80, 7.18, 6.80 [15:48:49] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.51, 3.08, 3.22 [15:52:09] RECOVERY - wiki.overwood.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.overwood.xyz' will expire on Sat 03 Feb 2024 14:22:46 GMT +0000. [15:53:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.05, 6.78, 6.76 [15:56:36] !log [@test131] starting deploy of {'config': True} to all [15:56:37] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [15:56:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:56:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:57:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.53, 6.94, 6.88 [15:57:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is CRITICAL: CRITICAL - load average: 8.15, 6.78, 6.39 [15:57:51] PROBLEM - miraheze.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.wiki' expires in 15 day(s) (Tue 21 Nov 2023 15:38:50 GMT +0000). [15:59:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.77, 6.47, 6.71 [16:01:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.89, 6.98, 6.57 [16:03:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.87, 7.58, 7.08 [16:03:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 5.59, 6.46, 6.42 [16:07:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.08, 7.49, 6.89 [16:08:46] RECOVERY - wiki.jsal.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.jsal.org' will expire on Sat 03 Feb 2024 14:39:59 GMT +0000. [16:09:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.50, 7.37, 7.22 [16:09:30] !log [@mwtask141] starting deploy of {'config': True} to all [16:09:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:09:42] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 11s [16:09:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:11:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 3.88, 6.01, 6.46 [16:14:11] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.47, 3.07, 3.01 [16:15:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.49, 6.23, 6.77 [16:16:06] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.85, 2.95, 2.98 [16:19:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.38, 6.71, 6.88 [16:27:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.04, 6.47, 6.72 [16:33:21] RECOVERY - wiki.tmyt105.leyhp.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tmyt105.leyhp.com' will expire on Sat 03 Feb 2024 14:35:32 GMT +0000. [16:33:51] PROBLEM - psycho.engineering - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'psycho.engineering' expires in 15 day(s) (Tue 21 Nov 2023 16:30:22 GMT +0000). [16:34:11] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/40bacdab4c91...ad6d61b2fc07 [16:34:12] [02miraheze/ssl] 07MirahezeSSLBot 03ad6d61b - Bot: Update SSL cert for psycho.engineering [16:38:04] PROBLEM - rodzinka.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rodzinka.wiki' expires in 15 day(s) (Tue 21 Nov 2023 16:29:32 GMT +0000). [16:38:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ad6d61b2fc07...e8b96476b3c5 [16:38:19] [02miraheze/ssl] 07MirahezeSSLBot 03e8b9647 - Bot: Update SSL cert for rodzinka.wiki [16:41:05] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.97, 6.57, 6.55 [16:43:01] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.44, 6.69, 6.62 [16:52:42] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.60, 7.46, 6.97 [16:56:35] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.47, 6.39, 6.65 [16:57:29] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/c35feb137d18 [16:57:31] [02miraheze/mw-config] 07paladox 03c35feb1 - Enable warmup for wikis beginning with f [16:57:34] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [16:57:39] [02mw-config] 07paladox opened pull request 03#5364: Enable warmup for wikis beginning with f - 13https://github.com/miraheze/mw-config/pull/5364 [16:58:09] [02mw-config] 07paladox closed pull request 03#5364: Enable warmup for wikis beginning with f - 13https://github.com/miraheze/mw-config/pull/5364 [16:58:10] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/5bb51ea8db98...0affad817dc0 [16:58:12] [02miraheze/mw-config] 07paladox 030affad8 - Enable warmup for wikis beginning with f (#5364) [16:58:15] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [16:58:17] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [16:58:38] miraheze/mw-config - paladox the build passed. [16:59:02] miraheze/mw-config - paladox the build passed. [17:00:57] PROBLEM - removededm.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'removededm.com' expires in 15 day(s) (Tue 21 Nov 2023 16:31:05 GMT +0000). [17:01:12] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e8b96476b3c5...404d54caaa59 [17:01:15] [02miraheze/ssl] 07MirahezeSSLBot 03404d54c - Bot: Update SSL cert for removededm.com [17:02:45] RECOVERY - psycho.engineering - LetsEncrypt on sslhost is OK: OK - Certificate 'psycho.engineering' will expire on Sat 03 Feb 2024 15:34:03 GMT +0000. [17:02:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.86, 6.77, 6.29 [17:04:21] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.83, 6.83, 6.66 [17:04:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.46, 3.17, 2.96 [17:04:45] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.32, 6.57, 6.27 [17:05:10] PROBLEM - matomo121 Current Load on matomo121 is CRITICAL: LOAD CRITICAL - total load average: 4.01, 3.42, 3.14 [17:06:17] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.38, 6.69, 6.63 [17:06:39] RECOVERY - rodzinka.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'rodzinka.wiki' will expire on Sat 03 Feb 2024 15:38:10 GMT +0000. [17:06:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.64, 3.07, 2.95 [17:07:10] RECOVERY - matomo121 Current Load on matomo121 is OK: LOAD OK - total load average: 2.82, 3.23, 3.11 [17:09:30] !log [@mwtask141] starting deploy of {'config': True} to all [17:09:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:09:43] !log [@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 12s [17:09:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:12:06] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.02, 6.87, 6.76 [17:15:59] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.76, 6.48, 6.65 [17:19:53] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 10.80, 8.22, 7.28 [17:21:49] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.49, 7.75, 7.21 [17:27:04] !log [@test131] starting deploy of {'config': True} to all [17:27:05] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [17:27:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:27:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:45:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.72, 6.36, 6.69 [17:51:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.03, 6.74, 6.74 [17:53:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.43, 6.40, 6.60 [17:58:57] RECOVERY - removededm.com - LetsEncrypt on sslhost is OK: OK - Certificate 'removededm.com' will expire on Sat 03 Feb 2024 16:01:05 GMT +0000. [18:02:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.94, 6.67, 6.55 [18:04:48] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.49, 3.36, 3.07 [18:06:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.54, 6.53, 6.51 [18:06:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.95, 3.32, 3.09 [18:10:07] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/3e38427b3118 [18:10:08] [02miraheze/mw-config] 07paladox 033e38427 - Enable warmup for wikis beginning with g [18:10:09] [02mw-config] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [18:10:11] [02mw-config] 07paladox opened pull request 03#5365: Enable warmup for wikis beginning with g - 13https://github.com/miraheze/mw-config/pull/5365 [18:10:19] [02mw-config] 07paladox closed pull request 03#5365: Enable warmup for wikis beginning with g - 13https://github.com/miraheze/mw-config/pull/5365 [18:10:20] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/0affad817dc0...f302e7d169db [18:10:21] [02miraheze/mw-config] 07paladox 03f302e7d - Enable warmup for wikis beginning with g (#5365) [18:10:24] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-4 [18:10:27] [02mw-config] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/mw-config [18:10:37] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [18:10:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:50] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 12s [18:10:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:10:57] !log [paladox@mwtask141] starting deploy of {'config': True} to all [18:11:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:11:04] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [18:11:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:11:20] miraheze/mw-config - paladox the build passed. [18:11:24] miraheze/mw-config - paladox the build passed. [18:11:56] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.16, 6.88, 6.69 [18:15:47] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.73, 6.61, 6.66 [18:26:50] !log [@test131] starting deploy of {'config': True} to all [18:26:51] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [18:27:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:27:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:34:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.57, 3.26, 2.94 [18:36:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.18, 2.90, 2.85 [18:37:02] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 11.53, 9.23, 7.43 [18:39:02] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 7.89, 8.61, 7.42 [19:00:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d5cd09984ad0...112258fcba27 [19:00:35] [02miraheze/puppet] 07paladox 03112258f - swift: remove conn_timeout and node timeout [19:11:46] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.62, 6.41, 6.13 [19:13:42] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.08, 6.21, 6.09 [19:34:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/112258fcba27...2105b2d7753c [19:34:27] [02miraheze/puppet] 07paladox 032105b2d - Swift: Look for x-client-ip/x-real-ip headers in rewrite [19:49:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.77, 6.85, 6.51 [19:55:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.15, 6.62, 6.63 [20:00:19] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/f302e7d169db...13d87505bbb1 [20:00:22] [02miraheze/mw-config] 07paladox 0313d8750 - Enable warmup for wikis beginning with h [20:00:28] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [20:00:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:40] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 11s [20:00:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:46] !log [paladox@mwtask141] starting deploy of {'config': True} to all [20:00:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:00:54] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [20:01:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:01:22] miraheze/mw-config - paladox the build passed. [20:06:08] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.73, 3.40, 3.08 [20:08:03] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.52, 3.05, 2.99 [20:18:11] RECOVERY - trains.cnrailwiki.org - reverse DNS on sslhost is OK: SSL OK - trains.cnrailwiki.org reverse DNS resolves to cp25.miraheze.org - CNAME OK [20:19:52] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2105b2d7753c...28b8c96f0a8b [20:19:54] [02miraheze/puppet] 07paladox 0328b8c96 - jobrunner-hi: Increase refreshLinks memory limits [20:24:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.24, 3.76, 3.37 [20:26:55] !log [@test131] starting deploy of {'config': True} to all [20:26:56] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [20:27:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:27:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:30:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.72, 3.80, 3.53 [20:32:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.70, 4.54, 3.83 [20:33:36] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 6.81, 6.40, 5.73 [20:35:31] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.39, 6.31, 5.77 [20:38:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.64, 3.94, 3.82 [20:40:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.61, 7.23, 6.65 [20:40:46] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.48, 4.08, 3.88 [20:42:00] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/8ed4789bf226 [20:42:02] [02miraheze/puppet] 07paladox 038ed4789 - jobrunner: Run smw.changePropagationClassUpdate in a separate runner [20:42:03] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [20:42:05] [02puppet] 07paladox opened pull request 03#3482: jobrunner: Run smw.changePropagationClassUpdate in a separate runner - 13https://github.com/miraheze/puppet/pull/3482 [20:42:21] PROBLEM - db131 Current Load on db131 is WARNING: WARNING - load average: 7.47, 5.51, 4.27 [20:42:30] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8ed4789bf226...f8fd307a2044 [20:42:33] [02miraheze/puppet] 07paladox 03f8fd307 - Update jobrunner.json.erb [20:42:35] [02puppet] 07paladox synchronize pull request 03#3482: jobrunner: Run smw.changePropagationClassUpdate in a separate runner - 13https://github.com/miraheze/puppet/pull/3482 [20:42:39] [02puppet] 07paladox closed pull request 03#3482: jobrunner: Run smw.changePropagationClassUpdate in a separate runner - 13https://github.com/miraheze/puppet/pull/3482 [20:42:40] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/puppet/compare/28b8c96f0a8b...61a2f44b9da7 [20:42:44] [02miraheze/puppet] 07paladox 0361a2f44 - jobrunner: Run smw.changePropagationClassUpdate in a separate runner (#3482) [20:42:46] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-4 [20:42:49] [02puppet] 07paladox deleted branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [20:44:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.73, 6.74, 6.60 [20:44:21] RECOVERY - db131 Current Load on db131 is OK: OK - load average: 4.27, 4.95, 4.21 [20:45:12] PROBLEM - allthetropes.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 21 Nov 2023 20:39:44 GMT +0000). [20:46:25] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/61a2f44b9da7...391b4f6400d8 [20:46:28] [02miraheze/puppet] 07paladox 03391b4f6 - jobrunner-hi: increase parseroidcache runners to 4 [20:46:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.29, 3.57, 3.79 [20:49:56] [02miraheze/dns] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/c5da7ade1bdf...17a35d06f162 [20:49:58] [02miraheze/dns] 07paladox 0317a35d0 - Add acme challenge for train.org [20:50:11] PROBLEM - trains.cnrailwiki.org - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for trains.cnrailwiki.org could not be found [20:50:12] [02dns] 07paladox commented on commit 0317a35d06f1621812e871ef799321b142d77f100c - 13https://github.com/miraheze/dns/commit/17a35d06f1621812e871ef799321b142d77f100c#commitcomment-131763018 [20:51:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/404d54caaa59...27e670872cbb [20:51:18] [02miraheze/ssl] 07MirahezeSSLBot 0327e6708 - Bot: Update SSL cert for orain.org [20:51:51] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.08, 6.68, 6.58 [20:52:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.91, 2.84, 3.37 [20:55:12] PROBLEM - isv.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 21 Nov 2023 20:39:44 GMT +0000). [20:55:43] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.38, 7.31, 6.84 [20:56:39] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/13d87505bbb1...7eb039c7f65a [20:56:41] [02miraheze/mw-config] 07paladox 037eb039c - Enable warmup for wikis beginning with i [20:56:44] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [20:56:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:56:50] PROBLEM - poserdazfreebies.orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 21 Nov 2023 20:39:44 GMT +0000). [20:57:07] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 23s [20:57:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:14] !log [paladox@mwtask141] starting deploy of {'config': True} to all [20:57:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:25] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 10s [20:57:31] miraheze/mw-config - paladox the build passed. [20:57:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [20:57:39] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.58, 6.91, 6.74 [20:59:35] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.72, 6.65, 6.65 [21:05:24] PROBLEM - orain.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'orain.org' expires in 15 day(s) (Tue 21 Nov 2023 20:39:44 GMT +0000). [21:09:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.19, 6.85, 6.74 [21:13:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 12.60, 8.36, 7.28 [21:17:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.27, 7.53, 7.20 [21:19:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 9.21, 8.71, 7.70 [21:21:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 5.60, 7.62, 7.43 [21:26:10] RECOVERY - poserdazfreebies.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Sat 03 Feb 2024 19:51:10 GMT +0000. [21:27:01] !log [@test131] starting deploy of {'config': True} to all [21:27:02] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 1s [21:27:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:27:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:28:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.90, 3.96, 3.37 [21:30:44] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.44, 3.72, 3.36 [21:31:50] PROBLEM - swiftobject101 Current Load on swiftobject101 is WARNING: WARNING - load average: 7.60, 6.97, 6.26 [21:32:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.01, 3.93, 3.48 [21:33:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.52, 6.89, 6.22 [21:33:50] RECOVERY - swiftobject101 Current Load on swiftobject101 is OK: OK - load average: 6.14, 6.56, 6.19 [21:34:18] RECOVERY - orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Sat 03 Feb 2024 19:51:10 GMT +0000. [21:34:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.69, 4.00, 3.58 [21:36:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 1.92, 3.20, 3.33 [21:37:00] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 6.22, 6.61, 6.26 [21:39:02] PROBLEM - os141 Current Load on os141 is CRITICAL: LOAD CRITICAL - total load average: 4.55, 3.32, 1.86 [21:41:00] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 7.98, 7.21, 6.57 [21:43:00] PROBLEM - mwtask141 Current Load on mwtask141 is CRITICAL: CRITICAL - load average: 8.27, 7.55, 6.78 [21:44:37] RECOVERY - allthetropes.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Sat 03 Feb 2024 19:51:10 GMT +0000. [21:45:01] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.19, 7.04, 6.69 [21:45:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.46, 6.31, 6.70 [21:46:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.22, 3.76, 3.53 [21:48:43] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 5.78, 4.63, 3.88 [21:49:00] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 6.02, 6.67, 6.64 [21:49:02] PROBLEM - os141 Current Load on os141 is WARNING: LOAD WARNING - total load average: 1.85, 3.58, 2.89 [21:51:02] RECOVERY - os141 Current Load on os141 is OK: LOAD OK - total load average: 0.53, 2.50, 2.58 [21:52:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.42, 3.87, 3.74 [21:54:04] PROBLEM - swiftproxy111 HTTPS on swiftproxy111 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/1.1 200 OK [21:55:11] RECOVERY - isv.orain.org - LetsEncrypt on sslhost is OK: OK - Certificate 'orain.org' will expire on Sat 03 Feb 2024 19:51:10 GMT +0000. [21:58:04] RECOVERY - swiftproxy111 HTTPS on swiftproxy111 is OK: HTTP OK: Status line output matched "HTTP/1.1 404" - 352 bytes in 0.061 second response time [21:58:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.46, 3.02, 3.39 [22:12:32] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.79, 6.78, 6.40 [22:13:44] PROBLEM - mw131 Current Load on mw131 is WARNING: WARNING - load average: 11.28, 11.36, 9.13 [22:14:28] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.43, 6.23, 6.25 [22:15:02] PROBLEM - mw132 Current Load on mw132 is WARNING: WARNING - load average: 8.95, 10.36, 8.93 [22:17:02] RECOVERY - mw132 Current Load on mw132 is OK: OK - load average: 8.68, 9.77, 8.89 [22:17:36] RECOVERY - mw131 Current Load on mw131 is OK: OK - load average: 8.19, 9.56, 8.90 [22:19:25] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.13, 3.41, 3.34 [22:21:20] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.78, 3.21, 3.28 [22:22:14] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.19, 7.12, 6.60 [22:23:12] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/be99379878e0 [22:23:15] [02miraheze/puppet] 07paladox 03be99379 - swift: Set "Connection: close" if using HEAD [22:23:16] [02puppet] 07paladox created branch 03paladox-patch-4 - 13https://github.com/miraheze/puppet [22:23:19] [02puppet] 07paladox opened pull request 03#3483: swift: Set "Connection: close" if using HEAD - 13https://github.com/miraheze/puppet/pull/3483 [22:24:10] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.50, 6.82, 6.52 [22:24:34] [02puppet] 07paladox synchronize pull request 03#3483: swift: Set "Connection: close" if using HEAD - 13https://github.com/miraheze/puppet/pull/3483 [22:24:35] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-4 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/be99379878e0...e2cf0750bd72 [22:24:39] [02miraheze/puppet] 07paladox 03e2cf075 - Update rewrite.py [22:26:06] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 6.08, 6.45, 6.42 [22:31:38] PROBLEM - mwtask141 Current Load on mwtask141 is WARNING: WARNING - load average: 6.88, 6.64, 6.41 [22:32:54] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/7eb039c7f65a...9e198dd6b945 [22:32:56] [02miraheze/mw-config] 07paladox 039e198dd - Enable warmup on wikis beginning with j [22:33:06] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [22:33:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:17] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 11s [22:33:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:33] RECOVERY - mwtask141 Current Load on mwtask141 is OK: OK - load average: 6.72, 6.63, 6.44 [22:33:52] miraheze/mw-config - paladox the build passed. [22:33:59] !log [paladox@mwtask141] starting deploy of {'config': True} to all [22:34:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:34:08] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 8s [22:34:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:34:48] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.40, 3.68, 3.39 [22:40:43] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 2.82, 3.75, 3.56 [22:46:33] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-12 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/4c14beafec8e [22:46:35] [02miraheze/puppet] 07paladox 034c14bea - swift: enable container_sync for proxy-server [22:46:38] [02puppet] 07paladox created branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [22:46:39] [02puppet] 07paladox opened pull request 03#3484: swift: enable container_sync for proxy-server - 13https://github.com/miraheze/puppet/pull/3484 [22:46:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.53, 3.20, 3.38 [22:46:54] [02puppet] 07paladox closed pull request 03#3484: swift: enable container_sync for proxy-server - 13https://github.com/miraheze/puppet/pull/3484 [22:46:57] [02miraheze/puppet] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/391b4f6400d8...afb2a1766415 [22:47:00] [02miraheze/puppet] 07paladox 03afb2a17 - swift: enable container_sync for proxy-server (#3484) [22:47:01] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-12 [22:47:02] [02puppet] 07paladox deleted branch 03paladox-patch-12 - 13https://github.com/miraheze/puppet [22:56:36] !log [@test131] starting deploy of {'config': True} to all [22:56:37] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [22:56:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:43] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/9e198dd6b945...c9c32224c8cc [22:56:44] [02miraheze/mw-config] 07paladox 03c9c3222 - Enable warmup for wikis beginning with k [22:56:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:49] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [22:56:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:56:58] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [22:57:00] !log [paladox@mwtask141] starting deploy of {'config': True} to all [22:57:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:57:05] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [22:57:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:57:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:57:37] miraheze/mw-config - paladox the build passed. [22:59:12] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c9c32224c8cc...3e0260c1adc0 [22:59:12] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [22:59:13] [02miraheze/mw-config] 07paladox 033e0260c - Enable warmup for wikis beginning with l [22:59:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:59:24] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 11s [22:59:26] !log [paladox@mwtask141] starting deploy of {'config': True} to all [22:59:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:59:33] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 6s [22:59:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:59:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:00:05] miraheze/mw-config - paladox the build passed. [23:01:16] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3e0260c1adc0...242b85a85719 [23:01:17] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:01:18] [02miraheze/mw-config] 07paladox 03242b85a - Enable warmup for wikis beginning with m [23:01:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:01:26] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 8s [23:01:29] !log [paladox@mwtask141] starting deploy of {'config': True} to all [23:01:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:01:35] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 5s [23:01:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:01:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:02:13] miraheze/mw-config - paladox the build passed. [23:11:06] PROBLEM - graylog131 Current Load on graylog131 is CRITICAL: LOAD CRITICAL - total load average: 4.61, 3.57, 3.26 [23:11:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is CRITICAL: CRITICAL - load average: 8.07, 7.18, 6.40 [23:13:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 6.51, 6.90, 6.40 [23:14:56] PROBLEM - graylog131 Current Load on graylog131 is WARNING: LOAD WARNING - total load average: 3.50, 3.95, 3.51 [23:15:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 5.59, 6.34, 6.25 [23:20:43] RECOVERY - graylog131 Current Load on graylog131 is OK: LOAD OK - total load average: 2.48, 3.16, 3.30 [23:26:31] !log [@test131] starting deploy of {'config': True} to all [23:26:32] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [23:26:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:26:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:27:20] PROBLEM - swiftobject122 Current Load on swiftobject122 is WARNING: WARNING - load average: 7.29, 6.82, 6.40 [23:29:20] RECOVERY - swiftobject122 Current Load on swiftobject122 is OK: OK - load average: 4.37, 6.04, 6.17 [23:44:19] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_41 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/06e622539328...a56bb1fd76a5 [23:44:22] [02miraheze/mediawiki] 07paladox 03a56bb1f - Update TabberNeue [23:44:59] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_40 [+0/-0/±1] 13https://github.com/miraheze/mediawiki/compare/9b549cc62b42...c6d9b15ed033 [23:45:01] [02miraheze/mediawiki] 07paladox 03c6d9b15 - Update TabberNeue [23:45:48] !log [paladox@mwtask141] starting deploy of {'world': True} to all [23:45:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:47:52] !log [paladox@mwtask141] finished deploy of {'world': True} to all - SUCCESS in 124s [23:47:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:48:57] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/242b85a85719...603be80c89a5 [23:48:58] [02miraheze/mw-config] 07paladox 03603be80 - Enable warmcache on wikis beginning with n [23:48:59] !log [paladox@mwtask141] starting deploy of {'pull': 'config', 'config': True} to all [23:49:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:10] !log [paladox@mwtask141] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 10s [23:49:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:16] !log [paladox@mwtask141] starting deploy of {'config': True} to all [23:49:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:23] !log [paladox@mwtask141] finished deploy of {'config': True} to all - SUCCESS in 7s [23:49:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:49:58] miraheze/mw-config - paladox the build passed. [23:56:11] !log [@test131] starting deploy of {'config': True} to all [23:56:12] !log [@test131] finished deploy of {'config': True} to all - SUCCESS in 0s [23:56:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:56:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log