[00:00:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:00:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:02:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:02:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 50% [00:04:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [00:06:09] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 20558MiB (23% inode=98%); [00:07:07] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 20665MiB (23% inode=98%); [00:38:30] PROBLEM - bast181 ferm_active on bast181 is CRITICAL: ERROR ferm input drop default policy not set, ferm might not have been started correctly [00:40:21] PROBLEM - db161 Backups SQL on db161 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864174 seconds old and 113006 bytes [00:48:26] RECOVERY - bast181 ferm_active on bast181 is OK: OK ferm input default policy is set [00:52:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 59% [00:54:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 61% [00:55:52] PROBLEM - legacygt.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'legacygt.wiki' expires in 15 day(s) (Sun 03 Mar 2024 12:47:38 AM GMT +0000). [00:56:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6cc8bedbb77b...efbf8bb88524 [00:56:05] [02miraheze/ssl] 07MirahezeSSLBot 03efbf8bb - Bot: Update SSL cert for legacygt.wiki [01:02:34] PROBLEM - ping6 on cp28 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 388.35 ms [01:08:32] RECOVERY - ping6 on cp28 is OK: PING OK - Packet loss = 0%, RTA = 167.08 ms [01:23:14] PROBLEM - wikipedia.yogfront.ooo - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikipedia.yogfront.ooo' expires in 15 day(s) (Sun 03 Mar 2024 01:16:10 AM GMT +0000). [01:25:16] PROBLEM - evil.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'evil.wiki' expires in 15 day(s) (Sun 03 Mar 2024 01:19:40 AM GMT +0000). [01:25:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/efbf8bb88524...4d9127810dc0 [01:25:30] [02miraheze/ssl] 07MirahezeSSLBot 034d91278 - Bot: Update SSL cert for evil.wiki [01:25:43] RECOVERY - legacygt.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'legacygt.wiki' will expire on Wed 15 May 2024 11:55:58 PM GMT +0000. [01:27:24] [02puppet] 07Universal-Omega closed pull request 03#3756: logbot: support SASL authentication - 13https://github.com/miraheze/puppet/pull/3756 [01:27:26] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/cbb29e7ab576...3119206add74 [01:27:29] [02miraheze/puppet] 07Universal-Omega 033119206 - logbot: support SASL authentication (#3756) [01:27:32] [02miraheze/puppet] 07Universal-Omega deleted branch 03logbot-sasl [01:27:33] PROBLEM - wiki.evscope.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.evscope.org' expires in 15 day(s) (Sun 03 Mar 2024 01:12:50 AM GMT +0000). [01:27:35] [02puppet] 07Universal-Omega deleted branch 03logbot-sasl - 13https://github.com/miraheze/puppet [01:27:37] [02puppet] 07Universal-Omega closed pull request 03#3758: ircrcbot: support SASL authentication - 13https://github.com/miraheze/puppet/pull/3758 [01:27:38] [02miraheze/puppet] 07Universal-Omega deleted branch 03ircrcbot-sasl [01:27:41] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3119206add74...4135a87f80d5 [01:27:43] [02miraheze/puppet] 07Universal-Omega 034135a87 - ircrcbot: support SASL authentication (#3758) [01:27:45] [02puppet] 07Universal-Omega deleted branch 03ircrcbot-sasl - 13https://github.com/miraheze/puppet [01:27:48] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4d9127810dc0...3b4a33e1b15e [01:27:51] [02miraheze/ssl] 07MirahezeSSLBot 033b4a33e - Bot: Update SSL cert for wiki.evscope.org [01:28:42] miraheze/puppet - Universal-Omega the build passed. [01:29:08] miraheze/puppet - Universal-Omega the build passed. [01:31:21] PROBLEM - bots171 IRC Log Bot on bots171 is CRITICAL: PROCS CRITICAL: 0 processes with args 'adminlogbot.py' [01:36:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 59% [01:40:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [01:43:21] RECOVERY - bots171 IRC Log Bot on bots171 is OK: PROCS OK: 1 process with args 'adminlogbot.py' [01:50:10] PROBLEM - wiki.kachow.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.kachow.org' expires in 15 day(s) (Sun 03 Mar 2024 01:42:41 AM GMT +0000). [01:50:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3b4a33e1b15e...a0522f5cbb5f [01:50:24] [02miraheze/ssl] 07MirahezeSSLBot 03a0522f5 - Bot: Update SSL cert for wiki.kachow.org [01:53:14] PROBLEM - mystiverse.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mystiverse.wiki' expires in 15 day(s) (Sun 03 Mar 2024 01:40:06 AM GMT +0000). [01:53:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a0522f5cbb5f...62ebadc0176d [01:53:31] [02miraheze/ssl] 07MirahezeSSLBot 0362ebadc - Bot: Update SSL cert for mystiverse.wiki [01:54:33] RECOVERY - evil.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'evil.wiki' will expire on Thu 16 May 2024 12:25:22 AM GMT +0000. [01:56:06] RECOVERY - wiki.evscope.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.evscope.org' will expire on Thu 16 May 2024 12:27:39 AM GMT +0000. [01:58:06] PROBLEM - wiki.recaptime.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.recaptime.eu.org' expires in 15 day(s) (Sun 03 Mar 2024 01:40:39 AM GMT +0000). [01:58:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/62ebadc0176d...67843d1da038 [01:58:26] [02miraheze/ssl] 07MirahezeSSLBot 0367843d1 - Bot: Update SSL cert for wiki.recaptime.eu.org [02:01:52] MacFan4000: please don't update requests to complete yet, I saved them in progress so I can track some migrations I haven't completely finished yet [02:03:05] oh, ok [02:04:15] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 59% [02:04:41] Thanks [02:06:06] sorry should have made that clear on the requests [02:08:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [02:10:09] PROBLEM - wiki.veloren.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.veloren.net' expires in 15 day(s) (Sun 03 Mar 2024 01:56:37 AM GMT +0000). [02:10:20] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/67843d1da038...34030aaacf34 [02:10:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [02:10:22] [02miraheze/ssl] 07MirahezeSSLBot 0334030aa - Bot: Update SSL cert for wiki.veloren.net [02:14:42] PROBLEM - dragdown.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'dragdown.wiki' expires in 15 day(s) (Sun 03 Mar 2024 02:08:49 AM GMT +0000). [02:14:53] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/34030aaacf34...91f57fd77577 [02:14:55] [02miraheze/ssl] 07MirahezeSSLBot 0391f57fd - Bot: Update SSL cert for dragdown.wiki [02:16:09] [02miraheze/RequestSSL] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RequestSSL/compare/7d58b792c553...182fe9860124 [02:16:11] [02miraheze/RequestSSL] 07MacFan4000 03182fe98 - grammer fixes [02:18:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [02:19:18] [02miraheze/MirahezeMagic] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/ddb5c0f0cf39...158e9c546ef5 [02:19:21] [02miraheze/MirahezeMagic] 07MacFan4000 03158e9c5 - grammer fixes in requestssl-text [02:19:22] PROBLEM - wikicompliance.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikicompliance.com' expires in 15 day(s) (Sun 03 Mar 2024 02:01:03 AM GMT +0000). [02:19:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/91f57fd77577...6948b8558a82 [02:19:32] [02miraheze/ssl] 07MirahezeSSLBot 036948b85 - Bot: Update SSL cert for wikicompliance.com [02:20:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 58% [02:21:57] RECOVERY - mystiverse.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'mystiverse.wiki' will expire on Thu 16 May 2024 12:53:22 AM GMT +0000. [02:22:50] miraheze/RequestSSL - MacFan4000 the build passed. [02:23:52] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [02:23:54] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [02:23:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:23:57] PROBLEM - persist.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'persist.wiki' expires in 15 day(s) (Sun 03 Mar 2024 02:12:41 AM GMT +0000). [02:23:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:24:02] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [02:24:03] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [02:24:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:24:05] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6948b8558a82...3a60568e4072 [02:24:07] [02miraheze/ssl] 07MirahezeSSLBot 033a60568 - Bot: Update SSL cert for persist.wiki [02:24:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:25:27] miraheze/MirahezeMagic - MacFan4000 the build passed. [02:25:41] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [02:27:13] RECOVERY - wiki.recaptime.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.recaptime.eu.org' will expire on Thu 16 May 2024 12:58:18 AM GMT +0000. [02:28:22] [02miraheze/RequestSSL] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RequestSSL/compare/182fe9860124...7171a8354b04 [02:28:24] [02miraheze/RequestSSL] 07MacFan4000 037171a83 - fix LogRestrictions incorrect permission [02:33:00] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [02:33:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:33:08] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 8s [02:33:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:34:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [02:34:51] miraheze/RequestSSL - MacFan4000 the build passed. [02:39:15] PROBLEM - mariopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'mariopedia.org' expires in 15 day(s) (Sun 03 Mar 2024 02:19:40 AM GMT +0000). [02:39:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3a60568e4072...dc8d09166334 [02:39:26] [02miraheze/ssl] 07MirahezeSSLBot 03dc8d091 - Bot: Update SSL cert for mariopedia.org [02:43:11] PROBLEM - swests.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'swests.org' expires in 15 day(s) (Sun 03 Mar 2024 02:25:13 AM GMT +0000). [02:43:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dc8d09166334...857518c020b9 [02:43:21] [02miraheze/ssl] 07MirahezeSSLBot 03857518c - Bot: Update SSL cert for swests.org [02:48:10] RECOVERY - wiki.kachow.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.kachow.org' will expire on Thu 16 May 2024 12:50:16 AM GMT +0000. [02:53:25] RECOVERY - persist.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'persist.wiki' will expire on Thu 16 May 2024 01:24:00 AM GMT +0000. [02:58:20] PROBLEM - knifepointhorror.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'knifepointhorror.wiki' expires in 15 day(s) (Sun 03 Mar 2024 02:34:15 AM GMT +0000). [02:58:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/857518c020b9...fd40c0fa8ac8 [02:58:33] [02miraheze/ssl] 07MirahezeSSLBot 03fd40c0f - Bot: Update SSL cert for knifepointhorror.wiki [03:04:18] PROBLEM - wiki.aoemods.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.aoemods.com' expires in 15 day(s) (Sun 03 Mar 2024 03:01:04 AM GMT +0000). [03:04:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/fd40c0fa8ac8...d98e87e83fda [03:04:31] [02miraheze/ssl] 07MirahezeSSLBot 03d98e87e - Bot: Update SSL cert for wiki.aoemods.com [03:05:35] [Grafana] !sre FIRING: The mediawiki job queue has more than 2500 unclaimed jobs https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [03:05:55] PROBLEM - wiki.yahyabd.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.yahyabd.xyz' expires in 15 day(s) (Sun 03 Mar 2024 02:44:57 AM GMT +0000). [03:08:24] PROBLEM - roll-of-arms.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'roll-of-arms.com' expires in 15 day(s) (Sun 03 Mar 2024 03:01:18 AM GMT +0000). [03:08:28] RECOVERY - wiki.veloren.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.veloren.net' will expire on Thu 16 May 2024 01:10:15 AM GMT +0000. [03:08:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d98e87e83fda...39c40da66b16 [03:08:35] [02miraheze/ssl] 07MirahezeSSLBot 0339c40da - Bot: Update SSL cert for roll-of-arms.com [03:09:00] RECOVERY - mariopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'mariopedia.org' will expire on Thu 16 May 2024 01:39:18 AM GMT +0000. [03:12:21] RECOVERY - swests.org - LetsEncrypt on sslhost is OK: OK - Certificate 'swests.org' will expire on Thu 16 May 2024 01:43:14 AM GMT +0000. [03:13:05] RECOVERY - dragdown.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'dragdown.wiki' will expire on Thu 16 May 2024 01:14:47 AM GMT +0000. [03:18:07] RECOVERY - wikicompliance.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wikicompliance.com' will expire on Thu 16 May 2024 01:19:25 AM GMT +0000. [03:23:11] PROBLEM - buildabearwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'buildabearwiki.info' expires in 15 day(s) (Sun 03 Mar 2024 03:10:27 AM GMT +0000). [03:23:23] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/39c40da66b16...bd3fa7075839 [03:23:24] [02miraheze/ssl] 07MirahezeSSLBot 03bd3fa70 - Bot: Update SSL cert for buildabearwiki.info [03:25:14] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:27:40] RECOVERY - knifepointhorror.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'knifepointhorror.wiki' will expire on Thu 16 May 2024 01:58:26 AM GMT +0000. [03:33:35] RECOVERY - wiki.aoemods.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.aoemods.com' will expire on Thu 16 May 2024 02:04:23 AM GMT +0000. [03:37:17] RECOVERY - roll-of-arms.com - LetsEncrypt on sslhost is OK: OK - Certificate 'roll-of-arms.com' will expire on Thu 16 May 2024 02:08:26 AM GMT +0000. [03:41:50] PROBLEM - iol.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'iol.wiki' expires in 15 day(s) (Sun 03 Mar 2024 03:32:54 AM GMT +0000). [03:42:02] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/bd3fa7075839...4a0f6901c543 [03:42:04] [02miraheze/ssl] 07MirahezeSSLBot 034a0f690 - Bot: Update SSL cert for iol.wiki [03:43:04] PROBLEM - en.omniversalis.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.omniversalis.org' expires in 15 day(s) (Sun 03 Mar 2024 03:38:26 AM GMT +0000). [03:43:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4a0f6901c543...418081ff58e1 [03:43:17] [02miraheze/ssl] 07MirahezeSSLBot 03418081f - Bot: Update SSL cert for en.omniversalis.org [03:50:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [03:51:38] PROBLEM - squareenixwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'squareenixwiki.com' expires in 15 day(s) (Sun 03 Mar 2024 03:25:04 AM GMT +0000). [03:51:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/418081ff58e1...c6b019827552 [03:51:49] [02miraheze/ssl] 07MirahezeSSLBot 03c6b0198 - Bot: Update SSL cert for squareenixwiki.com [03:51:53] RECOVERY - buildabearwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'buildabearwiki.info' will expire on Thu 16 May 2024 02:23:17 AM GMT +0000. [03:53:36] PROBLEM - royal-wiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'royal-wiki.org' expires in 15 day(s) (Sun 03 Mar 2024 03:38:49 AM GMT +0000). [03:53:45] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c6b019827552...785f0503297a [03:53:48] [02miraheze/ssl] 07MirahezeSSLBot 03785f050 - Bot: Update SSL cert for royal-wiki.org [03:54:48] PROBLEM - wiki.infomedia.co.uk - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.infomedia.co.uk' expires in 15 day(s) (Sun 03 Mar 2024 03:24:52 AM GMT +0000). [03:55:00] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/785f0503297a...92ae0e767ead [03:55:02] [02miraheze/ssl] 07MirahezeSSLBot 0392ae0e7 - Bot: Update SSL cert for wiki.infomedia.co.uk [04:06:13] [02mw-config] 07labster opened pull request 03#5482: $wgWhitelistReadRegexp for kanrikyarawiki (T11856) - 13https://github.com/miraheze/mw-config/pull/5482 [04:07:06] miraheze/mw-config - labster the build passed. [04:07:51] [02mw-config] 07Universal-Omega closed pull request 03#5482: $wgWhitelistReadRegexp for kanrikyarawiki (T11856) - 13https://github.com/miraheze/mw-config/pull/5482 [04:07:54] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/367762a34647...a786c39a8011 [04:07:56] [02miraheze/mw-config] 07labster 03a786c39 - $wgWhitelistReadRegexp for kanrikyarawiki (T11856) (#5482) [04:08:45] miraheze/mw-config - Universal-Omega the build passed. [04:11:43] Great, new Fatal code `7374697c3fc3c0641e411464` [04:16:02] PROBLEM - wiki.virtualjet.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.virtualjet.net' expires in 15 day(s) (Sun 03 Mar 2024 03:58:20 AM GMT +0000). [04:16:13] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/92ae0e767ead...50782c8e5145 [04:16:16] [02miraheze/ssl] 07MirahezeSSLBot 0350782c8 - Bot: Update SSL cert for wiki.virtualjet.net [04:18:18] PROBLEM - wiki.mostly.vet - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mostly.vet' expires in 15 day(s) (Sun 03 Mar 2024 04:05:49 AM GMT +0000). [04:18:31] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/50782c8e5145...dd6c63a298cf [04:18:33] [02miraheze/ssl] 07MirahezeSSLBot 03dd6c63a - Bot: Update SSL cert for wiki.mostly.vet [04:20:17] RECOVERY - squareenixwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'squareenixwiki.com' will expire on Thu 16 May 2024 02:51:41 AM GMT +0000. [04:22:43] RECOVERY - royal-wiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'royal-wiki.org' will expire on Thu 16 May 2024 02:53:39 AM GMT +0000. [04:23:36] RECOVERY - wiki.infomedia.co.uk - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.infomedia.co.uk' will expire on Thu 16 May 2024 02:54:54 AM GMT +0000. [04:32:05] !log [@mwtask181] starting deploy of {'config': True} to all [04:32:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:32:14] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [04:32:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [04:40:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [04:41:10] RECOVERY - iol.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'iol.wiki' will expire on Thu 16 May 2024 02:41:56 AM GMT +0000. [04:41:12] RECOVERY - en.omniversalis.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.omniversalis.org' will expire on Thu 16 May 2024 02:43:10 AM GMT +0000. [04:44:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [04:45:03] RECOVERY - wiki.virtualjet.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.virtualjet.net' will expire on Thu 16 May 2024 03:16:08 AM GMT +0000. [04:48:07] RECOVERY - wiki.mostly.vet - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mostly.vet' will expire on Thu 16 May 2024 03:18:24 AM GMT +0000. [04:58:41] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:42:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [05:45:35] [Grafana] !sre RESOLVED: High Job Queue Backlog https://grafana.miraheze.org/d/GtxbP1Xnk?orgId=1 [05:46:21] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [05:58:58] [02miraheze/MirahezeMagic] 07MacFan4000 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/158e9c546ef5...acd9761ccb71 [05:59:01] [02miraheze/MirahezeMagic] 07MacFan4000 03acd9761 - fix typo; phabricator -> phorge [06:02:25] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [06:02:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:02:33] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 8s [06:02:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:05:52] miraheze/MirahezeMagic - MacFan4000 the build passed. [06:08:39] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/invalidateUserSessions.php --wiki=kanrikyarawiki --user Labster (END - exit=0) [06:08:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:08:58] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/invalidateUserSessions.php --wiki=kanrikyarawiki --user Universal Omega (END - exit=0) [06:09:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:09:10] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/invalidateUserSessions.php --wiki=kanrikyarawiki --user Universal_Omega (END - exit=0) [06:09:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:11:41] [02CreateWiki] 07Reception123 closed pull request 03#457: add minimum length for wiki requests - 13https://github.com/miraheze/CreateWiki/pull/457 [06:11:44] [02miraheze/CreateWiki] 07Reception123 deleted branch 03Reception123-patch-1 [06:11:45] [02miraheze/CreateWiki] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/CreateWiki/compare/1742a9524957...32b9a356edac [06:11:48] [02miraheze/CreateWiki] 07Reception123 0332b9a35 - add minimum length for wiki requests (#457) [06:11:50] [02CreateWiki] 07Reception123 deleted branch 03Reception123-patch-1 - 13https://github.com/miraheze/CreateWiki [06:11:53] !log [reception@mwtask181] starting deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to all [06:11:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:12:03] !log [reception@mwtask181] finished deploy of {'versions': '1.41', 'upgrade_extensions': 'CreateWiki'} to all - SUCCESS in 9s [06:12:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:12:21] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/invalidateUserSessions.php --wiki=kanrikyarawiki --user InquisitiveRaven (END - exit=0) [06:12:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:16:27] miraheze/CreateWiki - Reception123 the build has errored. [06:18:09] [02miraheze/mw-config] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a786c39a8011...516f05aa63cf [06:18:11] [02miraheze/mw-config] 07Reception123 03516f05a - set minimum length for wiki requests to 250 chars [06:18:27] !log [reception@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [06:18:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:18:36] !log [reception@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [06:18:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:19:06] miraheze/mw-config - Reception123 the build passed. [06:23:16] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [06:23:17] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [06:23:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:23:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:23:26] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [06:23:27] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [06:23:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:23:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:28:02] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [06:28:07] !log reception@mwtask181:~$ sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=nintendowiki /srv/mediawiki/1.41/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql [06:28:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [06:29:14] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=nintendowiki /srv/mediawiki/1.41/extensions/CheckUser/schema/mysql/patch-cu_changes-add-cuc_only_for_read_old.sql (END - exit=0) [06:29:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:21:52] PROBLEM - wiki.albrusmusic.ru - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.albrusmusic.ru' expires in 15 day(s) (Sun 03 Mar 2024 11:11:59 AM GMT +0000). [11:22:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dd6c63a298cf...ebbb8b4b5197 [11:22:10] [02miraheze/ssl] 07MirahezeSSLBot 03ebbb8b4 - Bot: Update SSL cert for wiki.albrusmusic.ru [11:40:38] PROBLEM - wiki.msnld.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.msnld.com' expires in 15 day(s) (Sun 03 Mar 2024 11:21:09 AM GMT +0000). [11:40:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ebbb8b4b5197...8a7fccf2b1c1 [11:40:51] [02miraheze/ssl] 07MirahezeSSLBot 038a7fccf - Bot: Update SSL cert for wiki.msnld.com [11:44:56] PROBLEM - looneypyramids.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'looneypyramids.wiki' expires in 15 day(s) (Sun 03 Mar 2024 11:14:26 AM GMT +0000). [11:45:09] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8a7fccf2b1c1...48d05e0b8db0 [11:45:10] [02miraheze/ssl] 07MirahezeSSLBot 0348d05e0 - Bot: Update SSL cert for looneypyramids.wiki [11:46:00] PROBLEM - encyclopedia.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'encyclopedia.eu.org' expires in 15 day(s) (Sun 03 Mar 2024 11:43:11 AM GMT +0000). [11:46:15] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/48d05e0b8db0...b69eafa089a6 [11:46:16] [02miraheze/ssl] 07MirahezeSSLBot 03b69eafa - Bot: Update SSL cert for encyclopedia.eu.org [11:50:21] RECOVERY - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is OK: OK - NGINX Error Rate is 36% [11:50:45] RECOVERY - wiki.albrusmusic.ru - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.albrusmusic.ru' will expire on Thu 16 May 2024 10:22:01 AM GMT +0000. [12:10:16] RECOVERY - wiki.msnld.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.msnld.com' will expire on Thu 16 May 2024 10:40:44 AM GMT +0000. [12:14:21] RECOVERY - looneypyramids.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'looneypyramids.wiki' will expire on Thu 16 May 2024 10:45:04 AM GMT +0000. [12:15:01] RECOVERY - encyclopedia.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopedia.eu.org' will expire on Thu 16 May 2024 10:46:09 AM GMT +0000. [12:15:40] PROBLEM - wiki.meeusen.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.meeusen.net' expires in 15 day(s) (Sun 03 Mar 2024 11:47:06 AM GMT +0000). [12:15:53] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b69eafa089a6...c9d7ac4fbaaf [12:15:56] [02miraheze/ssl] 07MirahezeSSLBot 03c9d7ac4 - Bot: Update SSL cert for wiki.meeusen.net [12:28:08] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [12:43:11] PROBLEM - wiki.harlonserver.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.harlonserver.net' expires in 15 day(s) (Sun 03 Mar 2024 12:23:19 PM GMT +0000). [12:43:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c9d7ac4fbaaf...cc6a31c595f7 [12:43:25] [02miraheze/ssl] 07MirahezeSSLBot 03cc6a31c - Bot: Update SSL cert for wiki.harlonserver.net [12:52:13] PROBLEM - landofliberos.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'landofliberos.com' expires in 15 day(s) (Sun 03 Mar 2024 12:38:38 PM GMT +0000). [12:52:24] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cc6a31c595f7...f6dd40a7a2dd [12:52:25] [02miraheze/ssl] 07MirahezeSSLBot 03f6dd40a - Bot: Update SSL cert for landofliberos.com [12:57:55] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [13:12:03] [02MirahezeMagic] 07amire80 opened pull request 03#465: Improve requestssl-text - 13https://github.com/miraheze/MirahezeMagic/pull/465 [13:12:22] RECOVERY - wiki.harlonserver.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.harlonserver.net' will expire on Thu 16 May 2024 11:43:16 AM GMT +0000. [13:12:30] [02MirahezeMagic] 07amire80 commented on pull request 03#465: Improve requestssl-text - 13https://github.com/miraheze/MirahezeMagic/pull/465#issuecomment-1948364716 [13:15:05] RECOVERY - wiki.meeusen.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.meeusen.net' will expire on Thu 16 May 2024 11:15:46 AM GMT +0000. [13:16:14] miraheze/MirahezeMagic - amire80 the build has errored. [13:19:49] PROBLEM - wiki.site97.fr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.site97.fr' expires in 15 day(s) (Sun 03 Mar 2024 01:09:55 PM GMT +0000). [13:20:00] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f6dd40a7a2dd...3c92a700b3a6 [13:20:03] [02miraheze/ssl] 07MirahezeSSLBot 033c92a70 - Bot: Update SSL cert for wiki.site97.fr [13:21:26] RECOVERY - landofliberos.com - LetsEncrypt on sslhost is OK: OK - Certificate 'landofliberos.com' will expire on Thu 16 May 2024 11:52:19 AM GMT +0000. [13:27:39] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [13:32:11] PROBLEM - evilgeniuswiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'evilgeniuswiki.com' expires in 15 day(s) (Sun 03 Mar 2024 01:10:12 PM GMT +0000). [13:53:59] PROBLEM - abeyancewiki.bumblebee-studios.se - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'abeyancewiki.bumblebee-studios.se' expires in 15 day(s) (Sun 03 Mar 2024 01:24:26 PM GMT +0000). [13:54:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3c92a700b3a6...0cac5c4adea4 [13:54:12] [02miraheze/ssl] 07MirahezeSSLBot 030cac5c4 - Bot: Update SSL cert for abeyancewiki.bumblebee-studios.se [13:57:22] PROBLEM - www.cgradegames.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.cgradegames.net' expires in 15 day(s) (Sun 03 Mar 2024 01:51:39 PM GMT +0000). [13:57:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0cac5c4adea4...80304066cd4d [13:57:37] [02miraheze/ssl] 07MirahezeSSLBot 038030406 - Bot: Update SSL cert for www.cgradegames.net [14:03:48] PROBLEM - wiki.mastodon.kr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mastodon.kr' expires in 15 day(s) (Sun 03 Mar 2024 01:55:20 PM GMT +0000). [14:04:00] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/80304066cd4d...fdb44ca4c557 [14:04:03] [02miraheze/ssl] 07MirahezeSSLBot 03fdb44ca - Bot: Update SSL cert for wiki.mastodon.kr [14:09:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/fdb44ca4c557...d07834b5d9f4 [14:09:06] [02miraheze/ssl] 07MirahezeSSLBot 03d07834b - Bot: Add SSL cert for wiki.10key.plus [14:16:06] PROBLEM - itora.bunnies1230.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'itora.bunnies1230.com' expires in 15 day(s) (Sun 03 Mar 2024 01:54:30 PM GMT +0000). [14:16:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/d07834b5d9f4...1d1f60145bc1 [14:16:21] [02miraheze/ssl] 07MirahezeSSLBot 031d1f601 - Bot: Update SSL cert for itora.bunnies1230.com [14:16:22] PROBLEM - eng.archiopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'eng.archiopedia.org' expires in 15 day(s) (Sun 03 Mar 2024 02:02:41 PM GMT +0000). [14:16:34] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1d1f60145bc1...a055bc188b79 [14:16:35] [02miraheze/ssl] 07MirahezeSSLBot 03a055bc1 - Bot: Update SSL cert for eng.archiopedia.org [14:19:25] RECOVERY - wiki.site97.fr - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.site97.fr' will expire on Thu 16 May 2024 12:19:54 PM GMT +0000. [14:23:27] RECOVERY - abeyancewiki.bumblebee-studios.se - LetsEncrypt on sslhost is OK: OK - Certificate 'abeyancewiki.bumblebee-studios.se' will expire on Thu 16 May 2024 12:54:04 PM GMT +0000. [14:27:14] RECOVERY - www.cgradegames.net - LetsEncrypt on sslhost is OK: OK - Certificate 'www.cgradegames.net' will expire on Thu 16 May 2024 12:57:28 PM GMT +0000. [14:29:21] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:33:47] RECOVERY - wiki.mastodon.kr - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mastodon.kr' will expire on Thu 16 May 2024 01:03:54 PM GMT +0000. [14:33:57] PROBLEM - webkinzguide.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'webkinzguide.com' expires in 15 day(s) (Sun 03 Mar 2024 02:10:51 PM GMT +0000). [14:34:09] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a055bc188b79...58f8336430d7 [14:34:11] [02miraheze/ssl] 07MirahezeSSLBot 0358f8336 - Bot: Update SSL cert for webkinzguide.com [14:49:28] PROBLEM - wiki.otir.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.otir.nl' expires in 15 day(s) (Sun 03 Mar 2024 02:28:30 PM GMT +0000). [14:49:42] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/58f8336430d7...ca035e0b459f [14:49:43] [02miraheze/ssl] 07MirahezeSSLBot 03ca035e0 - Bot: Update SSL cert for wiki.otir.nl [15:02:54] RECOVERY - webkinzguide.com - LetsEncrypt on sslhost is OK: OK - Certificate 'webkinzguide.com' will expire on Thu 16 May 2024 01:34:03 PM GMT +0000. [15:06:00] PROBLEM - wiki.milkywayidle.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.milkywayidle.com' expires in 15 day(s) (Sun 03 Mar 2024 02:50:24 PM GMT +0000). [15:06:12] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ca035e0b459f...e9edd51874a6 [15:06:15] [02miraheze/ssl] 07MirahezeSSLBot 03e9edd51 - Bot: Update SSL cert for wiki.milkywayidle.com [15:06:59] PROBLEM - wiki.auraxis.co - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.auraxis.co' expires in 15 day(s) (Sun 03 Mar 2024 02:48:02 PM GMT +0000). [15:07:10] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e9edd51874a6...ad4d78dc7bcb [15:07:11] [02miraheze/ssl] 07MirahezeSSLBot 03ad4d78d - Bot: Update SSL cert for wiki.auraxis.co [15:08:47] PROBLEM - cp41 Varnish Backends on cp41 is CRITICAL: 1 backends are down. mw151 [15:08:59] PROBLEM - 321nails.crpteam.club - LetsEncrypt on sslhost is WARNING: WARNING - Certificate '321nails.crpteam.club' expires in 15 day(s) (Sun 03 Mar 2024 02:56:50 PM GMT +0000). [15:09:11] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ad4d78dc7bcb...ecbd561fcb3a [15:09:13] [02miraheze/ssl] 07MirahezeSSLBot 03ecbd561 - Bot: Update SSL cert for 321nails.crpteam.club [15:14:19] RECOVERY - itora.bunnies1230.com - LetsEncrypt on sslhost is OK: OK - Certificate 'itora.bunnies1230.com' will expire on Thu 16 May 2024 01:16:12 PM GMT +0000. [15:15:04] RECOVERY - eng.archiopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'eng.archiopedia.org' will expire on Thu 16 May 2024 01:16:27 PM GMT +0000. [15:15:17] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 40% [15:19:21] PROBLEM - solaswiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'solaswiki.org' expires in 15 day(s) (Sun 03 Mar 2024 03:14:28 PM GMT +0000). [15:19:27] RECOVERY - wiki.otir.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.otir.nl' will expire on Thu 16 May 2024 01:49:35 PM GMT +0000. [15:19:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ecbd561fcb3a...9270ed78fe82 [15:19:31] [02miraheze/ssl] 07MirahezeSSLBot 039270ed7 - Bot: Update SSL cert for solaswiki.org [15:22:22] PROBLEM - largedu.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'largedu.eu.org' expires in 15 day(s) (Sun 03 Mar 2024 02:54:24 PM GMT +0000). [15:22:25] PROBLEM - dkpedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'dkpedia.com' expires in 15 day(s) (Sun 03 Mar 2024 03:07:28 PM GMT +0000). [15:22:36] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9270ed78fe82...923052d43692 [15:22:38] [02miraheze/ssl] 07MirahezeSSLBot 03923052d - Bot: Update SSL cert for largedu.eu.org [15:22:40] PROBLEM - feathercare.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'feathercare.net' expires in 15 day(s) (Sun 03 Mar 2024 03:08:18 PM GMT +0000). [15:22:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/923052d43692...1eec814cea43 [15:22:50] [02miraheze/ssl] 07MirahezeSSLBot 031eec814 - Bot: Update SSL cert for dkpedia.com [15:22:57] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1eec814cea43...3a1303ba1223 [15:23:00] [02miraheze/ssl] 07MirahezeSSLBot 033a1303b - Bot: Update SSL cert for feathercare.net [15:28:07] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=321nailswiki (END - exit=2) [15:28:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:28:11] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=321nailswiki --no-updates --username-prefix=321nails /home/reception/321nailswiki-20240216134210.xml (START) [15:28:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:28:22] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [15:30:23] PROBLEM - wiki.songngu.xyz - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for wiki.songngu.xyz could not be found [15:32:41] PROBLEM - wiki.songngu.xyz - LetsEncrypt on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [15:35:39] RECOVERY - wiki.auraxis.co - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.auraxis.co' will expire on Thu 16 May 2024 02:07:05 PM GMT +0000. [15:35:42] RECOVERY - wiki.milkywayidle.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.milkywayidle.com' will expire on Thu 16 May 2024 02:06:06 PM GMT +0000. [15:38:14] RECOVERY - cp41 Varnish Backends on cp41 is OK: All 18 backends are healthy [15:40:32] [02RequestSSL] 07amire80 opened pull request 03#34: Improve line breaks use in messages - 13https://github.com/miraheze/RequestSSL/pull/34 [15:47:50] PROBLEM - cp36 Disk Space on cp36 is WARNING: DISK WARNING - free space: / 9699MiB (10% inode=98%); [15:49:58] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=321nailswiki --no-updates --username-prefix=321nails /home/reception/321nailswiki-20240216134210.xml (END - exit=0) [15:49:59] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=321nailswiki (START) [15:50:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:50:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:50:08] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildall.php --wiki=321nailswiki (END - exit=256) [15:50:09] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/initSiteStats.php --wiki=321nailswiki --active --update (END - exit=0) [15:50:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:50:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:51:22] RECOVERY - feathercare.net - LetsEncrypt on sslhost is OK: OK - Certificate 'feathercare.net' will expire on Thu 16 May 2024 02:22:51 PM GMT +0000. [15:51:39] RECOVERY - largedu.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'largedu.eu.org' will expire on Thu 16 May 2024 02:22:30 PM GMT +0000. [15:51:40] RECOVERY - dkpedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'dkpedia.com' will expire on Thu 16 May 2024 02:22:41 PM GMT +0000. [15:52:17] [02RequestSSL] 07AgentIsai closed pull request 03#34: Improve line breaks use in messages - 13https://github.com/miraheze/RequestSSL/pull/34 [15:52:19] [02miraheze/RequestSSL] 07AgentIsai pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/RequestSSL/compare/7171a8354b04...89a25ff8b334 [15:52:20] [02miraheze/RequestSSL] 07amire80 0389a25ff - Improve line breaks use in messages (#34) [15:56:28] miraheze/RequestSSL - AgentIsai the build has errored. [15:57:45] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:01:26] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 9700MiB (10% inode=98%); [16:03:42] PROBLEM - grk.archiopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'grk.archiopedia.org' expires in 15 day(s) (Sun 03 Mar 2024 03:59:47 PM GMT +0000). [16:03:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3a1303ba1223...339bb5d266a5 [16:03:56] [02miraheze/ssl] 07MirahezeSSLBot 03339bb5d - Bot: Update SSL cert for grk.archiopedia.org [16:05:39] [02mw-config] 07R4356th synchronize pull request 03#5481: Install UnlinkedWikibase - 13https://github.com/miraheze/mw-config/pull/5481 [16:06:09] RECOVERY - 321nails.crpteam.club - LetsEncrypt on sslhost is OK: OK - Certificate '321nails.crpteam.club' will expire on Thu 16 May 2024 02:09:05 PM GMT +0000. [16:06:36] miraheze/mw-config - R4356th the build passed. [16:15:13] PROBLEM - puritwiki.p-e.kr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'puritwiki.p-e.kr' expires in 15 day(s) (Sun 03 Mar 2024 04:03:07 PM GMT +0000). [16:15:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/339bb5d266a5...592e7c214a41 [16:15:32] [02miraheze/ssl] 07MirahezeSSLBot 03592e7c2 - Bot: Update SSL cert for puritwiki.p-e.kr [16:16:42] RECOVERY - solaswiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'solaswiki.org' will expire on Thu 16 May 2024 02:19:24 PM GMT +0000. [16:27:24] [02RequestSSL] 07amire80 opened pull request 03#35: Add GENDER hint to log entry messages - 13https://github.com/miraheze/RequestSSL/pull/35 [16:27:43] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [16:27:52] PROBLEM - grk.repository.archiopedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'grk.repository.archiopedia.org' expires in 15 day(s) (Sun 03 Mar 2024 04:00:59 PM GMT +0000). [16:28:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/592e7c214a41...029872071462 [16:28:09] [02miraheze/ssl] 07MirahezeSSLBot 030298720 - Bot: Update SSL cert for grk.repository.archiopedia.org [16:30:15] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=polandballwiki --no-updates polandballwiki_xml_e802d265aa505e24dd91.xml.gz (START) [16:30:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:01] miraheze/RequestSSL - amire80 the build has errored. [16:32:31] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=lgbtawiki --no-updates lgbtqiawiki_xml_3095cc1edb334194a72f.xml.gz (START) [16:32:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:32:46] RECOVERY - grk.archiopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'grk.archiopedia.org' will expire on Thu 16 May 2024 03:03:48 PM GMT +0000. [16:42:25] PROBLEM - slymods.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'slymods.info' expires in 15 day(s) (Sun 03 Mar 2024 04:37:02 PM GMT +0000). [16:42:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/029872071462...8958787e446f [16:42:38] [02miraheze/ssl] 07MirahezeSSLBot 038958787 - Bot: Update SSL cert for slymods.info [16:43:07] PROBLEM - privacywiki.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'privacywiki.xyz' expires in 15 day(s) (Sun 03 Mar 2024 04:19:02 PM GMT +0000). [16:50:15] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki --comment=[[phab:T11781|Requested]] --user=Maintenance script . (START) [16:50:15] https://meta.miraheze.org/wiki/phab:T11781 [16:50:16] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki --comment=[[phab:T11781|Requested]] --user=Maintenance script . (END - exit=32512) [16:50:16] https://meta.miraheze.org/wiki/phab:T11781 [16:50:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:50:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:50:45] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance script (START) [16:50:45] https://meta.miraheze.org/wiki/phab:T11781%5c [16:50:47] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance script (END - exit=256) [16:50:47] https://meta.miraheze.org/wiki/phab:T11781%5c [16:50:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:50:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:50:54] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance_script (START) [16:50:54] https://meta.miraheze.org/wiki/phab:T11781%5c [16:50:56] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance_script (END - exit=2) [16:50:56] https://meta.miraheze.org/wiki/phab:T11781%5c [16:50:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:51:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:51:09] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance_script (START) [16:51:09] https://meta.miraheze.org/wiki/phab:T11781%5c [16:51:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:57:49] RECOVERY - grk.repository.archiopedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'grk.repository.archiopedia.org' will expire on Thu 16 May 2024 03:28:01 PM GMT +0000. [16:58:24] PROBLEM - replication-ops.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'replication-ops.com' expires in 15 day(s) (Sun 03 Mar 2024 04:29:40 PM GMT +0000). [16:58:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8958787e446f...8dd10d18c562 [16:58:34] [02miraheze/ssl] 07MirahezeSSLBot 038dd10d1 - Bot: Update SSL cert for replication-ops.com [16:59:47] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [17:05:22] PROBLEM - db181 Current Load on db181 is CRITICAL: LOAD CRITICAL - total load average: 52.28, 29.68, 11.95 [17:05:34] PROBLEM - db181 PowerDNS Recursor on db181 is CRITICAL: CRITICAL - Plugin timed out while executing system call [17:06:18] PROBLEM - db181 Puppet on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:09:16] PROBLEM - db181 SSH on db181 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [17:10:11] RECOVERY - db181 PowerDNS Recursor on db181 is OK: DNS OK: 0.787 seconds response time. miraheze.org returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [17:11:36] PROBLEM - lab.wikimimo.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'lab.wikimimo.com' expires in 15 day(s) (Sun 03 Mar 2024 04:47:37 PM GMT +0000). [17:11:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8dd10d18c562...5ebe4da78941 [17:11:49] [02miraheze/ssl] 07MirahezeSSLBot 035ebe4da - Bot: Update SSL cert for lab.wikimimo.com [17:12:13] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=warcapadawiki /srv/mediawiki/1.41/extensions/CheckUser/schema/mysql/patch-cu_private_event-def.sql (END - exit=0) [17:12:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:12:18] RECOVERY - slymods.info - LetsEncrypt on sslhost is OK: OK - Certificate 'slymods.info' will expire on Thu 16 May 2024 03:42:31 PM GMT +0000. [17:12:34] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=warcapadawiki /srv/mediawiki/1.41/extensions/CheckUser/schema/mysql/patch-cu_changes-add-cuc_only_for_read_old.sql (END - exit=0) [17:12:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:13:22] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/4135a87f80d5...2de77451c98c [17:13:25] [02miraheze/puppet] 07Universal-Omega 032de7745 - relaybot: remove #miraheze-sre-security [17:14:27] RECOVERY - puritwiki.p-e.kr - LetsEncrypt on sslhost is OK: OK - Certificate 'puritwiki.p-e.kr' will expire on Thu 16 May 2024 03:15:22 PM GMT +0000. [17:16:31] PROBLEM - db181 PowerDNS Recursor on db181 is CRITICAL: CRITICAL - Plugin timed out while executing system call [17:17:46] PROBLEM - db181 conntrack_table_size on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:17:46] PROBLEM - db181 Backups SQL mhglobal on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:17:46] PROBLEM - db181 Backups SQL on db181 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [17:27:16] RECOVERY - db181 PowerDNS Recursor on db181 is OK: DNS OK: 0.285 seconds response time. miraheze.org returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [17:27:19] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=lgbtawiki --no-updates lgbtqiawiki_xml_3095cc1edb334194a72f.xml.gz (END - exit=256) [17:27:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:28:10] RECOVERY - db181 SSH on db181 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [17:28:11] RECOVERY - replication-ops.com - LetsEncrypt on sslhost is OK: OK - Certificate 'replication-ops.com' will expire on Thu 16 May 2024 03:58:27 PM GMT +0000. [17:29:04] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:29:07] RECOVERY - db181 conntrack_table_size on db181 is OK: OK: nf_conntrack is 0 % full [17:29:07] RECOVERY - db181 Backups SQL mhglobal on db181 is OK: FILE_AGE OK: /var/log/sql-mhglobal-backup-weekly.log is 476937 seconds old and 208 bytes [17:29:12] RECOVERY - db181 Backups SQL on db181 is OK: FILE_AGE OK: /var/log/sql-backup.log is 705503 seconds old and 112439 bytes [17:34:15] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 143.50 ms [17:36:13] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 144.10 ms [17:47:49] [02RequestSSL] 07redbluegreenhat closed pull request 03#35: Add GENDER hint to log entry messages - 13https://github.com/miraheze/RequestSSL/pull/35 [17:47:52] [02miraheze/RequestSSL] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/RequestSSL/compare/89a25ff8b334...29975b2d30a1 [17:47:53] [02miraheze/RequestSSL] 07amire80 0329975b2 - Add GENDER hint to log entry messages (#35) [17:52:06] miraheze/RequestSSL - redbluegreenhat the build has errored. [17:52:08] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 143.18 ms [17:54:06] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.15 ms [17:58:14] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [17:58:58] [02miraheze/ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5ebe4da78941...f3f3e417c849 [17:59:01] [02miraheze/ssl] 07AgentIsai 03f3f3e41 - Test adding a wiki.surf redirect [17:59:15] PROBLEM - db181 Current Load on db181 is WARNING: LOAD WARNING - total load average: 0.19, 0.34, 12.00 [18:01:53] [02miraheze/dns] 07AgentIsai pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/2f0f3c27b08d...e454dfbd92a3 [18:01:56] [02miraheze/dns] 07AgentIsai 03e454dfb - Add wiki.surf [18:03:15] RECOVERY - db181 Current Load on db181 is OK: LOAD OK - total load average: 0.13, 0.25, 9.30 [18:03:30] PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:03:33] [02miraheze/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/dns/compare/e454dfbd92a3...a7f045d27ad8 [18:03:36] [02miraheze/dns] 07AgentIsai 03a7f045d - Add ACME challenges [18:07:35] PROBLEM - cp41 Puppet on cp41 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:10:12] PROBLEM - cp27 Puppet on cp27 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:11:05] RECOVERY - lab.wikimimo.com - LetsEncrypt on sslhost is OK: OK - Certificate 'lab.wikimimo.com' will expire on Thu 16 May 2024 04:11:41 PM GMT +0000. [18:11:13] PROBLEM - cp28 Puppet on cp28 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:12:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/ssl/compare/f3f3e417c849...e4a54122fa49 [18:12:38] [02miraheze/ssl] 07MirahezeSSLBot 03e4a5412 - Add wiki.surf certificate [18:14:10] PROBLEM - petrawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.petrawiki.org' expires in 15 day(s) (Sun 03 Mar 2024 06:10:17 PM GMT +0000). [18:19:28] [02miraheze/ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e4a54122fa49...054e7978c4d0 [18:19:30] [02miraheze/ssl] 07AgentIsai 03054e797 - Fix typo [18:21:26] PROBLEM - wiki.cryptoroyale.one - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.cryptoroyale.one' expires in 15 day(s) (Sun 03 Mar 2024 05:58:02 PM GMT +0000). [18:23:57] PROBLEM - cp41 SSH on cp41 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [18:24:33] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 100% [18:24:36] PROBLEM - cp41 HTTPS on cp41 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Connection timed out after 10002 milliseconds [18:25:11] PROBLEM - Host cp41 is DOWN: PING CRITICAL - Packet loss = 100% [18:27:49] PROBLEM - segawiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'segawiki.com' expires in 15 day(s) (Sun 03 Mar 2024 06:18:42 PM GMT +0000). [18:27:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/054e7978c4d0...e0fcfb5f93b8 [18:28:00] [02miraheze/ssl] 07MirahezeSSLBot 03e0fcfb5 - Bot: Update SSL cert for segawiki.com [18:28:46] [02miraheze/mw-config] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/516f05aa63cf...9cee72bad7ab [18:28:48] [02miraheze/mw-config] 07AgentIsai 039cee72b - Add wiki.surf for metawiki url shortner [18:29:03] !log [agent@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [18:29:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:29:12] !log [agent@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 9s [18:29:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:29:45] miraheze/mw-config - AgentIsai the build passed. [18:30:54] [02miraheze/ssl] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e0fcfb5f93b8...f38e95db8309 [18:30:57] [02miraheze/ssl] 07AgentIsai 03f38e95d - Fix URLRedirector [18:31:47] RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:35:02] RECOVERY - Host cp41 is UP: PING WARNING - Packet loss = 86%, RTA = 139.70 ms [18:35:10] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 147.53 ms [18:36:06] PROBLEM - en.petrawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.petrawiki.org' expires in 15 day(s) (Sun 03 Mar 2024 06:10:17 PM GMT +0000). [18:36:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f38e95db8309...ab2017241a81 [18:36:20] [02miraheze/ssl] 07MirahezeSSLBot 03ab20172 - Bot: Update SSL cert for en.petrawiki.org [18:36:41] RECOVERY - cp41 SSH on cp41 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [18:36:49] RECOVERY - cp41 Puppet on cp41 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [18:37:06] RECOVERY - cp41 HTTPS on cp41 is OK: HTTP OK: HTTP/2 404 - Status line output matched "HTTP/2 404" - 3610 bytes in 0.909 second response time [18:37:20] RECOVERY - cp27 Puppet on cp27 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures [18:39:19] RECOVERY - cp28 Puppet on cp28 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:41:46] !log [macfan@mwtask181] starting deploy of {'l10n': True, 'versions': '1.41', 'upgrade_extensions': ['MirahezeMagic', 'RequestSSL']} to all [18:41:47] !log [macfan@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [18:41:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:41:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:42:52] !log [macfan@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [18:42:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:46:02] PROBLEM - www.pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.pornwiki.org' expires in 15 day(s) (Sun 03 Mar 2024 06:36:43 PM GMT +0000). [18:47:25] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ab2017241a81...0a2835e95f0f [18:47:27] [02miraheze/ssl] 07Reception123 030a2835e - force redirect for nintendowiki.wiki [18:48:49] [02miraheze/dns] 07Reception123 pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/dns/compare/a7f045d27ad8...57f676c76a81 [18:48:51] [02miraheze/dns] 07Reception123 0357f676c - add nintendowiki.wiki zone [18:49:35] !log [macfan@mwtask181] finished deploy of {'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all - SUCCESS in 403s [18:49:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:51:07] PROBLEM - en.pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.pornwiki.org' expires in 15 day(s) (Sun 03 Mar 2024 06:36:43 PM GMT +0000). [18:51:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/0a2835e95f0f...110fef5bfa8a [18:51:19] [02miraheze/ssl] 07MirahezeSSLBot 03110fef5 - Bot: Update SSL cert for en.pornwiki.org [18:53:00] PROBLEM - resources.africanvision.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'resources.africanvision.org' expires in 15 day(s) (Sun 03 Mar 2024 06:41:59 PM GMT +0000). [18:53:11] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/110fef5bfa8a...99445ed9e3fa [18:53:14] [02miraheze/ssl] 07MirahezeSSLBot 0399445ed - Bot: Update SSL cert for resources.africanvision.org [18:56:32] PROBLEM - pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'en.pornwiki.org' expires in 15 day(s) (Sun 03 Mar 2024 06:36:43 PM GMT +0000). [18:57:40] RECOVERY - segawiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'segawiki.com' will expire on Thu 16 May 2024 05:27:52 PM GMT +0000. [18:59:14] !log [macfan@mwtask181] starting deploy of {'pull': 'world', 'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [18:59:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:59:36] !log [macfan@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [18:59:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:00:27] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [19:04:44] RECOVERY - en.petrawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.petrawiki.org' will expire on Thu 16 May 2024 05:36:12 PM GMT +0000. [19:06:31] PROBLEM - wiki.artmechanicum.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.artmechanicum.com' expires in 15 day(s) (Sun 03 Mar 2024 06:44:28 PM GMT +0000). [19:06:43] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/99445ed9e3fa...278073dee6eb [19:06:44] [02miraheze/ssl] 07MirahezeSSLBot 03278073d - Bot: Update SSL cert for wiki.artmechanicum.com [19:07:17] PROBLEM - wiki.bluevertigo.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.bluevertigo.org' expires in 15 day(s) (Sun 03 Mar 2024 06:47:39 PM GMT +0000). [19:07:29] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/278073dee6eb...55e471f72aa7 [19:07:32] [02miraheze/ssl] 07MirahezeSSLBot 0355e471f - Bot: Update SSL cert for wiki.bluevertigo.org [19:11:44] RECOVERY - petrawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.petrawiki.org' will expire on Thu 16 May 2024 05:36:12 PM GMT +0000. [19:14:38] PROBLEM - wiki.zahira.me - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.zahira.me' expires in 15 day(s) (Sun 03 Mar 2024 07:04:59 PM GMT +0000). [19:14:41] !log [macfan@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [19:14:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:14:50] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/55e471f72aa7...a47b6588cfc7 [19:14:53] [02miraheze/ssl] 07MirahezeSSLBot 03a47b658 - Bot: Update SSL cert for wiki.zahira.me [19:15:10] !log [macfan@mwtask181] finished deploy of {'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all - SUCCESS in 29s [19:15:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:08] !log [macfan@mwtask181] starting deploy of {'l10n': True, 'force': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [19:16:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:16] !log [macfan@mwtask181] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [19:16:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:18:28] !log [macfan@mwtask181] starting deploy of {'l10n': True, 'force_upgrade': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all [19:18:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:19:51] RECOVERY - en.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.pornwiki.org' will expire on Thu 16 May 2024 05:51:10 PM GMT +0000. [19:21:25] PROBLEM - tech.exputra.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'tech.exputra.com' expires in 15 day(s) (Sun 03 Mar 2024 07:02:11 PM GMT +0000). [19:21:38] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a47b6588cfc7...2da4a5464e17 [19:21:40] RECOVERY - resources.africanvision.org - LetsEncrypt on sslhost is OK: OK - Certificate 'resources.africanvision.org' will expire on Thu 16 May 2024 05:53:05 PM GMT +0000. [19:21:41] [02miraheze/ssl] 07MirahezeSSLBot 032da4a54 - Bot: Update SSL cert for tech.exputra.com [19:22:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+1/-0/±1] 13https://github.com/miraheze/ssl/compare/2da4a5464e17...1d60487c469f [19:22:19] [02miraheze/ssl] 07MirahezeSSLBot 031d60487 - Bot: Add SSL cert for nintendowiki.wiki [19:23:47] PROBLEM - metroidpedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'metroidpedia.com' expires in 15 day(s) (Sun 03 Mar 2024 07:06:43 PM GMT +0000). [19:23:56] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1d60487c469f...cecd52b9b265 [19:23:57] [02miraheze/ssl] 07MirahezeSSLBot 03cecd52b - Bot: Update SSL cert for metroidpedia.com [19:25:07] !log [macfan@mwtask181] finished deploy of {'l10n': True, 'force_upgrade': True, 'versions': '1.41', 'upgrade_extensions': 'RequestSSL'} to all - SUCCESS in 398s [19:25:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:25:24] RECOVERY - pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.pornwiki.org' will expire on Thu 16 May 2024 05:51:10 PM GMT +0000. [19:31:56] PROBLEM - schizoidnightmares.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'schizoidnightmares.wiki' expires in 15 day(s) (Sun 03 Mar 2024 07:25:37 PM GMT +0000). [19:32:07] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/cecd52b9b265...ff93e74514f2 [19:32:08] [02miraheze/ssl] 07MirahezeSSLBot 03ff93e74 - Bot: Update SSL cert for schizoidnightmares.wiki [19:34:44] PROBLEM - crashspyrowiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'crashspyrowiki.com' expires in 15 day(s) (Sun 03 Mar 2024 07:25:11 PM GMT +0000). [19:34:52] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ff93e74514f2...93db814887c8 [19:34:53] [02miraheze/ssl] 07MirahezeSSLBot 0393db814 - Bot: Update SSL cert for crashspyrowiki.com [19:35:02] RECOVERY - wiki.artmechanicum.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.artmechanicum.com' will expire on Thu 16 May 2024 06:06:37 PM GMT +0000. [19:36:33] RECOVERY - wiki.bluevertigo.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.bluevertigo.org' will expire on Thu 16 May 2024 06:07:23 PM GMT +0000. [19:42:57] PROBLEM - www.sekaipedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.sekaipedia.org' expires in 15 day(s) (Sun 03 Mar 2024 07:22:28 PM GMT +0000). [19:43:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/93db814887c8...e5523ff3fec7 [19:43:10] [02miraheze/ssl] 07MirahezeSSLBot 03e5523ff - Bot: Update SSL cert for www.sekaipedia.org [19:43:46] RECOVERY - wiki.zahira.me - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.zahira.me' will expire on Thu 16 May 2024 06:14:44 PM GMT +0000. [19:44:28] PROBLEM - rarewarewiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'rarewarewiki.com' expires in 15 day(s) (Sun 03 Mar 2024 07:39:38 PM GMT +0000). [19:44:37] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e5523ff3fec7...5532dc6eb348 [19:44:40] [02miraheze/ssl] 07MirahezeSSLBot 035532dc6 - Bot: Update SSL cert for rarewarewiki.com [19:44:50] RECOVERY - www.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'en.pornwiki.org' will expire on Thu 16 May 2024 05:51:10 PM GMT +0000. [19:46:51] PROBLEM - apeirology.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'apeirology.com' expires in 15 day(s) (Sun 03 Mar 2024 07:27:09 PM GMT +0000). [19:46:59] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5532dc6eb348...3d204246b00f [19:47:00] [02miraheze/ssl] 07MirahezeSSLBot 033d20424 - Bot: Update SSL cert for apeirology.com [19:50:01] RECOVERY - tech.exputra.com - LetsEncrypt on sslhost is OK: OK - Certificate 'tech.exputra.com' will expire on Thu 16 May 2024 06:21:31 PM GMT +0000. [19:52:56] RECOVERY - metroidpedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'metroidpedia.com' will expire on Thu 16 May 2024 06:23:49 PM GMT +0000. [19:59:29] PROBLEM - sona.pona.la - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sona.pona.la' expires in 15 day(s) (Sun 03 Mar 2024 07:52:58 PM GMT +0000). [19:59:41] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3d204246b00f...42ca4032fdfc [19:59:42] [02miraheze/ssl] 07MirahezeSSLBot 0342ca403 - Bot: Update SSL cert for sona.pona.la [20:00:50] RECOVERY - schizoidnightmares.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'schizoidnightmares.wiki' will expire on Thu 16 May 2024 06:32:01 PM GMT +0000. [20:04:13] RECOVERY - crashspyrowiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'crashspyrowiki.com' will expire on Thu 16 May 2024 06:34:47 PM GMT +0000. [20:12:09] PROBLEM - miraheze.cf - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'miraheze.cf' expires in 15 day(s) (Sun 03 Mar 2024 07:46:34 PM GMT +0000). [20:22:56] PROBLEM - reviwiki.info - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'reviwiki.info' expires in 15 day(s) (Sun 03 Mar 2024 08:00:29 PM GMT +0000). [20:23:04] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/42ca4032fdfc...b6b0d9a200d1 [20:23:07] [02miraheze/ssl] 07MirahezeSSLBot 03b6b0d9a - Bot: Update SSL cert for reviwiki.info [20:24:06] PROBLEM - wiki.dwarvesrp.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.dwarvesrp.com' expires in 15 day(s) (Sun 03 Mar 2024 08:17:35 PM GMT +0000). [20:24:17] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b6b0d9a200d1...480b8f2aca95 [20:24:19] [02miraheze/ssl] 07MirahezeSSLBot 03480b8f2 - Bot: Update SSL cert for wiki.dwarvesrp.com [20:28:03] RECOVERY - sona.pona.la - LetsEncrypt on sslhost is OK: OK - Certificate 'sona.pona.la' will expire on Thu 16 May 2024 06:59:35 PM GMT +0000. [20:28:24] PROBLEM - yokaiwatchwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'yokaiwatchwiki.com' expires in 15 day(s) (Sun 03 Mar 2024 08:20:20 PM GMT +0000). [20:28:32] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/480b8f2aca95...a70010b76660 [20:28:34] [02miraheze/ssl] 07MirahezeSSLBot 03a70010b - Bot: Update SSL cert for yokaiwatchwiki.com [20:41:13] PROBLEM - files.pornwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'files.pornwiki.org' expires in 15 day(s) (Sun 03 Mar 2024 08:20:52 PM GMT +0000). [20:41:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a70010b76660...27ccf9ce1ca6 [20:41:25] [02miraheze/ssl] 07MirahezeSSLBot 0327ccf9c - Bot: Update SSL cert for files.pornwiki.org [20:41:33] RECOVERY - www.sekaipedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'www.sekaipedia.org' will expire on Thu 16 May 2024 06:43:03 PM GMT +0000. [20:42:46] RECOVERY - rarewarewiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'rarewarewiki.com' will expire on Thu 16 May 2024 06:44:31 PM GMT +0000. [20:44:21] RECOVERY - apeirology.com - LetsEncrypt on sslhost is OK: OK - Certificate 'apeirology.com' will expire on Thu 16 May 2024 06:46:53 PM GMT +0000. [20:51:41] RECOVERY - reviwiki.info - LetsEncrypt on sslhost is OK: OK - Certificate 'reviwiki.info' will expire on Thu 16 May 2024 07:22:58 PM GMT +0000. [20:54:04] RECOVERY - wiki.dwarvesrp.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.dwarvesrp.com' will expire on Thu 16 May 2024 07:24:11 PM GMT +0000. [20:57:34] RECOVERY - yokaiwatchwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'yokaiwatchwiki.com' will expire on Thu 16 May 2024 07:28:26 PM GMT +0000. [20:58:16] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [20:58:26] PROBLEM - freedomplanetwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'freedomplanetwiki.com' expires in 15 day(s) (Sun 03 Mar 2024 08:41:53 PM GMT +0000). [20:58:38] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/27ccf9ce1ca6...9c4004bbcb29 [20:58:39] [02miraheze/ssl] 07MirahezeSSLBot 039c4004b - Bot: Update SSL cert for freedomplanetwiki.com [21:08:29] PROBLEM - minescape.wiki - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'minescape.wiki' expires in 15 day(s) (Sun 03 Mar 2024 08:48:38 PM GMT +0000). [21:08:40] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/9c4004bbcb29...c4beee0f3562 [21:08:43] [02miraheze/ssl] 07MirahezeSSLBot 03c4beee0 - Bot: Update SSL cert for minescape.wiki [21:09:06] PROBLEM - ncuwiki.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ncuwiki.com' expires in 15 day(s) (Sun 03 Mar 2024 08:58:47 PM GMT +0000). [21:09:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/c4beee0f3562...b3dbeee1d1fe [21:09:19] [02miraheze/ssl] 07MirahezeSSLBot 03b3dbeee - Bot: Update SSL cert for ncuwiki.com [21:09:52] RECOVERY - files.pornwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'files.pornwiki.org' will expire on Thu 16 May 2024 07:41:16 PM GMT +0000. [21:26:21] PROBLEM - wiki.limaru.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.limaru.net' expires in 15 day(s) (Sun 03 Mar 2024 09:22:04 PM GMT +0000). [21:26:33] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/b3dbeee1d1fe...55acd4420d73 [21:26:36] [02miraheze/ssl] 07MirahezeSSLBot 0355acd44 - Bot: Update SSL cert for wiki.limaru.net [21:27:26] RECOVERY - freedomplanetwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'freedomplanetwiki.com' will expire on Thu 16 May 2024 07:58:31 PM GMT +0000. [21:28:11] PROBLEM - creativecommons.id - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'creativecommons.id' expires in 15 day(s) (Sun 03 Mar 2024 09:12:54 PM GMT +0000). [21:28:22] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/55acd4420d73...2c87db262aa7 [21:28:24] [02miraheze/ssl] 07MirahezeSSLBot 032c87db2 - Bot: Update SSL cert for creativecommons.id [21:31:08] PROBLEM - smafallfest.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'smafallfest.org' expires in 15 day(s) (Sun 03 Mar 2024 09:00:43 PM GMT +0000). [21:31:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/2c87db262aa7...8fac68ad2cba [21:31:22] [02miraheze/ssl] 07MirahezeSSLBot 038fac68a - Bot: Update SSL cert for smafallfest.org [21:32:08] PROBLEM - witchingworld.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'witchingworld.org' expires in 15 day(s) (Sun 03 Mar 2024 09:11:26 PM GMT +0000). [21:32:19] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8fac68ad2cba...ab3f09c19ea8 [21:32:20] [02miraheze/ssl] 07MirahezeSSLBot 03ab3f09c - Bot: Update SSL cert for witchingworld.org [21:37:41] RECOVERY - minescape.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'minescape.wiki' will expire on Thu 16 May 2024 08:08:35 PM GMT +0000. [21:52:11] PROBLEM - cp36 Disk Space on cp36 is CRITICAL: DISK CRITICAL - free space: / 5302MiB (5% inode=98%); [21:55:21] RECOVERY - wiki.limaru.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.limaru.net' will expire on Thu 16 May 2024 08:26:27 PM GMT +0000. [21:57:21] RECOVERY - creativecommons.id - LetsEncrypt on sslhost is OK: OK - Certificate 'creativecommons.id' will expire on Thu 16 May 2024 08:28:16 PM GMT +0000. [21:58:07] PROBLEM - elsterclopedie.marithaime.nl - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'elsterclopedie.marithaime.nl' expires in 15 day(s) (Sun 03 Mar 2024 09:41:53 PM GMT +0000). [21:58:18] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ab3f09c19ea8...f1d107edb70c [21:58:20] [02miraheze/ssl] 07MirahezeSSLBot 03f1d107e - Bot: Update SSL cert for elsterclopedie.marithaime.nl [22:00:12] RECOVERY - smafallfest.org - LetsEncrypt on sslhost is OK: OK - Certificate 'smafallfest.org' will expire on Thu 16 May 2024 08:31:14 PM GMT +0000. [22:00:54] PROBLEM - private.yahyabd.xyz - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'private.yahyabd.xyz' expires in 15 day(s) (Sun 03 Mar 2024 09:50:49 PM GMT +0000). [22:01:06] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f1d107edb70c...6ab8d20b324d [22:01:08] [02miraheze/ssl] 07MirahezeSSLBot 036ab8d20 - Bot: Update SSL cert for private.yahyabd.xyz [22:01:41] RECOVERY - witchingworld.org - LetsEncrypt on sslhost is OK: OK - Certificate 'witchingworld.org' will expire on Thu 16 May 2024 08:32:13 PM GMT +0000. [22:04:43] PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 5308MiB (5% inode=98%); [22:06:46] RECOVERY - ncuwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'ncuwiki.com' will expire on Thu 16 May 2024 08:09:12 PM GMT +0000. [22:11:46] PROBLEM - wiki.blockate.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.blockate.com' expires in 15 day(s) (Sun 03 Mar 2024 10:04:19 PM GMT +0000). [22:11:58] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6ab8d20b324d...07c3e9754c28 [22:12:01] [02miraheze/ssl] 07MirahezeSSLBot 0307c3e97 - Bot: Update SSL cert for wiki.blockate.com [22:21:16] PROBLEM - translate.petrawiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'translate.petrawiki.org' expires in 15 day(s) (Sun 03 Mar 2024 10:05:40 PM GMT +0000). [22:21:28] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/07c3e9754c28...975a43e4c206 [22:21:30] [02miraheze/ssl] 07MirahezeSSLBot 03975a43e - Bot: Update SSL cert for translate.petrawiki.org [22:27:04] PROBLEM - psychevos.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'psychevos.org' expires in 15 day(s) (Sun 03 Mar 2024 10:23:40 PM GMT +0000). [22:27:09] RECOVERY - elsterclopedie.marithaime.nl - LetsEncrypt on sslhost is OK: OK - Certificate 'elsterclopedie.marithaime.nl' will expire on Thu 16 May 2024 08:58:13 PM GMT +0000. [22:27:16] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/975a43e4c206...ff8a66bbb87d [22:27:19] [02miraheze/ssl] 07MirahezeSSLBot 03ff8a66b - Bot: Update SSL cert for psychevos.org [22:29:40] RECOVERY - private.yahyabd.xyz - LetsEncrypt on sslhost is OK: OK - Certificate 'private.yahyabd.xyz' will expire on Thu 16 May 2024 09:00:59 PM GMT +0000. [22:40:41] PROBLEM - thestories.kr - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'thestories.kr' expires in 15 day(s) (Sun 03 Mar 2024 10:18:10 PM GMT +0000). [22:41:28] PROBLEM - acgn.sfdev.eu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'acgn.sfdev.eu.org' expires in 15 day(s) (Sun 03 Mar 2024 10:21:38 PM GMT +0000). [22:41:49] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/ff8a66bbb87d...b3a5dc658d36 [22:41:51] [02miraheze/ssl] 07MirahezeSSLBot 03b3a5dc6 - Bot: Update SSL cert for acgn.sfdev.eu.org [22:45:06] !log [agent@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=1d6chanwiki /home/agent/images --comment=[[phab:T11781\|Requested]] --user=Maintenance_script (END - exit=0) [22:45:06] https://meta.miraheze.org/wiki/phab:T11781%5c [22:45:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:50:51] RECOVERY - translate.petrawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'translate.petrawiki.org' will expire on Thu 16 May 2024 09:21:22 PM GMT +0000. [22:57:02] RECOVERY - psychevos.org - LetsEncrypt on sslhost is OK: OK - Certificate 'psychevos.org' will expire on Thu 16 May 2024 09:27:10 PM GMT +0000. [23:10:18] RECOVERY - wiki.blockate.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.blockate.com' will expire on Thu 16 May 2024 09:11:52 PM GMT +0000. [23:11:19] RECOVERY - acgn.sfdev.eu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'acgn.sfdev.eu.org' will expire on Thu 16 May 2024 09:41:43 PM GMT +0000. [23:22:11] [02MirahezeMagic] 07amire80 opened pull request 03#466: Fixes in messages - 13https://github.com/miraheze/MirahezeMagic/pull/466 [23:25:08] [02MirahezeMagic] 07Universal-Omega closed pull request 03#466: Fixes in messages - 13https://github.com/miraheze/MirahezeMagic/pull/466 [23:25:09] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/MirahezeMagic/compare/acd9761ccb71...283621edf123 [23:25:12] [02miraheze/MirahezeMagic] 07amire80 03283621e - Fixes in messages (#466) [23:26:28] miraheze/MirahezeMagic - amire80 the build has errored. [23:29:07] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [23:29:49] miraheze/MirahezeMagic - Universal-Omega the build has errored. [23:32:52] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [23:32:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:33:01] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 8s [23:33:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:37:26] [02MirahezeMagic] 07amire80 opened pull request 03#467: Create qqq.json for managewiki - 13https://github.com/miraheze/MirahezeMagic/pull/467 [23:38:51] [02MirahezeMagic] 07Universal-Omega closed pull request 03#467: Create qqq.json for managewiki - 13https://github.com/miraheze/MirahezeMagic/pull/467 [23:38:52] [02miraheze/MirahezeMagic] 07Universal-Omega pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/MirahezeMagic/compare/283621edf123...139ba32b755f [23:38:55] [02miraheze/MirahezeMagic] 07amire80 03139ba32 - Create qqq.json for managewiki (#467) [23:41:29] miraheze/MirahezeMagic - amire80 the build has errored. [23:41:44] [02ImportDump] 07Universal-Omega closed pull request 03#48: Add gender hints to logentry messages - 13https://github.com/miraheze/ImportDump/pull/48 [23:41:46] [02miraheze/ImportDump] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/fd7fb50c3ea6...ba9a25d3bced [23:41:49] [02miraheze/ImportDump] 07amire80 03ba9a25d - Add gender hints to logentry messages (#48) [23:42:40] miraheze/MirahezeMagic - Universal-Omega the build has errored. [23:45:49] miraheze/ImportDump - Universal-Omega the build has errored. [23:50:22] PROBLEM - cp28 SSH on cp28 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [23:50:33] PROBLEM - cp28 Disk Space on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:50:41] PROBLEM - ping6 on cp28 is CRITICAL: PING CRITICAL - Packet loss = 100% [23:51:10] PROBLEM - Host cp28 is DOWN: PING CRITICAL - Packet loss = 100% [23:53:08] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [23:53:09] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [23:53:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:14] RECOVERY - Host cp28 is UP: PING OK - Packet loss = 0%, RTA = 159.63 ms [23:53:15] PROBLEM - cp28 Varnish Backends on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:53:18] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [23:53:19] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [23:53:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:22] PROBLEM - cp28 Nginx Backend for mw172 on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:53:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:27] PROBLEM - cp28 PowerDNS Recursor on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:53:28] PROBLEM - cp28 Nginx Backend for matomo151 on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:53:33] PROBLEM - cp28 Nginx Backend for mw161 on cp28 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [23:54:22] RECOVERY - cp28 SSH on cp28 is OK: SSH OK - OpenSSH_9.2p1 Debian-2+deb12u2 (protocol 2.0) [23:54:49] RECOVERY - ping6 on cp28 is OK: PING OK - Packet loss = 0%, RTA = 157.56 ms [23:55:14] RECOVERY - cp28 Varnish Backends on cp28 is OK: All 18 backends are healthy [23:55:20] RECOVERY - cp28 Nginx Backend for mw172 on cp28 is OK: TCP OK - 0.000 second response time on localhost port 8118 [23:55:26] RECOVERY - cp28 Nginx Backend for matomo151 on cp28 is OK: TCP OK - 0.000 second response time on localhost port 8203 [23:55:26] RECOVERY - cp28 PowerDNS Recursor on cp28 is OK: DNS OK: 0.087 seconds response time. miraheze.org returns 2602:294:0:b13::110,2602:294:0:b23::112,38.46.223.205,38.46.223.206 [23:55:27] PROBLEM - cp28 Disk Space on cp28 is WARNING: DISK WARNING - free space: / 30283MiB (6% inode=99%); [23:55:30] RECOVERY - cp28 Nginx Backend for mw161 on cp28 is OK: TCP OK - 0.000 second response time on localhost port 8115