[00:01:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:03:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is WARNING: WARNING - NGINX Error Rate is 50% [00:04:39] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21315MiB (24% inode=98%); [00:05:08] RECOVERY - cp41 Disk Space on cp41 is OK: DISK OK - free space: / 19032MiB (20% inode=98%); [00:05:09] PROBLEM - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is CRITICAL: CRITICAL - NGINX Error Rate is 67% [00:05:29] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21289MiB (24% inode=98%); [00:46:07] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/3fc0ce40e61d...ef518f2bdc7f [00:46:08] [02miraheze/mw-config] 07Universal-Omega 03ef518f2 - Remove support for tideways_xhprof [00:47:06] miraheze/mw-config - Universal-Omega the build passed. [00:47:28] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/ef518f2bdc7f...8bb4727100bc [00:47:29] [02miraheze/mw-config] 07Universal-Omega 038bb4727 - Always enable ParserMigration [00:48:21] miraheze/mw-config - Universal-Omega the build passed. [00:49:58] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8bb4727100bc...59709e9b0132 [00:50:01] [02miraheze/mw-config] 07Universal-Omega 0359709e9 - Remove redundant version check [00:50:51] miraheze/mw-config - Universal-Omega the build passed. [00:52:16] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03wgCommandLineMode [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/820ceb2a6590 [00:52:17] [02miraheze/mw-config] 07Universal-Omega 03820ceb2 - Replace deprecated wgCommandLineMode [00:52:19] [02mw-config] 07Universal-Omega created branch 03wgCommandLineMode - 13https://github.com/miraheze/mw-config [00:52:22] [02mw-config] 07Universal-Omega opened pull request 03#5526: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/mw-config/pull/5526 [00:52:59] !log [@test151] starting deploy of {'config': True} to test151 [00:53:00] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:53:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:53:09] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03wgCommandLineMode [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/820ceb2a6590...8a8485b877df [00:53:11] [02miraheze/mw-config] 07Universal-Omega 038a8485b - Replace [00:53:12] miraheze/mw-config - Universal-Omega the build passed. [00:53:12] [02mw-config] 07Universal-Omega synchronize pull request 03#5526: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/mw-config/pull/5526 [00:53:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:54:00] miraheze/mw-config - Universal-Omega the build passed. [00:54:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03wgCommandLineMode [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8a8485b877df...767f095f6a4d [00:54:36] [02miraheze/mw-config] 07Universal-Omega 03767f095 - Replace [00:54:37] [02mw-config] 07Universal-Omega synchronize pull request 03#5526: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/mw-config/pull/5526 [00:55:27] miraheze/mw-config - Universal-Omega the build passed. [00:59:21] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03wgCommandLineMode [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/commit/2d45fb68f139 [00:59:22] [02miraheze/ErrorPages] 07Universal-Omega 032d45fb6 - Replace deprecated wgCommandLineMode [00:59:24] [02ErrorPages] 07Universal-Omega created branch 03wgCommandLineMode - 13https://github.com/miraheze/ErrorPages [00:59:26] [02ErrorPages] 07Universal-Omega opened pull request 03#43: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/ErrorPages/pull/43 [01:00:06] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03wgCommandLineMode [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/2d45fb68f139...59ce5b5b0ae0 [01:00:07] [02miraheze/ErrorPages] 07Universal-Omega 0359ce5b5 - Replace [01:00:09] [02ErrorPages] 07Universal-Omega synchronize pull request 03#43: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/ErrorPages/pull/43 [01:00:32] miraheze/ErrorPages - Universal-Omega the build passed. [01:00:59] miraheze/ErrorPages - Universal-Omega the build passed. [01:02:50] !log [@mwtask181] starting deploy of {'config': True} to all [01:02:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:02:59] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [01:03:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:03:16] [02mw-config] 07Universal-Omega closed pull request 03#5526: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/mw-config/pull/5526 [01:03:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/mw-config/compare/59709e9b0132...8ab105a9a4bc [01:03:20] [02miraheze/mw-config] 07Universal-Omega 038ab105a - Replace deprecated wgCommandLineMode (#5526) [01:03:23] [02miraheze/mw-config] 07Universal-Omega deleted branch 03wgCommandLineMode [01:03:25] [02mw-config] 07Universal-Omega deleted branch 03wgCommandLineMode - 13https://github.com/miraheze/mw-config [01:03:28] [02ErrorPages] 07Universal-Omega closed pull request 03#43: Replace deprecated wgCommandLineMode - 13https://github.com/miraheze/ErrorPages/pull/43 [01:03:29] [02miraheze/ErrorPages] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ErrorPages/compare/78a450f31f30...755f46289fd4 [01:03:30] [02miraheze/ErrorPages] 07Universal-Omega 03755f462 - Replace deprecated wgCommandLineMode (#43) [01:03:33] [02ErrorPages] 07Universal-Omega deleted branch 03wgCommandLineMode - 13https://github.com/miraheze/ErrorPages [01:03:36] [02miraheze/ErrorPages] 07Universal-Omega deleted branch 03wgCommandLineMode [01:04:07] miraheze/mw-config - Universal-Omega the build passed. [01:04:17] miraheze/ErrorPages - Universal-Omega the build passed. [01:06:41] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0bd05ed16e71...45405ab248cf [01:06:43] [02miraheze/puppet] 07Universal-Omega 0345405ab - postgresql: update SSL support [01:07:08] !log [@mwtask171] starting deploy of {'config': True} to all [01:07:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:07:16] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 8s [01:07:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:07:55] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/45405ab248cf...ae0be3c81d3b [01:07:56] [02miraheze/puppet] 07Universal-Omega 03ae0be3c - mariadb: update SSL support [01:09:20] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ae0be3c81d3b...07c6478bd187 [01:09:21] [02miraheze/puppet] 07Universal-Omega 0307c6478 - mariadb: update SSL support [01:10:57] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/07c6478bd187...0f0e90ec3805 [01:11:00] [02miraheze/puppet] 07Universal-Omega 030f0e90e - mariadb: update cacert [01:12:08] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0f0e90ec3805...7eb656526601 [01:12:10] [02miraheze/puppet] 07Universal-Omega 037eb6565 - mariadb: update cacert [01:23:04] !log [@test151] starting deploy of {'config': True} to test151 [01:23:05] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [01:23:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:23:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:27:24] [02miraheze/puppet] 07Universal-Omega created branch 03renewssl 13https://github.com/miraheze/puppet/commit/7eb65652660131e259e6739c349ac01dd48074da [01:27:25] [02puppet] 07Universal-Omega created branch 03renewssl - 13https://github.com/miraheze/puppet [01:32:37] !log [@mwtask181] starting deploy of {'config': True} to all [01:32:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:32:46] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 9s [01:32:48] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03renewssl [+2/-2/±2] 13https://github.com/miraheze/puppet/compare/7eb656526601...02f94c4c2789 [01:32:49] [02miraheze/puppet] 07Universal-Omega 0302f94c4 - Rename mirahezerenewssl [01:33:08] [02puppet] 07Universal-Omega opened pull request 03#3822: Rename mirahezerenewssl - 13https://github.com/miraheze/puppet/pull/3822 [01:33:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:34:45] [02puppet] 07Universal-Omega synchronize pull request 03#3822: Rename mirahezerenewssl - 13https://github.com/miraheze/puppet/pull/3822 [01:34:47] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03renewssl [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/02f94c4c2789...477903138341 [01:34:48] [02miraheze/puppet] 07Universal-Omega 034779031 - Temp absent old service [01:35:06] [02puppet] 07Universal-Omega synchronize pull request 03#3822: Rename mirahezerenewssl - 13https://github.com/miraheze/puppet/pull/3822 [01:35:07] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03renewssl [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/477903138341...5857a77cee96 [01:35:10] [02miraheze/puppet] 07Universal-Omega 035857a77 - - [01:38:14] [02puppet] 07Universal-Omega closed pull request 03#3822: Rename mirahezerenewssl - 13https://github.com/miraheze/puppet/pull/3822 [01:38:16] [02miraheze/puppet] 07Universal-Omega deleted branch 03renewssl [01:38:17] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+2/-2/±2] 13https://github.com/miraheze/puppet/compare/7eb656526601...3279e88e54e6 [01:38:18] [02miraheze/puppet] 07Universal-Omega 033279e88 - Rename mirahezerenewssl (#3822) [01:38:19] [02puppet] 07Universal-Omega deleted branch 03renewssl - 13https://github.com/miraheze/puppet [01:38:29] miraheze/puppet - Universal-Omega the build passed. [01:39:50] miraheze/puppet - Universal-Omega the build passed. [01:40:13] miraheze/puppet - Universal-Omega the build passed. [01:40:27] miraheze/puppet - Universal-Omega the build passed. [01:42:52] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3279e88e54e6...cecd34819b41 [01:42:54] [02miraheze/puppet] 07Universal-Omega 03cecd348 - Remove absented service [01:55:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-1/±7] 13https://github.com/miraheze/puppet/compare/cecd34819b41...17ab16f31f2f [01:55:03] [02miraheze/puppet] 07Universal-Omega 0317ab16f - Rename miraheze-backup [02:05:08] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/17ab16f31f2f...6398f83ee334 [02:05:11] [02miraheze/puppet] 07Universal-Omega 036398f83 - squid: rename acl to match hostname and org [02:09:29] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/6398f83ee334...48c07695b289 [02:09:30] [02miraheze/puppet] 07Universal-Omega 0348c0769 - mediawiki: update cgroup description to match hostname [02:12:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/48c07695b289...86fd6fbc31ea [02:12:02] [02miraheze/puppet] 07Universal-Omega 0386fd6fb - irclogserverbot: update description to match hostname [02:12:54] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/86fd6fbc31ea...2e24422c33dd [02:12:56] [02miraheze/puppet] 07Universal-Omega 032e24422 - logbot: update description to match hostname [02:15:16] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2e24422c33dd...c8f0a2005375 [02:15:19] [02miraheze/puppet] 07Universal-Omega 03c8f0a20 - postgresql: update SSL support [02:16:29] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c8f0a2005375...86049fcd6d35 [02:16:32] [02miraheze/puppet] 07Universal-Omega 0386049fc - ircrcbot: update description to match hostname [02:34:23] PROBLEM - puppet181 Puppet on puppet181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_srv-ssl] [02:36:31] PROBLEM - ping on changeprop151 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 291.51 ms [02:38:31] RECOVERY - ping on changeprop151 is OK: PING OK - Packet loss = 0%, RTA = 0.22 ms [03:00:03] RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [03:01:00] RECOVERY - db171 Backups SQL on db171 is OK: FILE_AGE OK: /var/log/sql-backup.log is 59 seconds old and 0 bytes [03:03:04] PROBLEM - mon181 Backups Grafana on mon181 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864165 seconds old and 92 bytes [03:36:42] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [05:03:04] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [05:31:52] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [06:00:37] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [06:28:41] PROBLEM - ru-teirailway.f5.si - LetsEncrypt on sslhost is CRITICAL: No address associated with hostnameHTTP CRITICAL - Unable to open TCP socket [06:29:28] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 166.83 ms [06:31:29] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 174.62 ms [06:39:55] PROBLEM - ru-teirailway.f5.si - reverse DNS on sslhost is WARNING: rDNS WARNING - reverse DNS entry for ru-teirailway.f5.si could not be found [06:56:56] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 173.33 ms [06:58:57] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 171.15 ms [07:05:07] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 195.46 ms [07:07:09] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 183.62 ms [07:09:01] PROBLEM - wiki.shaazzz.ir - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.shaazzz.ir All nameservers failed to answer the query. [07:38:06] RECOVERY - wiki.shaazzz.ir - reverse DNS on sslhost is OK: SSL OK - wiki.shaazzz.ir reverse DNS resolves to cp36.wikitide.net - CNAME OK [07:57:59] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:26:47] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [08:54:57] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [09:23:55] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [10:25:10] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query wiki.mahdiruiz.line.pm. IN CNAME: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [11:03:52] PROBLEM - cloud16 Puppet on cloud16 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [11:22:46] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [11:31:53] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [11:48:04] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [12:15:49] [02miraheze/ErrorPages] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ErrorPages/compare/755f46289fd4...0c7c8aad22c7 [12:15:52] [02miraheze/ErrorPages] 07translatewiki 030c7c8aa - Localisation updates from https://translatewiki.net. [12:15:55] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±7] 13https://github.com/miraheze/CreateWiki/compare/18f9455b313a...f164d1dfdd6d [12:15:58] [02miraheze/CreateWiki] 07translatewiki 03f164d1d - Localisation updates from https://translatewiki.net. [12:15:59] [02miraheze/GlobalNewFiles] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/GlobalNewFiles/compare/6c9800458a2b...c2365a858cc6 [12:16:01] [02miraheze/GlobalNewFiles] 07translatewiki 03c2365a8 - Localisation updates from https://translatewiki.net. [12:16:03] [02miraheze/ImportDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/e761cc86ae92...094678094297 [12:16:05] [02miraheze/ImportDump] 07translatewiki 030946780 - Localisation updates from https://translatewiki.net. [12:16:06] [02miraheze/IncidentReporting] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/IncidentReporting/compare/92979c919b7f...5069d11193e8 [12:16:09] [02miraheze/IncidentReporting] 07translatewiki 035069d11 - Localisation updates from https://translatewiki.net. [12:16:11] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/d6c9729057dd...ee18abbf6425 [12:16:12] [02miraheze/DataDump] 07translatewiki 03ee18abb - Localisation updates from https://translatewiki.net. [12:16:13] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±4] 13https://github.com/miraheze/landing/compare/89d2647e6b19...89c129b8e830 [12:16:16] [02miraheze/landing] 07translatewiki 0389c129b - Localisation updates from https://translatewiki.net. [12:16:18] [02miraheze/ManageWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/ManageWiki/compare/e6b172c3b6cc...5e597f8c9032 [12:16:19] [02miraheze/ManageWiki] 07translatewiki 035e597f8 - Localisation updates from https://translatewiki.net. [12:16:21] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/b2ed41b77a04...bc8cda0e6ecc [12:16:22] [02miraheze/WikiDiscover] 07translatewiki 03bc8cda0 - Localisation updates from https://translatewiki.net. [12:16:24] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/MirahezeMagic/compare/198e71cea4cc...c387a9c37903 [12:16:26] [02miraheze/MirahezeMagic] 07translatewiki 03c387a9c - Localisation updates from https://translatewiki.net. [12:16:44] miraheze/ErrorPages - translatewiki the build passed. [12:17:09] miraheze/landing - translatewiki the build passed. [12:19:37] miraheze/MirahezeMagic - translatewiki the build has errored. [12:20:13] miraheze/DataDump - translatewiki the build passed. [12:20:25] miraheze/GlobalNewFiles - translatewiki the build passed. [12:20:29] miraheze/WikiDiscover - translatewiki the build passed. [12:20:39] miraheze/IncidentReporting - translatewiki the build passed. [12:20:41] miraheze/ManageWiki - translatewiki the build passed. [12:23:50] miraheze/ImportDump - translatewiki the build passed. [12:23:57] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [12:23:58] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:24:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:07] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [12:24:08] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:24:11] miraheze/CreateWiki - translatewiki the build passed. [12:24:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:32:51] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [12:32:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:00] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 9s [12:33:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:33:59] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [13:39:37] PROBLEM - test2.specialchat.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'test2.specialchat.org' expires in 15 day(s) (Sat 20 Apr 2024 01:32:28 PM GMT +0000). [13:39:49] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/8f998acbbb6c...5f11de19c265 [13:39:51] [02miraheze/ssl] 07WikiTideSSLBot 035f11de1 - Bot: Update SSL cert for test2.specialchat.org [13:42:52] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [14:31:59] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [14:37:37] RECOVERY - test2.specialchat.org - LetsEncrypt on sslhost is OK: OK - Certificate 'test2.specialchat.org' will expire on Wed 03 Jul 2024 12:39:43 PM GMT +0000. [15:40:32] PROBLEM - cp41 Disk Space on cp41 is WARNING: DISK WARNING - free space: / 10221MiB (10% inode=98%); [15:50:34] PROBLEM - wiki.orvyn.ca - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.orvyn.ca' expires in 15 day(s) (Sat 20 Apr 2024 03:47:57 PM GMT +0000). [15:50:46] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5f11de19c265...116b8f6359f7 [15:50:48] [02miraheze/ssl] 07WikiTideSSLBot 03116b8f6 - Bot: Update SSL cert for wiki.orvyn.ca [16:20:24] RECOVERY - wiki.orvyn.ca - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.orvyn.ca' will expire on Wed 03 Jul 2024 02:50:40 PM GMT +0000. [17:15:11] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03libssl1.1 [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/f4a33c902582 [17:15:12] [02miraheze/puppet] 07Universal-Omega 03f4a33c9 - Add apt source for libssl1.1 [17:15:15] [02puppet] 07Universal-Omega created branch 03libssl1.1 - 13https://github.com/miraheze/puppet [17:15:17] [02puppet] 07Universal-Omega opened pull request 03#3823: Add apt source for libssl1.1 - 13https://github.com/miraheze/puppet/pull/3823 [17:15:51] [02miraheze/puppet] 07github-actions[bot] pushed 031 commit to 03libssl1.1 [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f4a33c902582...6d0d9f4da626 [17:15:52] [02miraheze/puppet] 07github-actions 036d0d9f4 - CI: lint puppet code to standards [17:15:53] [02puppet] 07github-actions[bot] synchronize pull request 03#3823: Add apt source for libssl1.1 - 13https://github.com/miraheze/puppet/pull/3823 [17:27:44] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03changeprop-queues [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/d5d783a3769f [17:27:46] [02miraheze/puppet] 07Universal-Omega 03d5d783a - changeprop: add global new files and semantic mediawiki queues [17:27:49] [02puppet] 07Universal-Omega created branch 03changeprop-queues - 13https://github.com/miraheze/puppet [17:27:51] [02puppet] 07Universal-Omega opened pull request 03#3824: changeprop: add global new files and semantic mediawiki queues - 13https://github.com/miraheze/puppet/pull/3824 [17:32:22] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03changeprop-queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d5d783a3769f...d527f0cf8748 [17:32:24] [02miraheze/puppet] 07Universal-Omega 03d527f0c - Add [17:32:27] [02puppet] 07Universal-Omega synchronize pull request 03#3824: changeprop: add global new files and semantic mediawiki queues - 13https://github.com/miraheze/puppet/pull/3824 [17:37:51] PROBLEM - comprehensibleinputwiki.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'comprehensibleinputwiki.org' expires in 15 day(s) (Sat 20 Apr 2024 05:10:21 PM GMT +0000). [17:38:02] [02miraheze/ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/116b8f6359f7...50c1c020ee52 [17:38:03] [02miraheze/ssl] 07WikiTideSSLBot 0350c1c02 - Bot: Update SSL cert for comprehensibleinputwiki.org [17:44:07] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03changeprop-queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d527f0cf8748...fc15e4f0aefa [17:44:08] [02miraheze/puppet] 07Universal-Omega 03fc15e4f - Add [17:44:10] [02puppet] 07Universal-Omega synchronize pull request 03#3824: changeprop: add global new files and semantic mediawiki queues - 13https://github.com/miraheze/puppet/pull/3824 [17:49:58] [02puppet] 07Universal-Omega synchronize pull request 03#3824: changeprop: add global new files and semantic mediawiki queues - 13https://github.com/miraheze/puppet/pull/3824 [17:50:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03changeprop-queues [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fc15e4f0aefa...3326006fd3a8 [17:50:01] [02miraheze/puppet] 07Universal-Omega 033326006 - Update [17:51:52] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [17:52:31] [02puppet] 07Universal-Omega closed pull request 03#3824: changeprop: add global new files and semantic mediawiki queues - 13https://github.com/miraheze/puppet/pull/3824 [17:52:34] [02miraheze/puppet] 07Universal-Omega deleted branch 03changeprop-queues [17:52:36] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±3] 13https://github.com/miraheze/puppet/compare/86049fcd6d35...fd3aa5839a7c [17:52:37] [02miraheze/puppet] 07Universal-Omega 03fd3aa58 - changeprop: add global new files and semantic mediawiki queues (#3824) [17:52:40] [02puppet] 07Universal-Omega deleted branch 03changeprop-queues - 13https://github.com/miraheze/puppet [18:04:11] PROBLEM - cp51 Puppet on cp51 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [18:07:41] RECOVERY - comprehensibleinputwiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'comprehensibleinputwiki.org' will expire on Wed 03 Jul 2024 04:37:56 PM GMT +0000. [18:07:56] [02mw-config] 07colleirose opened pull request 03#5527: Change the license image to be an svg - 13https://github.com/miraheze/mw-config/pull/5527 [18:08:59] miraheze/mw-config - colleirose the build passed. [18:32:10] RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:49:48] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [18:51:53] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 171.56 ms [18:53:54] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 168.63 ms [19:03:42] [02CreateWiki] 07redbluegreenhat opened pull request 03#503: T12010: Introduce CreateWiki-specific suppression rights and suppression log - 13https://github.com/miraheze/CreateWiki/pull/503 [19:08:46] miraheze/CreateWiki - redbluegreenhat the build has errored. [21:11:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/commit/66269033f027 [21:11:58] [02miraheze/mw-config] 07Universal-Omega 036626903 - Add support for multiple primary domains [21:12:00] [02mw-config] 07Universal-Omega created branch 03domains - 13https://github.com/miraheze/mw-config [21:12:01] [02mw-config] 07Universal-Omega opened pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:12:57] miraheze/mw-config - Universal-Omega the build passed. [21:13:15] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:13:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/66269033f027...c11b498b682a [21:13:20] [02miraheze/mw-config] 07Universal-Omega 03c11b498 - Fix [21:14:12] miraheze/mw-config - Universal-Omega the build passed. [21:22:15] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:22:18] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/c11b498b682a...a4f743c0ca16 [21:22:19] [02miraheze/mw-config] 07Universal-Omega 03a4f743c - Order [21:22:55] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [21:23:09] miraheze/mw-config - Universal-Omega the build passed. [21:34:25] PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 4179 bytes in 0.226 second response time [21:38:25] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.475 second response time [21:39:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/a4f743c0ca16...6c0d482e243d [21:39:52] [02miraheze/mw-config] 07Universal-Omega 036c0d482 - Fix [21:39:55] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:40:50] miraheze/mw-config - Universal-Omega the build passed. [21:41:52] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:41:55] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/6c0d482e243d...8d75c8601a3c [21:41:58] [02miraheze/mw-config] 07Universal-Omega 038d75c86 - Add orain.org to beta temporarily for testing [21:42:46] miraheze/mw-config - Universal-Omega the build passed. [21:52:30] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:52:33] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8d75c8601a3c...8d8fa21d9ef8 [21:52:34] [02miraheze/mw-config] 07Universal-Omega 038d8fa21 - Fix [21:53:21] miraheze/mw-config - Universal-Omega the build passed. [21:55:48] [02mw-config] 07Universal-Omega synchronize pull request 03#5528: Add support for multiple primary domains - 13https://github.com/miraheze/mw-config/pull/5528 [21:55:51] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03domains [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/8d8fa21d9ef8...a23465311a8e [21:55:53] [02miraheze/mw-config] 07Universal-Omega 03a234653 - Remove orain.org [21:56:47] miraheze/mw-config - Universal-Omega the build passed. [22:01:11] PROBLEM - graphite151 NTP time on graphite151 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4|-6] [-w ] [-c ] [-v verbose] [-o