[00:27:34] [02mediawiki-repos] 07brandon-wm opened pull request 03#17: T12111: Add LinkCards extension - 13https://github.com/miraheze/mediawiki-repos/pull/17 [00:28:16] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request 03#17: T12111: Add LinkCards extension - 13https://github.com/miraheze/mediawiki-repos/pull/17#issuecomment-2123637123 [00:29:18] [02mediawiki-repos] 07coderabbitai[bot] edited pull request 03#17: T12111: Add LinkCards extension - 13https://github.com/miraheze/mediawiki-repos/pull/17 [00:29:21] [02mediawiki-repos] 07coderabbitai[bot] edited a comment on pull request 03#17: T12111: Add LinkCards extension - 13https://github.com/miraheze/mediawiki-repos/pull/17#issuecomment-2123637123 [00:30:56] [02mw-config] 07brandon-wm opened pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569 [00:31:02] [02mw-config] 07coderabbitai[bot] commented on pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569#issuecomment-2123644505 [00:31:49] miraheze/mw-config - brandon-wm the build has errored. [00:31:59] [02mw-config] 07coderabbitai[bot] edited pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569 [00:32:22] [02mw-config] 07coderabbitai[bot] edited a comment on pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569#issuecomment-2123644505 [00:32:34] [02mw-config] 07brandon-wm synchronize pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569 [00:32:39] [02mw-config] 07coderabbitai[bot] edited a comment on pull request 03#5569: T12111: Add LinkCards to ManageWikiExtensions - 13https://github.com/miraheze/mw-config/pull/5569#issuecomment-2123644505 [00:33:36] miraheze/mw-config - brandon-wm the build passed. [01:31:59] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 142.41 ms [01:34:00] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.01 ms [03:11:17] [02ImportDump] 07Universal-Omega created branch 03services 13https://github.com/miraheze/ImportDump/commit/84fa660b2ec16aa84179b84a226980e0c9595b59 [03:11:19] [02ImportDump] 07Universal-Omega created branch 03services - 13https://github.com/miraheze/ImportDump [03:12:09] [02ImportDump] 07Universal-Omega pushed 031 commit to 03services [+2/-2/±7] 13https://github.com/miraheze/ImportDump/compare/84fa660b2ec1...94cd33829684 [03:12:11] [02ImportDump] 07Universal-Omega 0394cd338 - Move ImportDumpRequestManager to Services namespace [03:12:45] [02ImportDump] 07Universal-Omega opened pull request 03#108: Move ImportDumpRequestManager to Services namespace - 13https://github.com/miraheze/ImportDump/pull/108 [03:12:50] [02ImportDump] 07coderabbitai[bot] commented on pull request 03#108: Move ImportDumpRequestManager to Services namespace - 13https://github.com/miraheze/ImportDump/pull/108#issuecomment-2123789833 [03:13:29] [02ImportDump] 07Universal-Omega synchronize pull request 03#108: Move ImportDumpRequestManager to Services namespace - 13https://github.com/miraheze/ImportDump/pull/108 [03:13:31] [02ImportDump] 07Universal-Omega pushed 031 commit to 03services [+0/-0/±1] 13https://github.com/miraheze/ImportDump/compare/94cd33829684...b4bf546d8541 [03:13:33] [02ImportDump] 07Universal-Omega 03b4bf546 - Fix [04:33:43] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.24 ms [04:35:45] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 143.59 ms [04:53:03] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.84 ms [05:01:11] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 144.02 ms [05:20:35] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 142.48 ms [05:22:37] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 142.69 ms [06:01:21] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 146.57 ms [06:03:22] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 137.44 ms [06:17:35] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.35 ms [06:19:37] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.17 ms [06:32:56] PROBLEM - cloud16 Puppet on cloud16 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[ulogd2] [07:00:56] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:22:20] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [07:31:24] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [07:42:20] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [07:45:51] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 147.26 ms [07:49:56] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 142.01 ms [07:50:31] [02mediawiki-repos] 07BlankEclair opened pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18 [07:50:40] [02mediawiki-repos] 07coderabbitai[bot] commented on pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18#issuecomment-2124105304 [07:51:02] [02mediawiki-repos] 07RhinosF1 commented on pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18#issuecomment-2124105956 [07:51:53] [02mediawiki-repos] 07BlankEclair commented on pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18#issuecomment-2124107657 [07:52:36] [02mediawiki-repos] 07RhinosF1 commented on pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18#issuecomment-2124108951 [07:53:12] [02mediawiki-repos] 07coderabbitai[bot] edited pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18 [07:53:15] [02mediawiki-repos] 07coderabbitai[bot] edited a comment on pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18#issuecomment-2124105304 [08:10:00] [02mw-config] 07BlankEclair opened pull request 03#5570: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mw-config/pull/5570 [08:10:06] [02mw-config] 07coderabbitai[bot] commented on pull request 03#5570: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mw-config/pull/5570#issuecomment-2124142426 [08:10:48] [02mw-config] 07coderabbitai[bot] edited pull request 03#5570: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mw-config/pull/5570 [08:11:36] [02mw-config] 07coderabbitai[bot] edited a comment on pull request 03#5570: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mw-config/pull/5570#issuecomment-2124142426 [08:30:14] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [08:58:12] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 142.64 ms [09:00:14] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 144.17 ms [09:34:14] PROBLEM - db151 Backups SQL on db151 is WARNING: FILE_AGE WARNING: /var/log/sql-backup.log is 864219 seconds old and 124661 bytes [09:41:53] [02mediawiki-repos] 07Universal-Omega closed pull request 03#18: T12146: Add CustomSubtitle - 13https://github.com/miraheze/mediawiki-repos/pull/18 [09:41:54] [02mediawiki-repos] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mediawiki-repos/compare/a4068cea430e...378c2c62718a [09:41:55] [02mediawiki-repos] 07BlankEclair 03378c2c6 - T12146: Add CustomSubtitle (#18) [09:42:30] [02mediawiki-repos] 07Universal-Omega closed pull request 03#17: T12111: Add LinkCards extension - 13https://github.com/miraheze/mediawiki-repos/pull/17 [09:42:33] [02mediawiki-repos] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mediawiki-repos/compare/378c2c62718a...8c3a2c0eb545 [09:42:36] [02mediawiki-repos] 07brandon-wm 038c3a2c0 - Add LinkCards extension (#17) [09:44:15] miraheze/mw-config - BlankEclair the build passed. [09:53:14] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.96 ms [09:55:15] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 145.24 ms [10:05:47] PROBLEM - mwtask181 Puppet on mwtask181 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_clone_MediaWiki-REL1_41 LinkCards],Exec[git_clone_MediaWiki-REL1_42 LinkCards] [10:07:28] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 194.21 ms [10:09:47] PROBLEM - mwtask171 Puppet on mwtask171 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 2 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[git_clone_MediaWiki-REL1_41 LinkCards],Exec[git_clone_MediaWiki-REL1_42 LinkCards] [10:15:38] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 194.72 ms [10:22:49] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 194.73 ms [10:24:50] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 194.18 ms [10:27:51] PROBLEM - test151 Puppet on test151 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 3 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[git_clone_MediaWiki-REL1_41 LinkCards],Exec[git_clone_MediaWiki-REL1_42 LinkCards],Exec[git_clone_MediaWiki-master LinkCards] [10:43:26] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 201.94 ms [10:47:30] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 195.77 ms [11:09:57] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.mahdiruiz.line.pm All nameservers failed to answer the query. [12:05:46] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.19 ms [12:07:47] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 137.87 ms [12:10:03] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [12:59:00] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.gab.pt.eu.org All nameservers failed to answer the query. [12:59:37] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 138.69 ms [13:03:40] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.46 ms [13:28:48] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [13:37:53] PROBLEM - cp41 Puppet on cp41 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [13:44:30] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [14:00:25] [02CreateWiki] 07pixDeVl synchronize pull request 03#511: T12075: Notify Wiki Creators about requesters with over X(5) request - 13https://github.com/miraheze/CreateWiki/pull/511 [14:00:32] [02CreateWiki] 07coderabbitai[bot] edited a comment on pull request 03#511: T12075: Notify Wiki Creators about requesters with over X(5) request - 13https://github.com/miraheze/CreateWiki/pull/511#issuecomment-2118166172 [14:03:26] can someone approve the workflow on https://github.com/miraheze/CreateWiki/pull/511 rq to see if CI passes? [14:03:33] [02CreateWiki] 07coderabbitai[bot] edited a comment on pull request 03#511: T12075: Notify Wiki Creators about requesters with over X(5) request - 13https://github.com/miraheze/CreateWiki/pull/511#issuecomment-2118166172 [14:05:52] RECOVERY - cp41 Puppet on cp41 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures [14:14:32] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [14:24:56] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 144.08 ms [14:27:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [14:29:00] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.05 ms [14:35:08] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.20 ms [14:37:09] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.41 ms [14:47:20] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.53 ms [14:49:21] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 140.59 ms [15:05:59] [02mw-config] 07pixDeVl opened pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571 [15:06:06] [02mw-config] 07coderabbitai[bot] commented on pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571#issuecomment-2125037016 [15:06:55] miraheze/mw-config - pixDeVl the build passed. [15:07:26] @reception123 also can you look over and merge https://github.com/miraheze/mw-config/pull/5571 rq? [15:07:33] [02mw-config] 07coderabbitai[bot] edited pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571 [15:08:13] [02mw-config] 07coderabbitai[bot] reviewed pull request 03#5571 commit - 13https://github.com/miraheze/mw-config/pull/5571#discussion_r1610175979 [15:08:16] [02mw-config] 07coderabbitai[bot] edited a comment on pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571#issuecomment-2125037016 [15:29:11] is it me or does their custom icon look like it says "Niraheze" with an N? [15:29:26] I'm not sure we should permit a custom icon that seeks to replace our own with an unclear name [15:29:49] true, looks miscropped maybe [15:29:58] you could ask them in the task to fix it [15:30:06] i can def see it now yeah [15:30:06] yeah, I will [15:30:35] ergh I admit I still haven't fixed my phorge yet (the raw text issue) and I'm a bit in a hurry here so could you ask? [15:30:52] sure [15:30:53] just say that we don't usually allow people to use another footer icon for Miraheze itself but if they do they do need to make sure our name "Miraheze" is clear [15:32:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [15:33:40] https://issue-tracker.miraheze.org/T12147#243389 [15:38:00] PROBLEM - fanonpedia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'fanonpedia.com' expires in 15 day(s) (Fri 07 Jun 2024 03:24:36 PM GMT +0000). [15:38:12] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/f4138d09c0e0...4eea6e3a932d [15:38:13] [02ssl] 07WikiTideSSLBot 034eea6e3 - Bot: Update SSL cert for fanonpedia.com [15:42:13] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.39 ms [15:46:17] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 141.66 ms [16:06:30] RECOVERY - fanonpedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'fanonpedia.com' will expire on Tue 20 Aug 2024 02:38:06 PM GMT +0000. [16:23:24] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [16:27:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [16:30:07] PROBLEM - ping6 on cp41 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 141.63 ms [16:32:08] RECOVERY - ping6 on cp41 is OK: PING OK - Packet loss = 0%, RTA = 140.58 ms [16:43:20] Perfect! [16:43:50] https://tenor.com/view/hanamaru-kindergarten-anime-hiiragi-blue-eyes-blue-hair-gif-17172184 [16:43:56] Thank you thank you [16:52:07] !log [alex@test151] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:52:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:53:21] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - private.yahyabd.xyz All nameservers failed to answer the query. [16:54:01] Upgraded mediawiki-skin-Citizen when [16:54:04] RECOVERY - test151 APT on test151 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:54:12] when 1.42 drops [16:54:29] True [16:55:27] !log [alex@mw151] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:55:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:55:38] also OS can you click the approve workflow button for the CI on CW PR 511 pwease? [16:55:56] !log [alex@mw152] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:56:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:56:05] PR looks like it should still need some changes [16:56:25] !log [alex@mw161] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:56:27] RECOVERY - mw161 APT on mw161 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:56:28] RECOVERY - mw151 APT on mw151 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:56:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:57:04] RECOVERY - mw152 APT on mw152 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:57:05] !log [alex@mw162] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:57:09] https://tenor.com/view/cat-thumbs-up-cat-thumbs-up-meme-stock-image-gif-27117830 [16:57:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:57:37] !log [alex@mw171] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:57:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:57:45] RECOVERY - mw171 APT on mw171 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:58:04] !log [alex@mw172] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:58:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:58:20] RECOVERY - mw162 APT on mw162 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [16:58:37] !log [alex@mwtask171] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:58:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:58:56] RECOVERY - mwtask171 APT on mwtask171 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [16:59:07] !log [alex@mw181] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:59:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:59:43] !log [alex@mw182] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [16:59:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:59:56] RECOVERY - mw172 APT on mw172 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [17:00:13] !log [alex@mwtask181] Upgraded packages ghostscript, libgs10, libgs10-common, and libgs-common [17:00:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:00:38] RECOVERY - mw182 APT on mw182 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [17:00:53] RECOVERY - mw181 APT on mw181 is OK: APT OK: 27 packages available for upgrade (0 critical updates). [17:02:08] RECOVERY - mwtask181 APT on mwtask181 is OK: APT OK: 1 packages available for upgrade (0 critical updates). [17:02:29] PROBLEM - www.gengbaike.top - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [17:02:41] I was going to say "workflows only need to be approved manually for new contributors" [17:02:58] I forgor 💀 this was your first commit on CW [17:03:19] [02CreateWiki] 07The-Voidwalker reviewed pull request 03#511 commit - 13https://github.com/miraheze/CreateWiki/pull/511#discussion_r1610355154 [17:03:20] [02CreateWiki] 07The-Voidwalker reviewed pull request 03#511 commit - 13https://github.com/miraheze/CreateWiki/pull/511#discussion_r1610344282 [17:04:30] Added review [17:04:47] @pixldev CI is running for your PR [17:05:17] yea lol [17:05:27] thanks! [17:06:06] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [17:07:11] [02ManageWiki] 07OAuthority pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/76790a9945b3...abbab22f2da2 [17:07:13] [02ManageWiki] 07OAuthority 03abbab22 - replace use of UserGroupMembership::getGroupName() [17:08:31] [02ManageWiki] 07OAuthority pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/abbab22f2da2...21f106a48cbe [17:08:33] [02ManageWiki] 07OAuthority 0321f106a - revert [17:08:34] it now has more issues in CI then before LMAO [17:08:36] [Sippy](https://cdn.discordapp.com/emojis/911739461526749254.webp?size=48&quality=lossless&name=Sippy) [17:08:49] miraheze/CreateWiki - pixDeVl the build has errored. [17:09:20] [02ManageWiki] 07OAuthority pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ManageWiki/compare/21f106a48cbe...7ba6e09a1834 [17:09:22] [02ManageWiki] 07OAuthority 037ba6e09 - ignore me [17:14:13] miraheze/ManageWiki - OAuthority the build passed. [17:15:42] [02mw-config] 07pixDeVl synchronize pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571 [17:15:46] [02mw-config] 07coderabbitai[bot] edited a comment on pull request 03#5571: T12147: Custom Footer Icons for tnowiki - 13https://github.com/miraheze/mw-config/pull/5571#issuecomment-2125037016 [17:16:40] miraheze/mw-config - pixDeVl the build passed. [17:23:24] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.402 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [17:31:14] RECOVERY - www.gengbaike.top - reverse DNS on sslhost is OK: SSL OK - www.gengbaike.top reverse DNS resolves to cp36.wikitide.net - CNAME OK [17:36:05] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.yahyabd.xyz All nameservers failed to answer the query. [17:53:23] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - private.yahyabd.xyz All nameservers failed to answer the query. [18:07:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [18:10:40] PROBLEM - wiki.greatphermesia.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.greatphermesia.com' expires in 15 day(s) (Fri 07 Jun 2024 06:01:12 PM GMT +0000). [18:10:52] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/4eea6e3a932d...553fcf62715e [18:10:54] [02ssl] 07WikiTideSSLBot 03553fcf6 - Bot: Update SSL cert for wiki.greatphermesia.com [18:23:24] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [18:36:06] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.405 seconds: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out. [18:42:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [18:53:23] PROBLEM - private.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - private.yahyabd.xyz All nameservers failed to answer the query. [18:57:09] PROBLEM - wiki.cogsci.live - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.cogsci.live' expires in 15 day(s) (Fri 07 Jun 2024 06:54:38 PM GMT +0000). [18:57:21] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/553fcf62715e...f62ac07b62ee [18:57:22] [02ssl] 07WikiTideSSLBot 03f62ac07 - Bot: Update SSL cert for wiki.cogsci.live [19:02:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [19:06:05] PROBLEM - wiki.yahyabd.xyz - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.yahyabd.xyz All nameservers failed to answer the query. [19:10:00] RECOVERY - wiki.greatphermesia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.greatphermesia.com' will expire on Tue 20 Aug 2024 05:10:45 PM GMT +0000. [19:26:48] RECOVERY - wiki.cogsci.live - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cogsci.live' will expire on Tue 20 Aug 2024 05:57:15 PM GMT +0000. [19:37:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [19:51:38] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [22:15:51] PROBLEM - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is CRITICAL: Temporary failure in name resolutionHTTP CRITICAL - Unable to open TCP socket [22:52:04] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [23:13:49] RECOVERY - wiki.mahdiruiz.line.pm - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mahdiruiz.line.pm' will expire on Fri 14 Jun 2024 04:28:50 PM GMT +0000. [23:37:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [23:55:24] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL