[00:00:27] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is UNKNOWN: UNKNOWN - NGINX Error Rate is UNKNOWN [00:00:34] miraheze/puppet - Universal-Omega the build passed. [00:00:51] [02puppet] 07Universal-Omega synchronize pull request 03#3784: Fix use string and fix puppet lint - 13https://github.com/miraheze/puppet/pull/3784 [00:00:53] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03fix [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/58042237c245...0b1838e1a791 [00:00:56] [02miraheze/puppet] 07Universal-Omega 030b1838e - Try again [00:02:15] miraheze/puppet - Universal-Omega the build passed. [00:02:23] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 100% [00:04:07] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 21582MiB (24% inode=98%); [00:04:43] RECOVERY - cp36 Disk Space on cp36 is OK: DISK OK - free space: / 21556MiB (24% inode=98%); [00:10:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 56% [00:16:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 62% [00:20:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is WARNING: WARNING - NGINX Error Rate is 59% [00:27:55] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/87f974e6fd72...9a0c30f3c4b5 [00:27:56] [02miraheze/puppet] 07Universal-Omega 039a0c30f - Fix [00:28:20] [02puppet] 07Universal-Omega closed pull request 03#3784: Fix use string and fix puppet lint - 13https://github.com/miraheze/puppet/pull/3784 [00:28:23] [02miraheze/puppet] 07Universal-Omega deleted branch 03fix [00:28:24] [02puppet] 07Universal-Omega deleted branch 03fix - 13https://github.com/miraheze/puppet [00:29:41] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/9a0c30f3c4b5...8069e9a0b4c7 [00:29:43] [02miraheze/puppet] 07Universal-Omega 038069e9a - Fix [00:32:02] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/8069e9a0b4c7...b3489a3ae55d [00:32:05] [02miraheze/puppet] 07Universal-Omega 03b3489a3 - Fix [00:32:20] PROBLEM - cp26 HTTP 4xx/5xx ERROR Rate on cp26 is CRITICAL: CRITICAL - NGINX Error Rate is 60% [00:34:32] RECOVERY - kafka181 Puppet on kafka181 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [00:36:11] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/b3489a3ae55d...7535c7694293 [00:36:12] [02miraheze/puppet] 07Universal-Omega 037535c76 - Add Kafka broker [00:39:40] PROBLEM - kafka181 ferm_active on kafka181 is CRITICAL: ERROR ferm input drop default policy not set, ferm might not have been started correctly [00:39:48] PROBLEM - phorge171 Current Load on phorge171 is WARNING: LOAD WARNING - total load average: 1.14, 1.71, 1.44 [00:43:48] RECOVERY - phorge171 Current Load on phorge171 is OK: LOAD OK - total load average: 1.31, 1.62, 1.47 [00:50:22] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7535c7694293...65f91348f84a [00:50:23] [02miraheze/puppet] 07Universal-Omega 0365f9134 - zookeeper: configure logging [00:51:46] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/65f91348f84a...9b71039fc5f8 [00:51:48] [02miraheze/puppet] 07Universal-Omega 039b71039 - Fix legacy fact [00:56:56] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/9b71039fc5f8...2d54b8db401e [00:56:58] [02miraheze/puppet] 07Universal-Omega 032d54b8d - zookeeper: configure servers [00:58:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/2d54b8db401e...0a754db6e1fd [00:58:01] [02miraheze/puppet] 07Universal-Omega 030a754db - Fix [01:13:14] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0a754db6e1fd...58a588d2f386 [01:13:16] [02miraheze/puppet] 07Universal-Omega 0358a588d - zookeeper: install java [01:14:56] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/58a588d2f386...84c3893d98e1 [01:14:59] [02miraheze/puppet] 07Universal-Omega 0384c3893 - Use 17 [01:37:14] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03manage_service_file [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/6e8676b4a85e [01:37:16] [02miraheze/puppet] 07Universal-Omega 036e8676b - zookeeper: set manage_service_file to true [01:37:18] [02puppet] 07Universal-Omega created branch 03manage_service_file - 13https://github.com/miraheze/puppet [01:37:20] [02puppet] 07Universal-Omega opened pull request 03#3785: zookeeper: set manage_service_file to true - 13https://github.com/miraheze/puppet/pull/3785 [01:39:07] [02puppet] 07Universal-Omega closed pull request 03#3785: zookeeper: set manage_service_file to true - 13https://github.com/miraheze/puppet/pull/3785 [01:39:09] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/84c3893d98e1...c89269aa4ddf [01:39:11] [02miraheze/puppet] 07Universal-Omega 03c89269a - zookeeper: set manage_service_file to true (#3785) [01:39:13] [02miraheze/puppet] 07Universal-Omega deleted branch 03manage_service_file [01:39:14] [02puppet] 07Universal-Omega deleted branch 03manage_service_file - 13https://github.com/miraheze/puppet [01:50:18] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c89269aa4ddf...ef43e60a1d8c [01:50:20] [02miraheze/puppet] 07Universal-Omega 03ef43e60 - Set zoo_dir [01:53:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1006141603 secs [02:05:40] RECOVERY - kafka181 ferm_active on kafka181 is OK: OK ferm input default policy is set [02:11:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09896039963 secs [02:23:22] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ef43e60a1d8c...a9a7a717cf1c [02:23:24] [02miraheze/puppet] 07Universal-Omega 03a9a7a71 - zookeeper.service: fix ExecStart [02:31:41] PROBLEM - puppet181 Puppet on puppet181 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_pull_puppet] [02:39:46] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a9a7a717cf1c...f2587b887ff0 [02:39:49] [02miraheze/puppet] 07Universal-Omega 03f2587b8 - zookeeper: try fixing service [02:41:17] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f2587b887ff0...081724d1ddd4 [02:41:20] [02miraheze/puppet] 07Universal-Omega 03081724d - Fix newlines [02:47:03] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/081724d1ddd4...ef4524be0a2b [02:47:05] [02miraheze/puppet] 07Universal-Omega 03ef4524b - Messy temp fix [02:48:45] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/ef4524be0a2b...fdd0e4d358b1 [02:48:46] [02miraheze/puppet] 07Universal-Omega 03fdd0e4d - revert [03:03:09] PROBLEM - mon181 Backups Grafana on mon181 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864170 seconds old and 93 bytes [03:10:59] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/fdd0e4d358b1...e8cb04633459 [03:11:01] [02miraheze/puppet] 07Universal-Omega 03e8cb046 - Fix [03:26:24] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e8cb04633459...e588549b2365 [03:26:27] [02miraheze/puppet] 07Universal-Omega 03e588549 - Minor cleanup [03:29:41] RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [03:39:33] Yay Kafka is FINALLY running! [07:19:57] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1143493056 secs [07:47:54] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 28%, RTA = 172.76 ms [07:49:57] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 195.91 ms [08:14:54] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 168.23 ms [08:16:57] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.26 ms [08:39:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09829673171 secs [08:54:23] !log sudo -u www-data php /srv/mediawiki/1.41/extensions/Translate/scripts/moveTranslatableBundle.php --wiki rainworldwiki "Looks_to_the_Moon_(character)/Dialogue" "Looks_to_the_Moon/Dialogue" "Reception123" --reason "Requested at T11976" [08:54:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [08:55:20] !log sudo -u www-data php /srv/mediawiki/1.41/extensions/Translate/scripts/moveTranslatableBundle.php --wiki rainworldwiki "Five_Pebbles_(character)/Dialogue" "Five_Pebbles/Dialogue" "Reception123" --reason "Requested at T11976" [08:55:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:31:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1005470157 secs [10:33:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09920886159 secs [10:39:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1023656428 secs [11:48:12] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 172.29 ms [11:52:20] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 169.17 ms [12:02:58] PROBLEM - theagonian.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'theagonian.com' expires in 15 day(s) (Sat 06 Apr 2024 11:44:22 AM GMT +0000). [12:03:09] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/7608845ab680...5f16d47a72f1 [12:03:11] [02miraheze/ssl] 07MirahezeSSLBot 035f16d47 - Bot: Update SSL cert for theagonian.com [12:04:02] miraheze/ssl - MirahezeSSLBot the build has errored. [12:07:58] RECOVERY - cp27 HTTP 4xx/5xx ERROR Rate on cp27 is OK: OK - NGINX Error Rate is 28% [12:08:45] PROBLEM - sonicpedia.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'sonicpedia.org' expires in 15 day(s) (Sat 06 Apr 2024 11:41:38 AM GMT +0000). [12:08:54] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/5f16d47a72f1...57d6bcff5598 [12:08:56] [02miraheze/ssl] 07MirahezeSSLBot 0357d6bcf - Bot: Update SSL cert for sonicpedia.org [12:09:53] miraheze/ssl - MirahezeSSLBot the build has errored. [12:14:42] [02miraheze/GlobalNewFiles] 07translatewiki pushed 031 commit to 03master [+1/-0/±0] 13https://github.com/miraheze/GlobalNewFiles/compare/34db58fbd088...6c9800458a2b [12:14:43] [02miraheze/GlobalNewFiles] 07translatewiki 036c98004 - Localisation updates from https://translatewiki.net. [12:14:44] [02miraheze/CreateWiki] 07translatewiki pushed 031 commit to 03master [+0/-0/±2] 13https://github.com/miraheze/CreateWiki/compare/526baf01b0e1...3aea2b9ddada [12:14:46] [02miraheze/CreateWiki] 07translatewiki 033aea2b9 - Localisation updates from https://translatewiki.net. [12:14:49] [02miraheze/RequestSSL] 07translatewiki pushed 031 commit to 03master [+2/-0/±1] 13https://github.com/miraheze/RequestSSL/compare/0e4759a0dd83...5de0765e6a12 [12:14:50] [02miraheze/RequestSSL] 07translatewiki 035de0765 - Localisation updates from https://translatewiki.net. [12:14:52] [02miraheze/landing] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/landing/compare/06e0daa7bd6e...5d1f7a851e8a [12:14:55] [02miraheze/landing] 07translatewiki 035d1f7a8 - Localisation updates from https://translatewiki.net. [12:14:58] [02miraheze/WikiDiscover] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/WikiDiscover/compare/f11df1aceb85...b2ed41b77a04 [12:15:01] [02miraheze/WikiDiscover] 07translatewiki 03b2ed41b - Localisation updates from https://translatewiki.net. [12:15:02] [02miraheze/MirahezeMagic] 07translatewiki pushed 031 commit to 03master [+1/-0/±4] 13https://github.com/miraheze/MirahezeMagic/compare/3c38baebed6b...8c82427d1f2b [12:15:05] [02miraheze/MirahezeMagic] 07translatewiki 038c82427 - Localisation updates from https://translatewiki.net. [12:15:06] [02miraheze/DataDump] 07translatewiki pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/DataDump/compare/7ff8b9eeae56...7275fcb9ae55 [12:15:07] [02miraheze/DataDump] 07translatewiki 037275fcb - Localisation updates from https://translatewiki.net. [12:15:47] miraheze/landing - translatewiki the build passed. [12:18:26] miraheze/MirahezeMagic - translatewiki the build has errored. [12:19:20] miraheze/GlobalNewFiles - translatewiki the build passed. [12:19:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09760925174 secs [12:19:52] miraheze/DataDump - translatewiki the build passed. [12:19:57] miraheze/WikiDiscover - translatewiki the build passed. [12:20:57] PROBLEM - wiki.susqu.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.susqu.org' expires in 15 day(s) (Sat 06 Apr 2024 12:14:47 PM GMT +0000). [12:21:08] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/57d6bcff5598...a2164115740f [12:21:10] [02miraheze/ssl] 07MirahezeSSLBot 03a216411 - Bot: Update SSL cert for wiki.susqu.org [12:22:04] miraheze/ssl - MirahezeSSLBot the build has errored. [12:22:41] miraheze/CreateWiki - translatewiki the build passed. [12:22:57] miraheze/RequestSSL - translatewiki the build passed. [12:23:36] !log [@test151] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 [12:23:37] !log [@test151] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:23:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1002447307 secs [12:23:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:23:45] !log [@test151] starting deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 [12:23:46] !log [@test151] finished deploy of {'folders': '1.42/extensions/MirahezeMagic'} to test151 - SUCCESS in 0s [12:23:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:23:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:24:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:25:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09475928545 secs [12:31:27] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml (START) [12:31:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:32:10] RECOVERY - theagonian.com - LetsEncrypt on sslhost is OK: OK - Certificate 'theagonian.com' will expire on Wed 19 Jun 2024 11:03:04 AM GMT +0000. [12:32:24] !log [@mwtask181] starting deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all [12:32:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:32:33] !log [@mwtask181] finished deploy of {'folders': '1.41/extensions/MirahezeMagic'} to all - SUCCESS in 9s [12:32:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:33] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml (END - exit=2) [12:33:37] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=themanaworldwiki --username-prefix=themanaworld /home/reception/themanaworld.xml --report 1 (START) [12:33:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:33:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:08] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/reception/themanaworld --search-recursively --summary=Imported from https://wiki.themanaworld.org/index.php/ (START) [12:35:12] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/reception/themanaworld --search-recursively --summary=Imported from https://wiki.themanaworld.org/index.php/ (END - exit=2) [12:35:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:30] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/reception/themanaworld --search-recursively --summary=Imported from https://wiki.themanaworld.org/index.php/ (START) [12:35:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:35:51] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=themanaworldwiki /home/reception/themanaworld --search-recursively --summary=Imported from https://wiki.themanaworld.org/index.php/ (END - exit=2) [12:35:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:38:00] RECOVERY - sonicpedia.org - LetsEncrypt on sslhost is OK: OK - Certificate 'sonicpedia.org' will expire on Wed 19 Jun 2024 11:08:48 AM GMT +0000. [12:46:02] PROBLEM - wiki.mc.marochuru.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.mc.marochuru.net' expires in 15 day(s) (Sat 06 Apr 2024 12:42:00 PM GMT +0000). [12:46:14] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a2164115740f...dd0342f86fd3 [12:46:16] [02miraheze/ssl] 07MirahezeSSLBot 03dd0342f - Bot: Update SSL cert for wiki.mc.marochuru.net [12:47:06] miraheze/ssl - MirahezeSSLBot the build has errored. [12:48:10] PROBLEM - wiki.ff6worldscollide.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.ff6worldscollide.com' expires in 15 day(s) (Sat 06 Apr 2024 12:28:32 PM GMT +0000). [12:48:21] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/dd0342f86fd3...1e4a4eb18c38 [12:48:23] [02miraheze/ssl] 07MirahezeSSLBot 031e4a4eb - Bot: Update SSL cert for wiki.ff6worldscollide.com [12:49:13] miraheze/ssl - MirahezeSSLBot the build has errored. [12:50:23] RECOVERY - wiki.susqu.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.susqu.org' will expire on Wed 19 Jun 2024 11:21:03 AM GMT +0000. [12:54:41] PROBLEM - ndgkb.nena.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'ndgkb.nena.org' expires in 15 day(s) (Sat 06 Apr 2024 12:33:18 PM GMT +0000). [12:54:52] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/1e4a4eb18c38...3e0be1284d88 [12:54:53] [02miraheze/ssl] 07MirahezeSSLBot 033e0be12 - Bot: Update SSL cert for ndgkb.nena.org [12:55:54] miraheze/ssl - MirahezeSSLBot the build has errored. [13:03:35] PROBLEM - kb.nena.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'kb.nena.org' expires in 15 day(s) (Sat 06 Apr 2024 12:41:45 PM GMT +0000). [13:03:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1031424105 secs [13:03:47] [02miraheze/ssl] 07MirahezeSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/3e0be1284d88...811fc269a7dd [13:03:50] [02miraheze/ssl] 07MirahezeSSLBot 03811fc26 - Bot: Update SSL cert for kb.nena.org [13:04:41] miraheze/ssl - MirahezeSSLBot the build has errored. [13:05:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.08674505353 secs [13:06:08] PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 28.15, 18.72, 12.49 [13:06:10] PROBLEM - mw171 Current Load on mw171 is CRITICAL: LOAD CRITICAL - total load average: 27.63, 18.21, 12.07 [13:08:09] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.76, 19.18, 13.45 [13:08:10] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 21.23, 19.74, 13.44 [13:09:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1066886187 secs [13:10:10] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 14.14, 17.75, 13.49 [13:11:38] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09659391642 secs [13:15:03] RECOVERY - wiki.mc.marochuru.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mc.marochuru.net' will expire on Wed 19 Jun 2024 11:46:08 AM GMT +0000. [13:15:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1076278687 secs [13:17:47] PROBLEM - mw152 Current Load on mw152 is CRITICAL: LOAD CRITICAL - total load average: 26.96, 17.66, 11.98 [13:17:50] RECOVERY - wiki.ff6worldscollide.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ff6worldscollide.com' will expire on Wed 19 Jun 2024 11:48:16 AM GMT +0000. [13:18:08] PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 30.15, 23.27, 16.96 [13:18:10] PROBLEM - mw171 Current Load on mw171 is CRITICAL: LOAD CRITICAL - total load average: 30.34, 23.46, 17.02 [13:19:24] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 29.44, 21.78, 14.64 [13:19:53] PROBLEM - cp41 Current Load on cp41 is CRITICAL: LOAD CRITICAL - total load average: 10.08, 8.30, 6.62 [13:19:53] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 25.27, 19.42, 13.45 [13:20:10] PROBLEM - mw161 Current Load on mw161 is CRITICAL: LOAD CRITICAL - total load average: 26.28, 18.54, 12.71 [13:21:53] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 21.60, 19.99, 14.39 [13:23:21] RECOVERY - ndgkb.nena.org - LetsEncrypt on sslhost is OK: OK - Certificate 'ndgkb.nena.org' will expire on Wed 19 Jun 2024 11:54:46 AM GMT +0000. [13:23:53] PROBLEM - cp41 Current Load on cp41 is WARNING: LOAD WARNING - total load average: 7.31, 7.83, 6.83 [13:23:53] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 28.18, 23.06, 16.19 [13:24:49] PROBLEM - mw162 Current Load on mw162 is CRITICAL: LOAD CRITICAL - total load average: 24.60, 20.84, 14.16 [13:25:36] PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 24.35, 21.44, 15.25 [13:25:52] PROBLEM - cp41 Current Load on cp41 is CRITICAL: LOAD CRITICAL - total load average: 11.17, 9.33, 7.51 [13:26:49] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 17.53, 20.09, 14.75 [13:27:36] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 21.93, 21.20, 15.91 [13:27:53] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 23.37, 22.88, 17.72 [13:29:36] PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 28.93, 24.11, 17.62 [13:29:53] PROBLEM - mw172 Current Load on mw172 is CRITICAL: LOAD CRITICAL - total load average: 28.62, 24.91, 19.08 [13:30:49] PROBLEM - mw162 Current Load on mw162 is CRITICAL: LOAD CRITICAL - total load average: 28.45, 24.32, 17.65 [13:31:36] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 22.59, 23.84, 18.35 [13:31:43] PROBLEM - cp26 Varnish Backends on cp26 is CRITICAL: 2 backends are down. mw152 mw181 [13:33:15] RECOVERY - kb.nena.org - LetsEncrypt on sslhost is OK: OK - Certificate 'kb.nena.org' will expire on Wed 19 Jun 2024 12:03:41 PM GMT +0000. [13:33:36] PROBLEM - mw151 Current Load on mw151 is CRITICAL: LOAD CRITICAL - total load average: 29.31, 25.94, 19.79 [13:33:40] PROBLEM - cp27 Varnish Backends on cp27 is CRITICAL: 3 backends are down. mw161 mw171 mw172 [13:37:23] PROBLEM - cp37 Varnish Backends on cp37 is CRITICAL: 2 backends are down. mw162 mw182 [13:37:29] @reception123 just how big is that XML? [13:37:40] RECOVERY - cp27 Varnish Backends on cp27 is OK: All 19 backends are healthy [13:38:56] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [13:39:19] RECOVERY - cp37 Varnish Backends on cp37 is OK: All 19 backends are healthy [13:39:38] RECOVERY - cp26 Varnish Backends on cp26 is OK: All 19 backends are healthy [13:40:05] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 5627 bytes in 0.012 second response time [13:42:10] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 8.434 second response time [13:45:30] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [13:45:42] PROBLEM - cp36 Varnish Backends on cp36 is CRITICAL: 2 backends are down. mw151 mw162 [13:48:35] PROBLEM - cp27 Varnish Backends on cp27 is CRITICAL: 1 backends are down. mw182 [13:49:38] RECOVERY - cp36 Varnish Backends on cp36 is OK: All 19 backends are healthy [13:50:33] RECOVERY - cp27 Varnish Backends on cp27 is OK: All 19 backends are healthy [13:50:49] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 16.61, 23.57, 23.90 [13:51:36] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 18.42, 23.21, 23.83 [13:51:53] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 14.05, 21.68, 23.54 [13:52:10] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 14.73, 21.31, 22.92 [13:53:52] PROBLEM - cp41 Current Load on cp41 is WARNING: LOAD WARNING - total load average: 6.35, 7.41, 7.81 [13:54:08] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 0.24, 0.05, 0.02 [13:56:49] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 11.71, 16.00, 20.39 [13:57:24] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 12.77, 18.66, 23.19 [13:57:36] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 9.12, 13.64, 19.30 [13:57:47] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 17.73, 19.54, 23.42 [13:57:53] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 12.29, 15.14, 19.97 [13:58:10] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 4.85, 1.14, 0.38 [13:58:10] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 11.08, 14.54, 19.33 [14:00:30] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [14:05:24] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 11.15, 14.75, 19.55 [14:05:47] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 12.52, 14.83, 19.42 [14:10:24] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 44%, RTA = 172.30 ms [14:11:52] RECOVERY - cp41 Current Load on cp41 is OK: LOAD OK - total load average: 4.81, 5.87, 6.64 [14:12:27] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.33 ms [14:25:44] just ~300MB [14:26:08] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-14/±0] 13https://github.com/miraheze/puppet/commit/924161b56482 [14:26:10] [02miraheze/puppet] 07Universal-Omega 03924161b - Cleanup kafka and zookeeper [14:26:12] [02puppet] 07Universal-Omega created branch 03cleanup - 13https://github.com/miraheze/puppet [14:26:15] [02puppet] 07Universal-Omega opened pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:26:44] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-6/±0] 13https://github.com/miraheze/puppet/compare/924161b56482...44ff0cf7bd1b [14:26:45] [02miraheze/puppet] 07Universal-Omega 0344ff0cf - - [14:26:48] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:27:35] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/44ff0cf7bd1b...8e913f6d1742 [14:27:37] [02miraheze/puppet] 07Universal-Omega 038e913f6 - - [14:27:38] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:27:54] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/8e913f6d1742...83abac1f9b60 [14:27:56] [02miraheze/puppet] 07Universal-Omega 0383abac1 - - [14:27:57] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:28:11] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/83abac1f9b60...6a8938366cc4 [14:28:14] [02miraheze/puppet] 07Universal-Omega 036a89383 - - [14:28:16] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:28:26] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/6a8938366cc4...c6f6c0753f10 [14:28:28] [02miraheze/puppet] 07Universal-Omega 03c6f6c07 - - [14:28:30] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:28:39] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-22/±0] 13https://github.com/miraheze/puppet/compare/c6f6c0753f10...52c75717316d [14:28:42] [02miraheze/puppet] 07Universal-Omega 0352c7571 - - [14:28:44] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:33:04] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/52c75717316d...554d994e4c55 [14:33:05] [02miraheze/puppet] 07Universal-Omega 03554d994 - - [14:33:08] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:33:39] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+1/-1/±0] 13https://github.com/miraheze/puppet/compare/554d994e4c55...761a2b90f7b2 [14:33:42] [02miraheze/puppet] 07Universal-Omega 03761a2b9 - rename [14:33:43] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:33:55] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/761a2b90f7b2...c4f2357a2b86 [14:33:58] [02miraheze/puppet] 07Universal-Omega 03c4f2357 - - [14:34:00] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:34:09] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/c4f2357a2b86...d0ad73acdaaf [14:34:10] [02miraheze/puppet] 07Universal-Omega 03d0ad73a - - [14:34:12] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:37:26] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp37.wikitide.net - CNAME OK [14:44:48] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/d0ad73acdaaf...e69aa95c37b8 [14:44:50] [02miraheze/puppet] 07Universal-Omega 03e69aa95 - Update [14:44:52] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:47:35] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/e69aa95c37b8...0592225a1c96 [14:47:38] [02miraheze/puppet] 07Universal-Omega 030592225 - Fix [14:47:41] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:48:55] miraheze/puppet - Universal-Omega the build passed. [14:50:59] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:51:00] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0592225a1c96...0d685e43d5ed [14:51:03] [02miraheze/puppet] 07Universal-Omega 030d685e4 - Fix [14:52:15] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:52:16] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0d685e43d5ed...0379604be003 [14:52:18] [02miraheze/puppet] 07Universal-Omega 030379604 - Fix [14:52:31] miraheze/puppet - Universal-Omega the build passed. [14:53:58] miraheze/puppet - Universal-Omega the build passed. [14:54:28] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:54:30] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/0379604be003...c55c7ad17a41 [14:54:32] [02miraheze/puppet] 07Universal-Omega 03c55c7ad - Fix [14:55:51] miraheze/puppet - Universal-Omega the build passed. [14:56:51] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:56:52] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/c55c7ad17a41...a71995e7b1ad [14:56:54] [02miraheze/puppet] 07Universal-Omega 03a71995e - Fix [14:57:10] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/refreshLinks.php --wiki=sublimelibrarywiki (START) [14:57:14] !log [reception@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/refreshLinks.php --wiki=sublimelibrarywiki (END - exit=0) [14:57:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:57:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:58:17] miraheze/puppet - Universal-Omega the build passed. [14:58:46] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [14:58:49] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a71995e7b1ad...e5a2076ba936 [14:58:52] [02miraheze/puppet] 07Universal-Omega 03e5a2076 - Fix [14:59:15] [02miraheze/puppet] 07github-actions[bot] pushed 031 commit to 03cleanup [+0/-0/±38] 13https://github.com/miraheze/puppet/compare/e5a2076ba936...3a557f31c64a [14:59:18] [02miraheze/puppet] 07github-actions 033a557f3 - CI: lint puppet code to standards [14:59:20] [02puppet] 07github-actions[bot] synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:00:01] miraheze/puppet - Universal-Omega the build passed. [15:08:58] PROBLEM - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query line.pm. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered The DNS operation timed out.; Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [15:09:21] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/3a557f31c64a...a653a7cbfb3d [15:09:23] [02miraheze/puppet] 07Universal-Omega 03a653a7c - Revert [15:09:24] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:10:53] miraheze/puppet - Universal-Omega the build passed. [15:27:16] Btw OS did the manual reattach work for #Incorrect username or password entered. Please try again. - when trying to log into a new wiki? [15:27:27] not yet [15:27:45] It’s a yet to try, not didn’t work? [15:27:50] I plan on doing it today, hopefully it will go well first try [15:28:14] it's a yet to try [15:28:34] Bet [15:31:07] PROBLEM - ping6 on cp51 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 177.01 ms [15:33:11] RECOVERY - ping6 on cp51 is OK: PING OK - Packet loss = 0%, RTA = 170.74 ms [15:37:13] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a653a7cbfb3d...1a8f6681d029 [15:37:16] [02miraheze/puppet] 07Universal-Omega 031a8f668 - - [15:37:18] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:38:16] RECOVERY - wiki.mahdiruiz.line.pm - reverse DNS on sslhost is OK: SSL OK - wiki.mahdiruiz.line.pm reverse DNS resolves to cp36.wikitide.net - CNAME OK [15:38:34] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/1a8f6681d029...38b813cb7d05 [15:38:35] miraheze/puppet - Universal-Omega the build passed. [15:38:35] [02miraheze/puppet] 07Universal-Omega 0338b813c - - [15:38:37] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:39:39] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-1/±0] 13https://github.com/miraheze/puppet/compare/38b813cb7d05...a273e2370102 [15:39:40] [02miraheze/puppet] 07Universal-Omega 03a273e23 - - [15:39:41] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:39:55] miraheze/puppet - Universal-Omega the build passed. [15:41:13] miraheze/puppet - Universal-Omega the build passed. [15:50:05] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/a273e2370102...45dd105e95e8 [15:50:07] [02miraheze/puppet] 07Universal-Omega 0345dd105 - - [15:50:10] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:51:46] miraheze/puppet - Universal-Omega the build passed. [15:54:22] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/45dd105e95e8...7badb2113a64 [15:54:25] [02miraheze/puppet] 07Universal-Omega 037badb21 - - [15:54:26] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [15:55:57] miraheze/puppet - Universal-Omega the build passed. [15:58:46] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/7badb2113a64...f8b4734e6de6 [15:58:47] [02miraheze/puppet] 07Universal-Omega 03f8b4734 - Update [15:58:48] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [16:00:08] miraheze/puppet - Universal-Omega the build passed. [16:00:25] [02puppet] 07Universal-Omega synchronize pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [16:00:27] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03cleanup [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/f8b4734e6de6...c9edbb40efa0 [16:00:28] [02miraheze/puppet] 07Universal-Omega 03c9edbb4 - Revert changes [16:02:08] [02puppet] 07Universal-Omega closed pull request 03#3786: Cleanup kafka and zookeeper - 13https://github.com/miraheze/puppet/pull/3786 [16:02:11] [02miraheze/puppet] 07Universal-Omega deleted branch 03cleanup [16:02:12] [02miraheze/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-50/±36] 13https://github.com/miraheze/puppet/compare/e588549b2365...bdc28f3e2d58 [16:02:13] [02miraheze/puppet] 07Universal-Omega 03bdc28f3 - Cleanup kafka and zookeeper (#3786) [16:02:15] [02puppet] 07Universal-Omega deleted branch 03cleanup - 13https://github.com/miraheze/puppet [16:17:38] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.08393141627 secs [16:20:10] PROBLEM - wiki.gab.pt.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query pt.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [16:21:37] PROBLEM - cp41 NTP time on cp41 is WARNING: NTP WARNING: Offset 0.1131973863 secs [16:29:37] RECOVERY - cp41 NTP time on cp41 is OK: NTP OK: Offset 0.09903749824 secs [16:48:49] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp37.wikitide.net - CNAME OK [17:11:16] time to go big or go home [17:15:43] @rhinosf1 you said that I should manually add entries to localuser and localnames, however I'm not sure I understand what information localnames is supposed to have (https://www.mediawiki.org/wiki/Extension:CentralAuth/localnames_table) [17:15:48] can you help me here? [17:20:10] seems ln_name is supposed to match the name on globalnames' gn_name [17:20:19] Yes [17:20:31] aight, I get it now [17:21:19] localnames table should be enough to stimulate CentralAuth [17:21:53] I'll try adding just localnames and running the maint script, let's see what CA thinks now [17:29:19] !log mhglobal: INSERT INTO localnames (ln_wiki, ln_name) VALUES ('kylariswiki', 'Thrace'); [17:29:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:29:37] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal (END - exit=2) [17:29:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:30:28] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/sql.php --wiki=loginwiki --wikidb=mhglobal (END - exit=2) [17:30:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:31:10] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CentralAuth/maintenance/migrateAccount.php --wiki=kylariswiki --username=Thrace --auto --attachmissing --attachbroken --homewiki=metawiki (END - exit=0) [17:31:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:32:37] https://meta.miraheze.org/wiki/Special:CentralAuth/Thrace hey, we're getting somewhere! [17:32:59] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CentralAuth/maintenance/attachAccount.php --wiki=kylariswiki --userlist=/home/alex/Thrace.txt (END - exit=0) [17:33:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:33:13]