[00:01:11] PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 27.04, 21.52, 18.62 [00:03:07] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.98, 22.00, 19.16 [00:07:03] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 16.83, 19.99, 19.06 [02:02:46] PROBLEM - wiki.alathramc.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.alathramc.com All nameservers failed to answer the query. [02:03:00] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [02:03:22] PROBLEM - isocasg.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - isocasg.org All nameservers failed to answer the query. [02:06:29] PROBLEM - phorge171 Puppet on phorge171 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:34] PROBLEM - mw172 Puppet on mw172 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:35] PROBLEM - ns2 GDNSD Datacenters on ns2 is CRITICAL: CRITICAL - 2 datacenters are down: 38.46.223.206/cpweb, 2602:294:0:b23::112/cpweb [02:06:41] PROBLEM - mem151 Puppet on mem151 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:41] PROBLEM - db171 Puppet on db171 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:41] PROBLEM - www.publictestwiki.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.publictestwiki.com All nameservers failed to answer the query. [02:06:42] PROBLEM - www.kinitopedia.lol - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - www.kinitopedia.lol All nameservers failed to answer the query. [02:06:42] PROBLEM - wiki.kscucf.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.kscucf.org All nameservers failed to answer the query. [02:06:42] PROBLEM - db181 Puppet on db181 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:42] PROBLEM - os161 Puppet on os161 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [02:06:42] PROBLEM - iol.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - iol.wiki All nameservers failed to answer the query. [02:06:42] PROBLEM - rippaverse.wikitide.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - rippaverse.wikitide.org All nameservers failed to answer the query. [02:06:43] PROBLEM - ndgkb.nena.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - ndgkb.nena.org All nameservers failed to answer the query. [02:06:44] PROBLEM - wiki.ricochetuniverse.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.ricochetuniverse.com All nameservers failed to answer the query. [02:06:44] PROBLEM - exordium.wiki - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - exordium.wiki All nameservers failed to answer the query. [02:06:44] PROBLEM - wiki.jeremyboyd.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.jeremyboyd.org All nameservers failed to answer the query. [02:06:45] PROBLEM - wiki.cube-conflict.com - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.cube-conflict.com All nameservers failed to answer the query. [02:08:36] [02mw-config] 07The-Voidwalker pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/mw-config/compare/2feb73d406fd...0efa51f2e8b5 [02:08:39] [02mw-config] 07The-Voidwalker 030efa51f - remove maintenance sitenotice [02:09:32] miraheze/mw-config - The-Voidwalker the build passed. [02:11:09] RECOVERY - cloud18 Puppet on cloud18 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [02:13:06] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [02:13:32] RECOVERY - swiftobject171 Puppet on swiftobject171 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [02:13:53] RECOVERY - cloud17 Puppet on cloud17 is OK: OK: Puppet is currently enabled, last run 10 seconds ago with 0 failures [02:14:17] !log [void@cloud18.wikitide.net] set vm 101 (bast181) onboot [02:14:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:17:09] PROBLEM - graylog161 Current Load on graylog161 is WARNING: LOAD WARNING - total load average: 7.92, 7.78, 4.98 [02:17:34] PROBLEM - mon181 Current Load on mon181 is WARNING: LOAD WARNING - total load average: 0.56, 4.93, 7.21 [02:17:51] RECOVERY - db151 Puppet on db151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:18:32] RECOVERY - swiftobject151 Puppet on swiftobject151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:19:19] PROBLEM - mw182 Puppet on mw182 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_femiwiki-deploy-1.41],Exec[git_pull_femiwiki-deploy-1.42],Exec[git_pull_3d2png] [02:19:28] PROBLEM - puppet181 Puppet on puppet181 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): Exec[git_pull_puppet],Exec[git_pull_ssl],Exec[git_pull_mediawiki-repos] [02:19:30] RECOVERY - cp36 Puppet on cp36 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [02:19:32] RECOVERY - mon181 Current Load on mon181 is OK: LOAD OK - total load average: 0.23, 3.43, 6.39 [02:20:58] RECOVERY - graylog161 Current Load on graylog161 is OK: LOAD OK - total load average: 3.31, 6.41, 5.12 [02:23:08] RECOVERY - cp26 Puppet on cp26 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:23:52] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:25:08] RECOVERY - graphite151 Puppet on graphite151 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures [02:25:47] RECOVERY - matomo151 Puppet on matomo151 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [02:26:05] RECOVERY - swiftobject161 Puppet on swiftobject161 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures [02:26:44] RECOVERY - swiftac171 Puppet on swiftac171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:27:12] RECOVERY - db171 Puppet on db171 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [02:27:38] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [02:27:38] RECOVERY - reports171 Puppet on reports171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:29:07] RECOVERY - mem161 Puppet on mem161 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [02:29:16] RECOVERY - os161 Puppet on os161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:29:28] RECOVERY - rdb151 Puppet on rdb151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:30:11] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [02:30:23] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [02:31:51] RECOVERY - swiftproxy161 Puppet on swiftproxy161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:32:12] RECOVERY - cp51 Puppet on cp51 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:32:38] RECOVERY - swiftproxy171 Puppet on swiftproxy171 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [02:34:48] RECOVERY - mem151 Puppet on mem151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [02:59:08] RECOVERY - bast161 PowerDNS Recursor on bast161 is OK: DNS OK: 0.317 seconds response time. wikitide.net returns 2602:294:0:b13::110,38.46.223.205 [02:59:48] RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [03:00:01] RECOVERY - db161 Backups SQL on db161 is OK: FILE_AGE OK: /var/log/sql-backup.log is 0 seconds old and 0 bytes [03:00:29] PROBLEM - isocasg.org - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['ns71.domaincontrol.com.', 'ns72.domaincontrol.com.'], 'CNAME': None} [03:01:03] RECOVERY - bast161 Puppet on bast161 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures [03:01:06] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:01:26] PROBLEM - wiki.alathramc.com - reverse DNS on sslhost is WARNING: SSL WARNING - rDNS OK but records conflict. {'NS': ['damon.ns.cloudflare.com.', 'gigi.ns.cloudflare.com.'], 'CNAME': 'wiki.alathra.com.'} [03:01:38] RECOVERY - db181 Backups SQL on db181 is OK: FILE_AGE OK: /var/log/sql-backup.log is 97 seconds old and 0 bytes [03:02:33] !log [@mwtask181] starting deploy of {'config': True} to all [03:02:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:02:43] !log [@mwtask181] finished deploy of {'config': True} to all - SUCCESS in 10s [03:02:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:03:45] PROBLEM - mon181 Backups Grafana on mon181 is WARNING: FILE_AGE WARNING: /var/log/grafana-backup.log is 864205 seconds old and 93 bytes [03:03:47] RECOVERY - www.publictestwiki.com - reverse DNS on sslhost is OK: SSL OK - www.publictestwiki.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:03:50] RECOVERY - www.kinitopedia.lol - reverse DNS on sslhost is OK: SSL OK - www.kinitopedia.lol reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:03:56] RECOVERY - wiki.dwarvesrp.com - reverse DNS on sslhost is OK: SSL OK - wiki.dwarvesrp.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:03:59] RECOVERY - www.streamerlore.com - reverse DNS on sslhost is OK: SSL OK - www.streamerlore.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:01] RECOVERY - rockrevolution.wiki - reverse DNS on sslhost is OK: SSL OK - rockrevolution.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:02] RECOVERY - iol.wiki - reverse DNS on sslhost is OK: SSL OK - iol.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:03] RECOVERY - wiki.eversd.com - reverse DNS on sslhost is OK: SSL OK - wiki.eversd.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:04] RECOVERY - ndgkb.nena.org - reverse DNS on sslhost is OK: SSL OK - ndgkb.nena.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:04] RECOVERY - wiki.jeremyboyd.org - reverse DNS on sslhost is OK: SSL OK - wiki.jeremyboyd.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:05] RECOVERY - ru.countryhumans.polandball.wiki - reverse DNS on sslhost is OK: SSL OK - ru.countryhumans.polandball.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:08] RECOVERY - www.johanloopmans.nl - reverse DNS on sslhost is OK: SSL OK - www.johanloopmans.nl reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:10] RECOVERY - www.puzzles.wiki - reverse DNS on sslhost is OK: SSL OK - www.puzzles.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:11] RECOVERY - journeytheword.wiki - reverse DNS on sslhost is OK: SSL OK - journeytheword.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:15] RECOVERY - looneypyramids.wiki - reverse DNS on sslhost is OK: SSL OK - looneypyramids.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:15] RECOVERY - wiki.pulsus.cc - reverse DNS on sslhost is OK: SSL OK - wiki.pulsus.cc reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:18] RECOVERY - with.cpt-ra.bid - reverse DNS on sslhost is OK: SSL OK - with.cpt-ra.bid reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:19] RECOVERY - wiki.kscucf.org - reverse DNS on sslhost is OK: SSL OK - wiki.kscucf.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:20] RECOVERY - rainverse.wiki - reverse DNS on sslhost is OK: SSL OK - rainverse.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:21] RECOVERY - moddedminecraft.wiki - reverse DNS on sslhost is OK: SSL OK - moddedminecraft.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:04:21] RECOVERY - wiki.manos.pt - reverse DNS on sslhost is OK: SSL OK - wiki.manos.pt reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:25] RECOVERY - www.burnout.wiki - reverse DNS on sslhost is OK: SSL OK - www.burnout.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:25] RECOVERY - uk.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - uk.religiononfire.mar.in.ua reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:27] RECOVERY - test2.specialchat.org - reverse DNS on sslhost is OK: SSL OK - test2.specialchat.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:27] RECOVERY - wiki.chicory.pizza - reverse DNS on sslhost is OK: SSL OK - wiki.chicory.pizza reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:29] RECOVERY - wiki.eastsidegamers.no - reverse DNS on sslhost is OK: SSL OK - wiki.eastsidegamers.no reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:30] RECOVERY - minescape.wiki - reverse DNS on sslhost is OK: SSL OK - minescape.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:04:33] RECOVERY - wiki.potabi.com - reverse DNS on sslhost is OK: SSL OK - wiki.potabi.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:34] RECOVERY - wiki.hwfib.com - reverse DNS on sslhost is OK: SSL OK - wiki.hwfib.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:36] RECOVERY - dkpedia.com - reverse DNS on sslhost is OK: SSL OK - dkpedia.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:36] RECOVERY - speleo.wiki - reverse DNS on sslhost is OK: SSL OK - speleo.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:38] RECOVERY - wiki.thesimswiki.com - reverse DNS on sslhost is OK: SSL OK - wiki.thesimswiki.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:38] RECOVERY - touhouplace.wiki - reverse DNS on sslhost is OK: SSL OK - touhouplace.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:39] RECOVERY - wikitide.net - reverse DNS on sslhost is OK: SSL OK - wikitide.net reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:40] RECOVERY - wiki.msnld.com - reverse DNS on sslhost is OK: SSL OK - wiki.msnld.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:40] RECOVERY - ncuwiki.com - reverse DNS on sslhost is OK: SSL OK - ncuwiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:41] RECOVERY - chobots.wiki - reverse DNS on sslhost is OK: SSL OK - chobots.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:04:42] RECOVERY - wiki.mxlinuxusers.de - reverse DNS on sslhost is OK: SSL OK - wiki.mxlinuxusers.de reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:43] RECOVERY - en.religiononfire.mar.in.ua - reverse DNS on sslhost is OK: SSL OK - en.religiononfire.mar.in.ua reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:43] RECOVERY - wiki.s23.org - reverse DNS on sslhost is OK: SSL OK - wiki.s23.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:43] RECOVERY - small.polandballwiki.com - reverse DNS on sslhost is OK: SSL OK - small.polandballwiki.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:44] RECOVERY - holocron.net - reverse DNS on sslhost is OK: SSL OK - holocron.net reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:45] RECOVERY - spiceandwolf.wiki - reverse DNS on sslhost is OK: SSL OK - spiceandwolf.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:46] RECOVERY - wiki.theatlas.live - reverse DNS on sslhost is OK: SSL OK - wiki.theatlas.live reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:47] RECOVERY - wiki.overwood.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.overwood.xyz reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:48] RECOVERY - revi.wiki - reverse DNS on sslhost is OK: SSL OK - revi.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:04:48] RECOVERY - wiki.counterculturelabs.org - reverse DNS on sslhost is OK: SSL OK - wiki.counterculturelabs.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:48] RECOVERY - wiki.esnmilanostatale.it - reverse DNS on sslhost is OK: SSL OK - wiki.esnmilanostatale.it reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:48] RECOVERY - beyondtheveilrp.com - reverse DNS on sslhost is OK: SSL OK - beyondtheveilrp.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:48] RECOVERY - storytime.jdstroy.cf - reverse DNS on sslhost is OK: SSL OK - storytime.jdstroy.cf reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:51] RECOVERY - wiki.rlab.org.uk - reverse DNS on sslhost is OK: SSL OK - wiki.rlab.org.uk reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:51] RECOVERY - www.allthetropes.org - reverse DNS on sslhost is OK: SSL OK - www.allthetropes.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:52] RECOVERY - kombinat.eu.org - reverse DNS on sslhost is OK: SSL OK - kombinat.eu.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:53] RECOVERY - wiki.lifesprogress.com - reverse DNS on sslhost is OK: SSL OK - wiki.lifesprogress.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:53] RECOVERY - wiki.ricochetuniverse.com - reverse DNS on sslhost is OK: SSL OK - wiki.ricochetuniverse.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:54] RECOVERY - psychevos.org - reverse DNS on sslhost is OK: SSL OK - psychevos.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:56] RECOVERY - wiki.altilunium.my.id - reverse DNS on sslhost is OK: SSL OK - wiki.altilunium.my.id reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:04:56] RECOVERY - pandorastale.wiki - reverse DNS on sslhost is OK: SSL OK - pandorastale.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:04:57] RECOVERY - wiki.strangereons.com - reverse DNS on sslhost is OK: SSL OK - wiki.strangereons.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:08] RECOVERY - www.glitchcity.wiki - reverse DNS on sslhost is OK: SSL OK - www.glitchcity.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:08] RECOVERY - evade.robtospro.com - reverse DNS on sslhost is OK: SSL OK - evade.robtospro.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:09] RECOVERY - drachronicles.voxel.gay - reverse DNS on sslhost is OK: SSL OK - drachronicles.voxel.gay reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:09] RECOVERY - wiki.tmyt105.leyhp.com - reverse DNS on sslhost is OK: SSL OK - wiki.tmyt105.leyhp.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:09] RECOVERY - wiki.globasa.net - reverse DNS on sslhost is OK: SSL OK - wiki.globasa.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:09] RECOVERY - www.pomologia.org - reverse DNS on sslhost is OK: SSL OK - www.pomologia.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:10] RECOVERY - valsora.tep.wiki - reverse DNS on sslhost is OK: SSL OK - valsora.tep.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:11] RECOVERY - www.satepedia.com - reverse DNS on sslhost is OK: SSL OK - www.satepedia.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:11] RECOVERY - wiki.gab.pt.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.gab.pt.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:11] RECOVERY - www.dovearchives.wiki - reverse DNS on sslhost is OK: SSL OK - www.dovearchives.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:11] RECOVERY - enc.for.uz - reverse DNS on sslhost is OK: SSL OK - enc.for.uz reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:12] RECOVERY - wiki.mastodon.kr - reverse DNS on sslhost is OK: SSL OK - wiki.mastodon.kr reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:12] RECOVERY - metroidpedia.com - reverse DNS on sslhost is OK: SSL OK - metroidpedia.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:13] RECOVERY - wiki.beergeeks.co.il - reverse DNS on sslhost is OK: SSL OK - wiki.beergeeks.co.il reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:24] RECOVERY - removededm.com - reverse DNS on sslhost is OK: SSL OK - removededm.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:25] RECOVERY - dc.wikitide.org - reverse DNS on sslhost is OK: SSL OK - dc.wikitide.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:25] RECOVERY - sonicpedia.org - reverse DNS on sslhost is OK: SSL OK - sonicpedia.org reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:05:25] RECOVERY - zomvilles.pixiemeta.com - reverse DNS on sslhost is OK: SSL OK - zomvilles.pixiemeta.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:25] RECOVERY - freedomplanetwiki.com - reverse DNS on sslhost is OK: SSL OK - freedomplanetwiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:26] RECOVERY - knifepointhorror.wiki - reverse DNS on sslhost is OK: SSL OK - knifepointhorror.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:26] RECOVERY - history.sdtef.org - reverse DNS on sslhost is OK: SSL OK - history.sdtef.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:27] RECOVERY - steamdecklinux.wiki - reverse DNS on sslhost is OK: SSL OK - steamdecklinux.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:28] RECOVERY - forecasting.wiki - reverse DNS on sslhost is OK: SSL OK - forecasting.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:05:28] RECOVERY - archive.stellurgists.wiki - reverse DNS on sslhost is OK: SSL OK - archive.stellurgists.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:30] RECOVERY - wiki.worlduniversityandschool.org - reverse DNS on sslhost is OK: SSL OK - wiki.worlduniversityandschool.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:30] RECOVERY - vitriol.spoon.army - reverse DNS on sslhost is OK: SSL OK - vitriol.spoon.army reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:31] RECOVERY - wiki.cor.fyi - reverse DNS on sslhost is OK: SSL OK - wiki.cor.fyi reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:31] RECOVERY - worldless.wiki - reverse DNS on sslhost is OK: SSL OK - worldless.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:31] RECOVERY - elsterclopedie.marithaime.nl - reverse DNS on sslhost is OK: SSL OK - elsterclopedie.marithaime.nl reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:31] RECOVERY - wiki.ate42.ru - reverse DNS on sslhost is OK: SSL OK - wiki.ate42.ru reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:42] RECOVERY - trollpasta.com - reverse DNS on sslhost is OK: SSL OK - trollpasta.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:42] RECOVERY - wiki.solarconquest.net - reverse DNS on sslhost is OK: SSL OK - wiki.solarconquest.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:42] RECOVERY - www.mwcosmos.com - reverse DNS on sslhost is OK: SSL OK - www.mwcosmos.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:43] RECOVERY - www.hebammenwiki.de - reverse DNS on sslhost is OK: SSL OK - www.hebammenwiki.de reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:43] RECOVERY - wiki.otir.nl - reverse DNS on sslhost is OK: SSL OK - wiki.otir.nl reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:44] RECOVERY - exordium.wiki - reverse DNS on sslhost is OK: SSL OK - exordium.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:44] RECOVERY - wiki.sukotuze.com - reverse DNS on sslhost is OK: SSL OK - wiki.sukotuze.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:45] RECOVERY - baubles.me - reverse DNS on sslhost is OK: SSL OK - baubles.me reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:45] RECOVERY - eosdev.xyz - reverse DNS on sslhost is OK: SSL OK - eosdev.xyz reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:46] RECOVERY - wiki.fbpml.org - reverse DNS on sslhost is OK: SSL OK - wiki.fbpml.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:46] RECOVERY - osfirsttimer.wiki - reverse DNS on sslhost is OK: SSL OK - osfirsttimer.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:48] RECOVERY - osdev.wiki - reverse DNS on sslhost is OK: SSL OK - osdev.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:49] RECOVERY - hardcore.wiki - reverse DNS on sslhost is OK: SSL OK - hardcore.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:49] RECOVERY - infectowiki.com - reverse DNS on sslhost is OK: SSL OK - infectowiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:05:49] RECOVERY - wiki.moores.tech - reverse DNS on sslhost is OK: SSL OK - wiki.moores.tech reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:49] RECOVERY - wiki.projectdiablo2.com - reverse DNS on sslhost is OK: SSL OK - wiki.projectdiablo2.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:05:50] RECOVERY - tfuk.app - reverse DNS on sslhost is OK: SSL OK - tfuk.app reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:05:50] RECOVERY - translate.petrawiki.org - reverse DNS on sslhost is OK: SSL OK - translate.petrawiki.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:02] RECOVERY - wiki.widwwa.co.uk - reverse DNS on sslhost is OK: SSL OK - wiki.widwwa.co.uk reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:03] RECOVERY - pwiki.arkcls.com - reverse DNS on sslhost is OK: SSL OK - pwiki.arkcls.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:04] RECOVERY - wikimas.kr - reverse DNS on sslhost is OK: SSL OK - wikimas.kr reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:04] RECOVERY - o.kaoyu666.cn - reverse DNS on sslhost is OK: SSL OK - o.kaoyu666.cn reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:04] RECOVERY - www.rippaversewiki.com - reverse DNS on sslhost is OK: SSL OK - www.rippaversewiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:05] RECOVERY - dkwiki.org - reverse DNS on sslhost is OK: SSL OK - dkwiki.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:06] RECOVERY - kodiak.wiki - reverse DNS on sslhost is OK: SSL OK - kodiak.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:06] RECOVERY - lgbtqia.wiki - reverse DNS on sslhost is OK: SSL OK - lgbtqia.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:06:07] RECOVERY - electowiki.org - reverse DNS on sslhost is OK: SSL OK - electowiki.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:07] RECOVERY - history.estill.org - reverse DNS on sslhost is OK: SSL OK - history.estill.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:07] RECOVERY - miraheze.com - reverse DNS on sslhost is OK: SSL OK - miraheze.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:07] RECOVERY - files.petrawiki.org - reverse DNS on sslhost is OK: SSL OK - files.petrawiki.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:08] RECOVERY - puritwiki.p-e.kr - reverse DNS on sslhost is OK: SSL OK - puritwiki.p-e.kr reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:09] RECOVERY - www.ethene.wiki - reverse DNS on sslhost is OK: SSL OK - www.ethene.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:09] RECOVERY - mirabeta.org - reverse DNS on sslhost is OK: SSL OK - mirabeta.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:10] RECOVERY - bluearchive.wiki - reverse DNS on sslhost is OK: SSL OK - bluearchive.wiki reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:10] RECOVERY - dutki.kro.kr - reverse DNS on sslhost is OK: SSL OK - dutki.kro.kr reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:11] RECOVERY - wiki.insideearth.info - reverse DNS on sslhost is OK: SSL OK - wiki.insideearth.info reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:12] RECOVERY - wiki.mc.marochuru.net - reverse DNS on sslhost is OK: SSL OK - wiki.mc.marochuru.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:12] RECOVERY - wiki.wildport.com - reverse DNS on sslhost is OK: SSL OK - wiki.wildport.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:24] RECOVERY - segawiki.com - reverse DNS on sslhost is OK: SSL OK - segawiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:24] RECOVERY - beta.sagan4.org - reverse DNS on sslhost is OK: SSL OK - beta.sagan4.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:25] RECOVERY - wiki.kachow.org - reverse DNS on sslhost is OK: SSL OK - wiki.kachow.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:25] RECOVERY - wiki.cube-conflict.com - reverse DNS on sslhost is OK: SSL OK - wiki.cube-conflict.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:26] RECOVERY - wiki.joust.ro - reverse DNS on sslhost is OK: SSL OK - wiki.joust.ro reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:26] RECOVERY - pwiki.drydraytonvillagehall.org.uk - reverse DNS on sslhost is OK: SSL OK - pwiki.drydraytonvillagehall.org.uk reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:27] RECOVERY - wiki.cutefame.net - reverse DNS on sslhost is OK: SSL OK - wiki.cutefame.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:27] RECOVERY - www.bluepageswiki.org - reverse DNS on sslhost is OK: SSL OK - www.bluepageswiki.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:28] RECOVERY - mhdh.pj568.eu.org - reverse DNS on sslhost is OK: SSL OK - mhdh.pj568.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:28] RECOVERY - webkinzguide.com - reverse DNS on sslhost is OK: SSL OK - webkinzguide.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:29] RECOVERY - 9b9t.wiki - reverse DNS on sslhost is OK: SSL OK - 9b9t.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:29] RECOVERY - www.winenjoy.net - reverse DNS on sslhost is OK: SSL OK - www.winenjoy.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:30] RECOVERY - wiki.elgeis.com - reverse DNS on sslhost is OK: SSL OK - wiki.elgeis.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:30] RECOVERY - lophocmatngu.wiki - reverse DNS on sslhost is OK: SSL OK - lophocmatngu.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:06:31] RECOVERY - agesofconflict.wiki - reverse DNS on sslhost is OK: SSL OK - agesofconflict.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:31] RECOVERY - wiki.wubbygame.com - reverse DNS on sslhost is OK: SSL OK - wiki.wubbygame.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:33] RECOVERY - wiki.junkstore.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.junkstore.xyz reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:33] RECOVERY - dariawiki.org - reverse DNS on sslhost is OK: SSL OK - dariawiki.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:44] RECOVERY - wiki.sequoiafabrica.org - reverse DNS on sslhost is OK: SSL OK - wiki.sequoiafabrica.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:44] RECOVERY - you.r-fit.cc - reverse DNS on sslhost is OK: SSL OK - you.r-fit.cc reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:45] RECOVERY - fanonpedia.com - reverse DNS on sslhost is OK: SSL OK - fanonpedia.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:45] RECOVERY - wiki.villagecollaborative.net - reverse DNS on sslhost is OK: SSL OK - wiki.villagecollaborative.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:46] RECOVERY - smafallfest.org - reverse DNS on sslhost is OK: SSL OK - smafallfest.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:46] RECOVERY - rippaverse.wikitide.org - reverse DNS on sslhost is OK: SSL OK - rippaverse.wikitide.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:47] RECOVERY - wiki.dragonsmp.pro - reverse DNS on sslhost is OK: SSL OK - wiki.dragonsmp.pro reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:47] RECOVERY - guia.cineastas.pt - reverse DNS on sslhost is OK: SSL OK - guia.cineastas.pt reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:48] RECOVERY - www.hoolehistoryheritagesociety.org.uk - reverse DNS on sslhost is OK: SSL OK - www.hoolehistoryheritagesociety.org.uk reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:48] RECOVERY - wiki.luemir.xyz - reverse DNS on sslhost is OK: SSL OK - wiki.luemir.xyz reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:49] RECOVERY - www.pyramidgames.wiki - reverse DNS on sslhost is OK: SSL OK - www.pyramidgames.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:49] RECOVERY - wiltshire.xyz - reverse DNS on sslhost is OK: SSL OK - wiltshire.xyz reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:50] RECOVERY - poserdazfreebies.orain.org - reverse DNS on sslhost is OK: SSL OK - poserdazfreebies.orain.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:50] RECOVERY - yokaiwatchwiki.com - reverse DNS on sslhost is OK: SSL OK - yokaiwatchwiki.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:06:51] RECOVERY - wiki.cogsci.live - reverse DNS on sslhost is OK: SSL OK - wiki.cogsci.live reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:51] RECOVERY - wiki.gamergeeked.us - reverse DNS on sslhost is OK: SSL OK - wiki.gamergeeked.us reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:52] RECOVERY - de.berlinwiki.org - reverse DNS on sslhost is OK: SSL OK - de.berlinwiki.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:06:52] RECOVERY - wiki.lefrenchmelee.fr - reverse DNS on sslhost is OK: SSL OK - wiki.lefrenchmelee.fr reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:04] RECOVERY - opendatascot.org - reverse DNS on sslhost is OK: SSL OK - opendatascot.org reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:07:04] RECOVERY - wiki.cdntennis.ca - reverse DNS on sslhost is OK: SSL OK - wiki.cdntennis.ca reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:05] RECOVERY - wiki.gestaltonline.ca - reverse DNS on sslhost is OK: SSL OK - wiki.gestaltonline.ca reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:05] RECOVERY - burnout.wiki - reverse DNS on sslhost is OK: SSL OK - burnout.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:07:06] RECOVERY - wiki.pokrew.net - reverse DNS on sslhost is OK: SSL OK - wiki.pokrew.net reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:06] RECOVERY - tdr.wiki - reverse DNS on sslhost is OK: SSL OK - tdr.wiki reverse DNS resolves to cp36.wikitide.net - CNAME FLAT [03:07:08] RECOVERY - www.fossilfighterswiki.com - reverse DNS on sslhost is OK: SSL OK - www.fossilfighterswiki.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:08] RECOVERY - wiki.lesunk.com - reverse DNS on sslhost is OK: SSL OK - wiki.lesunk.com reverse DNS resolves to cp36.wikitide.net - CNAME OK [03:07:08] RECOVERY - wizardia.wiki - reverse DNS on sslhost is OK: SSL OK - wizardia.wiki reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:07:08] RECOVERY - project-patterns.com - reverse DNS on sslhost is OK: SSL OK - project-patterns.com reverse DNS resolves to cp36.wikitide.net - NS RECORDS OK [03:07:37] !log [@mwtask171] starting deploy of {'config': True} to all [03:07:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:07:47] !log [@mwtask171] finished deploy of {'config': True} to all - SUCCESS in 9s [03:07:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:09:29] PROBLEM - mon181 Current Load on mon181 is CRITICAL: LOAD CRITICAL - total load average: 15.04, 13.50, 6.37 [03:09:45] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:13:25] RECOVERY - mon181 Current Load on mon181 is OK: LOAD OK - total load average: 0.56, 6.21, 4.98 [03:16:19] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 46 seconds ago with 0 failures [03:17:11] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [03:24:05] !log [@test151] starting deploy of {'config': True} to test151 [03:24:06] !log [@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [03:24:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:25:12] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:25:43] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [03:25:58] RECOVERY - bots171 Puppet on bots171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [05:02:29] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [05:59:56] This is allowed, also you're asking in the wrong channel [06:01:18] they also asked in #general and probably #tech as well [06:25:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.79, 19.19, 16.51 [06:27:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 15.74, 18.33, 16.56 [06:31:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.65, 21.47, 18.27 [06:33:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.33, 19.57, 17.95 [06:37:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.40, 20.72, 18.78 [06:39:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.10, 22.04, 19.50 [06:43:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.41, 22.27, 20.21 [06:47:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.72, 24.26, 21.46 [06:51:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.33, 22.74, 21.62 [06:59:34] PROBLEM - education.gced.in - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'education.gced.in' expires in 15 day(s) (Sun 21 Jul 2024 06:31:00 AM GMT +0000). [06:59:45] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/e24cb9e52a74...a2fd23cc6f97 [06:59:48] [02ssl] 07WikiTideSSLBot 03a2fd23c - Bot: Update SSL cert for education.gced.in [06:59:54] PROBLEM - wiki.chicory.pizza - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.chicory.pizza' expires in 15 day(s) (Sun 21 Jul 2024 06:52:15 AM GMT +0000). [07:00:07] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/a2fd23cc6f97...6775ffcbaf19 [07:00:09] [02ssl] 07WikiTideSSLBot 036775ffc - Bot: Update SSL cert for wiki.chicory.pizza [07:01:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 20.36, 19.15, 20.36 [07:16:09] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.91, 20.80, 20.25 [07:18:05] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 27.19, 23.16, 21.18 [07:23:53] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 17.02, 22.46, 21.75 [07:28:57] RECOVERY - education.gced.in - LetsEncrypt on sslhost is OK: OK - Certificate 'education.gced.in' will expire on Thu 03 Oct 2024 05:59:39 AM GMT +0000. [07:29:26] RECOVERY - wiki.chicory.pizza - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.chicory.pizza' will expire on Thu 03 Oct 2024 06:00:01 AM GMT +0000. [07:29:40] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 14.79, 18.20, 20.17 [07:46:46] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [08:12:41] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 0 seconds ago with 0 failures [08:17:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [08:35:28] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query uk.eu.org. IN NS: Server 2606:4700:4700::1111 UDP port 53 answered SERVFAIL [08:54:14] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.10, 20.87, 19.10 [08:56:10] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 20.31, 20.04, 18.97 [09:35:30] PROBLEM - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - wiki.andreijiroh.uk.eu.org All nameservers failed to answer the query. [09:38:45] [02puppet] 07redbluegreenhat pushed 031 commit to 03pywikibot-install-path-no-recurse [+0/-0/±1] 13https://github.com/miraheze/puppet/commit/e9f9b765a3f2 [09:38:48] [02puppet] 07redbluegreenhat 03e9f9b76 - pywikibot: do not recursively apply mode 644 on $install_path [09:38:51] [02puppet] 07redbluegreenhat created branch 03pywikibot-install-path-no-recurse - 13https://github.com/miraheze/puppet [09:39:10] [02puppet] 07redbluegreenhat opened pull request 03#3878: pywikibot: do not recursively apply mode 644 on $install_path - 13https://github.com/miraheze/puppet/pull/3878 [09:39:13] [02puppet] 07coderabbitai[bot] commented on pull request 03#3878: pywikibot: do not recursively apply mode 644 on $install_path - 13https://github.com/miraheze/puppet/pull/3878#issuecomment-2210549303 [09:39:25] [02puppet] 07redbluegreenhat edited pull request 03#3878: pywikibot: do not recursively apply mode 644 on $install_path - 13https://github.com/miraheze/puppet/pull/3878 [09:41:31] [02puppet] 07redbluegreenhat closed pull request 03#3878: pywikibot: do not recursively apply mode 644 on $install_path - 13https://github.com/miraheze/puppet/pull/3878 [09:41:33] [02puppet] 07redbluegreenhat pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/puppet/compare/9626916a9e60...706206b92cb4 [09:41:35] [02puppet] 07redbluegreenhat 03706206b - pywikibot: do not recursively apply mode 644 on $install_path (#3878) [09:41:38] [02puppet] 07redbluegreenhat deleted branch 03pywikibot-install-path-no-recurse [09:41:41] [02puppet] 07redbluegreenhat deleted branch 03pywikibot-install-path-no-recurse - 13https://github.com/miraheze/puppet [10:43:26] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=indomitawiki indomita.wikitide.org_1_40-20230924-history.xml (START) [10:43:31] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=indomitawiki indomita.wikitide.org_1_40-20230924-history.xml (END - exit=2) [10:43:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:43:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:44:37] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=indomitawiki indomita.wikitide.org_1_40-20230924-history.xml (START) [10:44:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:46:01] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importDump.php --wiki=indomitawiki indomita.wikitide.org_1_40-20230924-history.xml (END - exit=0) [10:46:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:46:53] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/rebuildrecentchanges.php --wiki=indomitawiki (END - exit=0) [10:46:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:47:11] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/initSiteStats.php --wiki=indomitawiki (END - exit=0) [10:47:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:54:04] !log [alex@mwtask171] installed the 7zip package [10:54:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:56:54] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=indomitawiki --search-recursively --skip-dupes --summary=[[phorge:T11779]] . (START) [10:56:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [10:57:16] !log [alex@mwtask171] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/importImages.php --wiki=indomitawiki --search-recursively --skip-dupes --summary=[[phorge:T11779]] . (END - exit=0) [10:57:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:24:56] PROBLEM - www.ebcat.me - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'www.ebcat.me' expires in 15 day(s) (Sun 21 Jul 2024 11:21:50 AM GMT +0000). [11:25:07] [02ssl] 07WikiTideSSLBot pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/miraheze/ssl/compare/6775ffcbaf19...41acdc69ff48 [11:25:10] [02ssl] 07WikiTideSSLBot 0341acdc6 - Bot: Update SSL cert for www.ebcat.me [11:31:51] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/reassignEdits.php --wiki=indomitawiki Lithium Miraheze_maintenance_script (END - exit=0) [11:31:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:33:13] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/extensions/CentralAuth/maintenance/createLocalAccount.php --wiki=indomitawiki Miraheze_maintenance_script (END - exit=0) [11:33:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:33:22] !log [alex@mwtask181] sudo -u www-data php /srv/mediawiki/1.41/maintenance/run.php /srv/mediawiki/1.41/maintenance/reassignEdits.php --wiki=indomitawiki Lithium Miraheze_maintenance_script (END - exit=0) [11:33:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [11:34:10] forgot to do the import with --username-prefix, did this so that we don't wrongly attribute edits to an existing Miraheze user. [11:34:33] Miraheze_maintenance_script just happened to be the best account at hand [11:35:32] RECOVERY - wiki.andreijiroh.uk.eu.org - reverse DNS on sslhost is OK: SSL OK - wiki.andreijiroh.uk.eu.org reverse DNS resolves to cp36.wikitide.net - CNAME OK [11:53:51] RECOVERY - www.ebcat.me - LetsEncrypt on sslhost is OK: OK - Certificate 'www.ebcat.me' will expire on Thu 03 Oct 2024 10:25:01 AM GMT +0000. [12:17:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [13:22:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [13:51:32] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.20, 19.77, 16.89 [13:53:28] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.50, 20.09, 17.34 [13:59:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.03, 19.95, 18.35 [14:16:25] PROBLEM - ns2 Puppet on ns2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [14:17:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [14:25:20] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 28.44, 22.70, 19.51 [14:27:20] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 19.92, 21.68, 19.56 [14:29:20] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.83, 20.32, 19.31 [14:32:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [14:44:23] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures [15:17:25] [Grafana] !tech FIRING: The mediawiki job queue has more than 500 unclaimed jobs https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [15:55:19] PROBLEM - isocasg.org - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'isocasg.org' expires in 15 day(s) (Sun 21 Jul 2024 03:43:22 PM GMT +0000). [15:57:25] [Grafana] !tech RESOLVED: High Job Queue Backlog https://grafana.wikitide.net/d/GtxbP1Xnk?orgId=1 [16:07:39] PROBLEM - ns2 NTP time on ns2 is UNKNOWN: check_ntp_time: Invalid hostname/address - time.cloudflare.comUsage: check_ntp_time -H [-4|-6] [-w ] [-c ] [-v verbose] [-o