[00:00:55] PROBLEM - wiki.fraterniterebellion.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.fraterniterebellion.com' expires in 7 day(s) (Tue 19 Oct 2021 23:58:49 GMT +0000). [00:01:23] [02MirahezeMagic] 07dmehus commented on pull request 03#291: MirahezeMagic override message for global abuse filter - 13https://git.io/JopNg [00:01:25] [02MirahezeMagic] 07dmehus closed pull request 03#291: MirahezeMagic override message for global abuse filter - 13https://git.io/Jop9k [00:02:55] PROBLEM - graylog2 Puppet on graylog2 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [00:05:16] PROBLEM - wiki.patriam.cc - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.patriam.cc' expires in 7 day(s) (Tue 19 Oct 2021 23:59:52 GMT +0000). [00:05:24] PROBLEM - www.fraterniterebellion.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.fraterniterebellion.com' expires in 7 day(s) (Tue 19 Oct 2021 23:58:49 GMT +0000). [00:06:39] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.58, 6.31, 4.98 [00:06:41] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.61, 7.24, 4.92 [00:07:20] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 30.83, 25.18, 19.98 [00:07:39] PROBLEM - wiki.cdntennis.ca - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.cdntennis.ca' expires in 7 day(s) (Wed 20 Oct 2021 00:05:50 GMT +0000). [00:08:40] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.62, 6.91, 5.35 [00:08:43] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.83, 6.51, 4.93 [00:10:39] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.07, 6.05, 5.22 [00:10:46] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.34, 5.87, 4.96 [00:12:47] PROBLEM - wiki.graalmilitary.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.graalmilitary.com' expires in 7 day(s) (Wed 20 Oct 2021 00:11:28 GMT +0000). [00:13:17] PROBLEM - wiki.grottocenter.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.grottocenter.org' expires in 7 day(s) (Wed 20 Oct 2021 00:09:24 GMT +0000). [00:24:08] PROBLEM - miraheze.ml - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'miraheze.ml' expires in 7 day(s) (Wed 20 Oct 2021 00:17:43 GMT +0000). [00:25:20] PROBLEM - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'encyclopediaofastrobiology.org' expires in 7 day(s) (Wed 20 Oct 2021 00:24:07 GMT +0000). [00:33:16] RECOVERY - graylog2 Puppet on graylog2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [00:36:30] PROBLEM - pubwiki.lab.co.pl - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'pubwiki.lab.co.pl' expires in 7 day(s) (Wed 20 Oct 2021 00:30:48 GMT +0000). [00:39:26] [02MirahezeMagic] 07Universal-Omega commented on pull request 03#291: MirahezeMagic override message for global abuse filter - 13https://git.io/JohY9 [00:39:57] PROBLEM - es.nocyclo.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'es.nocyclo.tk' expires in 7 day(s) (Wed 20 Oct 2021 00:37:29 GMT +0000). [00:42:11] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.43, 3.58, 3.23 [00:44:13] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.15, 3.31, 3.17 [00:45:09] PROBLEM - madzebrascience.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'madzebrascience.wiki' expires in 7 day(s) (Wed 20 Oct 2021 00:42:33 GMT +0000). [00:55:28] PROBLEM - wiki.conworlds.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.conworlds.org' expires in 7 day(s) (Wed 20 Oct 2021 00:53:44 GMT +0000). [01:00:18] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.48, 3.48, 3.24 [01:01:28] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 12.88, 18.64, 23.72 [01:01:58] PROBLEM - index.pdcommunity.ir - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'index.pdcommunity.ir' expires in 7 day(s) (Wed 20 Oct 2021 00:55:24 GMT +0000). [01:02:17] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.66, 3.13, 3.14 [01:03:58] PROBLEM - wiki.sdbx.dev - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.sdbx.dev' expires in 7 day(s) (Wed 20 Oct 2021 01:00:54 GMT +0000). [01:05:30] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.33, 22.44, 24.10 [01:07:26] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.99, 21.57, 23.59 [01:11:16] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.18, 3.79, 3.44 [01:11:16] PROBLEM - munwiki.info - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'munwiki.info' expires in 7 day(s) (Wed 20 Oct 2021 01:10:45 GMT +0000). [01:12:32] PROBLEM - lg.gyaanipedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'lg.gyaanipedia.com' expires in 7 day(s) (Wed 20 Oct 2021 01:11:18 GMT +0000). [01:13:18] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.57, 3.90, 3.54 [01:13:53] PROBLEM - wiki.funkey-project.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.funkey-project.com' expires in 7 day(s) (Wed 20 Oct 2021 01:07:22 GMT +0000). [01:16:36] PROBLEM - wiki.thehall.xyz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.thehall.xyz' expires in 7 day(s) (Wed 20 Oct 2021 01:16:00 GMT +0000). [01:19:19] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.40, 3.23, 3.34 [01:23:52] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.71, 3.53, 3.44 [01:25:52] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 5.72, 4.22, 3.70 [01:26:27] PROBLEM - beaconspace.unrestrictedlorefare.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'beaconspace.unrestrictedlorefare.com' expires in 7 day(s) (Wed 20 Oct 2021 01:20:50 GMT +0000). [01:27:59] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.25, 3.82, 3.62 [01:28:07] PROBLEM - wiki.openhatch.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.openhatch.org' expires in 7 day(s) (Wed 20 Oct 2021 01:24:07 GMT +0000). [01:31:41] PROBLEM - guia.cineastas.pt - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'guia.cineastas.pt' expires in 7 day(s) (Wed 20 Oct 2021 01:30:21 GMT +0000). [01:32:03] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.38, 2.93, 3.33 [01:35:00] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.97, 5.44, 5.91 [01:36:25] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.77, 3.83, 3.59 [01:37:11] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.66, 5.98, 6.05 [01:38:32] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.95, 3.86, 3.63 [01:39:14] PROBLEM - wiki.responsibly.ai - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.responsibly.ai' expires in 7 day(s) (Wed 20 Oct 2021 01:33:08 GMT +0000). [01:39:30] PROBLEM - wiki.twilightsignal.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.twilightsignal.com' expires in 7 day(s) (Wed 20 Oct 2021 01:37:19 GMT +0000). [01:40:50] PROBLEM - n64brew.dev - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'n64brew.dev' expires in 7 day(s) (Wed 20 Oct 2021 01:37:44 GMT +0000). [01:40:58] PROBLEM - spiral.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'spiral.wiki' expires in 7 day(s) (Wed 20 Oct 2021 01:40:39 GMT +0000). [01:43:56] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.19, 22.41, 21.94 [01:45:52] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 16.53, 20.00, 21.12 [01:49:50] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.45, 18.79, 20.39 [01:50:56] PROBLEM - enc.for.uz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'enc.for.uz' expires in 7 day(s) (Wed 20 Oct 2021 01:46:52 GMT +0000). [01:51:01] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.55, 5.43, 5.90 [01:52:57] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 5.70, 5.87, 6.03 [01:53:53] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.55, 21.19, 20.90 [01:54:40] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.19, 3.22, 3.38 [01:55:53] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.30, 19.14, 20.19 [01:56:52] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.09, 5.46, 5.82 [01:58:52] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.91, 6.17, 6.04 [02:00:03] PROBLEM - wiki.anglish.info - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.anglish.info' expires in 7 day(s) (Wed 20 Oct 2021 01:53:29 GMT +0000). [02:02:45] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.62, 3.44, 3.40 [02:03:27] PROBLEM - vise.dayid.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'vise.dayid.org' expires in 7 day(s) (Wed 20 Oct 2021 01:59:30 GMT +0000). [02:04:46] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.27, 3.24, 3.32 [02:05:53] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.82, 20.97, 20.64 [02:07:18] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 22.59, 19.62, 17.13 [02:08:02] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.22, 7.12, 5.39 [02:08:45] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.37, 5.59, 5.90 [02:09:19] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 16.68, 17.97, 16.81 [02:09:55] PROBLEM - vedopedia.witches-empire.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'vedopedia.witches-empire.com' expires in 7 day(s) (Wed 20 Oct 2021 02:06:35 GMT +0000). [02:10:03] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.38, 6.05, 5.20 [02:11:48] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 16.55, 19.64, 20.33 [02:16:17] PROBLEM - wiki.starship.digital - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.starship.digital' expires in 7 day(s) (Wed 20 Oct 2021 02:14:03 GMT +0000). [02:18:50] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.62, 6.63, 5.99 [02:19:29] PROBLEM - kunwok.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'kunwok.org' expires in 7 day(s) (Wed 20 Oct 2021 02:13:08 GMT +0000). [02:21:57] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 19.74, 21.84, 20.83 [02:22:28] PROBLEM - thelonsdalebattalion.co.uk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'thelonsdalebattalion.co.uk' expires in 7 day(s) (Wed 20 Oct 2021 02:16:39 GMT +0000). [02:24:45] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.62, 5.43, 5.74 [02:25:56] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 18.58, 19.92, 20.28 [02:28:45] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.18, 6.15, 5.91 [02:29:13] PROBLEM - antiguabarbudacalypso.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'antiguabarbudacalypso.com' expires in 7 day(s) (Wed 20 Oct 2021 02:22:32 GMT +0000). [02:31:57] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.83, 20.71, 20.52 [02:32:02] PROBLEM - en.nocyclo.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'en.nocyclo.tk' expires in 7 day(s) (Wed 20 Oct 2021 02:26:59 GMT +0000). [02:32:45] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.73, 5.63, 5.79 [02:33:53] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.26, 21.78, 20.91 [02:34:46] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.71, 6.00, 5.90 [02:35:56] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.07, 22.21, 21.18 [02:36:52] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.62, 5.02, 5.54 [02:37:20] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 13.94, 9.08, 6.03 [02:37:31] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 24.62, 22.61, 18.88 [02:38:01] PROBLEM - wiki.rebirthofthenight.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.rebirthofthenight.com' expires in 7 day(s) (Wed 20 Oct 2021 02:33:58 GMT +0000). [02:39:17] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 4.57, 7.26, 5.73 [02:39:32] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 15.28, 19.88, 18.34 [02:39:59] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.45, 18.84, 20.04 [02:40:38] PROBLEM - wiki.worlduniversityandschool.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.worlduniversityandschool.org' expires in 7 day(s) (Wed 20 Oct 2021 02:38:33 GMT +0000). [02:40:39] PROBLEM - aman.awiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'aman.awiki.org' expires in 7 day(s) (Wed 20 Oct 2021 02:39:59 GMT +0000). [02:41:14] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 3.64, 5.96, 5.43 [02:42:45] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.06, 3.79, 4.86 [02:50:53] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.19, 5.27, 5.02 [02:51:01] PROBLEM - wiki.seamly.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.seamly.net' expires in 7 day(s) (Wed 20 Oct 2021 02:47:33 GMT +0000). [02:52:49] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 5.07, 5.08, 4.97 [02:53:01] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.26, 3.45, 3.27 [02:55:02] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.08, 3.21, 3.20 [02:55:52] PROBLEM - tensegritywiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'tensegritywiki.com' expires in 7 day(s) (Wed 20 Oct 2021 02:53:12 GMT +0000). [02:58:53] PROBLEM - wiki.helioss.co - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.helioss.co' expires in 7 day(s) (Wed 20 Oct 2021 02:53:45 GMT +0000). [03:04:29] PROBLEM - electowiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'electowiki.org' expires in 7 day(s) (Wed 20 Oct 2021 03:01:07 GMT +0000). [03:06:00] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 25.60, 20.96, 19.27 [03:08:08] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.00, 5.44, 4.87 [03:08:17] PROBLEM - data.nonbinary.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'data.nonbinary.wiki' expires in 7 day(s) (Wed 20 Oct 2021 03:02:44 GMT +0000). [03:08:29] PROBLEM - hololive.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'hololive.wiki' expires in 7 day(s) (Wed 20 Oct 2021 03:08:26 GMT +0000). [03:08:55] PROBLEM - mw12 APT on mw12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [03:10:03] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 18.86, 23.61, 21.07 [03:10:04] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.12, 4.86, 4.73 [03:10:56] RECOVERY - mw12 APT on mw12 is OK: APT OK: 16 packages available for upgrade (0 critical updates). [03:13:59] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 15.12, 18.75, 19.69 [03:15:10] PROBLEM - warmage.necrocity.co.uk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'warmage.necrocity.co.uk' expires in 7 day(s) (Wed 20 Oct 2021 03:15:07 GMT +0000). [03:18:13] PROBLEM - wiki.candela.digital - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.candela.digital' expires in 7 day(s) (Wed 20 Oct 2021 03:13:29 GMT +0000). [03:28:34] PROBLEM - miraheze.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'miraheze.tk' expires in 7 day(s) (Wed 20 Oct 2021 03:22:35 GMT +0000). [03:28:45] PROBLEM - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'cyberlaw.ccdcoe.org' expires in 7 day(s) (Wed 20 Oct 2021 03:25:52 GMT +0000). [03:29:17] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.54, 3.23, 3.04 [03:31:02] PROBLEM - wiki.invinciblewellbeing.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.invinciblewellbeing.com' expires in 7 day(s) (Wed 20 Oct 2021 03:29:22 GMT +0000). [03:31:17] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.24, 3.15, 3.03 [03:33:11] PROBLEM - planetnomads.speedbomb.de - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'planetnomads.speedbomb.de' expires in 7 day(s) (Wed 20 Oct 2021 03:32:14 GMT +0000). [03:35:34] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.10, 7.07, 5.42 [03:37:31] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.59, 6.35, 5.38 [03:37:37] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 26.31, 19.92, 17.49 [03:39:34] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.69, 20.05, 17.82 [03:41:32] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.66, 17.54, 17.16 [03:42:41] PROBLEM - history.sdtef.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'history.sdtef.org' expires in 7 day(s) (Wed 20 Oct 2021 03:36:15 GMT +0000). [03:54:59] PROBLEM - wiki.joust.ro - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.joust.ro' expires in 7 day(s) (Wed 20 Oct 2021 03:50:08 GMT +0000). [03:57:29] PROBLEM - wiki.elgeis.com - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.elgeis.com' expires in 15 day(s) (Thu 28 Oct 2021 03:49:16 GMT +0000). [04:01:57] PROBLEM - wiki.pablojprieto.com.es - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.pablojprieto.com.es' expires in 7 day(s) (Wed 20 Oct 2021 03:55:35 GMT +0000). [04:02:16] PROBLEM - wiki.park-city.club - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wiki.park-city.club' expires in 15 day(s) (Thu 28 Oct 2021 03:53:45 GMT +0000). [04:05:51] PROBLEM - wiki.insideearth.info - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.insideearth.info' expires in 7 day(s) (Wed 20 Oct 2021 04:03:43 GMT +0000). [04:08:45] PROBLEM - infectowiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'infectowiki.com' expires in 7 day(s) (Wed 20 Oct 2021 04:08:09 GMT +0000). [04:11:54] [02MirahezeMagic] 07dmehus commented on pull request 03#291: MirahezeMagic override message for global abuse filter - 13https://git.io/Jojoi [04:20:01] PROBLEM - meta.nocyclo.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'meta.nocyclo.tk' expires in 7 day(s) (Wed 20 Oct 2021 04:13:26 GMT +0000). [04:20:57] PROBLEM - wiki.shaazzz.ir - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.shaazzz.ir' expires in 7 day(s) (Wed 20 Oct 2021 04:18:35 GMT +0000). [04:25:48] PROBLEM - stablestate.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'stablestate.org' expires in 7 day(s) (Wed 20 Oct 2021 04:24:34 GMT +0000). [04:29:20] PROBLEM - tep.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'tep.wiki' expires in 7 day(s) (Wed 20 Oct 2021 04:26:11 GMT +0000). [04:37:17] PROBLEM - halloween.gyaanipedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'halloween.gyaanipedia.com' expires in 7 day(s) (Wed 20 Oct 2021 04:32:34 GMT +0000). [04:40:16] PROBLEM - tl.awiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'tl.awiki.org' expires in 7 day(s) (Wed 20 Oct 2021 04:39:56 GMT +0000). [04:40:53] PROBLEM - pwiki.arkcls.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'pwiki.arkcls.com' expires in 7 day(s) (Wed 20 Oct 2021 04:38:24 GMT +0000). [04:45:57] PROBLEM - wiki.teessidehackspace.org.uk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.teessidehackspace.org.uk' expires in 7 day(s) (Wed 20 Oct 2021 04:45:23 GMT +0000). [04:55:10] PROBLEM - wiki.scvo.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.scvo.org' expires in 7 day(s) (Wed 20 Oct 2021 04:52:52 GMT +0000). [04:58:36] !log created LDAP account for agent [04:58:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [05:01:07] PROBLEM - www.programming.red - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'programming.red' expires in 7 day(s) (Wed 20 Oct 2021 05:00:42 GMT +0000). [05:03:50] PROBLEM - programming.red - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'programming.red' expires in 7 day(s) (Wed 20 Oct 2021 05:00:42 GMT +0000). [05:06:23] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.90, 3.40, 2.97 [05:06:34] PROBLEM - mw10 APT on mw10 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [05:07:11] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 9.70, 7.43, 5.09 [05:08:24] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.61, 3.20, 2.96 [05:08:32] RECOVERY - mw10 APT on mw10 is OK: APT OK: 16 packages available for upgrade (0 critical updates). [05:09:08] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.74, 5.99, 4.85 [05:09:10] PROBLEM - www.publictestwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'publictestwiki.com' expires in 7 day(s) (Wed 20 Oct 2021 05:07:03 GMT +0000). [05:12:21] PROBLEM - es.publictestwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'publictestwiki.com' expires in 7 day(s) (Wed 20 Oct 2021 05:07:03 GMT +0000). [05:13:50] PROBLEM - publictestwiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'publictestwiki.com' expires in 7 day(s) (Wed 20 Oct 2021 05:07:03 GMT +0000). [05:19:47] PROBLEM - wiki.mikrodev.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mikrodev.com' expires in 7 day(s) (Wed 20 Oct 2021 05:16:53 GMT +0000). [05:20:20] PROBLEM - wiki.ciptamedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.ciptamedia.org' expires in 7 day(s) (Wed 20 Oct 2021 05:13:35 GMT +0000). [05:25:54] PROBLEM - wiki.whentheycry.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.whentheycry.org' expires in 7 day(s) (Wed 20 Oct 2021 05:22:25 GMT +0000). [05:32:24] PROBLEM - wiki.climatechange.ai - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.climatechange.ai' expires in 7 day(s) (Wed 20 Oct 2021 05:28:07 GMT +0000). [05:36:41] PROBLEM - wiki.triplescripts.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.triplescripts.org' expires in 7 day(s) (Wed 20 Oct 2021 05:36:07 GMT +0000). [05:40:17] PROBLEM - wiki.lefrenchmelee.fr - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.lefrenchmelee.fr' expires in 7 day(s) (Wed 20 Oct 2021 05:33:31 GMT +0000). [05:45:09] PROBLEM - guia.esporo.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'guia.esporo.net' expires in 7 day(s) (Wed 20 Oct 2021 05:42:23 GMT +0000). [05:46:53] PROBLEM - spcodex.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'spcodex.wiki' expires in 7 day(s) (Wed 20 Oct 2021 05:43:44 GMT +0000). [05:56:38] PROBLEM - savagepedia.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'savagepedia.wiki' expires in 7 day(s) (Wed 20 Oct 2021 05:50:52 GMT +0000). [05:57:06] PROBLEM - wiki.raghuveer.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.raghuveer.net' expires in 7 day(s) (Wed 20 Oct 2021 05:56:29 GMT +0000). [06:05:55] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+1/-0/±8] 13https://git.io/JKerA [06:05:56] [02miraheze/ssl] 07Reception123 036c08183 - add scpvncommunitywiki.xyz cert [06:05:58] [02miraheze/ssl] 07Reception123 03d6daee7 - renew SSL certs [06:06:27] PROBLEM - wiki.s23.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.s23.org' expires in 7 day(s) (Wed 20 Oct 2021 06:02:20 GMT +0000). [06:08:11] PROBLEM - wiki.kourouklides.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.kourouklides.com' expires in 7 day(s) (Wed 20 Oct 2021 06:07:15 GMT +0000). [06:11:44] morning RhinosF1 [06:11:57] I'm actually just heading to bed, but thought I'd say hi before I left [06:12:25] Ok [06:12:28] Hi [06:13:21] PROBLEM - mwtask1 Puppet on mwtask1 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [06:13:36] good night dmehus :) [06:13:48] night, Agent :) [06:14:01] PROBLEM - wiki.mcpirevival.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mcpirevival.tk' expires in 7 day(s) (Wed 20 Oct 2021 06:09:35 GMT +0000). [06:15:03] Reception123: can I assume nginx alert is you? [06:15:31] RhinosF1: yes it's those private keys not matching the certs again [06:15:36] Ok [06:16:04] It will fix when puppet runs [06:17:46] Agent: you can join #miraheze-mediawiki-engineers [06:17:59] Assuming I'm awake and typed it right [06:19:51] PROBLEM - fc.songcontests.eu - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'fc.songcontests.eu' expires in 7 day(s) (Wed 20 Oct 2021 06:14:33 GMT +0000). [06:27:51] PROBLEM - runzeppelin.ru - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'runzeppelin.ru' expires in 7 day(s) (Wed 20 Oct 2021 06:21:53 GMT +0000). [06:29:29] RECOVERY - es.nocyclo.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'es.nocyclo.tk' will expire on Mon 10 Jan 2022 05:03:48 GMT +0000. [06:34:38] PROBLEM - wiki.cyberfurs.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.cyberfurs.org' expires in 7 day(s) (Wed 20 Oct 2021 06:31:20 GMT +0000). [06:35:05] PROBLEM - storytime.jdstroy.cf - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'storytime.jdstroy.cf' expires in 7 day(s) (Wed 20 Oct 2021 06:28:49 GMT +0000). [06:35:15] PROBLEM - wiki.valkyrienskies.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.valkyrienskies.org' expires in 7 day(s) (Wed 20 Oct 2021 06:34:55 GMT +0000). [06:36:00] PROBLEM - en.gyaanipedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'en.gyaanipedia.com' expires in 7 day(s) (Wed 20 Oct 2021 06:32:48 GMT +0000). [06:37:44] RECOVERY - history.sdtef.org - LetsEncrypt on sslhost is OK: OK - Certificate 'history.sdtef.org' will expire on Mon 10 Jan 2022 04:52:34 GMT +0000. [06:39:16] RECOVERY - guia.cineastas.pt - LetsEncrypt on sslhost is OK: OK - Certificate 'guia.cineastas.pt' will expire on Mon 10 Jan 2022 04:59:32 GMT +0000. [06:39:37] PROBLEM - franchise.franchising.org.ua - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'franchise.franchising.org.ua' expires in 7 day(s) (Wed 20 Oct 2021 06:33:38 GMT +0000). [06:40:36] RECOVERY - guia.esporo.net - LetsEncrypt on sslhost is OK: OK - Certificate 'guia.esporo.net' will expire on Mon 10 Jan 2022 04:56:14 GMT +0000. [06:41:33] RECOVERY - fr.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'fr.gyaanipedia.com' will expire on Mon 10 Jan 2022 05:01:26 GMT +0000. [06:41:50] PROBLEM - wiki.seredrau.xyz - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.seredrau.xyz' expires in 7 day(s) (Wed 20 Oct 2021 06:38:19 GMT +0000). [06:41:53] RECOVERY - encyclopediaofastrobiology.org - LetsEncrypt on sslhost is OK: OK - Certificate 'encyclopediaofastrobiology.org' will expire on Mon 10 Jan 2022 05:04:27 GMT +0000. [06:42:27] RECOVERY - mwtask1 Puppet on mwtask1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [06:42:46] RECOVERY - halloween.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'halloween.gyaanipedia.com' will expire on Mon 10 Jan 2022 04:54:59 GMT +0000. [06:45:03] PROBLEM - wiki.mastodon.kr - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mastodon.kr' expires in 7 day(s) (Wed 20 Oct 2021 06:42:45 GMT +0000). [06:46:55] PROBLEM - ao90.pinho.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'ao90.pinho.org' expires in 7 day(s) (Wed 20 Oct 2021 06:43:36 GMT +0000). [06:52:39] PROBLEM - wiki.villagecollaborative.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.villagecollaborative.net' expires in 7 day(s) (Wed 20 Oct 2021 06:50:42 GMT +0000). [06:57:05] PROBLEM - wiki.mobilityengineer.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mobilityengineer.com' expires in 7 day(s) (Wed 20 Oct 2021 06:54:28 GMT +0000). [07:05:12] PROBLEM - nonciclopedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'nonciclopedia.org' expires in 7 day(s) (Wed 20 Oct 2021 07:01:22 GMT +0000). [07:10:19] PROBLEM - icclopedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'icclopedia.org' expires in 7 day(s) (Wed 20 Oct 2021 07:08:29 GMT +0000). [07:10:36] PROBLEM - wiki.arnab.at - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.arnab.at' expires in 7 day(s) (Wed 20 Oct 2021 07:07:03 GMT +0000). [07:12:22] [02miraheze/ssl] 07Reception123 pushed 032 commits to 03master [+0/-0/±21] 13https://git.io/JKvqv [07:12:24] [02miraheze/ssl] 07Reception123 031ad481f - renew publicteswiki.com and remove es.* redirect [07:12:25] [02miraheze/ssl] 07Reception123 03fa70095 - renew SSL certs [07:12:41] PROBLEM - miraheze.ga - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'miraheze.ga' expires in 7 day(s) (Wed 20 Oct 2021 07:12:11 GMT +0000). [07:16:36] PROBLEM - wiki.nowchess.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.nowchess.org' expires in 7 day(s) (Wed 20 Oct 2021 07:13:13 GMT +0000). [07:17:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.78, 9.29, 5.95 [07:17:17] PROBLEM - cp13 Puppet on cp13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:18:18] PROBLEM - cp14 Puppet on cp14 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:18:29] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:18:40] PROBLEM - cp15 Puppet on cp15 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:18:45] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:18:49] PROBLEM - mw13 Puppet on mw13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:19:06] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:19:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 6.82, 8.00, 5.87 [07:19:29] PROBLEM - mw12 Puppet on mw12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [07:20:20] RECOVERY - cp14 Puppet on cp14 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [07:20:23] RECOVERY - www.dariawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dariawiki.org' will expire on Mon 10 Jan 2022 06:06:14 GMT +0000. [07:20:40] RECOVERY - www.publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Mon 10 Jan 2022 05:57:56 GMT +0000. [07:20:49] ah that's what I like to see [07:20:56] RECOVERY - antiguabarbudacalypso.com - LetsEncrypt on sslhost is OK: OK - Certificate 'antiguabarbudacalypso.com' will expire on Mon 10 Jan 2022 06:09:34 GMT +0000. [07:21:01] RECOVERY - mw13 Puppet on mw13 is OK: OK: Puppet is currently enabled, last run 34 seconds ago with 0 failures [07:21:18] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [07:21:20] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 11.96, 10.27, 7.03 [07:21:22] RECOVERY - franchise.franchising.org.ua - LetsEncrypt on sslhost is OK: OK - Certificate 'franchise.franchising.org.ua' will expire on Mon 10 Jan 2022 06:01:27 GMT +0000. [07:21:36] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 28.38, 24.04, 18.32 [07:21:37] RECOVERY - electowiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'electowiki.org' will expire on Mon 10 Jan 2022 06:04:54 GMT +0000. [07:22:16] RECOVERY - ao90.pinho.org - LetsEncrypt on sslhost is OK: OK - Certificate 'ao90.pinho.org' will expire on Mon 10 Jan 2022 06:08:59 GMT +0000. [07:22:25] RECOVERY - dariawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'dariawiki.org' will expire on Mon 10 Jan 2022 06:06:14 GMT +0000. [07:22:26] RECOVERY - mw12 Puppet on mw12 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures [07:22:27] RECOVERY - cp13 Puppet on cp13 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [07:22:43] RECOVERY - adadevelopersacademy.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'adadevelopersacademy.wiki' will expire on Mon 10 Jan 2022 06:11:54 GMT +0000. [07:22:53] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:22:57] RECOVERY - cp15 Puppet on cp15 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:23:00] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [07:23:08] RECOVERY - fc.songcontests.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'fc.songcontests.eu' will expire on Mon 10 Jan 2022 06:02:16 GMT +0000. [07:23:11] RECOVERY - en.gyaanipedia.com - LetsEncrypt on sslhost is OK: OK - Certificate 'en.gyaanipedia.com' will expire on Mon 10 Jan 2022 06:04:23 GMT +0000. [07:23:21] RECOVERY - beaconspace.unrestrictedlorefare.com - LetsEncrypt on sslhost is OK: OK - Certificate 'beaconspace.unrestrictedlorefare.com' will expire on Mon 10 Jan 2022 06:07:39 GMT +0000. [07:23:23] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 3.93, 7.86, 6.55 [07:23:30] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.68, 20.19, 17.60 [07:24:17] RECOVERY - archive.stellurgists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'archive.stellurgists.wiki' will expire on Mon 10 Jan 2022 06:08:22 GMT +0000. [07:24:36] RECOVERY - en.nocyclo.tk - LetsEncrypt on sslhost is OK: OK - Certificate 'en.nocyclo.tk' will expire on Mon 10 Jan 2022 06:03:44 GMT +0000. [07:24:44] PROBLEM - wiki.velaan.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.velaan.org' expires in 7 day(s) (Wed 20 Oct 2021 07:23:28 GMT +0000). [07:25:09] RECOVERY - www.allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Mon 10 Jan 2022 06:11:18 GMT +0000. [07:25:09] RECOVERY - cyberlaw.ccdcoe.org - LetsEncrypt on sslhost is OK: OK - Certificate 'cyberlaw.ccdcoe.org' will expire on Mon 10 Jan 2022 06:07:01 GMT +0000. [07:25:12] RECOVERY - housing.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'housing.wiki' will expire on Mon 10 Jan 2022 05:59:53 GMT +0000. [07:25:23] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 2.47, 6.07, 6.06 [07:25:42] RECOVERY - enc.for.uz - LetsEncrypt on sslhost is OK: OK - Certificate 'enc.for.uz' will expire on Mon 10 Jan 2022 06:02:55 GMT +0000. [07:25:45] RECOVERY - publictestwiki.com - LetsEncrypt on sslhost is OK: OK - Certificate 'publictestwiki.com' will expire on Mon 10 Jan 2022 05:57:56 GMT +0000. [07:25:47] RECOVERY - data.nonbinary.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'data.nonbinary.wiki' will expire on Mon 10 Jan 2022 06:05:39 GMT +0000. [07:25:53] PROBLEM - wiki.warframestat.us - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.warframestat.us' expires in 7 day(s) (Wed 20 Oct 2021 07:19:36 GMT +0000). [07:26:14] RECOVERY - hololive.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'hololive.wiki' will expire on Mon 10 Jan 2022 06:00:29 GMT +0000. [07:26:34] RECOVERY - aman.awiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'aman.awiki.org' will expire on Mon 10 Jan 2022 06:10:13 GMT +0000. [07:26:35] RECOVERY - index.pdcommunity.ir - LetsEncrypt on sslhost is OK: OK - Certificate 'index.pdcommunity.ir' will expire on Mon 10 Jan 2022 05:59:14 GMT +0000. [07:26:45] RECOVERY - allthetropes.org - LetsEncrypt on sslhost is OK: OK - Certificate 'allthetropes.org' will expire on Mon 10 Jan 2022 06:11:18 GMT +0000. [07:34:11] PROBLEM - dc-multiverse.dcwikis.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'dc-multiverse.dcwikis.com' expires in 7 day(s) (Wed 20 Oct 2021 07:30:56 GMT +0000). [07:38:20] PROBLEM - www.opendatascot.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'opendatascot.org' expires in 7 day(s) (Wed 20 Oct 2021 07:37:37 GMT +0000). [07:42:53] PROBLEM - opendatascot.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'opendatascot.org' expires in 7 day(s) (Wed 20 Oct 2021 07:37:37 GMT +0000). [07:45:18] PROBLEM - battlecats.f5.si - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'battlecats.f5.si' expires in 7 day(s) (Wed 20 Oct 2021 07:42:49 GMT +0000). [07:52:36] PROBLEM - history.estill.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'history.estill.org' expires in 7 day(s) (Wed 20 Oct 2021 07:46:26 GMT +0000). [07:56:46] PROBLEM - www.marinebiodiversitymatrix.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'marinebiodiversitymatrix.org' expires in 7 day(s) (Wed 20 Oct 2021 07:54:09 GMT +0000). [07:57:25] PROBLEM - es.gyaanipedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'es.gyaanipedia.com' expires in 7 day(s) (Wed 20 Oct 2021 07:55:00 GMT +0000). [08:00:10] PROBLEM - marinebiodiversitymatrix.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'marinebiodiversitymatrix.org' expires in 7 day(s) (Wed 20 Oct 2021 07:54:09 GMT +0000). [08:00:17] PROBLEM - wiki.ripto.gq - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.ripto.gq' expires in 7 day(s) (Wed 20 Oct 2021 07:58:32 GMT +0000). [08:06:10] PROBLEM - lama.madooa.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'lama.madooa.org' expires in 7 day(s) (Wed 20 Oct 2021 08:04:30 GMT +0000). [08:16:55] PROBLEM - wiki.minkyu.kim - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.minkyu.kim' expires in 7 day(s) (Wed 20 Oct 2021 08:13:18 GMT +0000). [08:17:01] PROBLEM - wiki.mxlinuxusers.de - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mxlinuxusers.de' expires in 7 day(s) (Wed 20 Oct 2021 08:11:12 GMT +0000). [08:31:03] PROBLEM - vmcodex.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'vmcodex.net' expires in 7 day(s) (Wed 20 Oct 2021 08:25:37 GMT +0000). [08:32:01] PROBLEM - bebaskanpengetahuan.id - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'bebaskanpengetahuan.id' expires in 7 day(s) (Wed 20 Oct 2021 08:27:11 GMT +0000). [08:39:26] PROBLEM - wiki.kali-team.cn - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.kali-team.cn' expires in 7 day(s) (Wed 20 Oct 2021 08:33:24 GMT +0000). [08:42:50] PROBLEM - wiki.macc.nyc - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.macc.nyc' expires in 7 day(s) (Wed 20 Oct 2021 08:40:43 GMT +0000). [08:55:30] PROBLEM - wiki.counterculturelabs.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.counterculturelabs.org' expires in 7 day(s) (Wed 20 Oct 2021 08:50:36 GMT +0000). [09:01:32] PROBLEM - wiki.minecraftathome.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.minecraftathome.com' expires in 7 day(s) (Wed 20 Oct 2021 08:57:08 GMT +0000). [09:05:25] PROBLEM - de.gyaanipedia.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'de.gyaanipedia.com' expires in 7 day(s) (Wed 20 Oct 2021 09:04:41 GMT +0000). [09:07:13] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.98, 6.70, 4.29 [09:09:16] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 2.88, 5.15, 4.01 [09:15:25] PROBLEM - miraheze.gq - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'miraheze.gq' expires in 7 day(s) (Wed 20 Oct 2021 09:09:01 GMT +0000). [09:19:20] PROBLEM - podpedia.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'podpedia.org' expires in 7 day(s) (Wed 20 Oct 2021 09:16:52 GMT +0000). [09:26:42] PROBLEM - pastport.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'pastport.org' expires in 7 day(s) (Wed 20 Oct 2021 09:21:58 GMT +0000). [09:30:32] PROBLEM - idleon.info - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'idleon.info' expires in 7 day(s) (Wed 20 Oct 2021 09:27:12 GMT +0000). [09:33:47] PROBLEM - wiki.tulpa.info - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.tulpa.info' expires in 7 day(s) (Wed 20 Oct 2021 09:32:46 GMT +0000). [09:38:58] PROBLEM - www.mh142.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'mh142.com' expires in 7 day(s) (Wed 20 Oct 2021 09:36:09 GMT +0000). [09:40:35] PROBLEM - mh142.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'mh142.com' expires in 7 day(s) (Wed 20 Oct 2021 09:36:09 GMT +0000). [09:48:54] PROBLEM - oecumene.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'oecumene.org' expires in 7 day(s) (Wed 20 Oct 2021 09:43:18 GMT +0000). [09:53:36] PROBLEM - wiki.ssij.se - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.ssij.se' expires in 7 day(s) (Wed 20 Oct 2021 09:50:55 GMT +0000). [09:58:28] PROBLEM - wiki.mussa.id - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mussa.id' expires in 7 day(s) (Wed 20 Oct 2021 09:52:43 GMT +0000). [10:07:01] PROBLEM - bharatwiki.online - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'bharatwiki.online' expires in 7 day(s) (Wed 20 Oct 2021 10:00:10 GMT +0000). [10:13:23] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.31, 4.01, 3.42 [10:15:23] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.13, 3.33, 3.25 [10:16:17] PROBLEM - nenawiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'nenawiki.org' expires in 7 day(s) (Wed 20 Oct 2021 10:15:44 GMT +0000). [10:19:03] PROBLEM - www.permanentfuturelab.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'permanentfuturelab.wiki' expires in 7 day(s) (Wed 20 Oct 2021 10:18:52 GMT +0000). [10:19:53] PROBLEM - permanentfuturelab.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'permanentfuturelab.wiki' expires in 7 day(s) (Wed 20 Oct 2021 10:18:52 GMT +0000). [10:21:52] PROBLEM - wiki.redeemer.live - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.redeemer.live' expires in 7 day(s) (Wed 20 Oct 2021 10:19:55 GMT +0000). [10:22:23] PROBLEM - ndg.nenawiki.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'nenawiki.org' expires in 7 day(s) (Wed 20 Oct 2021 10:15:44 GMT +0000). [10:26:46] PROBLEM - wiki.make717.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.make717.org' expires in 7 day(s) (Wed 20 Oct 2021 10:20:43 GMT +0000). [10:32:08] PROBLEM - wiki.mineland.eu - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.mineland.eu' expires in 7 day(s) (Wed 20 Oct 2021 10:27:23 GMT +0000). [10:35:02] PROBLEM - wiki.thunis.eu - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.thunis.eu' expires in 7 day(s) (Wed 20 Oct 2021 10:34:06 GMT +0000). [10:37:30] PROBLEM - speleo.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'speleo.wiki' expires in 7 day(s) (Wed 20 Oct 2021 10:31:05 GMT +0000). [10:37:45] PROBLEM - wiki.danvs.net - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.danvs.net' expires in 7 day(s) (Wed 20 Oct 2021 10:36:05 GMT +0000). [10:44:22] PROBLEM - nonbinary.wiki - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'nonbinary.wiki' expires in 7 day(s) (Wed 20 Oct 2021 10:43:12 GMT +0000). [10:49:10] PROBLEM - iesd.mobagenie.my.id - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'iesd.mobagenie.my.id' expires in 7 day(s) (Wed 20 Oct 2021 10:48:07 GMT +0000). [12:33:12] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 5.96, 6.08, 4.65 [12:35:11] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.68, 5.50, 4.60 [12:37:11] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.98, 4.65, 4.40 [12:37:19] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.19, 20.89, 17.14 [12:37:30] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.26, 6.36, 4.65 [12:39:18] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 13.97, 18.66, 16.82 [12:39:28] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 5.59, 6.23, 4.83 [13:03:10] RECOVERY - froggy.info - LetsEncrypt on sslhost is OK: OK - Certificate 'froggy.info' will expire on Mon 10 Jan 2022 11:56:12 GMT +0000. [13:20:33] PROBLEM - archive.a2b2.org - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'archive.a2b2.org' expires in 7 day(s) (Wed 20 Oct 2021 13:17:46 GMT +0000). [13:37:50] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 6.22, 7.21, 5.39 [13:39:49] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.36, 6.35, 5.28 [14:00:24] !log continue testing on db12 for https://phabricator.miraheze.org/T5877 [14:00:25] [url] ⚓ T5877 Revise MariaDB backup strategy | phabricator.miraheze.org [14:00:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:08:17] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.11, 8.38, 5.93 [14:10:07] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 7.30, 4.84, 3.94 [14:10:16] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.68, 7.99, 6.07 [14:12:05] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.62, 5.06, 4.13 [14:14:04] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.51, 4.77, 4.13 [14:14:14] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.60, 6.34, 5.85 [14:26:06] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.60, 5.44, 4.72 [14:28:06] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.20, 7.13, 5.44 [14:35:42] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 29.54, 23.71, 20.27 [14:37:36] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 20.10, 22.63, 20.32 [14:38:13] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.42, 7.48, 6.10 [14:38:14] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.63, 5.74, 5.53 [14:40:11] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.65, 6.42, 5.87 [14:42:13] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.71, 6.33, 5.75 [14:42:49] !log adjust w/.git/config on test3 to pull REL1_37 [14:42:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [14:44:15] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 20.92, 19.18, 17.07 [14:45:19] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 19.11, 19.46, 19.50 [14:46:11] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.70, 18.63, 17.12 [14:46:12] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.47, 5.60, 5.60 [14:52:12] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.60, 4.18, 4.96 [14:55:30] !log checkout 1.37 to diff/grep sql changes [14:55:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:10:16] [02miraheze/puppet] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKT05 [15:10:18] [02miraheze/puppet] 07Reception123 03228a3de - REL1_37 for test3 [15:10:19] ^ RhinosF1 [15:12:44] !log rhinos@test3:/srv/mediawiki-staging/w/maintenance/archives$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=test3wiki patch-objectcache-modtoken.sql [15:12:47] Reception123: pulled? [15:12:49] PROBLEM - test3 Puppet on test3 is WARNING: WARNING: Puppet is currently disabled, message: Rf1, last run 9 minutes ago with 0 failures [15:12:56] !log rhinos@test3:/srv/mediawiki-staging/w/maintenance/archives$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php --wiki=test3wiki patch-oldimage-oi_timestamp.sql [15:13:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:13:07] Reception123: them 2 can be ran on all wikis [15:13:15] RhinosF1: just now yes [15:13:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:13:50] !log reclone w to tidy up [15:14:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:15:08] RECOVERY - test3 Puppet on test3 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [15:15:23] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 34.22, 29.86, 22.64 [15:15:33] !log updating test3 staging to 1.37 properly [15:15:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:15:53] PROBLEM - mw11 Current Load on mw11 is CRITICAL: CRITICAL - load average: 10.69, 10.44, 7.07 [15:16:21] PROBLEM - mw13 Current Load on mw13 is CRITICAL: CRITICAL - load average: 8.35, 9.47, 6.79 [15:19:16] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 17.19, 23.83, 21.87 [15:19:46] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 5.19, 7.94, 6.85 [15:20:21] PROBLEM - mw13 Current Load on mw13 is WARNING: WARNING - load average: 5.83, 7.71, 6.67 [15:21:16] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 29.31, 27.03, 23.30 [15:21:29] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKTVa [15:21:31] [02miraheze/mw-config] 07RhinosF1 03a7a7dbc - Revert "Revert "Update LocalSettings.php"" [15:21:38] Reception123: there ^ [15:21:53] the message is wrong but it'll do it [15:22:06] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.84, 3.90, 3.38 [15:22:22] ack [15:22:41] miraheze/mw-config - RhinosF1 the build passed. [15:23:13] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 19.18, 23.95, 22.61 [15:23:46] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 4.72, 6.22, 6.40 [15:24:02] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.04, 3.77, 3.39 [15:24:10] RECOVERY - mw13 Current Load on mw13 is OK: OK - load average: 5.16, 6.70, 6.53 [15:25:10] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 27.04, 25.54, 23.36 [15:25:58] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.57, 3.67, 3.40 [15:27:44] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 7.09, 6.46, 5.72 [15:27:55] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.21, 3.19, 3.25 [15:29:07] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 20.94, 23.42, 23.02 [15:29:53] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 10.02, 7.16, 6.03 [15:31:11] !log [@test3] starting deploy of {'config': True} to skip [15:31:12] !log [@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [15:31:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:31:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:31:52] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 4.76, 6.41, 5.92 [15:31:57] !log [rhinos@test3] starting deploy of {'world': True, 'l10n': True, 'gitinfo': True, 'force': True} to skip [15:32:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:32:13] Agent, Reception123: ^ [15:35:02] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 25.35, 21.43, 22.03 [15:35:04] !log [@mw11] starting deploy of {'config': True} to all [15:35:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:35:39] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 34s [15:35:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:02] !log [rhinos@test3] finished deploy of {'world': True, 'l10n': True, 'gitinfo': True, 'force': True} to skip - FAIL: [0, 0, 0, 65280, 65280, 65280] in 244s [15:36:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:36:39] Reception123: Error: Call to a member function readCache() on null [15:36:57] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.84, 22.35, 22.36 [15:37:49] :( [15:37:53] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 4.91, 7.03, 6.39 [15:38:17] PROBLEM - test3 MediaWiki Rendering on test3 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 638 bytes in 0.120 second response time [15:38:53] oh wait [15:38:55] update CW [15:39:51] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.56, 5.87, 6.04 [15:41:22] RhinosF1: ^ [15:42:11] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKTy5 [15:42:13] [02miraheze/mediawiki] 07RhinosF1 03da65bef - Update CreateWiki [15:42:28] !log [rhinos@test3] starting deploy of {'world': True, 'l10n': True, 'gitinfo': True, 'force': True} to skip [15:42:32] rerunning [15:42:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:42:47] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 14.42, 16.72, 19.78 [15:45:39] * RhinosF1 is surprised we're not up [15:46:24] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKTHu [15:46:26] [02miraheze/mediawiki] 07paladox 03913f095 - Update DynamicPageList3 [15:49:31] paladox: should I abort & rerun? [15:49:37] No [15:49:47] ok [15:49:57] !log [rhinos@test3] finished deploy of {'world': True, 'l10n': True, 'gitinfo': True, 'force': True} to skip - SUCCESS in 449s [15:50:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:50:02] sync is up to da65bef - Update CreateWiki [15:50:07] new error [15:50:08] https://graylog.miraheze.org/messages/graylog_331/9ed734a0-2b73-11ec-a12b-0200001a24a4 [15:50:21] wikibase related [15:50:37] that's wikibase [15:51:12] https://github.com/wikimedia/Wikibase/commit/c452d99f7ef447ee327bd1e579ec18b8b61ca68c [15:51:12] [url] Remove WikibaseRepo foreignRepositories setting · wikimedia/Wikibase@c452d99 · GitHub | github.com [15:51:46] i'll disable for now [15:53:36] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.89, 3.45, 3.17 [15:53:39] paladox: can't disable because of the fatal [15:53:46] yeh [15:53:53] * RhinosF1 needs to drp [15:53:55] drop* [15:53:57] you can cross it out in the config tho [15:54:02] Reception123: ^ [15:56:18] hm? can't you? [15:56:23] https://github.com/wikimedia/Wikibase/commit/c452d99f7ef447ee327bd1e579ec18b8b61ca68c#diff-0774a6eb0ebde59937be3faab5fe91a308cde433fa3299df542469983934e62e [15:56:24] [url] Remove WikibaseRepo foreignRepositories setting · wikimedia/Wikibase@c452d99 · GitHub | github.com [15:56:28] Reception123: need to make tea [15:56:39] you just need to hack the config on test32 [15:56:41] *test3 [15:57:06] oh [15:57:07] wait [15:57:11] things were changed [15:57:18] so you can no longer do that [15:57:25] MW/CW automatically load extensions [15:57:28] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.01, 3.17, 3.12 [15:57:55] ah yes, with CosmicAlpha 's changes r [15:58:43] paladox: I know MW auto loads [15:58:50] But why is that at fault [15:59:12] what do you mean at fault? [15:59:44] paladox: how as MW/CW autoloading got to do with WikiBase [15:59:54] you carn't disable it easily [16:00:11] you'll need to manually do it in the db. [16:00:23] Reception123: can you comment out like 8/9 [16:00:30] paladox: disabling still works [16:00:39] From the script [16:00:48] It's just fataling before setup [16:00:56] don't think so as it'll still fail in the script [16:01:18] Reception123: line*, WikiBase.php [16:01:19] Locally [16:01:38] paladox: yeah but that's because it's a setup failure [16:01:46] As far as I know that would have happened before [16:02:23] currently looking at something else, it's not that urgent so you can get to it after you make your tea unless I finish before that, but tbh I'm not sure what I need to be commenting out anyway [16:03:15] Ok [16:03:39] Ok [16:04:49] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JKTp7 [16:04:51] [02miraheze/mw-config] 07paladox 03c8fb06c - Add support for MW 1.37 in wikibase [16:04:52] [02mw-config] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbvb3 [16:04:54] [02mw-config] 07paladox opened pull request 03#4143: Add support for MW 1.37 in wikibase - 13https://git.io/JKTpF [16:05:54] miraheze/mw-config - paladox the build passed. [16:05:57] paladox: I think entityNamespaces is affected too from the commit [16:06:06] yeh it was removed [16:06:20] but we can keep it without changing anything as it is basically a noop. [16:06:24] Ok [16:06:30] Feel free to merge & sync [16:07:12] Well run puppet [16:08:34] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.19, 7.68, 5.42 [16:08:56] * RhinosF1 is going to merge [16:09:07] [02mw-config] 07RhinosF1 closed pull request 03#4143: Add support for MW 1.37 in wikibase - 13https://git.io/JKTpF [16:09:08] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKket [16:09:10] [02miraheze/mw-config] 07paladox 03e9cfbf6 - Add support for MW 1.37 in wikibase (#4143) [16:09:11] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JKkeq [16:09:13] [02miraheze/mw-config] 07paladox 03315046b - Update Wikibase.php [16:09:48] huh [16:09:51] [02mw-config] 07RhinosF1 opened pull request 03#4144: Paladox patch 1 - 13https://git.io/JKkea [16:09:54] RhinosF1: why did you merge that [16:10:12] miraheze/mw-config - RhinosF1 the build passed. [16:10:30] paladox: what do you want me to do [16:10:32] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.44, 6.08, 5.11 [16:11:00] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JKkv8 [16:11:01] [02miraheze/mw-config] 07paladox 038722f90 - Update Wikibase.php [16:11:03] [02mw-config] 07paladox created branch 03paladox-patch-2 - 13https://git.io/vbvb3 [16:11:04] [02mw-config] 07paladox opened pull request 03#4145: Update Wikibase.php - 13https://git.io/JKkvR [16:12:03] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.18, 4.97, 4.23 [16:12:04] miraheze/mw-config - paladox the build passed. [16:13:22] new error https://graylog.miraheze.org/messages/graylog_331/2dcec171-2b77-11ec-a12b-0200001a24a4 [16:13:37] [02miraheze/mw-config] 07paladox pushed 031 commit to 03paladox-patch-2 [+0/-0/±1] 13https://git.io/JKkf9 [16:13:39] [02miraheze/mw-config] 07paladox 03df5b48a - Update Wikibase.php [16:13:40] [02mw-config] 07paladox synchronize pull request 03#4145: Update Wikibase.php - 13https://git.io/JKkvR [16:13:46] [02mw-config] 07paladox closed pull request 03#4145: Update Wikibase.php - 13https://git.io/JKkvR [16:13:47] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKkfb [16:13:49] [02miraheze/mw-config] 07paladox 03f78792a - Update Wikibase.php (#4145) [16:13:50] [02mw-config] 07paladox deleted branch 03paladox-patch-2 - 13https://git.io/vbvb3 [16:13:52] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-2 [16:14:02] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 2.26, 4.03, 3.98 [16:14:15] Reception123: RhinosF1 https://graylog.miraheze.org/messages/graylog_331/2dcec171-2b77-11ec-a12b-0200001a24a4 [16:14:15] paladox: don't you need to sync the WikiBase fixes? [16:14:19] But what I'm mobile [16:14:25] no [16:14:29] i just manually did it [16:14:38] paladox: but we pull to staging [16:14:42] because cannot be arsed syncing when i can perfectly test by hacking [16:14:45] miraheze/mw-config - paladox the build passed. [16:14:46] Ok [16:14:48] this is not a production server [16:14:49] Fair enough [16:14:52] if you want to sync you can [16:14:56] Meh [16:14:57] miraheze/mw-config - paladox the build passed. [16:15:00] As long as it's deployed [16:15:11] Puppet will make itself happy [16:15:22] paladox: can you paste as I'm mobile [16:15:39] I’ve just gone mobile [16:15:48] But basically was an error in mw core [16:15:53] Looks like php types issues [16:16:09] Ok [16:16:10] Fun [16:16:24] TypeError [16:16:34] That's probably bad [16:16:36] Argument 3 passed to MediaWiki\Watchlist\WatchlistManager::__construct() must be an instance of MediaWiki\Permissions\PermissionManager, instance of ReadOnlyMode given, called in /srv/mediawiki/w/includes/ServiceWiring.php on line 1804 [16:17:31] paladox: hmm [16:18:01] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 23.58, 20.46, 18.56 [16:18:04] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 16.74, 9.35, 5.84 [16:18:49] PROBLEM - mw8 Current Load on mw8 is CRITICAL: CRITICAL - load average: 9.27, 9.19, 5.87 [16:18:59] Can we just turn read only off [16:20:05] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.97, 19.73, 18.55 [16:21:00] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 4.15, 7.18, 5.58 [16:22:25] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 11.76, 10.19, 6.50 [16:22:27] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 3.53, 7.33, 6.01 [16:22:33] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 26.19, 19.97, 17.60 [16:23:03] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.28, 5.82, 5.27 [16:23:29] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.68, 3.06, 2.74 [16:24:26] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.17, 5.95, 5.66 [16:24:28] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 17.65, 18.60, 17.36 [16:26:23] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 3.06, 6.86, 6.00 [16:27:21] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.04, 3.16, 2.86 [16:28:24] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.45, 5.69, 5.68 [16:33:10] !log [@test3] starting deploy of {'config': True} to skip [16:33:11] !log [@test3] DEPLOY ABORTED: Canary check failed for localhost [16:33:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:33:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:25] !log [@mw11] starting deploy of {'config': True} to all [16:34:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:34:44] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 18s [16:34:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:35:27] PROBLEM - test3 Puppet on test3 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[MediaWiki Config Sync] [16:35:55] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKkO2 [16:35:56] [02miraheze/mw-config] 07RhinosF1 03c17cedc - Revert "Revert "Revert "Update LocalSettings.php""" [16:36:20] bringing it out of read only [16:36:49] !log [@test3] starting deploy of {'config': True} to skip [16:36:50] !log [@test3] DEPLOY ABORTED: Canary check failed for localhost [16:36:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:36:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:37:01] miraheze/mw-config - RhinosF1 the build passed. [16:43:57] paladox: I've pulled out of ReadOnlyMode and it still happens [16:44:53] Guess a bug still happens somewhere [16:47:39] paladox: why would an instance of ReadOnlyMode be passed in read-writte [16:47:42] majavah: ? [16:47:51] https://phabricator.wikimedia.org/T293113 [16:47:52] [url] ⚓ T293113 Argument 3 passed to MediaWiki\Watchlist\WatchlistManager::__construct() must be an instance of MediaWiki\Permissions\PermissionManager, instance of ReadOnlyMode given, called in /srv/mediaw | phabricator.wikimedia.org [16:48:36] ReadOnlyMode is the service that tells other components whether MW is in RO or not [16:48:43] sounds like something has an outdated constructor [16:48:53] majavah: I reverted the change though [16:48:59] it's in RW [16:49:03] but weird [16:49:20] oh you mean it's a check [16:49:30] ReadOnlyMode will be constructed and used even if MW is in RW, that class literally is the logic that checks for that [16:49:35] ah [16:50:30] majavah: any idea how to know what? [16:51:22] which branch is that on? [16:51:52] majavah: REL1_37 [16:52:17] https://github.com/miraheze/mediawiki/tree/REL1_37 [16:52:18] [url] GitHub - miraheze/mediawiki at REL1_37 | github.com [16:52:20] REL1_37 "looks" correct [16:52:25] [02miraheze/mediawiki] 07RhinosF1 pushed 033 commits to 03REL1_37 [+0/-0/±6] 13https://git.io/JKkWH [16:52:27] [02miraheze/mediawiki] 07hartman 0373d9a57 - Update git submodules [16:52:28] [02miraheze/mediawiki] 07cscott 0328a5ecc - Doc update: MW 1.36 included parsoid 0.13.1; we're now at 0.14.0-a19 [16:52:30] [02miraheze/mediawiki] 07RhinosF1 0312b7e26 - Merge branch 'wikimedia:REL1_37' into REL1_37 [16:53:04] if I had to guess, you have includes/ServiceWiring.php from 1.36 but includes/watchlist/WatchlistManager.php from 1.37 [16:53:10] hmm [16:53:22] * RhinosF1 reruns the deploy tool [16:53:43] I suggest you look at the actual files first to see if that is the case [16:53:48] would forgetting --ignore-time cause that [16:54:06] !log [rhinos@test3] DEPLOY ABORTED: Canary check failed for localhost [16:54:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:54:17] not sure about how your deploy system works, sorry [16:54:57] You may be right [16:55:19] Still a type error on a second deploy [16:55:45] I will diff [17:00:04] !log testing on db12, slightly more intensive conditions this time - closely monitoring load/cpu/memory usages [17:00:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:01:31] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKkRR [17:01:32] [02miraheze/mediawiki] 07RhinosF1 032cd310c - manually apply missing changes [17:02:27] !log [rhinos@test3] starting deploy of {'files': 'w/includes/watchlist/WatchlistManager.php'} to skip [17:02:29] !log [rhinos@test3] finished deploy of {'files': 'w/includes/watchlist/WatchlistManager.php'} to skip - SUCCESS in 1s [17:02:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:02:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:03:16] RECOVERY - test3 MediaWiki Rendering on test3 is OK: HTTP OK: HTTP/1.1 200 OK - 20557 bytes in 0.454 second response time [17:03:22] RECOVERY - test3 Puppet on test3 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures [17:03:53] majavah: ^ [17:04:14] now Call to undefined method MagicWord::get() from /srv/mediawiki/w/extensions/NumberedHeadings/NumberedHeadings.class.php:28 [17:04:17] !log [@mw11] starting deploy of {'config': True} to all [17:04:17] fix your monitoring, https://test3.miraheze.org/wiki/Main_Page still gives me a "Fatal exception of type "Error"" [17:04:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:04:47] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 30s [17:04:51] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:04:56] majavah: what status code does mediawiki give [17:05:08] 500 [17:05:27] yeah weird [17:05:32] icinga-miraheze: you stupid [17:05:47] majavah: is that a broken extension i should murder [17:05:47] ah, your check is for login.miraheze.org, which probably does not have that extension [17:06:25] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 7.43, 5.58, 4.15 [17:06:56] you're pulling that extension from a github repo untouched since 2019, so maybe [17:08:16] **** now Call to undefined method MagicWord::get() from /srv/mediawiki/w/extensions/NumberedHeadings/NumberedHeadings.class.php:28 [17:08:23] Check out the pull I did for it [17:08:24] paladox: yes [17:08:27] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.38, 7.00, 4.86 [17:08:29] where? [17:08:44] although https://github.com/wikimedia/mediawiki/commit/112b6f305b01ef8e50676ca6d7f7cd5b8b43610c should have happened a long time ago, so not sure why it was not broken in 1.36 [17:08:44] [url] Drop MagicWord static getters, deprecated in 1.32 · wikimedia/mediawiki@112b6f3 · GitHub | github.com [17:10:29] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 3.46, 5.58, 4.60 [17:10:36] https://github.com/staspika/mediawiki-numberedheadings/pull/3 [17:10:37] [url] MagicWord -> MagicWordFactory by paladox · Pull Request #3 · staspika/mediawiki-numberedheadings · GitHub | github.com [17:11:51] RhinosF1: [17:12:33] paladox: how do you unset an extension [17:12:45] You carnt [17:12:54] I've removed from MWE, ext-list, cache and it still shows itself [17:13:35] Remove it in the db for that wiki? [17:13:47] paladox: but it's not even installed [17:13:57] MW or CW load all extensions [17:14:08] in the 'mediawiki' repo [17:14:11] Need a blacklist function [17:14:30] *config [17:14:42] ergh [17:15:50] Why not just fix the problem [17:15:54] Then trying to work around [17:16:01] I have a pull [17:16:10] Check out the branch like we did on the 1.36 [17:16:28] using forks should be logged [17:18:02] Not really [17:18:21] how are we supposed to know you hacked a patch in [17:20:04] Looking at the history? Tbf that is hard but my first look wouldn’t be looking at the log because to me that’s for actions done on a server not a branch checked out within a git repo [17:20:51] not SAL [17:20:56] but somewhere [17:21:04] like the non-rel list we now have [17:21:16] (it didn't exist then fwiw) [17:21:24] Oh [17:22:25] * RhinosF1 is still confused how it errors with the code currently deleted [17:23:11] [02miraheze/ssl] 07Reception123 pushed 037 commits to 03master [+0/-6/±20] 13https://git.io/JKkod [17:23:13] [02miraheze/ssl] 07Reception123 031fe4161 - renew SSL certs [17:23:14] [02miraheze/ssl] 07Reception123 0373b6bf5 - remove iesd.mobagenie.my.id (wiki deleted) [17:23:16] [02miraheze/ssl] 07Reception123 031ba8708 - remove wiki.mussa.id cert (wiki deleted) [17:23:17] [02miraheze/ssl] ... and 4 more commits. [17:24:13] RhinosF1: what error [17:25:04] paladox: on test3 [17:25:24] ok, i didn't ask which server i said what error :) [17:25:45] paladox: the big fatal [17:26:16] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKk6B [17:26:18] [02miraheze/mediawiki] 07paladox 03a621367 - Update NumberedHeadings to ada69e7fd80bc01bcfd518588559a4ff23f51834 [17:27:30] CosmicAlpha: can you deploy ? ^ [17:28:00] !log [paladox@test3] starting deploy of {'world': True, 'gitinfo': True} to skip [17:28:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:28:07] Or Paladox [17:28:12] paladox: do config too [17:29:34] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JKkPU [17:29:36] [02miraheze/mediawiki] 07RhinosF1 03bfcdfce - Update non-rel-extensions [17:30:40] !log [paladox@test3] finished deploy of {'world': True, 'gitinfo': True} to skip - SUCCESS in 159s [17:30:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:30:56] new error [17:30:57] https://graylog.miraheze.org/messages/graylog_331/1cded070-2b82-11ec-a12b-0200001a24a4 [17:31:00] [02miraheze/mediawiki] 07RhinosF1 pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKkP9 [17:31:02] [02miraheze/mediawiki] 07RhinosF1 037f00d9b - Update non-rel-extensions [17:31:17] https://www.irccloud.com/pastebin/3bGt40fd/ [17:31:36] That's a schema change [17:32:30] PROBLEM - cp13 Puppet on cp13 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 3 minutes ago with 3 failures. Failed resources (up to 3 shown): File[madgenderscience.wiki_private],File[iesd.mobagenie.my.id_private],File[wiki.mussa.id_private] [17:33:50] paladox: not sure why Diff didn't see it [17:33:55] https://github.com/wikimedia/mediawiki-extensions-CommentStreams/blob/REL1_37/sql/mysql/cs_comments.sql [17:33:56] [url] mediawiki-extensions-CommentStreams/cs_comments.sql at REL1_37 · wikimedia/mediawiki-extensions-CommentStreams · GitHub | github.com [17:34:21] bit weird how test3 has none of the cs_* tables [17:34:29] oh [17:34:30] it does [17:34:39] CosmicAlpha: how do we migrate [17:35:02] PROBLEM - wiki.erisly.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.erisly.com' expires in 7 day(s) (Wed 20 Oct 2021 17:30:49 GMT +0000). [17:35:35] https://github.com/wikimedia/mediawiki-extensions-CommentStreams/commit/1d486daaec0ffb8d828d502f8229e955bfb869f1#diff-783685e84d43b6c3a188d5ea83525671cc775fd083b474fed90c9f36de89566b [17:35:35] [url] Convert to abstract schema and split comments and replies in DB · wikimedia/mediawiki-extensions-CommentStreams@1d486da · GitHub | github.com [17:35:51] running [17:35:55] * RhinosF1 has to go [17:36:02] paladox: there's the list in security [17:36:13] But they are all defaults & indexes so should be fine [17:36:18] !log db12: Create cs_comments for test3wiki [17:36:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:37:00] paladox: https://github.com/wikimedia/mediawiki-extensions-CommentStreams/blob/REL1_37/maintenance/MigrateToAbstractSchema.php I think. [17:37:01] [url] mediawiki-extensions-CommentStreams/MigrateToAbstractSchema.php at REL1_37 · wikimedia/mediawiki-extensions-CommentStreams · GitHub | github.com [17:37:17] RECOVERY - nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Mon 10 Jan 2022 16:08:25 GMT +0000. [17:37:34] !log [paladox@test3] sudo -u www-data php /srv/mediawiki/w/extensions/CommentStreams/maintenance/MigrateToAbstractSchema.php --wiki test3wiki [17:37:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:37:46] RECOVERY - wiki.danvs.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.danvs.net' will expire on Mon 10 Jan 2022 16:04:26 GMT +0000. [17:37:54] RECOVERY - oecumene.org - LetsEncrypt on sslhost is OK: OK - Certificate 'oecumene.org' will expire on Mon 10 Jan 2022 16:14:03 GMT +0000. [17:37:56] Test3 works now [17:38:50] RECOVERY - wiki.mussa.id - LetsEncrypt on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [17:39:12] /usr/lib/python3/dist-packages/urllib3/connectionpool.py:1015: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme-v02.api.letsencrypt.org'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings [17:39:12] [url] Advanced Usage - urllib3 2.0.0.dev0 documentation | urllib3.readthedocs.io [17:39:14] RECOVERY - wiki.mineland.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.mineland.eu' will expire on Mon 10 Jan 2022 16:06:27 GMT +0000. [17:39:19] ^ paladox I wonder, why have we never got rid of this warning? [17:39:32] RECOVERY - archive.a2b2.org - LetsEncrypt on sslhost is OK: OK - Certificate 'archive.a2b2.org' will expire on Mon 10 Jan 2022 16:01:51 GMT +0000. [17:39:55] i haven't looked into it as it was harmless [17:40:04] RECOVERY - www.permanentfuturelab.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'permanentfuturelab.wiki' will expire on Mon 10 Jan 2022 16:11:54 GMT +0000. [17:40:25] guess we could have a low prio task to get it done at some point [17:40:45] RECOVERY - wiki.make717.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.make717.org' will expire on Mon 10 Jan 2022 16:07:09 GMT +0000. [17:40:53] RECOVERY - permanentfuturelab.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'permanentfuturelab.wiki' will expire on Mon 10 Jan 2022 16:11:54 GMT +0000. [17:41:02] CosmicAlpha: do you want me to do the rest of the schema changes this evening [17:41:20] RECOVERY - speleo.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'speleo.wiki' will expire on Mon 10 Jan 2022 16:05:11 GMT +0000. [17:41:53] RECOVERY - www.mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Mon 10 Jan 2022 16:16:05 GMT +0000. [17:42:01] RECOVERY - wiki.thunis.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.thunis.eu' will expire on Mon 10 Jan 2022 16:05:49 GMT +0000. [17:42:09] why doesn't test3 not use the new search experence [17:42:10] RECOVERY - iesd.mobagenie.my.id - LetsEncrypt on sslhost is OK: OK - Certificate '*.miraheze.org' will expire on Wed 09 Nov 2022 23:59:59 GMT +0000. [17:42:29] RECOVERY - bharatwiki.online - LetsEncrypt on sslhost is OK: OK - Certificate 'bharatwiki.online' will expire on Mon 10 Jan 2022 16:12:31 GMT +0000. [17:42:34] RECOVERY - wiki.ssij.se - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ssij.se' will expire on Mon 10 Jan 2022 16:13:21 GMT +0000. [17:43:04] RECOVERY - ndg.nenawiki.org - LetsEncrypt on sslhost is OK: OK - Certificate 'nenawiki.org' will expire on Mon 10 Jan 2022 16:08:25 GMT +0000. [17:43:05] RECOVERY - mh142.com - LetsEncrypt on sslhost is OK: OK - Certificate 'mh142.com' will expire on Mon 10 Jan 2022 16:16:05 GMT +0000. [17:43:09] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±12] 13https://git.io/JKkSH [17:43:10] [02miraheze/ssl] 07Reception123 034ec0dba - renew SSL certs [17:43:26] paladox: you mean the Vue one? [17:43:28] RECOVERY - wiki.tulpa.info - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.tulpa.info' will expire on Mon 10 Jan 2022 16:16:36 GMT +0000. [17:43:31] yeh [17:43:59] paladox: is it behind config [17:43:59] RECOVERY - nonbinary.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'nonbinary.wiki' will expire on Mon 10 Jan 2022 16:03:42 GMT +0000. [17:44:10] yes but it defaults to true [17:44:13] from what i can tell [17:44:19] Weird [17:46:20] [02miraheze/ssl] 07Reception123 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKkHm [17:46:21] [02miraheze/ssl] 07Reception123 037291daa - remove redirect to inexistent wiki [17:46:53] https://www.irccloud.com/pastebin/RcARJAKs/ [17:46:57] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JKkHR [17:46:58] [02miraheze/mw-config] 07Universal-Omega 03d37b87b - Set wgVectorUseWvuiSearch to true by default [17:47:00] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [17:47:01] [02mw-config] 07Universal-Omega opened pull request 03#4146: Set wgVectorUseWvuiSearch to true by default - 13https://git.io/JKkHz [17:47:38] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JKkHQ [17:47:39] [02miraheze/mw-config] 07Universal-Omega 031d0cb79 - Update ManageWikiSettings.php [17:47:41] [02mw-config] 07Universal-Omega synchronize pull request 03#4146: Set wgVectorUseWvuiSearch to true by default - 13https://git.io/JKkHz [17:47:48] PROBLEM - mw11 Puppet on mw11 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:47:56] paladox:^ [17:48:00] PROBLEM - cp14 Puppet on cp14 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:48:01] paladox: can you run drop flaggedimages please per https://gerrit.wikimedia.org/r/c/mediawiki/extensions/FlaggedRevs/+/717445 [17:48:01] miraheze/mw-config - Universal-Omega the build passed. [17:48:04] https://github.com/snapwiki/SnapProjectEmbed/pull/60 [17:48:05] [url] Fix "Deprecated: Non-static method EmbedSnap::renderEmbedSnap()" by paladox · Pull Request #60 · snapwiki/SnapProjectEmbed · GitHub | github.com [17:48:10] PROBLEM - cp12 Puppet on cp12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:48:17] PROBLEM - mw13 Puppet on mw13 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:48:37] @R4356th ^ [17:48:47] miraheze/mw-config - Universal-Omega the build passed. [17:48:56] hmm they isn't on discord [17:49:17] [02mw-config] 07paladox closed pull request 03#4146: Set wgVectorUseWvuiSearch to true by default - 13https://git.io/JKkHz [17:49:18] [02miraheze/mw-config] 07paladox deleted branch 03Universal-Omega-patch-2 [17:49:19] PROBLEM - mw10 Puppet on mw10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:49:20] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±2] 13https://git.io/JKkQ6 [17:49:21] RhinosF1: running [17:49:21] [02miraheze/mw-config] 07Universal-Omega 0324a6672 - Set wgVectorUseWvuiSearch to true by default (#4146) [17:49:23] [02mw-config] 07paladox deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [17:49:26] paladox: ty [17:49:39] PROBLEM - mw9 Puppet on mw9 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:49:39] PROBLEM - mw8 Puppet on mw8 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:49:52] PROBLEM - mw12 Puppet on mw12 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:49:53] !log db12: drop flaggedimages on test3wiki [17:49:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:49:58] PROBLEM - cp15 Puppet on cp15 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[nginx] [17:50:21] !log [paladox@test3] starting deploy of {'config': True, 'gitinfo': True} to skip [17:50:25] miraheze/mw-config - paladox the build passed. [17:50:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:50:26] !log [paladox@test3] finished deploy of {'config': True, 'gitinfo': True} to skip - SUCCESS in 4s [17:50:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:50:55] oh [17:50:57] hmm [17:50:58] https://test3.miraheze.org/w/rest.php/v1/search/title?q=m&limit=10 [17:51:00] fails [17:51:00] [url] Internal error - Test3 | test3.miraheze.org [17:51:06] paladox: they are Redmin on discord. [17:51:30] found it [17:51:30] https://graylog.miraheze.org/messages/graylog_331/f82451d1-2b84-11ec-a12b-0200001a24a4 [17:51:35] CosmicAlpha: ah, thanks! [17:51:48] @Redmin could you review https://github.com/snapwiki/SnapProjectEmbed/pull/60 pls [17:51:49] [url] Fix "Deprecated: Non-static method EmbedSnap::renderEmbedSnap()" by paladox · Pull Request #60 · snapwiki/SnapProjectEmbed · GitHub | github.com [17:54:15] RECOVERY - cp12 Puppet on cp12 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:54:43] PROBLEM - wikiru.wildterra2.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikiru.wildterra2.com' expires in 7 day(s) (Wed 20 Oct 2021 17:52:02 GMT +0000). [17:56:29] [02miraheze/mediawiki] 07paladox pushed 032 commits to 03REL1_37 [+0/-0/±2] 13https://git.io/JKkFT [17:56:31] [02miraheze/mediawiki] 07paladox 0317ada70 - Update ShortDescription to 2f3e5f0fb7775f1e8415399cdc266ddefa1bd620 [17:56:31] RhinosF1: ^ [17:56:32] [02miraheze/mediawiki] 07paladox 033dd2c69 - Merge branch 'REL1_37' of github.com:miraheze/mediawiki into REL1_37 [17:56:48] Ok [17:56:57] [02ManageWiki] 07Universal-Omega closed pull request 03#300: Update workflow - 13https://git.io/Jz4ax [17:56:58] [02miraheze/ManageWiki] 07Universal-Omega pushed 031 commit to 03master [+5/-2/±8] 13https://git.io/JKkFW [17:57:00] [02miraheze/ManageWiki] 07Universal-Omega 031312a44 - Update workflow (#300) [17:57:01] !log [paladox@test3] starting deploy of {'world': True, 'gitinfo': True} to skip [17:57:01] [02ManageWiki] 07Universal-Omega deleted branch 03add-ci - 13https://git.io/vpSns [17:57:03] [02miraheze/ManageWiki] 07Universal-Omega deleted branch 03add-ci [17:57:07] PROBLEM - wikien.wildterra2.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikien.wildterra2.com' expires in 7 day(s) (Wed 20 Oct 2021 17:55:20 GMT +0000). [17:57:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:57:28] RECOVERY - wiki.patriam.cc - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.patriam.cc' will expire on Mon 10 Jan 2022 16:36:21 GMT +0000. [17:57:50] RECOVERY - monarchists.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'monarchists.wiki' will expire on Mon 10 Jan 2022 16:26:15 GMT +0000. [17:58:15] !log [@mw11] starting deploy of {'config': True} to all [17:58:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:58:24] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 9s [17:58:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [17:58:28] RECOVERY - wiki.ricochetuniverse.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.ricochetuniverse.com' will expire on Mon 10 Jan 2022 16:28:54 GMT +0000. [17:58:34] RECOVERY - cp14 Puppet on cp14 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:58:40] RECOVERY - wiki.graalmilitary.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.graalmilitary.com' will expire on Mon 10 Jan 2022 16:39:55 GMT +0000. [17:58:42] RECOVERY - mw13 Puppet on mw13 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [17:58:44] RECOVERY - mw10 Puppet on mw10 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [17:58:59] RECOVERY - cp13 Puppet on cp13 is OK: OK: Puppet is currently enabled, last run 7 seconds ago with 0 failures [17:59:03] PROBLEM - mw12 APT on mw12 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 10 seconds. [17:59:11] paladox: if you need to then add it to the non rel list [17:59:44] RECOVERY - miraheze.ml - LetsEncrypt on sslhost is OK: OK - Certificate 'miraheze.ml' will expire on Mon 10 Jan 2022 16:42:13 GMT +0000. [17:59:48] miraheze/ManageWiki - Universal-Omega the build has errored. [17:59:51] !log [paladox@test3] finished deploy of {'world': True, 'gitinfo': True} to skip - SUCCESS in 169s [17:59:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:00:00] RECOVERY - mw9 Puppet on mw9 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:00:01] RECOVERY - mw8 Puppet on mw8 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:00:13] RECOVERY - mw12 Puppet on mw12 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [18:00:15] RECOVERY - mw11 Puppet on mw11 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:00:16] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKkNL [18:00:17] [02miraheze/mediawiki] 07paladox 033befb9a - Update non-rel-extensions [18:00:18] RECOVERY - cp15 Puppet on cp15 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [18:00:28] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.25, 3.65, 3.31 [18:00:51] it works now (searching) [18:01:02] RECOVERY - mw12 APT on mw12 is OK: APT OK: 17 packages available for upgrade (0 critical updates). [18:01:22] Parsoid has been updated so we can bump VE [18:01:33] .in 2hours uodate VE if no one has [18:01:33] RhinosF1: Okay, will remind at 2021-10-12 - 21:01:33BST [18:01:59] RECOVERY - wiki.grottocenter.org - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.grottocenter.org' will expire on Mon 10 Jan 2022 16:41:34 GMT +0000. [18:02:16] RECOVERY - wiki.fraterniterebellion.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fraterniterebellion.com' will expire on Mon 10 Jan 2022 16:37:25 GMT +0000. [18:02:23] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.71, 3.31, 3.22 [18:02:46] RECOVERY - studentwiki.ddns.net - LetsEncrypt on sslhost is OK: OK - Certificate 'studentwiki.ddns.net' will expire on Mon 10 Jan 2022 16:30:16 GMT +0000. [18:03:14] RECOVERY - wiki.cdntennis.ca - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.cdntennis.ca' will expire on Mon 10 Jan 2022 16:38:41 GMT +0000. [18:03:18] RECOVERY - www.fraterniterebellion.com - LetsEncrypt on sslhost is OK: OK - Certificate 'wiki.fraterniterebellion.com' will expire on Mon 10 Jan 2022 16:37:25 GMT +0000. [18:03:18] RECOVERY - mrlove.wiki - LetsEncrypt on sslhost is OK: OK - Certificate 'mrlove.wiki' will expire on Mon 10 Jan 2022 16:32:51 GMT +0000. [18:03:31] RECOVERY - vitriol.spoon.army - LetsEncrypt on sslhost is OK: OK - Certificate 'vitriol.spoon.army' will expire on Mon 10 Jan 2022 16:27:28 GMT +0000. [18:03:44] RECOVERY - mediawiki.ecfin.eu - LetsEncrypt on sslhost is OK: OK - Certificate 'mediawiki.ecfin.eu' will expire on Mon 10 Jan 2022 16:31:24 GMT +0000. [18:03:44] paladox: ManageWiki needs updating also. I fixed an issue that broke formatting on 1.37 with it. I can do in a few minutes though. [18:03:55] ack [18:04:27] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKkxs [18:04:29] [02miraheze/mediawiki] 07paladox 037bcf732 - Update VisualEditor [18:04:51] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKkx8 [18:04:52] [02miraheze/mediawiki] 07paladox 032779c5a - Update ManageWiki [18:05:12] !log [paladox@test3] starting deploy of {'world': True, 'gitinfo': True} to skip [18:05:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:05:22] CosmicAlpha: done [18:05:50] Oh thanks! [18:07:54] !log [paladox@test3] finished deploy of {'world': True, 'gitinfo': True} to skip - SUCCESS in 161s [18:07:59] miraheze/ManageWiki - Universal-Omega the build passed. [18:08:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:08:15] PROBLEM - wiki.etn.link - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.etn.link' expires in 7 day(s) (Wed 20 Oct 2021 18:08:11 GMT +0000). [18:09:01] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 21.68, 19.94, 18.16 [18:11:00] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 19.25, 19.72, 18.30 [18:12:03] PROBLEM - wiki.yunachannel.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.yunachannel.com' expires in 7 day(s) (Wed 20 Oct 2021 18:05:14 GMT +0000). [18:12:46] PROBLEM - mwtask1 Disk Space on mwtask1 is WARNING: DISK WARNING - free space: / 5117 MB (10% inode=89%); [18:14:05] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 2.97, 3.42, 3.32 [18:14:43] RECOVERY - mwtask1 Disk Space on mwtask1 is OK: DISK OK - free space: / 15158 MB (32% inode=89%); [18:15:15] heh, +10GB on mwtask1 in less than 15 secs [18:16:02] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 1.00, 2.55, 3.01 [18:16:24] PROBLEM - wiki.thesimswiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'www.thesimswiki.com' expires in 7 day(s) (Wed 20 Oct 2021 18:16:08 GMT +0000). [18:17:03] PROBLEM - db12 Current Load on db12 is CRITICAL: CRITICAL - load average: 14.80, 10.40, 6.94 [18:17:51] :O [18:18:01] https://test3.miraheze.org/w/index.php?title=Main_Page&useskin=liberty https://test3.miraheze.org/w/index.php?title=Main_Page&useskin=medik liberty and medik fatal. https://test3.miraheze.org/w/index.php?title=Main_Page&useskin=nostalgia doesn't render resources it seems. All other skins seem to work. [18:18:03] [url] Test3 | test3.miraheze.org [18:18:04] [url] Test3 | test3.miraheze.org [18:19:52] PROBLEM - www.thesimswiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'www.thesimswiki.com' expires in 7 day(s) (Wed 20 Oct 2021 18:16:08 GMT +0000). [18:20:13] PROBLEM - thesimswiki.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'www.thesimswiki.com' expires in 7 day(s) (Wed 20 Oct 2021 18:16:08 GMT +0000). [18:21:00] PROBLEM - db12 Current Load on db12 is WARNING: WARNING - load average: 4.96, 7.92, 6.75 [18:22:03] CosmicAlpha: https://graylog.miraheze.org/messages/graylog_331/309bbef0-2b89-11ec-a12b-0200001a24a4 [18:22:07] revision class not found [18:22:15] PROBLEM - www.wikilukas.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikilukas.tk' expires in 7 day(s) (Wed 20 Oct 2021 18:21:19 GMT +0000). [18:22:36] PROBLEM - wikilukas.tk - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikilukas.tk' expires in 7 day(s) (Wed 20 Oct 2021 18:21:19 GMT +0000). [18:24:57] RECOVERY - db12 Current Load on db12 is OK: OK - load average: 4.45, 6.04, 6.24 [18:30:21] PROBLEM - zw.fontainebleau-avon.fr - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'zw.fontainebleau-avon.fr' expires in 7 day(s) (Wed 20 Oct 2021 18:27:27 GMT +0000). [18:30:33] PROBLEM - wiki.xysspon.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.xysspon.com' expires in 7 day(s) (Wed 20 Oct 2021 18:25:19 GMT +0000). [18:35:29] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.85, 3.48, 3.20 [18:35:39] PROBLEM - wiki.zymonic.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wiki.zymonic.com' expires in 7 day(s) (Wed 20 Oct 2021 18:34:20 GMT +0000). [18:37:25] PROBLEM - mon2 Current Load on mon2 is CRITICAL: CRITICAL - load average: 4.22, 3.49, 3.22 [18:37:40] paladox: https://gitlab.com/librewiki/Liberty-MW-Skin/-/merge_requests/121 [18:37:41] [url] Replace usages of Revision::RAW with RevisionRecord::RAW (!121) · Merge requests · LibreWiki / Liberty MediaWiki Skin · GitLab | gitlab.com [18:38:11] thanks! CosmicAlpha also ContentHandler::getContentText is deprecated in 1.37 [18:38:24] https://github.com/wikimedia/mediawiki/blob/2bcc75c9b418be92e869110213ece748d38a5062/RELEASE-NOTES-1.37#L621 [18:38:24] [url] mediawiki/RELEASE-NOTES-1.37 at 2bcc75c9b418be92e869110213ece748d38a5062 · wikimedia/mediawiki · GitHub | github.com [18:39:20] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.16, 3.62, 3.32 [18:39:40] PROBLEM - wikipariksha.com - LetsEncrypt on sslhost is CRITICAL: CRITICAL - Certificate 'wikipariksha.com' expires in 7 day(s) (Wed 20 Oct 2021 18:38:08 GMT +0000). [18:39:54] paladox: I know, there are alot of deprecated stuff in Liberty which is why I just fixed the fatal for now. [18:40:03] ahhh [18:41:04] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JKIOj [18:41:05] [02miraheze/mediawiki] 07paladox 03542c91f - Update ShortDescription to 2f3e5f0fb7775f1e8415399cdc266ddefa1bd620 [18:43:10] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.02, 2.95, 3.12 [18:43:49] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_37 [+0/-0/±1] 13https://git.io/JKIsC [18:43:50] [02miraheze/mediawiki] 07paladox 0375d3501 - Update liberty to a0f656aa76e5146b1c856e35cfa97b0ebd8de88d [18:44:08] !log [paladox@test3] starting deploy of {'world': True, 'gitinfo': True} to skip [18:44:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:46:47] !log [paladox@test3] finished deploy of {'world': True, 'gitinfo': True} to skip - SUCCESS in 158s [18:46:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:47:34] paladox: loads now without fatal but resources don't load. [18:47:41] yeh [18:47:43] that's weird [18:48:39] !log [paladox@test3] starting deploy of {'world': True, 'gitinfo': True} to skip [18:48:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:49:20] !log [paladox@test3] finished deploy of {'world': True, 'gitinfo': True} to skip - FAIL: [2, 2, 2, 2] in 40s [18:49:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [18:51:15] oh [18:51:16] https://gitlab.com/librewiki/Liberty-MW-Skin/-/merge_requests/120/diffs [18:51:17] [url] Add support for 1.35>= (!120) · Merge requests · LibreWiki / Liberty MediaWiki Skin · GitLab | gitlab.com [18:51:19] CosmicAlpha: [18:51:25] Oh [18:51:37] How did we not catch that before now then? [18:52:54] hmm [18:52:55] Maybe we should consider removing this skin? [18:53:19] That other one has been open for 9 months [18:53:34] maybe [18:58:44] maybe create another duplicate patch that isn't supposed to be merged but so we can at least see if it fixes the problems [18:59:12] CosmicAlpha: also nostalgia is supposed to look like that [18:59:49] i think we want to shut this https://graylog.miraheze.org/messages/graylog_331/7440ffd4-2b8e-11ec-a12b-0200001a24a4 log up CosmicAlpha ? [19:01:14] https://graylog.miraheze.org/messages/graylog_331/74534f51-2b8e-11ec-a12b-0200001a24a4 [19:01:21] Unsupported operand types [19:05:25] oh yeah... [19:06:06] RECOVERY - dbbackup1 Disk Space on dbbackup1 is OK: DISK OK - free space: / 310564 MB (49% inode=97%); [19:07:22] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-2 [+0/-0/±1] 13https://git.io/JKIRN [19:07:24] [02miraheze/mw-config] 07Universal-Omega 03a4992dc - Profiler: set `running` to `true` [19:07:25] [02mw-config] 07Universal-Omega created branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [19:07:27] [02mw-config] 07Universal-Omega opened pull request 03#4147: Profiler: set `running` to `true` - 13https://git.io/JKIRh [19:08:30] miraheze/mw-config - Universal-Omega the build passed. [19:08:46] [02mw-config] 07Universal-Omega closed pull request 03#4147: Profiler: set `running` to `true` - 13https://git.io/JKIRh [19:08:48] [02miraheze/mw-config] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKI0S [19:08:49] [02miraheze/mw-config] 07Universal-Omega 030fe3b2d - Profiler: set `running` to `true` (#4147) [19:08:50] [02miraheze/mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 [19:08:52] [02mw-config] 07Universal-Omega deleted branch 03Universal-Omega-patch-2 - 13https://git.io/vbvb3 [19:09:54] miraheze/mw-config - Universal-Omega the build passed. [19:10:12] !log conducting db12 -> dbbackup1 testing for T5877 [19:10:15] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:16:35] https://bitbucket.org/wikiskripta/medik/pull-requests/2 [19:16:36] [url] Bitbucket | bitbucket.org [19:19:02] that didn't fix it :/ [19:19:52] [02miraheze/mw-config] 07paladox pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKIaP [19:19:54] [02miraheze/mw-config] 07paladox 03ac20c03 - Disable ActionFactory channel [19:20:16] !log [paladox@test3] starting deploy of {'config': True, 'gitinfo': True} to skip [19:20:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:21] !log [paladox@test3] finished deploy of {'config': True, 'gitinfo': True} to skip - SUCCESS in 5s [19:20:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:20:59] miraheze/mw-config - paladox the build passed. [19:28:16] CosmicAlpha: this https://bitbucket.org/wikiskripta/medik/pull-requests/3 fixes it [19:28:17] [url] Bitbucket | bitbucket.org [19:30:46] Great! [19:34:40] !log [@mw11] starting deploy of {'config': True} to all [19:34:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:35:25] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 44s [19:35:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:36:16] Profiling fully works now! It does include stuff from mw-config and everything else it didn't before. So it's more accurate now. [19:36:24] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 7.80, 6.86, 4.64 [19:38:20] CosmicAlpha: i cannot seem to check the branch out [19:38:24] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 3.67, 5.58, 4.44 [19:38:30] as bitbucket only makes the branch under your fork [19:39:29] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.80, 3.67, 3.22 [19:40:19] Oh yeah.... I forgot about that. It's more difficult to do that there. I don't use bitbucket much, maybe once or twice ever, so can't be to much help there. [19:45:17] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 2.57, 3.11, 3.14 [19:58:18] [02miraheze/mediawiki] 07paladox pushed 031 commit to 03REL1_36 [+0/-0/±1] 13https://git.io/JKI5R [19:58:20] [02miraheze/mediawiki] 07paladox 032ecbfc8 - Update non-rel-extensions [19:59:39] I'm now going to leave the MW team to handle this [19:59:45] i have other work to get doing. [20:01:34] RhinosF1: uodate VE if no one has [20:01:48] I'll do schema work later [20:01:54] And lots of fun [20:01:59] CosmicAlpha: can you record blockers [20:06:49] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.13, 18.74, 16.42 [20:09:03] PROBLEM - cloud4 Current Load on cloud4 is CRITICAL: CRITICAL - load average: 24.33, 20.33, 17.28 [20:11:00] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.95, 8.96, 6.29 [20:11:02] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.90, 19.57, 17.39 [20:11:55] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 6.12, 7.11, 5.29 [20:11:57] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 5.35, 7.29, 5.79 [20:13:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 5.12, 7.57, 6.11 [20:14:03] PROBLEM - mw9 Current Load on mw9 is CRITICAL: CRITICAL - load average: 8.22, 7.07, 5.86 [20:14:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 4.64, 6.06, 5.12 [20:15:06] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.11, 8.55, 6.65 [20:16:01] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 14.86, 9.33, 6.95 [20:19:06] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 3.84, 6.90, 6.52 [20:19:55] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 4.57, 7.05, 6.61 [20:19:55] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.21, 6.80, 6.33 [20:21:07] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.83, 6.26, 6.33 [20:21:52] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 3.14, 5.73, 6.18 [20:38:38] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/JKLqH [20:38:39] [02miraheze/mw-config] 07RhinosF1 03c6efc40 - Test3: set json&split metadata [20:38:41] [02mw-config] 07RhinosF1 created branch 03RhinosF1-patch-1 - 13https://git.io/vbvb3 [20:38:48] [02mw-config] 07RhinosF1 opened pull request 03#4148: Test3: set json&split metadata - 13https://git.io/JKLqN [20:39:05] [02miraheze/mw-config] 07github-actions[bot] pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/JKLmk [20:39:06] [02miraheze/mw-config] 07github-actions 035568657 - CI: lint code to MediaWiki standards [20:39:08] [02mw-config] 07github-actions[bot] synchronize pull request 03#4148: Test3: set json&split metadata - 13https://git.io/JKLqN [20:39:52] miraheze/mw-config - RhinosF1 the build passed. [20:49:07] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 9.35, 6.77, 5.59 [20:51:06] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.87, 5.95, 5.44 [21:26:42] !log (test3) delete from logging where log_type = 'review' and log_action = 'approve-a' - 160 rows [21:26:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:28:05] !log rhinos@test3:~$ sudo -u www-data php /srv/mediawiki/w/extensions/FlaggedRevs/maintenance/pruneRevData.php --wiki=test3wiki --prune [21:28:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:29:41] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03RhinosF1-patch-1 [+0/-0/±1] 13https://git.io/JKLKN [21:29:43] [02miraheze/mw-config] 07RhinosF1 035d563b7 - Update LocalWiki.php [21:29:44] [02mw-config] 07RhinosF1 synchronize pull request 03#4148: Test3: set json&split metadata - 13https://git.io/JKLqN [21:30:38] [02mw-config] 07RhinosF1 closed pull request 03#4148: Test3: set json&split metadata - 13https://git.io/JKLqN [21:30:39] [02mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-1 - 13https://git.io/vbvb3 [21:30:41] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKL6W [21:30:42] [02miraheze/mw-config] 07RhinosF1 03501d40a - Test3: set json&split metadata (#4148) [21:30:44] miraheze/mw-config - RhinosF1 the build passed. [21:30:44] [02miraheze/mw-config] 07RhinosF1 deleted branch 03RhinosF1-patch-1 [21:30:45] [02miraheze/SpriteSheet] 07Universal-Omega pushed 031 commit to 03Universal-Omega-patch-1 [+0/-0/±1] 13https://git.io/JKL6l [21:30:47] [02miraheze/SpriteSheet] 07Universal-Omega 037cc8e74 - Replace usages of `$wgUser` [21:30:48] [02SpriteSheet] 07Universal-Omega created branch 03Universal-Omega-patch-1 - 13https://git.io/JTPvX [21:30:50] [02SpriteSheet] 07Universal-Omega opened pull request 03#8: Replace usages of `$wgUser` - 13https://git.io/JKL64 [21:31:06] !log [rhinos@test3] starting deploy of {'config': True} to skip [21:31:07] !log [rhinos@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [21:31:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:31:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:31:43] miraheze/SpriteSheet - Universal-Omega the build passed. [21:31:46] miraheze/mw-config - RhinosF1 the build passed. [21:32:07] .op [21:32:07] Attempting to OP... [21:32:09] https://graylog.miraheze.org/messages/graylog_331/c083f5e0-2ba3-11ec-a12b-0200001a24a4 [21:32:11] RhinosF1: [21:32:21] Class name must be a valid object or a string [21:32:23] .deop [21:32:23] Attempting to OP... [21:33:01] paladox: me [21:34:21] RhinosF1: can miraheze-github_ be exempted from ban? [21:34:24] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 9.12, 6.12, 4.75 [21:34:51] !log [@mw11] starting deploy of {'config': True} to all [21:34:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:35:16] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 25s [21:35:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:36:58] guess you have to copy https://github.com/wikimedia/mediawiki/blob/3b5c598143efc604eb74db7f6bdd7b4999efbe9d/includes/Setup.php#L358 [21:36:59] [url] mediawiki/Setup.php at 3b5c598143efc604eb74db7f6bdd7b4999efbe9d · wikimedia/mediawiki · GitHub | github.com [21:39:24] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKLMh [21:39:25] [02miraheze/mw-config] 07RhinosF1 030ac360f - Update LocalWiki.php [21:39:36] !log [rhinos@test3] starting deploy of {'config': True} to skip [21:39:37] !log [rhinos@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [21:39:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:39:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:39:48] paladox: yep [21:40:25] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 3.37, 5.28, 4.91 [21:40:27] miraheze/mw-config - RhinosF1 the build passed. [21:40:43] !log rhinos@test3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/refreshImageMetadata.php --wiki=test3wiki --force --verbose [21:40:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:41:05] Warning: Syntax error [Called from MediaWiki\Extension\EmbedVideo\Media\FFProbe\FFProbe::invokeFFProbe in /srv/mediawiki/w/extensions/EmbedVideo/includes/Media/FFProbe/FFProbe.php at line 165] in /srv/mediawiki/w/includes/debug/MWDebug.php on line 498 [21:42:25] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 4.22, 5.02, 4.86 [21:47:58] !log (test3wiki) > ALTER TABLE revision ALTER rev_timestamp DROP DEFAULT; [21:48:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:48:15] !log rhinos@test3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/maintenance/archives/patch-revision-rename-index.sql --wiki=test3wiki [21:48:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:48:24] !log rhinos@test3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/maintenance/archives/patch-page-rename-name_title-index.sql --wiki=test3wiki [21:48:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:48:33] !log rhinos@test3:~$ sudo -u www-data php /srv/mediawiki/w/maintenance/sql.php /srv/mediawiki/w/maintenance/archives/patch-change_tag-rename-indexes.sql --wiki=test3wiki [21:48:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:50:15] paladox: schema work done [21:50:22] ok [21:50:26] hopefully a slightly smaller disk [21:57:00] !log [rhinos@test3] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/flaggedrevs.json /srv/mediawiki/w/extensions/FlaggedRevs/maintenance/pruneRevData.php --prune (END - exit=256) [21:57:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:59:35] !log [rhinos@test3] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/flaggedrevs.json /srv/mediawiki/w/maintenance/sql.php cleanfrlogs.sql (END - exit=0) [21:59:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:03:05] !log clean up old flagged revs. auto approve. logs. and. template. garbage on production [22:03:14] !log [rhinos@test3] sudo -u www-data /usr/local/bin/foreachwikiindblist /home/rhinos/flaggedrevs.json /srv/mediawiki/w/maintenance/sql.php cleanfrlogs.sql (END - exit=0) [22:03:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:03:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:04:18] !log [@mw11] starting deploy of {'config': True} to all [22:04:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:04:30] !log [@mw11] finished deploy of {'config': True} to all - SUCCESS in 11s [22:04:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:06:16] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+1/-0/±0] 13https://git.io/JKLj2 [22:06:18] [02miraheze/mw-config] 07RhinosF1 03ed35601 - Create 137updates.sql [22:07:13] [02miraheze/mw-config] 07RhinosF1 pushed 031 commit to 03master [+0/-0/±1] 13https://git.io/JKtef [22:07:14] [02miraheze/mw-config] 07RhinosF1 03fdf648f - Update LocalSettings.php [22:07:17] paladox: ^ [22:07:20] miraheze/mw-config - RhinosF1 the build passed. [22:07:24] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JKtem [22:07:25] [02miraheze/puppet] 07paladox 03a59f1a6 - nginx: Reduce keepalive timeout to 2 [22:07:27] [02puppet] 07paladox created branch 03paladox-patch-1 - 13https://git.io/vbiAS [22:07:28] [02puppet] 07paladox opened pull request 03#2026: nginx: Reduce keepalive timeout to 2 - 13https://git.io/JKteY [22:07:57] RhinosF1: i'm confused why that was commited? [22:08:09] [02mw-config] 07paladox closed pull request 03#4144: Paladox patch 1 - 13https://git.io/JKkea [22:08:11] [02miraheze/mw-config] 07paladox deleted branch 03paladox-patch-1 [22:08:13] [02mw-config] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbvb3 [22:08:18] miraheze/mw-config - RhinosF1 the build passed. [22:08:24] paladox: they are the changes safe to apply in advance [22:09:12] ok [22:09:24] will run in prod in a. few. seconds [22:09:39] !log starting production schema. updates [22:09:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:09:59] !log [rhinos@mw11] starting deploy of {'files': '137updates.sql'} to all [22:10:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:10:05] !log [rhinos@mw11] finished deploy of {'files': '137updates.sql'} to all - FAIL: [5888, 5888, 5888, 5888, 5888, 5888, 5888] in 5s [22:10:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:10:19] !log [rhinos@mw11] starting deploy of {'files': 'config/137updates.sql'} to all [22:10:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:10:23] !log [rhinos@mw11] finished deploy of {'files': 'config/137updates.sql'} to all - SUCCESS in 4s [22:10:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:11:51] !log [rhinos@mw11] starting deploy of {'files': 'config/LocalSettings.php'} to all [22:11:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:11:59] !log [rhinos@mw11] finished deploy of {'files': 'config/LocalSettings.php'} to all - SUCCESS in 7s [22:12:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:12:12] !log [rhinos@mw11] starting deploy of {'config': True} to all [22:12:23] !log [rhinos@mw11] finished deploy of {'config': True} to all - SUCCESS in 10s [22:12:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:12:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:13:13] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JKtUU [22:13:14] [02miraheze/puppet] 07paladox 0369e9c9c - Update nginx.conf.erb [22:13:16] [02puppet] 07paladox synchronize pull request 03#2026: nginx: Reduce keepalive timeout to 2 - 13https://git.io/JKteY [22:14:20] [02miraheze/puppet] 07paladox pushed 031 commit to 03paladox-patch-1 [+0/-0/±1] 13https://git.io/JKtUx [22:14:21] [02miraheze/puppet] 07paladox 031540c1d - Update init.pp [22:14:23] [02puppet] 07paladox synchronize pull request 03#2026: nginx: Reduce keepalive timeout to 2 - 13https://git.io/JKteY [22:15:09] [02puppet] 07paladox closed pull request 03#2026: nginx: Reduce keepalive timeout to 2 - 13https://git.io/JKteY [22:15:10] [02miraheze/puppet] 07paladox pushed 034 commits to 03master [+0/-0/±5] 13https://git.io/JKtT2 [22:15:12] [02miraheze/puppet] 07paladox 03a59f1a6 - nginx: Reduce keepalive timeout to 2 [22:15:13] [02miraheze/puppet] 07paladox 0369e9c9c - Update nginx.conf.erb [22:15:15] [02miraheze/puppet] 07paladox 031540c1d - Update init.pp [22:15:16] [02miraheze/puppet] 07paladox 034a34b05 - Merge pull request #2026 from miraheze/paladox-patch-1 [22:15:18] [02puppet] 07paladox deleted branch 03paladox-patch-1 - 13https://git.io/vbiAS [22:15:19] [02miraheze/puppet] 07paladox deleted branch 03paladox-patch-1 [22:15:22] !log start actual changes in a screen [22:15:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:18:22] paladox: did my flaggedrevs cleanup just save 2GB? [22:18:24] https://grafana.miraheze.org/d/W9MIkA7iz/miraheze-cluster?orgId=1&var-job=node&var-node=db11.miraheze.org&var-port=9100&viewPanel=152 [22:18:25] [url] Grafana | grafana.miraheze.org [22:18:30] unoptimised [22:18:53] I have no idea how much we were storing before [22:19:31] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.62, 6.07, 4.61 [22:19:43] paladox: I'm impressed if it did [22:19:53] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.20, 20.94, 17.79 [22:20:25] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 8.73, 8.49, 5.94 [22:21:27] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 3.58, 5.22, 4.49 [22:21:52] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 17.06, 19.59, 17.69 [22:22:23] PROBLEM - mw12 Current Load on mw12 is WARNING: WARNING - load average: 4.92, 7.06, 5.71 [22:24:24] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.52, 6.12, 5.53 [22:33:18] !log [@test3] starting deploy of {'config': True} to skip [22:33:19] !log [@test3] finished deploy of {'config': True} to skip - SUCCESS in 0s [22:33:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:33:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:39:17] https://phabricator.miraheze.org/T8159 You can disregard this now, as I realized that @Blazikeye has been doing the deletion for all of us. [22:39:18] [url] ⚓ T8159 Mass deletion of celebrity pages on `terribletvshowswiki` is needed | phabricator.miraheze.org [22:39:34] Close as invalid then [22:41:21] {{done}} [22:41:22] https://meta.miraheze.org/wiki/Template:Done [22:41:32] [url] Template:Done - Miraheze Meta | meta.miraheze.org [22:52:31] PROBLEM - mon2 Current Load on mon2 is WARNING: WARNING - load average: 3.69, 3.13, 2.62 [22:54:32] RECOVERY - mon2 Current Load on mon2 is OK: OK - load average: 3.03, 2.93, 2.60 [22:55:48] RhinosF1, Reception123, On Skin Minerva Neue, we're currently loading skins.minerva.content.styles, whereas we're supposed to be loading skins.minerva.base.styles. Please how do we load this skins.minerva.base.styles instead of the former, or possibly both. This is a reason why many wikis have issues with Mobile appearance. I'm speaking strictly on famepedia. [22:57:54] [02puppet] 07Universal-Omega opened pull request 03#2027: Add landing and ErrorPages to staging - 13https://git.io/JKtV6 [23:02:36] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.89, 5.33, 4.26 [23:04:57] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.84, 4.78, 4.22 [23:06:02] PROBLEM - mw8 Current Load on mw8 is WARNING: WARNING - load average: 7.45, 7.07, 4.81 [23:07:02] PROBLEM - mw12 Current Load on mw12 is CRITICAL: CRITICAL - load average: 12.71, 9.40, 6.03 [23:07:10] PROBLEM - mw9 Current Load on mw9 is WARNING: WARNING - load average: 6.07, 6.92, 5.03 [23:07:25] PROBLEM - mw10 Current Load on mw10 is CRITICAL: CRITICAL - load average: 12.10, 9.06, 5.83 [23:07:34] PROBLEM - cloud4 Current Load on cloud4 is WARNING: WARNING - load average: 21.49, 21.43, 17.51 [23:07:41] PROBLEM - cloud5 Current Load on cloud5 is CRITICAL: CRITICAL - load average: 26.09, 23.87, 18.42 [23:08:05] RECOVERY - mw8 Current Load on mw8 is OK: OK - load average: 5.42, 6.21, 4.77 [23:09:04] RECOVERY - mw9 Current Load on mw9 is OK: OK - load average: 4.18, 6.00, 4.92 [23:09:22] PROBLEM - mw10 Current Load on mw10 is WARNING: WARNING - load average: 3.67, 6.91, 5.42 [23:09:41] PROBLEM - cloud5 Current Load on cloud5 is WARNING: WARNING - load average: 15.76, 20.82, 17.97 [23:11:00] RECOVERY - mw12 Current Load on mw12 is OK: OK - load average: 4.81, 6.73, 5.72 [23:11:14] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 8.79, 6.22, 4.99 [23:11:20] RECOVERY - mw10 Current Load on mw10 is OK: OK - load average: 5.12, 6.22, 5.32 [23:13:13] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 4.22, 5.71, 4.98 [23:13:29] RECOVERY - cloud4 Current Load on cloud4 is OK: OK - load average: 14.54, 18.76, 17.76 [23:13:39] RECOVERY - cloud5 Current Load on cloud5 is OK: OK - load average: 13.95, 18.19, 17.62 [23:15:16] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.43, 4.88, 4.76 [23:21:14] PROBLEM - gluster3 Current Load on gluster3 is CRITICAL: CRITICAL - load average: 6.87, 5.62, 5.02 [23:23:14] PROBLEM - gluster3 Current Load on gluster3 is WARNING: WARNING - load average: 5.61, 5.49, 5.04 [23:25:13] RECOVERY - gluster3 Current Load on gluster3 is OK: OK - load average: 3.01, 4.63, 4.79 [23:37:14] PROBLEM - mw11 Current Load on mw11 is WARNING: WARNING - load average: 7.20, 6.02, 4.71 [23:39:10] RECOVERY - mw11 Current Load on mw11 is OK: OK - load average: 3.94, 5.21, 4.57