[00:17:49] PROBLEM - os201 Current Load on os201 is WARNING: LOAD WARNING - total load average: 2.75, 3.43, 3.36 [00:19:48] RECOVERY - os201 Current Load on os201 is OK: LOAD OK - total load average: 3.10, 3.40, 3.35 [00:19:51] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 44287MiB (9.8% inode=100%); [00:25:42] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 44431MiB (9.8% inode=100%); [00:27:51] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 86869MiB (19.1% inode=100%); [00:30:35] RECOVERY - speedscope211 Check unit status of speedscope_daily_aggregation on speedscope211 is OK: OK: Status of the systemd unit speedscope_daily_aggregation [00:33:42] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 87350MiB (19.2% inode=100%); [00:35:30] PROBLEM - os201 Current Load on os201 is WARNING: LOAD WARNING - total load average: 3.87, 3.38, 3.31 [00:37:28] RECOVERY - os201 Current Load on os201 is OK: LOAD OK - total load average: 2.95, 3.27, 3.29 [00:43:28] PROBLEM - os201 Current Load on os201 is CRITICAL: LOAD CRITICAL - total load average: 4.12, 3.52, 3.35 [00:47:28] PROBLEM - os201 Current Load on os201 is WARNING: LOAD WARNING - total load average: 3.75, 3.71, 3.46 [00:48:35] PROBLEM - speedscope211 Check unit status of speedscope_daily_aggregation on speedscope211 is CRITICAL: CRITICAL: Status of the systemd unit speedscope_daily_aggregation [00:51:28] RECOVERY - os201 Current Load on os201 is OK: LOAD OK - total load average: 1.67, 2.84, 3.19 [00:58:36] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 44387MiB (9.8% inode=100%); [01:10:36] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 86426MiB (19.1% inode=100%); [01:15:59] PROBLEM - os151 Current Load on os151 is WARNING: LOAD WARNING - total load average: 3.44, 3.06, 2.40 [01:17:59] RECOVERY - os151 Current Load on os151 is OK: LOAD OK - total load average: 3.12, 3.10, 2.50 [01:58:12] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44548MiB (10.0% inode=100%); [02:29:19] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43545MiB (9.7% inode=100%); [02:29:25] PROBLEM - os161 Current Load on os161 is WARNING: LOAD WARNING - total load average: 3.74, 2.86, 1.85 [02:30:00] PROBLEM - os191 Current Load on os191 is CRITICAL: LOAD CRITICAL - total load average: 4.70, 3.21, 2.03 [02:30:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 43803MiB (9.8% inode=100%); [02:31:19] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 46484MiB (10.4% inode=100%); [02:31:25] RECOVERY - os161 Current Load on os161 is OK: LOAD OK - total load average: 3.09, 2.84, 1.96 [02:33:57] PROBLEM - os191 Current Load on os191 is WARNING: LOAD WARNING - total load average: 3.30, 3.62, 2.50 [02:34:56] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 47537MiB (10.7% inode=100%); [02:36:02] RECOVERY - os191 Current Load on os191 is OK: LOAD OK - total load average: 3.10, 3.33, 2.53 [02:57:54] PROBLEM - os191 Current Load on os191 is WARNING: LOAD WARNING - total load average: 3.42, 2.95, 2.60 [02:59:53] RECOVERY - os191 Current Load on os191 is OK: LOAD OK - total load average: 2.38, 2.71, 2.56 [03:04:47] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43819MiB (9.8% inode=100%); [03:11:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 48661MiB (10.9% inode=100%); [03:12:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 47669MiB (10.7% inode=100%); [03:12:37] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 48080MiB (10.8% inode=100%); [03:19:56] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 12.69, 9.81, 6.17 [03:21:05] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.78, 16.42, 10.64 [03:23:05] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 14.04, 15.18, 10.89 [03:25:43] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 11.33, 11.65, 8.25 [03:29:35] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 6.83, 9.32, 8.08 [03:34:46] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.70, 18.81, 14.73 [03:35:23] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 25.86, 21.27, 16.74 [03:36:46] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 13.76, 16.80, 14.49 [03:37:19] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 17.09, 19.61, 16.68 [03:58:30] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 44035MiB (9.8% inode=100%); [04:04:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 42965MiB (9.7% inode=100%); [04:12:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 45183MiB (10.2% inode=100%); [04:16:06] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45130MiB (10.1% inode=100%); [04:16:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 42112MiB (9.5% inode=100%); [04:18:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 46139MiB (10.4% inode=100%); [04:33:00] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 43830MiB (9.8% inode=100%); [04:43:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43419MiB (9.8% inode=100%); [04:49:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44517MiB (10.0% inode=100%); [04:53:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43813MiB (9.9% inode=100%); [05:27:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 45249MiB (10.2% inode=100%); [05:42:37] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43527MiB (9.8% inode=100%); [06:02:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 44733MiB (10.0% inode=100%); [06:05:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 45736MiB (10.3% inode=100%); [06:35:03] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43649MiB (9.8% inode=100%); [06:37:03] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45425MiB (10.2% inode=100%); [06:45:03] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 44527MiB (10.0% inode=100%); [06:58:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 44468MiB (10.0% inode=100%); [07:31:03] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45370MiB (10.1% inode=100%); [07:32:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 44905MiB (10.1% inode=100%); [07:38:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 42789MiB (9.6% inode=100%); [07:39:03] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 44481MiB (9.9% inode=100%); [07:44:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 45748MiB (10.3% inode=100%); [07:48:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 43049MiB (9.7% inode=100%); [07:50:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 45581MiB (10.2% inode=100%); [07:51:03] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 46099MiB (10.3% inode=100%); [07:51:48] i think the disk was warning [07:54:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 43214MiB (9.7% inode=100%); [07:56:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 47126MiB (10.6% inode=100%); [08:20:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44463MiB (9.9% inode=100%); [08:29:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43375MiB (9.8% inode=100%); [08:31:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44704MiB (10.1% inode=100%); [08:41:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 44034MiB (9.9% inode=100%); [09:04:13] PROBLEM - os151 Current Load on os151 is WARNING: LOAD WARNING - total load average: 3.65, 3.16, 2.69 [09:10:13] RECOVERY - os151 Current Load on os151 is OK: LOAD OK - total load average: 3.29, 3.36, 2.95 [09:11:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44911MiB (10.1% inode=100%); [09:14:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 44978MiB (10.1% inode=100%); [09:17:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43080MiB (9.7% inode=100%); [09:18:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44694MiB (10.0% inode=100%); [09:25:23] PROBLEM - os201 Current Load on os201 is CRITICAL: LOAD CRITICAL - total load average: 4.03, 3.59, 3.07 [09:27:22] PROBLEM - os201 Current Load on os201 is WARNING: LOAD WARNING - total load average: 3.33, 3.47, 3.09 [09:29:21] RECOVERY - os201 Current Load on os201 is OK: LOAD OK - total load average: 3.07, 3.34, 3.09 [09:29:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44839MiB (10.1% inode=100%); [09:35:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43388MiB (9.8% inode=100%); [10:03:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44758MiB (10.1% inode=100%); [10:18:38] [02mw-config] 07dravyabbs opened pull request #6576: Update LocalSettings.php (07miraheze:03main...07dravyabbs:03patch-1) 13https://github.com/miraheze/mw-config/pull/6576 [10:29:20] [02mw-config] 07dravyabbs opened pull request #6577: Defining new protection level template editor (07miraheze:03main...07dravyabbs:03patch-2) 13https://github.com/miraheze/mw-config/pull/6577 [10:29:39] [02mw-config] 07dravyabbs closed pull request #6576: Update LocalSettings.php (07miraheze:03main...07dravyabbs:03patch-1) 13https://github.com/miraheze/mw-config/pull/6576 [10:39:50] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 44524MiB (10.0% inode=100%); [10:56:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 47759MiB (10.7% inode=100%); [10:57:26] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 47123MiB (10.5% inode=100%); [11:38:34] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.59, 22.05, 18.57 [11:40:30] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 14.25, 19.34, 18.01 [11:41:55] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43477MiB (9.7% inode=100%); [11:50:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 44333MiB (10.0% inode=100%); [12:01:29] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45523MiB (10.2% inode=100%); [12:06:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 46557MiB (10.5% inode=100%); [12:22:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44060MiB (9.9% inode=100%); [12:33:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 44435MiB (10.0% inode=100%); [13:07:41] PROBLEM - os201 Current Load on os201 is WARNING: LOAD WARNING - total load average: 3.73, 3.41, 3.01 [13:09:41] RECOVERY - os201 Current Load on os201 is OK: LOAD OK - total load average: 2.55, 3.00, 2.90 [13:11:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 45708MiB (10.3% inode=100%); [13:14:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 44839MiB (10.0% inode=100%); [13:22:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44313MiB (9.9% inode=100%); [13:48:12] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 41644MiB (9.4% inode=100%); [13:57:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 45346MiB (10.2% inode=100%); [14:27:01] PROBLEM - mw202 Current Load on mw202 is CRITICAL: LOAD CRITICAL - total load average: 31.98, 19.32, 12.58 [14:29:01] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 16.28, 17.86, 12.89 [14:39:16] PROBLEM - os191 Current Load on os191 is WARNING: LOAD WARNING - total load average: 3.68, 3.15, 2.73 [14:41:15] RECOVERY - os191 Current Load on os191 is OK: LOAD OK - total load average: 3.18, 3.18, 2.79 [14:46:45] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 44374MiB (9.9% inode=100%); [15:00:28] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/eb386d54a323ba8c5ef4797f2c163dc00b4c6326 [15:00:29] 02ssl/03main 07WikiTideBot 03eb386d5 Bot: Auto-update domain lists [15:02:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 48319MiB (10.8% inode=100%); [15:02:24] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 46847MiB (10.5% inode=100%); [15:18:50] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.43, 18.35, 14.45 [15:22:50] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 17.53, 18.61, 15.50 [15:26:50] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 24.50, 22.17, 17.69 [15:27:27] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.03, 19.87, 16.18 [15:28:50] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 23.41, 22.71, 18.44 [15:30:22] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/3ddc424fe62357fc35a113f92a7332cbbd63b324 [15:30:22] 02ssl/03main 07WikiTideBot 033ddc424 Bot: Auto-update domain lists [15:31:27] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 20.05, 19.93, 17.09 [15:32:50] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 25.33, 22.60, 19.22 [15:34:46] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.14, 20.54, 16.82 [15:35:27] PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 24.06, 22.15, 18.65 [15:35:54] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 22.90, 18.57, 14.58 [15:36:04] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 23.10, 19.49, 15.92 [15:37:01] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.47, 19.41, 15.70 [15:37:05] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.06, 19.13, 15.25 [15:37:27] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.63, 22.63, 19.26 [15:37:27] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.78, 19.16, 15.59 [15:37:52] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 21.66, 19.08, 15.10 [15:38:02] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 23.31, 18.85, 14.19 [15:38:07] PROBLEM - mw152 Current Load on mw152 is WARNING: LOAD WARNING - total load average: 23.66, 19.67, 14.84 [15:38:09] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 23.55, 20.01, 16.20 [15:38:17] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 23.30, 19.58, 15.52 [15:38:31] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43677MiB (9.8% inode=100%); [15:38:42] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 23.57, 21.30, 17.43 [15:39:02] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 22.15, 18.73, 14.25 [15:39:08] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 23.03, 19.87, 15.72 [15:39:09] PROBLEM - mw161 Current Load on mw161 is WARNING: LOAD WARNING - total load average: 23.21, 19.06, 14.41 [15:39:28] PROBLEM - mw162 Current Load on mw162 is CRITICAL: LOAD CRITICAL - total load average: 24.12, 20.25, 15.30 [15:39:54] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 19.19, 20.08, 16.20 [15:40:02] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 17.16, 18.53, 14.68 [15:40:07] RECOVERY - mw152 Current Load on mw152 is OK: LOAD OK - total load average: 18.24, 19.19, 15.27 [15:41:01] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 15.45, 19.24, 16.62 [15:41:02] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 11.84, 16.34, 13.91 [15:41:05] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 15.01, 19.12, 16.30 [15:41:08] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 14.50, 18.44, 15.75 [15:41:09] RECOVERY - mw161 Current Load on mw161 is OK: LOAD OK - total load average: 16.64, 18.83, 14.95 [15:41:21] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 13.38, 18.67, 16.38 [15:41:28] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 11.15, 17.25, 14.85 [15:41:52] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 10.78, 16.54, 15.15 [15:42:04] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 10.75, 17.88, 16.78 [15:42:09] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 11.52, 17.72, 16.35 [15:42:17] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 9.70, 16.51, 15.42 [15:42:42] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 11.29, 17.36, 16.88 [15:42:46] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 9.68, 16.81, 17.17 [15:42:50] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 11.91, 19.84, 20.39 [15:43:27] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 11.69, 17.11, 18.19 [15:46:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 42603MiB (9.6% inode=100%); [15:48:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 45999MiB (10.3% inode=100%); [15:52:55] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 43060MiB (9.7% inode=100%); [15:54:09] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45568MiB (10.2% inode=100%); [15:54:55] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 46985MiB (10.6% inode=100%); [15:58:03] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 41943MiB (9.4% inode=100%); [16:00:01] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45961MiB (10.3% inode=100%); [16:18:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44550MiB (10.0% inode=100%); [16:22:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 44774MiB (10.0% inode=100%); [16:28:07] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 43569MiB (9.7% inode=100%); [16:37:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 44183MiB (9.9% inode=100%); [16:41:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44676MiB (10.0% inode=100%); [16:45:49] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 44203MiB (9.9% inode=100%); [16:51:49] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 46718MiB (10.5% inode=100%); [16:52:07] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 46221MiB (10.3% inode=100%); [17:00:23] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/a471e927f3c034616b29e80fa5d144d6d7f2ad15 [17:00:24] 02ssl/03main 07WikiTideBot 03a471e92 Bot: Auto-update domain lists [17:09:54] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 44194MiB (9.9% inode=100%); [17:28:37] PROBLEM - os201 Disk Space on os201 is WARNING: DISK WARNING - free space: / 43101MiB (9.7% inode=100%); [17:55:40] RECOVERY - os201 Disk Space on os201 is OK: DISK OK - free space: / 44793MiB (10.1% inode=100%); [18:24:27] [02ManageWiki] 07originalauthority commented on pull request #816: LGTM 🫡 13https://github.com/miraheze/ManageWiki/pull/816#issuecomment-5972145722 [18:30:24] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/8380fa858a2107278741a318ab4f71316e0c78a0 [18:30:24] 02ssl/03main 07WikiTideBot 038380fa8 Bot: Auto-update domain lists [18:37:41] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43487MiB (9.7% inode=100%); [18:41:38] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 48014MiB (10.7% inode=100%); [18:42:30] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 47805MiB (10.7% inode=100%); [19:17:00] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 24.90, 18.79, 13.86 [19:17:10] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.63, 17.46, 13.07 [19:18:11] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 20.97, 16.78, 12.33 [19:19:08] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.93, 17.99, 13.79 [19:20:43] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 21.25, 18.46, 13.34 [19:22:42] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 19.95, 18.79, 14.07 [19:23:45] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.01, 19.91, 14.65 [19:24:06] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.02, 19.36, 14.96 [19:25:00] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 23.87, 23.98, 18.51 [19:25:41] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 17.89, 18.90, 14.91 [19:29:00] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 24.54, 22.90, 19.21 [19:30:38] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43979MiB (9.8% inode=100%); [19:30:48] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 23.03, 21.04, 16.81 [19:32:57] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.35, 20.48, 17.12 [19:34:38] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 18.95, 19.79, 17.26 [19:34:52] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.39, 21.14, 18.37 [19:35:35] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.41, 20.56, 17.51 [19:36:38] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45398MiB (10.2% inode=100%); [19:36:56] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 23.08, 20.91, 17.48 [19:37:25] PROBLEM - mw173 Current Load on mw173 is CRITICAL: LOAD CRITICAL - total load average: 24.19, 21.40, 17.78 [19:37:30] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.87, 20.44, 17.46 [19:37:31] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 21.98, 20.22, 16.84 [19:38:28] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 23.28, 21.38, 18.42 [19:38:44] PROBLEM - mw151 Current Load on mw151 is WARNING: LOAD WARNING - total load average: 23.01, 20.62, 17.00 [19:38:52] PROBLEM - mw182 Current Load on mw182 is CRITICAL: LOAD CRITICAL - total load average: 24.02, 22.68, 19.64 [19:39:20] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 23.87, 22.40, 18.60 [19:40:14] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.78, 21.01, 18.21 [19:40:33] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 23.59, 21.13, 17.53 [19:40:40] PROBLEM - mw162 Current Load on mw162 is WARNING: LOAD WARNING - total load average: 22.38, 20.20, 16.27 [19:40:52] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.89, 23.07, 20.16 [19:41:27] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 14.86, 19.71, 18.00 [19:41:31] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 13.68, 18.78, 17.19 [19:41:50] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1791052880000&orgId=1&to=1791056510032 [19:42:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 12.54, 17.97, 17.45 [19:42:18] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 11.76, 18.56, 18.12 [19:42:31] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 10.36, 16.75, 16.34 [19:42:40] RECOVERY - mw162 Current Load on mw162 is OK: LOAD OK - total load average: 10.51, 16.50, 15.40 [19:42:40] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 11.76, 18.11, 17.68 [19:42:41] RECOVERY - mw151 Current Load on mw151 is OK: LOAD OK - total load average: 9.73, 16.51, 16.35 [19:42:49] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 10.30, 16.55, 17.41 [19:42:52] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 10.90, 18.36, 18.78 [19:43:00] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 12.49, 20.82, 21.64 [19:43:11] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 11.36, 18.00, 17.81 [19:43:35] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 9.14, 16.28, 17.29 [19:45:00] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 9.69, 16.90, 20.09 [19:45:30] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 43833MiB (9.8% inode=100%); [19:46:50] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1791052910000&orgId=1&to=1791056510000 [19:52:27] !log [universalomega@test151] Starting deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to test151 [19:52:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195228-6b168625 [19:52:56] PROBLEM - os191 Disk Space on os191 is WARNING: DISK WARNING - free space: / 44047MiB (9.9% inode=100%); [19:53:30] !log [universalomega@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [19:53:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195330-ad661551 [19:55:01] !log [universalomega@test151] Starting deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to test151 [19:55:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195501-6b168625 [19:55:30] !log [universalomega@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [19:55:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195530-ad661551 [19:55:40] !log [universalomega@test151] Starting deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to test151 [19:55:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195540-6b168625 [19:56:08] !log [universalomega@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [19:56:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195608-ad661551 [19:56:51] !log [universalomega@test151] Starting deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to test151 [19:56:53] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195651-6b168625 [19:57:04] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 44962MiB (10.1% inode=100%); [19:58:05] !log [universalomega@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [19:58:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003195805-ad661551 [20:00:57] PROBLEM - os161 Disk Space on os161 is WARNING: DISK WARNING - free space: / 41353MiB (9.2% inode=100%); [20:02:52] RECOVERY - os161 Disk Space on os161 is OK: DISK OK - free space: / 45376MiB (10.1% inode=100%); [20:12:29] RECOVERY - os191 Disk Space on os191 is OK: DISK OK - free space: / 46390MiB (10.4% inode=100%); [20:18:32] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/225784d3cdef81ccdb2db55c7b88548322fb7258 [20:18:33] 02puppet/03main 07CosmicAlpha 03225784d mwdeploy: exclude LS [20:18:41] PROBLEM - os151 Disk Space on os151 is WARNING: DISK WARNING - free space: / 43749MiB (9.8% inode=100%); [20:19:31] [02mw-config] 07redbluegreenhat opened pull request #6578: T16083: Add $wgVisualEditorTabPosition to ManageWikiSettings (07miraheze:03main...07redbluegreenhat:03T16083) 13https://github.com/miraheze/mw-config/pull/6578 [20:20:10] miraheze/puppet - Universal-Omega the build has errored. [20:20:34] miraheze/mw-config - redbluegreenhat the build passed. [20:20:55] now that's a message I haven't seen in a while [20:22:32] RECOVERY - os151 Disk Space on os151 is OK: DISK OK - free space: / 44756MiB (10.0% inode=100%); [20:26:20] !log [void@puppet181] Upgraded packages on bast161: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:26:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202620-87692e37 [20:26:28] RECOVERY - bast161 APT on bast161 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [20:26:40] !log [void@puppet181] Upgraded packages on cloud15: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:26:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202640-89c2a7b5 [20:26:56] !log [void@puppet181] Upgraded packages on cloud21: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:26:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202656-75c77fc2 [20:27:16] !log [void@puppet181] Upgraded packages on cp161: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:27:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202716-1898e421 [20:27:33] !log [void@puppet181] Upgraded packages on bots171: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:27:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202733-ab245338 [20:27:54] !log [void@puppet181] Upgraded packages on cloud19: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:27:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202754-bdc72fcc [20:28:13] !log [void@puppet181] Upgraded packages on cp171: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:28:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202813-b76f729a [20:28:19] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/21d1bcf902e443b51a65b5d62d958066a9f543dc [20:28:19] 02puppet/03main 07CosmicAlpha 0321d1bcf mwdeploy: fix [20:28:23] RECOVERY - cloud15 APT on cloud15 is OK: APT OK: 192 packages available for upgrade (0 critical updates). [20:28:28] RECOVERY - cloud19 APT on cloud19 is OK: APT OK: 192 packages available for upgrade (0 critical updates). [20:28:33] !log [void@puppet181] Upgraded packages on cloud18: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:28:33] RECOVERY - bots171 APT on bots171 is OK: APT OK: 125 packages available for upgrade (0 critical updates). [20:28:35] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202833-c52a93df [20:28:52] RECOVERY - cloud21 APT on cloud21 is OK: APT OK: 184 packages available for upgrade (0 critical updates). [20:28:53] !log [void@puppet181] Upgraded packages on cloud16: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:28:55] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202853-9135ec1a [20:29:14] RECOVERY - cp161 APT on cp161 is OK: APT OK: 112 packages available for upgrade (0 critical updates). [20:29:14] RECOVERY - cloud16 APT on cloud16 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [20:29:14] !log [void@puppet181] Upgraded packages on bast181: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:29:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202914-b1b1ab1f [20:29:19] RECOVERY - cloud18 APT on cloud18 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [20:29:32] RECOVERY - bast181 APT on bast181 is OK: APT OK: 116 packages available for upgrade (0 critical updates). [20:29:34] !log [void@puppet181] Upgraded packages on db151: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:29:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202934-b3e48018 [20:29:49] miraheze/puppet - Universal-Omega the build has errored. [20:29:54] !log [void@puppet181] Upgraded packages on db161: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:29:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003202954-92a35285 [20:29:57] RECOVERY - cp171 APT on cp171 is OK: APT OK: 114 packages available for upgrade (0 critical updates). [20:30:14] !log [void@puppet181] Upgraded packages on db171: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:30:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203014-363296e5 [20:30:35] RECOVERY - db151 APT on db151 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [20:30:38] !log [void@puppet181] Upgraded packages on db201: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:30:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203038-9685859c [20:30:58] !log [void@puppet181] Upgraded packages on cloud17: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:31:00] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203058-e00b6e80 [20:31:01] RECOVERY - db171 APT on db171 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [20:31:24] !log [void@puppet181] Upgraded packages on changeprop202: openssl-provider-legacy, libssl-dev, libssl3t64, redis-server, redis-tools, openssl, and libpng16-16t64 [20:31:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203124-e00c0df3 [20:31:38] RECOVERY - db161 APT on db161 is OK: APT OK: 114 packages available for upgrade (0 critical updates). [20:31:43] !log [void@puppet181] Upgraded packages on db172: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:31:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203143-cf923239 [20:31:52] RECOVERY - cloud17 APT on cloud17 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [20:32:04] !log [void@puppet181] Upgraded packages on cloud20: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:32:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203204-80561d5c [20:32:09] RECOVERY - cloud20 APT on cloud20 is OK: APT OK: 192 packages available for upgrade (0 critical updates). [20:32:23] !log [void@puppet181] Upgraded packages on cp191: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:32:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203223-d8335abc [20:32:36] RECOVERY - changeprop202 APT on changeprop202 is OK: APT OK: 121 packages available for upgrade (0 critical updates). [20:32:41] RECOVERY - db201 APT on db201 is OK: APT OK: 121 packages available for upgrade (0 critical updates). [20:32:46] !log [void@puppet181] Upgraded packages on cp201: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:32:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203246-edb33b8d [20:32:58] RECOVERY - cp201 APT on cp201 is OK: APT OK: 114 packages available for upgrade (0 critical updates). [20:33:07] !log [void@puppet181] Upgraded packages on db182: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:33:10] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203307-48d3e5ab [20:33:20] RECOVERY - db182 APT on db182 is OK: APT OK: 116 packages available for upgrade (0 critical updates). [20:33:28] RECOVERY - db172 APT on db172 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [20:33:30] !log [void@puppet181] Upgraded packages on db181: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:33:32] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203330-f8f6a147 [20:33:50] !log [void@puppet181] Upgraded packages on eventgate182: openssl-provider-legacy, libssl-dev, libssl3t64, openssl, and libpng16-16t64 [20:33:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203350-e73b654a [20:34:02] RECOVERY - cp191 APT on cp191 is OK: APT OK: 114 packages available for upgrade (0 critical updates). [20:34:14] !log [void@puppet181] Upgraded packages on db192: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:34:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203414-c1b74bda [20:34:25] RECOVERY - db181 APT on db181 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [20:34:35] !log [void@puppet181] Upgraded packages on graylog161: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:34:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203435-2d340d9d [20:34:48] !log [void@puppet181] Upgraded packages on mattermost2: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:34:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203448-db77a920 [20:35:03] RECOVERY - eventgate182 APT on eventgate182 is OK: APT OK: 120 packages available for upgrade (0 critical updates). [20:35:09] RECOVERY - db192 APT on db192 is OK: APT OK: 121 packages available for upgrade (0 critical updates). [20:35:12] !log [void@puppet181] Upgraded packages on kafka182: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:35:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203512-058f246b [20:35:30] !log [void@puppet181] Upgraded packages on ldap171: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:35:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203530-8a65023b [20:35:37] RECOVERY - mattermost2 APT on mattermost2 is OK: APT OK: 114 packages available for upgrade (0 critical updates). [20:35:49] !log [void@puppet181] Upgraded packages on matomo211: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:35:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203549-7e34f479 [20:36:01] RECOVERY - graylog161 APT on graylog161 is OK: APT OK: 120 packages available for upgrade (0 critical updates). [20:36:18] !log [void@puppet181] Upgraded packages on matomo151: openssl-provider-legacy, libssl3t64, redis-server, redis-tools, libpng16-16t64, and openssl [20:36:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203618-6b5b5a72 [20:36:35] !log [void@puppet181] Upgraded packages on mem151: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:36:38] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203635-388d54ae [20:36:54] !log [void@puppet181] Upgraded packages on mem161: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:36:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203654-dcf90c8d [20:36:57] RECOVERY - kafka182 APT on kafka182 is OK: APT OK: 124 packages available for upgrade (0 critical updates). [20:36:58] RECOVERY - ldap171 APT on ldap171 is OK: APT OK: 111 packages available for upgrade (0 critical updates). [20:36:58] !log [universalomega@mwtask181] Starting deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to mwtask181 [20:37:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203658-54b7c613 [20:37:03] RECOVERY - matomo211 APT on matomo211 is OK: APT OK: 69 packages available for upgrade (0 critical updates). [20:37:18] !log [void@puppet181] Upgraded packages on mw151: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:37:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203718-fa2244da [20:37:37] !log [void@puppet181] Upgraded packages on llm191: openssl-provider-legacy, libssl3t64, openssl, and libpng16-16t64 [20:37:40] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203737-cb57ba2c [20:37:52] RECOVERY - matomo151 APT on matomo151 is OK: APT OK: 111 packages available for upgrade (0 critical updates). [20:37:59] !log [void@puppet181] Upgraded packages on mon181: openssl-provider-legacy, libssl3t64, redis-server, redis-tools, libpng16-16t64, and openssl [20:38:01] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203759-f8ce75df [20:38:02] RECOVERY - llm191 APT on llm191 is OK: APT OK: 124 packages available for upgrade (0 critical updates). [20:38:04] RECOVERY - mem161 APT on mem161 is OK: APT OK: 109 packages available for upgrade (0 critical updates). [20:38:19] !log [void@puppet181] Upgraded packages on mem201: openssl-provider-legacy, libssl3t64, libpng16-16t64, and openssl [20:38:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203819-2113e3e1 [20:38:28] RECOVERY - mon181 APT on mon181 is OK: APT OK: 106 packages available for upgrade (0 critical updates). [20:38:29] RECOVERY - mem151 APT on mem151 is OK: APT OK: 109 packages available for upgrade (0 critical updates). [20:38:43] !log [void@puppet181] Upgraded packages on mw162: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:38:45] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203843-e6cc2fe9 [20:38:46] PROBLEM - matomo151 Check unit status of matomo-queuedtracking-0 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-queuedtracking-0 [20:39:07] !log [void@puppet181] Upgraded packages on mw171: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:39:08] RECOVERY - mw151 APT on mw151 is OK: APT OK: 179 packages available for upgrade (0 critical updates). [20:39:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203907-6fc9aa17 [20:39:11] RECOVERY - mem201 APT on mem201 is OK: APT OK: 109 packages available for upgrade (0 critical updates). [20:39:28] PROBLEM - matomo151 Check unit status of matomo-queuedtracking-1 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-queuedtracking-1 [20:39:31] !log [void@puppet181] Upgraded packages on mw153: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:39:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203931-dcfd6c27 [20:39:40] PROBLEM - matomo211 Check unit status of matomo-queuedtracking-3 on matomo211 is CRITICAL: CRITICAL: Status of the systemd unit matomo-queuedtracking-3 [20:39:41] RECOVERY - mw162 APT on mw162 is OK: APT OK: 179 packages available for upgrade (0 critical updates). [20:39:56] !log [void@puppet181] Upgraded packages on mw172: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:39:58] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003203956-0efac45e [20:40:01] RECOVERY - mw171 APT on mw171 is OK: APT OK: 179 packages available for upgrade (0 critical updates). [20:40:03] PROBLEM - matomo211 Check unit status of matomo-queuedtracking-2 on matomo211 is CRITICAL: CRITICAL: Status of the systemd unit matomo-queuedtracking-2 [20:40:06] !log [universalomega@mwtask181] finished deploy of {'upgrade_world': True, 'versions': 'all', 'batch': True, 'debug': True} to mwtask181 - SUCCESS in 187s [20:40:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003204006-e48823e8 [20:40:22] !log [void@puppet181] Upgraded packages on mw152: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:40:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261003204022-f685d06e [20:40:41] RECOVERY - mw153 APT on mw153 is OK: APT OK: 179 packages available for upgrade (0 critical updates). [20:40:46] !log [void@puppet181] Upgraded packages on mw161: openssl-provider-legacy, libssl3t64, libpng-dev, libpng-tools, libpng16-16t64, and openssl [20:40:48]