[00:16:22] RECOVERY - cp37 Disk Space on cp37 is OK: DISK OK - free space: / 50758MiB (11% inode=99%); [00:42:22] PROBLEM - cp37 Disk Space on cp37 is WARNING: DISK WARNING - free space: / 49780MiB (10% inode=99%); [00:56:54] PROBLEM - matomo151 Current Load on matomo151 is WARNING: LOAD WARNING - total load average: 6.81, 5.99, 4.57 [00:58:54] RECOVERY - matomo151 Current Load on matomo151 is OK: LOAD OK - total load average: 4.98, 5.43, 4.53 [06:36:50] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 31.10 ms [06:38:52] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.09 ms [06:45:14] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [06:51:27] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.15 ms [09:12:20] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:20:42] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.09 ms [09:25:04] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:29:13] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.09 ms [09:33:35] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [09:44:02] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.10 ms [11:18:22] PROBLEM - cp37 Disk Space on cp37 is CRITICAL: DISK CRITICAL - free space: / 27220MiB (5% inode=99%); [11:29:33] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [11:37:55] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.09 ms [11:44:19] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [11:50:34] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.13 ms [11:58:57] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 70%, RTA = 31.19 ms [11:59:11] Oh fuck off mattermost [12:03:01] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.19 ms [12:15:31] PROBLEM - wiki.cowgl.xyz - Cloudflare on sslhost is CRITICAL: Name or service not knownHTTP CRITICAL - Unable to open TCP socket [13:08:07] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 90%, RTA = 31.06 ms [13:12:13] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.14 ms [13:18:35] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [13:24:49] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.16 ms [13:31:11] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [13:37:27] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.14 ms [14:50:39] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 50%, RTA = 31.13 ms [14:52:40] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.14 ms [14:59:03] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [15:13:46] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.16 ms [15:20:08] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [15:24:16] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.13 ms [15:35:55] PROBLEM - mw154 APT on mw154 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:36:00] PROBLEM - mem151 APT on mem151 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (2 critical updates). [15:36:04] PROBLEM - cloud15 APT on cloud15 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [15:37:24] PROBLEM - mw151 APT on mw151 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:38:12] PROBLEM - mw164 APT on mw164 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:39:21] PROBLEM - jobchron171 APT on jobchron171 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:39:36] PROBLEM - ping6 on ns2 is CRITICAL: PING CRITICAL - Packet loss = 16%, RTA = 139.92 ms [15:39:42] PROBLEM - ldap171 APT on ldap171 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (2 critical updates). [15:39:53] PROBLEM - mw161 APT on mw161 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:40:04] PROBLEM - mwtask171 APT on mwtask171 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:40:12] PROBLEM - mw162 APT on mw162 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:40:59] PROBLEM - mw184 APT on mw184 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:41:09] PROBLEM - phorge171 APT on phorge171 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [15:41:34] RECOVERY - ping6 on ns2 is OK: PING OK - Packet loss = 0%, RTA = 139.92 ms [15:41:36] PROBLEM - os162 APT on os162 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (2 critical updates). [15:42:29] PROBLEM - cloud18 APT on cloud18 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [15:43:34] PROBLEM - mw174 APT on mw174 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:43:51] PROBLEM - kafka181 APT on kafka181 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:44:37] PROBLEM - mw173 APT on mw173 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:45:13] PROBLEM - mattermost1 APT on mattermost1 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (2 critical updates). [15:45:48] PROBLEM - cloud17 APT on cloud17 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [15:46:10] PROBLEM - ns2 APT on ns2 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (2 critical updates). [15:46:13] PROBLEM - swiftobject171 APT on swiftobject171 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (2 critical updates). [15:46:36] PROBLEM - mw153 APT on mw153 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:46:36] PROBLEM - mw171 APT on mw171 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:47:48] PROBLEM - prometheus151 APT on prometheus151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:48:16] PROBLEM - mon181 APT on mon181 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:48:48] PROBLEM - mw183 APT on mw183 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:49:06] PROBLEM - mw172 APT on mw172 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:49:09] PROBLEM - swiftproxy161 APT on swiftproxy161 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (2 critical updates). [15:49:24] PROBLEM - db151 APT on db151 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (2 critical updates). [15:49:30] PROBLEM - swiftobject151 APT on swiftobject151 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (2 critical updates). [15:50:50] PROBLEM - mwtask161 APT on mwtask161 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:51:10] PROBLEM - mwtask151 APT on mwtask151 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:51:45] PROBLEM - cp36 APT on cp36 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [15:52:16] PROBLEM - changeprop151 APT on changeprop151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [15:54:10] PROBLEM - ns1 APT on ns1 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (2 critical updates). [15:55:12] PROBLEM - mw152 APT on mw152 is CRITICAL: APT CRITICAL: 69 packages available for upgrade (2 critical updates). [15:56:16] PROBLEM - test151 APT on test151 is CRITICAL: APT CRITICAL: 71 packages available for upgrade (2 critical updates). [15:56:23] PROBLEM - mw181 APT on mw181 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [15:57:32] PROBLEM - bast181 APT on bast181 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (2 critical updates). [15:57:38] PROBLEM - bots171 APT on bots171 is CRITICAL: APT CRITICAL: 72 packages available for upgrade (2 critical updates). [15:57:38] PROBLEM - matomo151 APT on matomo151 is CRITICAL: APT CRITICAL: 46 packages available for upgrade (2 critical updates). [15:57:46] PROBLEM - swiftac171 APT on swiftac171 is CRITICAL: APT CRITICAL: 66 packages available for upgrade (2 critical updates). [15:57:56] PROBLEM - graphite151 APT on graphite151 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (2 critical updates). [15:58:07] PROBLEM - db172 APT on db172 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [15:58:27] PROBLEM - swiftobject161 APT on swiftobject161 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (2 critical updates). [15:58:48] PROBLEM - reports171 APT on reports171 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [15:59:32] PROBLEM - db161 APT on db161 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [15:59:34] PROBLEM - swiftobject181 APT on swiftobject181 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [15:59:58] PROBLEM - mw182 APT on mw182 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [16:00:25] PROBLEM - mem161 APT on mem161 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (2 critical updates). [16:00:35] PROBLEM - db171 APT on db171 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [16:00:51] PROBLEM - bast161 APT on bast161 is CRITICAL: APT CRITICAL: 60 packages available for upgrade (2 critical updates). [16:00:55] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 80%, RTA = 31.14 ms [16:00:58] PROBLEM - rdb151 APT on rdb151 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [16:01:01] PROBLEM - os161 APT on os161 is CRITICAL: APT CRITICAL: 61 packages available for upgrade (2 critical updates). [16:01:13] PROBLEM - eventgate181 APT on eventgate181 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [16:01:54] PROBLEM - cp37 APT on cp37 is CRITICAL: APT CRITICAL: 43 packages available for upgrade (2 critical updates). [16:02:02] PROBLEM - puppet181 APT on puppet181 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [16:02:05] PROBLEM - graylog161 APT on graylog161 is CRITICAL: APT CRITICAL: 64 packages available for upgrade (2 critical updates). [16:02:53] PROBLEM - db181 APT on db181 is CRITICAL: APT CRITICAL: 2 packages available for upgrade (2 critical updates). [16:02:59] PROBLEM - cloud16 APT on cloud16 is CRITICAL: APT CRITICAL: 65 packages available for upgrade (2 critical updates). [16:03:12] PROBLEM - mw163 APT on mw163 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [16:04:05] PROBLEM - db182 APT on db182 is CRITICAL: APT CRITICAL: 44 packages available for upgrade (2 critical updates). [16:04:12] PROBLEM - mwtask181 APT on mwtask181 is CRITICAL: APT CRITICAL: 70 packages available for upgrade (2 critical updates). [16:04:42] PROBLEM - swiftproxy171 APT on swiftproxy171 is CRITICAL: APT CRITICAL: 67 packages available for upgrade (2 critical updates). [16:04:58] PROBLEM - os151 APT on os151 is CRITICAL: APT CRITICAL: 62 packages available for upgrade (2 critical updates). [16:05:00] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.12 ms [16:11:22] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [16:17:37] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.10 ms [16:23:59] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [16:30:12] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.20 ms [18:09:36] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [18:20:00] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.13 ms [19:28:57] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [19:30:58] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.08 ms [19:39:22] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [20:21:35] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.10 ms [21:58:09] PROBLEM - changeprop151 Current Load on changeprop151 is CRITICAL: LOAD CRITICAL - total load average: 17.69, 17.69, 9.66 [22:00:09] RECOVERY - changeprop151 Current Load on changeprop151 is OK: LOAD OK - total load average: 2.97, 12.09, 8.58 [22:09:59] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [22:16:14] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.11 ms [22:20:35] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [22:21:13] [02mw-config] 07OAuthority pushed 1 new commit to 03master 13https://github.com/miraheze/mw-config/commit/6de09e365b36bd5c7475d625a05816c3569c462b [22:21:13] 02mw-config/03master 07Luke 036de09e3 Increase $wgUploadStash if Moderation is enabled… [22:22:11] miraheze/mw-config - OAuthority the build passed. [22:28:57] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.12 ms [23:16:51] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [23:25:13] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.11 ms [23:27:53] [02mw-config] 07edwardspec opened pull request #5791: Fix typo in variable name ($wgUploadStashMaxAge) (07miraheze:03master...07edwardspec:03master) 13https://github.com/miraheze/mw-config/pull/5791 [23:43:42] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100% [23:52:04] RECOVERY - ping6 on mattermost1 is OK: PING OK - Packet loss = 0%, RTA = 31.05 ms [23:56:26] PROBLEM - ping6 on mattermost1 is CRITICAL: PING CRITICAL - Packet loss = 100%