[01:27:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: minFraud_endpoint_critical 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[01:32:17] RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[11:58:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7813 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 62 days 10 hours - memory use is 6.50M (peak 9.13M, 0.14% of max, fragmentation 1.75%), connected_slaves is 3, donations is 4, jobs is 0, jobs-adyen is 0, jobs-paypal is 124, payments-antifraud is 11, payments-init is 0, pending is 2, refund is 0, unsubscribe is 1 https://icinga.wikimed
[11:58:21] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:03:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7554 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 10 hours - memory use is 6.33M (peak 9.13M, 0.13% of max, fragmentation 1.75%), connected_slaves is 3, donations is 10, jobs is 0, jobs-adyen is 0, jobs-paypal is 87, payments-antifraud is 9, payments-init is 7, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedi
[12:03:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:08:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8058 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 5 keys, up 62 days 11 hours - memory use is 6.67M (peak 9.13M, 0.14% of max, fragmentation 1.66%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 98, payments-antifraud is 7, payments-init is 0, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedia
[12:08:21] bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:13:17] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8528 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 11 hours - memory use is 6.94M (peak 9.13M, 0.14% of max, fragmentation 1.67%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 103, payments-antifraud is 10, payments-init is 8, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimed
[12:13:17] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:18:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9004 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 11 hours - memory use is 7.13M (peak 9.13M, 0.15% of max, fragmentation 1.68%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 106, payments-antifraud is 3, payments-init is 0, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedi
[12:18:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:23:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8679 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 11 hours - memory use is 7.31M (peak 9.13M, 0.14% of max, fragmentation 1.66%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 117, payments-antifraud is 15, payments-init is 10, pending is 1, refund is 0, unsubscribe is 0 https://icinga.wikime
[12:23:21] gi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:28:17] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9149 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 11 hours - memory use is 7.27M (peak 9.13M, 0.14% of max, fragmentation 1.64%), connected_slaves is 3, donations is 77, jobs is 0, jobs-adyen is 1, jobs-paypal is 101, payments-antifraud is 8, payments-init is 0, pending is 0, refund is 1, unsubscribe is 0 https://icinga.wikimed
[12:28:17] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:33:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9685 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 62 days 11 hours - memory use is 7.62M (peak 9.13M, 0.15% of max, fragmentation 1.63%), connected_slaves is 3, donations is 36, jobs is 0, jobs-adyen is 0, jobs-paypal is 115, payments-antifraud is 16, payments-init is 10, pending is 1, refund is 0, unsubscribe is 2 https://icinga.wikim
[12:33:21] cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:38:17] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 10239 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 11 hours - memory use is 7.94M (peak 9.13M, 0.15% of max, fragmentation 1.58%), connected_slaves is 3, donations is 37, jobs is 0, jobs-adyen is 0, jobs-paypal is 119, payments-antifraud is 3, payments-init is 2, pending is 0, refund is 0, unsubscribe is 2 https://icinga.wikime
[12:38:17] gi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:43:17] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9994 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 11 hours - memory use is 7.73M (peak 9.13M, 0.15% of max, fragmentation 1.60%), connected_slaves is 3, donations is 4, jobs is 0, jobs-adyen is 0, jobs-paypal is 105, payments-antifraud is 14, payments-init is 9, pending is 0, refund is 0, unsubscribe is 2 https://icinga.wikimed
[12:43:17] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:48:17] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 10216 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 5 keys, up 62 days 11 hours - memory use is 7.64M (peak 9.13M, 0.15% of max, fragmentation 1.61%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 1, payments-init is 0, pending is 0, refund is 0, unsubscribe is 2 https://icinga.wikimedia
[12:48:17] bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:53:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 10216 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 11 hours - memory use is 7.73M (peak 9.13M, 0.15% of max, fragmentation 1.60%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 12, payments-init is 7, pending is 0, refund is 0, unsubscribe is 2 https://icinga.wikimedi
[12:53:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[12:58:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 10219 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 11 hours - memory use is 7.73M (peak 9.13M, 0.15% of max, fragmentation 1.58%), connected_slaves is 3, donations is 3, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 11, payments-init is 0, pending is 0, refund is 0, unsubscribe is 2 https://icinga.wikimedi
[12:58:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:03:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9433 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 11 hours - memory use is 7.26M (peak 9.13M, 0.15% of max, fragmentation 1.66%), connected_slaves is 3, donations is 24, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 10, payments-init is 7, pending is 1, refund is 0, unsubscribe is 0 https://icinga.wikimedi
[13:03:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:08:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9435 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 5 keys, up 62 days 12 hours - memory use is 7.21M (peak 9.13M, 0.14% of max, fragmentation 1.64%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 6, payments-init is 0, pending is 1, refund is 0, unsubscribe is 0 https://icinga.wikimedia.
[13:08:21] in/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:13:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9436 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 12 hours - memory use is 7.29M (peak 9.13M, 0.15% of max, fragmentation 1.64%), connected_slaves is 3, donations is 1, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 13, payments-init is 10, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedi
[13:13:21] -bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:18:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 9437 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 12 hours - memory use is 7.21M (peak 9.13M, 0.14% of max, fragmentation 1.64%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 3, payments-antifraud is 3, payments-init is 2, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedia.
[13:18:21] in/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:23:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8652 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 8 keys, up 62 days 12 hours - memory use is 6.96M (peak 9.13M, 0.14% of max, fragmentation 1.67%), connected_slaves is 3, donations is 11, jobs is 0, jobs-adyen is 0, jobs-paypal is 1, payments-antifraud is 11, payments-init is 10, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimed
[13:23:21] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:28:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8652 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 12 hours - memory use is 6.87M (peak 9.13M, 0.14% of max, fragmentation 1.63%), connected_slaves is 3, donations is 79, jobs is 0, jobs-adyen is 0, jobs-paypal is 3, payments-antifraud is 2, payments-init is 6, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedia
[13:28:21] bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:33:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8655 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 12 hours - memory use is 6.91M (peak 9.13M, 0.14% of max, fragmentation 1.68%), connected_slaves is 3, donations is 38, jobs is 0, jobs-adyen is 1, jobs-paypal is 0, payments-antifraud is 12, payments-init is 11, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimed
[13:33:21] i-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:38:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 8655 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 12 hours - memory use is 6.97M (peak 9.13M, 0.14% of max, fragmentation 1.64%), connected_slaves is 3, donations is 31, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 7, payments-init is 3, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikimedia
[13:38:21] bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:43:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7864 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 9 keys, up 62 days 12 hours - memory use is 6.40M (peak 9.13M, 0.13% of max, fragmentation 1.72%), connected_slaves is 3, donations is 13, jobs is 0, jobs-adyen is 33, jobs-paypal is 1, payments-antifraud is 12, payments-init is 10, pending is 0, refund is 0, unsubscribe is 0 https://icinga.wikime
[13:43:21] gi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:48:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7864 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 12 hours - memory use is 6.36M (peak 9.13M, 0.13% of max, fragmentation 1.71%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 3, payments-init is 1, pending is 1, refund is 0, unsubscribe is 0 https://icinga.wikimedia.
[13:48:21] in/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:53:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7865 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 12 hours - memory use is 6.57M (peak 9.13M, 0.13% of max, fragmentation 1.70%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 17, payments-init is 8, pending is 2, refund is 0, unsubscribe is 0 https://icinga.wikimedia
[13:53:21] bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[13:58:21] PROBLEM - check_redis on frqueue1003 is CRITICAL: CRITICAL: recurring is 7866 7500 - REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 6 keys, up 62 days 12 hours - memory use is 6.33M (peak 9.13M, 0.13% of max, fragmentation 1.73%), connected_slaves is 3, donations is 0, jobs is 0, jobs-adyen is 0, jobs-paypal is 1, payments-antifraud is 6, payments-init is 0, pending is 1, refund is 0, unsubscribe is 0 https://icinga.wikimedia.
[13:58:21] in/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_redis
[14:23:21] RECOVERY - check_redis on frqueue1003 is OK: OK: REDIS 5.0.14 on 127.0.0.1:6379 has 1 databases (db0) with 7 keys, up 62 days 13 hours - memory use is 5.54M (peak 9.13M, 0.12% of max, fragmentation 1.84%), connected_slaves is 3, donations is 2, jobs is 0, jobs-adyen is 0, jobs-paypal is 0, payments-antifraud is 16, payments-init is 10, pending is 0, recurring is 6270, refund is 0, unsubscribe is 0 https://icinga.wikimedia.org/cgi-bin/ici
[14:23:21] fo.cgi?type=2&host=frqueue1003&service=check_redis
[22:32:25] (03PS1) 10Zabe: Use namespaced CentralAuthUser [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/752315 (https://phabricator.wikimedia.org/T298840)