[00:05:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-email-preferences: Changing other options in email prefs centre while already snoozed re-snoozes - https://phabricator.wikimedia.org/T434711#12218189 (10dmiranda) 05Open→03In progress a:03dmiranda [00:24:04] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first faiure email template syntax issue - https://phabricator.wikimedia.org/T434965 (10AnnWF) 03NEW [00:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [00:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [00:39:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first faiure email template syntax issue - https://phabricator.wikimedia.org/T434965#12218224 (10AnnWF) Fixed by update the template to using {if "{contact.first_name}"} instead of {if '{contact.first_name}'}. We should not have first name contains any... [00:39:38] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first faiure email template syntax issue - https://phabricator.wikimedia.org/T434965#12218225 (10AnnWF) a:03AnnWF [01:52:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first failure email template syntax issue - https://phabricator.wikimedia.org/T434965#12218262 (10Reedy) [01:52:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first failure email template syntax issue - https://phabricator.wikimedia.org/T434965#12218263 (10Reedy) [04:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [04:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [08:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [08:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [12:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [12:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [16:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [16:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [19:00:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [19:01:39] FIRING: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [19:01:39] FIRING: HighPaymentMethodFailures: Average cc payment method failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [19:01:39] FIRING: HighPaymentSchemeFailures: Average payment method scheme failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [19:05:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.1] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [19:06:39] RESOLVED: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [19:06:39] RESOLVED: HighPaymentMethodFailures: Average cc payment method failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [19:06:39] RESOLVED: HighPaymentSchemeFailures: Average payment method scheme failures are high [5.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [19:10:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [19:20:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [11.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [19:26:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [19:31:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.1] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [20:24:35] FIRING: [2x] KafkaBrokerDown: Kafka Broker kafka-jumbo1018 is down: [0] - https://alerts.wikimedia.org/?q=alertname%3DKafkaBrokerDown [20:24:35] FIRING: [2x] KafkaJumboNoData: Kafka Broker hasn't received data from kafka-jumbo1018 in 60 minutes - https://alerts.wikimedia.org/?q=alertname%3DKafkaJumboNoData [23:25:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [11.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages