[00:00:15] PROBLEM - check_mysql on payments2006 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2006&service=check_mysql [00:00:19] PROBLEM - check_mysql on payments2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2005&service=check_mysql [00:01:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [00:01:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [00:02:11] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [00:02:11] PROBLEM - check_mysql on frdata2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:02:15] PROBLEM - check_mysql on payments2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_mysql [00:02:17] PROBLEM - check_redis on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 1532 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 1532, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis [00:02:19] PROBLEM - check_redis_donor_prefs on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 1538 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 1538, memory use is 2.20M (peak 2.24M, 0.11% of max, fragmentation 4.00%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis_donor_prefs [00:02:19] PROBLEM - check_redis on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 1534 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 1534, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis [00:02:21] PROBLEM - check_redis_donor_prefs on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 1541 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 1541, memory use is 2.18M (peak 2.24M, 0.11% of max, fragmentation 4.12%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis_donor_prefs [00:03:11] PROBLEM - check_mysql on frdb1007 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1007&service=check_mysql [00:04:21] (03CR) 10Eileen: Add updateAnnualDonors API Action This ended up being too complex to handle with an API call directly (let alone from cv), so I've added an (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203508 (https://phabricator.wikimedia.org/T377438) (owner: 10Lars SG) [00:04:54] (03PS3) 10Eileen: Fix comparison to ignore float crazies [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203558 (https://phabricator.wikimedia.org/T399782) [00:05:13] PROBLEM - check_mysql on payments2006 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2006&service=check_mysql [00:05:17] PROBLEM - check_mysql on payments2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2005&service=check_mysql [00:06:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [00:06:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [00:07:09] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [00:07:11] PROBLEM - check_mysql on frdata2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:07:15] PROBLEM - check_mysql on payments2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_mysql [00:07:17] PROBLEM - check_redis on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 1832 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 1832, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis [00:07:19] PROBLEM - check_redis_donor_prefs on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 1838 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 1838, memory use is 2.20M (peak 2.24M, 0.11% of max, fragmentation 3.99%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis_donor_prefs [00:07:19] PROBLEM - check_redis on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 1835 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 1835, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis [00:07:21] PROBLEM - check_redis_donor_prefs on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 1841 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 1841, memory use is 2.18M (peak 2.24M, 0.11% of max, fragmentation 4.12%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis_donor_prefs [00:08:11] PROBLEM - check_mysql on frdb1007 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1007&service=check_mysql [00:09:06] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1203565 [00:10:13] PROBLEM - check_mysql on payments2006 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2006&service=check_mysql [00:10:15] PROBLEM - check_mysql on payments2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2005&service=check_mysql [00:11:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [00:11:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [00:12:11] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [00:12:11] PROBLEM - check_mysql on frdata2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:12:15] PROBLEM - check_mysql on payments2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_mysql [00:12:17] PROBLEM - check_redis on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 2132 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 2132, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis [00:12:19] PROBLEM - check_redis_donor_prefs on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 2138 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 2138, memory use is 2.20M (peak 2.24M, 0.11% of max, fragmentation 3.99%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis_donor_prefs [00:12:19] PROBLEM - check_redis on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 2134 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 2134, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis [00:12:21] PROBLEM - check_redis_donor_prefs on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 2140 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 2140, memory use is 2.18M (peak 2.24M, 0.11% of max, fragmentation 4.12%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis_donor_prefs [00:12:56] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1203565 (owner: 10Eileen) [00:13:13] PROBLEM - check_mysql on frdb1007 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1007&service=check_mysql [00:15:13] PROBLEM - check_mysql on payments2006 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2006&service=check_mysql [00:15:17] PROBLEM - check_mysql on payments2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2005&service=check_mysql [00:16:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [00:16:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [00:17:11] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [00:17:11] PROBLEM - check_mysql on frdata2002 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:17:15] PROBLEM - check_mysql on payments2004 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_mysql [00:17:17] PROBLEM - check_redis on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 2432 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 2432, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis [00:17:19] PROBLEM - check_redis_donor_prefs on frqueue2002 is CRITICAL: CRITICAL: replication_delay is 2438 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 2438, memory use is 2.20M (peak 2.24M, 0.11% of max, fragmentation 3.99%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis_donor_prefs [00:17:19] PROBLEM - check_redis on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 2434 300 - REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 2434, memory use is 259.38M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis [00:17:21] PROBLEM - check_redis_donor_prefs on frqueue2003 is CRITICAL: CRITICAL: replication_delay is 2440 300 - REDIS 7.0.15 on 127.0.0.1:6380 has 0 databases (), up 12 days 1 hours - replication_delay is 2440, memory use is 2.18M (peak 2.24M, 0.11% of max, fragmentation 4.12%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis_donor_prefs [00:18:13] PROBLEM - check_mysql on frdb1007 is CRITICAL: Slave IO: Connecting Slave SQL: Yes Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1007&service=check_mysql [00:20:13] RECOVERY - check_mysql on payments2006 is OK: Uptime: 1051203 Threads: 6 Questions: 4620750 Slow queries: 0 Opens: 145 Open tables: 139 Queries per second avg: 4.395 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2006&service=check_mysql [00:20:15] RECOVERY - check_mysql on payments2005 is OK: Uptime: 1051677 Threads: 5 Questions: 4632495 Slow queries: 0 Opens: 144 Open tables: 138 Queries per second avg: 4.404 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2005&service=check_mysql [00:21:09] RECOVERY - check_mysql on frdb2004 is OK: Uptime: 1048360 Threads: 4 Questions: 95633782 Slow queries: 567 Opens: 5021 Open tables: 1410 Queries per second avg: 91.222 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [00:21:13] RECOVERY - check_mysql on frdb2005 is OK: Uptime: 1048307 Threads: 4 Questions: 95632889 Slow queries: 570 Opens: 4799 Open tables: 1306 Queries per second avg: 91.226 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [00:22:11] RECOVERY - check_mysql on frdb2003 is OK: Uptime: 364935 Threads: 279 Questions: 102117738 Slow queries: 2445 Opens: 2959 Open tables: 2271 Queries per second avg: 279.824 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 181 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [00:22:11] RECOVERY - check_mysql on frdata2002 is OK: Uptime: 1048439 Threads: 3 Questions: 479272 Slow queries: 0 Opens: 135 Open tables: 129 Queries per second avg: 0.457 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2002&service=check_mysql [00:22:15] RECOVERY - check_mysql on payments2004 is OK: Uptime: 1052070 Threads: 6 Questions: 4629303 Slow queries: 0 Opens: 144 Open tables: 138 Queries per second avg: 4.400 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_mysql [00:22:19] RECOVERY - check_redis on frqueue2002 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 31 keys, up 12 days 1 hours - replication_delay is 1, memory use is 259.07M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis [00:22:19] RECOVERY - check_redis_donor_prefs on frqueue2002 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6380 has 1 databases (db0) with 1 keys, up 12 days 1 hours - replication_delay is 7, memory use is 2.18M (peak 2.24M, 0.10% of max, fragmentation 4.02%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_redis_donor_prefs [00:22:19] RECOVERY - check_redis on frqueue2003 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6379 has 1 databases (db0) with 32 keys, up 12 days 1 hours - replication_delay is 0, memory use is 259.11M (peak 518.51M, 3.36% of max, fragmentation 1.06%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis [00:22:19] RECOVERY - check_redis_donor_prefs on frqueue2003 is OK: OK: REDIS 7.0.15 on 127.0.0.1:6380 has 1 databases (db0) with 1 keys, up 12 days 1 hours - replication_delay is 8, memory use is 2.20M (peak 2.24M, 0.11% of max, fragmentation 4.07%) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2003&service=check_redis_donor_prefs [00:23:13] RECOVERY - check_mysql on frdb1007 is OK: Uptime: 952582 Threads: 4 Questions: 670886319 Slow queries: 197 Opens: 6029 Open tables: 2060 Queries per second avg: 704.281 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 620 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1007&service=check_mysql [00:42:26] .!log civicrm upgraded from 0952d4ab to a519e045 [01:21:00] 06Fundraising-Backlog, 10fundraising-tech-ops, 10Observability-Alerting, 13Patch-For-Review: Shift frack alerting to use prometheus-alertmanager instead of icinga - https://phabricator.wikimedia.org/T367370#11361652 (10Dwisehaupt) @fgiunchedi We (fr-tech) are getting close to live testing with some alerts.... [02:01:58] (03CR) 10Wfan: [C:03+2] Send activity tracking for recurring pause + cancel [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1200169 (https://phabricator.wikimedia.org/T407644) (owner: 10Ejegg) [02:04:46] (03Merged) 10jenkins-bot: Send activity tracking for recurring pause + cancel [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1200169 (https://phabricator.wikimedia.org/T407644) (owner: 10Ejegg) [02:11:48] (03CR) 10Wfan: [C:03+2] Add test for activity tracking parameter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1200200 (https://phabricator.wikimedia.org/T407644) (owner: 10Ejegg) [02:17:37] thanks wfan! [02:18:10] there were a couple of parent patches for that test one that need C+2 too [02:53:21] wfan I'm working on something that will make the donation-specific donation_rules and the rates available in the JS, so we can use some of this same logic to validate minimums: https://phabricator.wikimedia.org/diffusion/EDOI/browse/master/modules/js/ext.donationInterface.monthlyConvert.js#L103 [02:57:27] since there are processor-specific overrides, we'll need to add the processor name to the donor summary [03:00:09] (03PS1) 10Ejegg: Add payment processor name to summary output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203585 (https://phabricator.wikimedia.org/T388753) [03:08:15] (03PS2) 10Ejegg: Add payment processor & country to summary output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203585 (https://phabricator.wikimedia.org/T388753) [03:08:33] right, could have a different country per recurring, too ^^^ [03:18:41] (03PS1) 10Ejegg: Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) [03:21:06] (03CR) 10CI reject: [V:04-1] Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) (owner: 10Ejegg) [03:33:57] wfan ooops I think I missed one change to extension.json for that trackingParams patch [03:38:03] (03PS2) 10Ejegg: Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) [03:38:03] (03PS1) 10Ejegg: Add trackingParams to packageFiles [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203588 (https://phabricator.wikimedia.org/T407644) [03:39:00] wfan: I needed that last one ^^^ to actually load portal pages once the 'Send act [03:39:16] ivity tracking for recurring pause + cancel' patch merged [03:39:21] (03PS3) 10Ejegg: Add properties for DonorPortal amount validation [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1203587 (https://phabricator.wikimedia.org/T388753) [03:39:29] k, really |away now [03:43:37] (03CR) 10Eileen: [C:03+2] Add payment processor & country to summary output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203585 (https://phabricator.wikimedia.org/T388753) (owner: 10Ejegg) [04:01:26] (03Merged) 10jenkins-bot: Add payment processor & country to summary output [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203585 (https://phabricator.wikimedia.org/T388753) (owner: 10Ejegg) [04:20:21] (03PS1) 10Eileen: Set batch status to verified_total if it matches the expected settlement [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203593 (https://phabricator.wikimedia.org/T399782) [04:39:16] (03CR) 10CI reject: [V:04-1] Set batch status to verified_total if it matches the expected settlement [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203593 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [07:48:11] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1203649 (owner: 10L10n-bot) [08:41:04] 06Fundraising-Backlog, 10fundraising-tech-ops, 10Observability-Alerting, 13Patch-For-Review: Shift frack alerting to use prometheus-alertmanager instead of icinga - https://phabricator.wikimedia.org/T367370#11362021 (10fgiunchedi) Thank you for following up @Dwisehaupt and I'm glad to know you are making p... [09:19:41] 06Fundraising-Backlog: Add (check?) currency minimums on gravy - https://phabricator.wikimedia.org/T409522#11362109 (10RKumar_WMF) Hi Anil - I have shared the minimum cost scenarios document with you. It does say: Please note: 10 INR is the systemic minimum for India [09:40:29] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1203649 (owner: 10L10n-bot) [10:03:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi Unsubscribe Description Not Updating - https://phabricator.wikimedia.org/T409478#11362288 (10krobinson) @Lars Does that mean any donor who has unsubscribed themselves using the link (not just... [11:05:06] !log payments-wiki upgraded from 1d4b0d2a to b67adca1 [11:05:08] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [14:36:32] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-current-sprint: Prevent [ change email ] from breaking onto multiple lines in email preferences centre - https://phabricator.wikimedia.org/T409576#11363247 (10AKanji-WMF) [14:36:38] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10FR-Donor-portal, 07fr-current-sprint, 13Patch-For-Review: Donor Portal - Problems Logging In Flow - https://phabricator.wikimedia.org/T407453#11363248 (10AKanji-WMF) [14:36:40] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10FR-Donor-portal, 07fr-current-sprint, 13Patch-For-Review: Add donation link for restart recurring - https://phabricator.wikimedia.org/T409590#11363250 (10AKanji-WMF) [14:36:44] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10FR-Donor-portal, 07fr-current-sprint, 13Patch-For-Review: Implement interim "change payment type" flow - https://phabricator.wikimedia.org/T396271#11363251 (10AKanji-WMF) [14:59:22] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 07fr-payments-ACH, 07payments-orchestration: Update Civi Record on status Pending from Webhook - https://phabricator.wikimedia.org/T409503#11363352 (10AKanji-WMF) [14:59:25] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 07fr-payments-ACH, 07payments-orchestration: Create Civi Contribution with Pending Status for ACH - https://phabricator.wikimedia.org/T409502#11363354 (10AKanji-WMF) [14:59:31] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 07fr-payments-ACH, 07payments-orchestration: Add support for a new parameter to show a Pending Thank You (TY) page - https://phabricator.wikimedia.org/T409500#11363356 (10AKanji-WMF) [15:48:28] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing, 10FR-WMF-Audit: Automate - MGGO appeal update - https://phabricator.wikimedia.org/T406662#11363528 (10RLewis) @Eileenmcnaughton did you need the list of appeal codes we use in MGFE per fiscal yea... [16:21:04] 03Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10FR-Donor-portal, 10FR-email-preferences, and 2 others: Implement recurring donation amount modification for donor portal - https://phabricator.wikimedia.org/T388755#11363634 (10RLopez-WMF) @AnnWF i belive we can keep the alert for now, but I can put tog... [17:58:08] (03PS1) 10Reedy: build: Update MediaWiki requirement to 1.46 [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1203901 (https://phabricator.wikimedia.org/T409239) [18:01:08] (03PS1) 10Reedy: build: Update MediaWiki requirement to 1.46 [extensions/FundraisingTranslateWorkflow] - 10https://gerrit.wikimedia.org/r/1203941 (https://phabricator.wikimedia.org/T409239) [18:03:14] (03PS1) 10Reedy: build: Update MediaWiki requirement to 1.46 [extensions/LandingCheck] - 10https://gerrit.wikimedia.org/r/1203966 (https://phabricator.wikimedia.org/T409239) [21:59:10] (03PS4) 10Eileen: Fix comparison to ignore float crazies [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203558 (https://phabricator.wikimedia.org/T399782) [22:15:15] (03CR) 10CI reject: [V:04-1] Fix comparison to ignore float crazies [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203558 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [22:42:50] (03PS2) 10Eileen: Do not attempt to calculate contribution_extra for Adyen fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203554 (https://phabricator.wikimedia.org/T399782) [22:42:50] (03PS2) 10Eileen: Address deprecation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203560 (https://phabricator.wikimedia.org/T399782) [22:42:50] (03PS2) 10Eileen: Add fail information to test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203561 [22:42:50] (03PS5) 10Eileen: Fix comparison to ignore float crazies [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203558 (https://phabricator.wikimedia.org/T399782) [22:59:06] (03CR) 10CI reject: [V:04-1] Address deprecation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203560 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [23:01:16] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1203560 (https://phabricator.wikimedia.org/T399782) (owner: 10Eileen) [23:22:12] PROBLEM - check_nginx on payments1005 is CRITICAL: PROCS CRITICAL: 101 processes with command name nginx https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1005&service=check_nginx [23:27:10] RECOVERY - check_nginx on payments1005 is OK: PROCS OK: 52 processes with command name nginx https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1005&service=check_nginx