[00:00:55] (03PS1) 10Ejegg: Backfill payment_instrument_id for monthly convert [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031095 [00:01:22] fr-tech I think that will work to backfill the IDs we need ^^^ [00:01:46] does that limit of 'created since 2024-03-01' seem OK? [00:02:11] Should be easy enough to go back further if we have older messages to requeue [00:06:23] (03PS1) 10Eileen: Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) [00:06:28] (03PS1) 10Eileen: Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) [00:07:15] ejegg: I'll take a look - I haven't seen any more since that last deploy which slight matches my impression they were failing straight after the recur was created [00:07:40] eileen: oh weird, I thought those were monthly converts! [00:08:04] which are charged for the first time a month after the creation [00:08:20] I'm not too sure - I didn't look that carefully but had kinda formed that impression [00:08:20] so we still want to requeue the existing messages that are sitting in damaged [00:08:24] yep [00:08:32] let's look at one of the damaged ones [00:08:37] & maybe there are some NOT in damaged? before we fixed that? [00:09:10] /civicrm/damaged/edit?action=update&id=633403&reset=1 [00:09:40] ejegg: yeah - so your upgrade wouldn't work for the ones I looked at as they did not have a contribution - which is why it couldnt' find it [00:11:43] yep, the corresponding recur row for that damaged message has start_date 1 month after create_date so it's a monthly convert [00:12:00] eileen so you can't join on contribution_recur_id for these [00:12:17] because it's a one-time donation, then a recurring one that starts a month later [00:12:41] instead, we make the recurring processor_id be the same as the one-time contribution's gateway_txn_id [00:13:04] so in this case, we have contribution_recur.processor_id=ZXVJ9PCKQ95CKMG2 [00:13:09] ah ok - we could do that in the message class too - but I guess the theory is we won't need to going forwards [00:13:32] and there is indeed a row where wmf_contribution_extra.gateway_txn_id=ZXVJ9PCKQ95CKMG2 [00:13:47] yeah, this is just to record all the things that are in the damaged table right now [00:13:57] and the ones that might come in the next month [00:25:46] (03CR) 10CI reject: [V:04-1] Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [00:26:05] (03CR) 10CI reject: [V:04-1] Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [00:27:55] (03CR) 10Eileen: "So maybe it doesn't matter but when I tried the underlying query I find 24k contacts - but only 9k-ish if I restrict to where the contribu" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031095 (owner: 10Ejegg) [00:44:35] (03PS2) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) [00:44:35] (03PS1) 10Eileen: Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) [00:44:36] (03PS1) 10Eileen: Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280501) [00:57:13] (03PS2) 10Eileen: Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) [01:03:42] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [01:04:35] (03CR) 10CI reject: [V:04-1] Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [01:10:24] (03CR) 10CI reject: [V:04-1] Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [01:13:53] (03CR) 10Ejegg: "Ah, I guess it'll pick up some that are not monthly converts as well. But in any case, if the recurring row's processor_id matches a contr" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031095 (owner: 10Ejegg) [01:14:57] (03CR) 10Eileen: [C:03+2] "OK - I'm good to give it a whirl then" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031095 (owner: 10Ejegg) [01:15:04] thanks eileen ! [01:15:13] notice what I did there - it's already merged [01:15:35] ahh i didn't see the initial C+2 [01:15:37] thanks again [01:15:40] will deploy [01:16:22] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031112 [01:16:24] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031112 (owner: 10Ejegg) [01:17:23] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031112 (owner: 10Ejegg) [01:18:27] !log fundraising civicrm upgraded from c854dd3a to c7b0dfbb [01:18:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:23:36] ok, let's requeue a couple of those messages and see if they get through [01:24:12] that's two requeued [01:24:25] ah i should have looked at the IDs [01:24:29] lemme do a couple more [01:27:04] (03PS3) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) [01:30:53] oh dang, the last run was full of deadlocks [01:31:18] i guess maybe while I was running the script [01:31:24] let's see if those got requeued [01:32:19] they did! [01:32:30] so that's nice at least [01:34:20] so have we got some that are not in damaged from before we fixed the deadlock handling? [01:34:34] maybe, but those will be hard to find [01:34:42] I guess grepping logs? [01:34:57] what will happen if we don't find them/. [01:35:09] there's a chance we will get them from the audit files [01:35:26] but I haven't seen exactly what the audit rows look like for autorescues [01:35:27] probably a high chance? [01:35:39] we should look for em in the audits! [01:37:58] ok, nice, those requeued ones seem to have been processed correctly this time [01:38:03] let's take a look in the Civi UI [01:38:51] (03PS2) 10Eileen: Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) [01:39:42] yep, looks good there [01:40:08] ok, I'll requeue the rest of those [01:40:41] looks like 136 with a rescue_reference and no retry_date [01:40:46] I'll just set the retry_date [01:41:15] oh cool [01:41:19] ok, committed 136 rows [01:50:30] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [01:50:35] (03PS4) 10Eileen: Remove calls to metric_reporting via module_invoke [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030750 (https://phabricator.wikimedia.org/T270668) [01:50:40] (03CR) 10Ejegg: [C:03+2] Remove calls to metric_reporting via module_invoke [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030750 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [01:51:09] hmm ejegg not quite right huh - the message looks informative tho [01:52:09] (03PS3) 10Eileen: Fully remove drush queue.consume action, last call to metrics drupal fn [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030751 (https://phabricator.wikimedia.org/T362206) [01:53:44] oh did we just get more failmail? [01:54:04] phooey, a lot [01:54:26] oh, a bunch are deadlocks! [01:55:16] well dang it, why didn't those get caught in the deadlock-retry logic? [01:57:54] I guess the catch in _wmf_civicrm_message_contribution_insert isn't seeing the right strings in $e->getErrorCode() ? [01:58:13] (03CR) 10CI reject: [V:04-1] Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [01:58:39] should we just add -31 to that list of codes? [01:58:45] >> Source: array ( 'error_code' => -31, 'sql' => 'IN [01:59:50] hmm, is that defined as a constant someplace? [02:03:36] ejegg: which one are you looking at? [02:03:38] (03PS1) 10Ejegg: Detect deadlocks with error code -31 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031116 [02:03:45] https://civicrm.wikimedia.org/civicrm/damaged/edit?id=633450 [02:06:20] so do we currently have a locking query? [02:07:04] innotop doesn't show anything [02:07:09] I'll queue 'em up again [02:10:18] (03Merged) 10jenkins-bot: Remove calls to metric_reporting via module_invoke [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030750 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [02:11:25] ejegg: I'm a bit mystified on that deadlock - they should be mapped - in civicrm/vender/pear/db/DB.php [02:11:43] 06Fundraising-Backlog: Japan Email Campaign Records - Incorrect utm_campaign Causing Duplicate Records - https://phabricator.wikimedia.org/T364515#9792928 (10Cstone) Thanks for the spreadsheet it was really helpful! I updated these utm_campaigns: |Country | mailing | sent | wrong | fixed | amount updated | CZ |... [02:13:49] oops, now i've requeued them all and forgot to save the full stack trace [02:17:59] (03PS2) 10Eileen: Remove remaining code from metrics_reporting module [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031094 (https://phabricator.wikimedia.org/T270668) [02:25:54] (03CR) 10Ejegg: [C:03+2] Fully remove drush queue.consume action, last call to metrics drupal fn (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030751 (https://phabricator.wikimedia.org/T362206) (owner: 10Eileen) [02:45:32] (03Merged) 10jenkins-bot: Fully remove drush queue.consume action, last call to metrics drupal fn [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030751 (https://phabricator.wikimedia.org/T362206) (owner: 10Eileen) [02:48:41] (03PS3) 10Eileen: Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) [02:48:52] (03PS2) 10Eileen: Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) [02:53:37] 14Fundraising Spring - anybodyKnowWhatThisDoes(), 06Fundraising-Backlog, 10Wikipedia-Android-App-Backlog (Android Release - FY2023-24), 10Wikipedia-iOS-App-Backlog (iOS Release FY2023-24): Standardize utm_source from app donations without a banner and clea... - https://phabricator.wikimedia.org/T350919#9792941 [02:54:39] (03PS1) 10Eileen: Move address DonationQueue tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031126 [02:54:52] (03CR) 10Cstone: [C:03+2] Remove BitPay file [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031087 (https://phabricator.wikimedia.org/T364784) (owner: 10Eileen) [02:55:19] yay more code gone [02:55:36] my brain is dead but i could handle a nice delete review :) [03:02:53] lol [03:08:03] (03PS1) 10Eileen: Move Relationship import test to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031128 [03:08:03] (03PS1) 10Eileen: Another test move [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031129 [03:08:20] well those should be brain-dead ones to merge ... if they pass [03:13:48] (03Merged) 10jenkins-bot: Remove BitPay file [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031087 (https://phabricator.wikimedia.org/T364784) (owner: 10Eileen) [05:15:20] (03PS1) 10Eileen: Improve deadlock handling on contribution Insert [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031162 [05:16:50] (03CR) 10Eileen: "I dug into this & it's not just error 31 that is not being caught - it's all 3." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031116 (owner: 10Ejegg) [05:30:03] (03PS5) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [05:35:17] (03CR) 10CI reject: [V:04-1] Improve deadlock handling on contribution Insert [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031162 (owner: 10Eileen) [05:40:45] (03PS4) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) [05:48:15] (03CR) 10CI reject: [V:04-1] Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [05:59:43] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [06:08:02] (03PS5) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) [06:28:05] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [07:25:49] (03PS8) 10Eileen: Wip 5.74 rc, stock" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1027983 [07:26:10] (03PS9) 10Eileen: 5.74 rc, stock" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1027983 [07:44:24] (03PS5) 10Eileen: Move handling of field 'partner' to shared function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030613 [07:44:24] (03PS5) 10Eileen: Copy handleUpdate back to ChecksFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030616 [07:44:24] (03PS7) 10Eileen: Minor simplification [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030617 [07:44:25] (03PS8) 10Eileen: Move unpacking of external identifiers to message class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030614 [07:44:26] (03PS2) 10Eileen: Remove legacy-code function is_addess_valid [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031092 [07:44:27] (03PS4) 10Eileen: Remove last call to handleUpdate from outside the class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 [08:02:10] (03CR) 10Lucas Werkmeister (WMDE): Switch to a slightly less magic way of denoting how address should be updated (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [08:15:26] (03PS6) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280581) [08:27:29] (03PS2) 10Eileen: Improve deadlock handling on contribution Insert [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031162 [08:37:53] (03PS6) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [08:38:32] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280581) (owner: 10Eileen) [08:38:39] (03PS2) 10Eileen: Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280581) [08:39:11] (03CR) 10CI reject: [V:04-1] Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280581) (owner: 10Eileen) [08:39:55] (03PS3) 10Eileen: Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280581) [08:40:25] (03CR) 10CI reject: [V:04-1] Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280581) (owner: 10Eileen) [08:47:12] (03CR) 10CI reject: [V:04-1] Improve deadlock handling on contribution Insert [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031162 (owner: 10Eileen) [08:52:17] (03CR) 10Urbanecm: [C:04-1] Move the main message import to the DonationQueue handler (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280581) (owner: 10Eileen) [08:52:44] (03CR) 10Urbanecm: [C:04-1] Finally push the recurring donation to the donation queue (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280581) (owner: 10Eileen) [09:31:58] (03PS7) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T240581) [09:32:37] (03PS3) 10Eileen: Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T240581) [09:33:10] (03CR) 10CI reject: [V:04-1] Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [09:34:51] (03CR) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [09:35:38] (03CR) 10Eileen: Move the main message import to the DonationQueue handler (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [09:36:23] (03PS4) 10Eileen: Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T240581) [09:36:28] (03CR) 10Eileen: Finally push the recurring donation to the donation queue (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [09:36:51] (03CR) 10CI reject: [V:04-1] Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [09:51:38] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T240581) (owner: 10Eileen) [13:53:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Figure out what to do when Adyen autorescues a Chargebacked payment - https://phabricator.wikimedia.org/T364860 (10Damilare) 03NEW [14:28:19] 06Fundraising-Backlog: Japan Email Campaign Records - Incorrect utm_campaign Causing Duplicate Records - https://phabricator.wikimedia.org/T364515#9794920 (10NCreasy) Thank you! I am still seeing a duplication against one of the JP sends: country: JP mailing id: sp74192289 incorrect utm_campaign: C22324_Email1... [14:28:57] 06Fundraising-Backlog, 10FR-donorservices, 07payments-orchestration: Investigation: Cancel Paypal Recurring doesn't always cancel on processor side - https://phabricator.wikimedia.org/T341346#9794921 (10AKanji-WMF) In discussion with @MBeat33 , adding this to orchestration tag as potentially gravy could addr... [14:37:13] (03PS3) 10Eileen: Remove remaining code from metrics_reporting module [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031094 (https://phabricator.wikimedia.org/T270668) [14:37:20] (03CR) 10Ejegg: [C:03+2] Remove remaining code from metrics_reporting module [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031094 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [14:43:25] 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9794974 (10EMartin) Dlocal indicate: Love Kolapo 6:54 AM Hello @Rakhi and @emartin Thank you for your patience . After further investigation with our operations team, we can see we are not receiving the c... [14:51:42] (03PS4) 10Eileen: Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) [14:51:47] (03CR) 10Ejegg: [C:03+2] Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [14:54:41] (03PS3) 10Eileen: Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) [14:54:43] (03CR) 10Ejegg: [C:03+2] Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [14:57:50] (03Merged) 10jenkins-bot: Remove remaining code from metrics_reporting module [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031094 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [15:02:00] greg-g: the "fr-tech-devs" are now a thing and they can ssh to crm2001 [15:03:36] mutante: sweet, thank you! [15:05:20] 06Fundraising-Backlog: Search kit fields for pledges - https://phabricator.wikimedia.org/T361635#9795121 (10MDemosWMF) @Eileenmcnaughton I spoke to the major gifts team and they have raised priority on this so that we can implement during this year's Endowment campaign - if we can aim for end of fiscal year? The... [15:12:21] (03Merged) 10jenkins-bot: Move DonationStats Collector to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031096 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [15:12:21] (03CR) 10CI reject: [V:04-1] Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [15:13:13] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9795160 (10Damilare) a:03Damilare [16:17:12] (03CR) 10Ejegg: Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [16:17:19] (03CR) 10Ejegg: [C:03+2] "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [16:38:45] 10fundraising-tech-ops: Fundraising access request for KMorrow - https://phabricator.wikimedia.org/T363937#9796085 (10Dwisehaupt) 05Open→03Resolved Verified scripts that call mysql are working. [16:39:37] (03Merged) 10jenkins-bot: Move Queue2civicrm to the extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031097 (https://phabricator.wikimedia.org/T270668) (owner: 10Eileen) [16:47:13] (03CR) 10Ejegg: [C:04-1] "Refactor looks really good! Let's just not add changes to the do_not_trade field here." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [16:48:06] 14Fundraising Spring - anybodyKnowWhatThisDoes(), 06Fundraising-Backlog, 10Wikipedia-Android-App-Backlog (Android Release - FY2023-24), 10Wikipedia-iOS-App-Backlog (iOS Release FY2023-24): Standardize utm_source from app donations without a banner and clea... - https://phabricator.wikimedia.org/T350919#9796171 [16:52:06] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9796198 (10Damilare) Hi @EMartin, looks like this was due to a custom filter we added to validate the payments. We've updated the settings and the new payments are now being app... [16:55:15] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9796237 (10EMartin) Thanks Dami! [17:12:33] 06Fundraising-Backlog: Japan Email Campaign Records - Incorrect utm_campaign Causing Duplicate Records - https://phabricator.wikimedia.org/T364515#9796310 (10NCreasy) The Italy one from above currently has a blank utm_campaign: country: IT mailing id: sp74116362 incorrect utm_campaign: correct utm_campaign: C2... [17:36:42] (03CR) 10Bartosz Dziewoński: [C:03+2] "Supposedly fixed, let's try again." [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1027595 (owner: 10Umherirrender) [17:38:12] (03PS1) 10Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1031521 [17:38:15] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1031521 (owner: 10Ejegg) [17:40:55] (03Merged) 10jenkins-bot: Migrate IReadableDatabase::buildGroupConcatField to SelectQueryBuilder [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1027595 (owner: 10Umherirrender) [17:40:56] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1031521 (owner: 10Ejegg) [17:43:35] (03PS1) 10Ejegg: Update composer.lock [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1031524 [17:43:41] (03CR) 10Ejegg: [C:03+2] Update composer.lock [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1031524 (owner: 10Ejegg) [17:47:26] !log donorwiki upgraded from b005071a to fa7de70f [17:47:28] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [17:49:41] oh dang, donor prefs links went all the way to the bottom of the contact summary again [17:49:45] how did that happen? [17:51:03] (03Merged) 10jenkins-bot: Update composer.lock [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1031524 (owner: 10Ejegg) [17:52:04] ok, submitted a snooze request for my CID [18:02:50] (03PS6) 10Eileen: Move handling of field 'partner' to shared function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030613 [18:02:52] (03CR) 10Ejegg: [C:03+2] Move handling of field 'partner' to shared function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030613 (owner: 10Eileen) [18:18:06] (03PS7) 10Eileen: Move handling of field 'partner' to shared function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030613 [18:18:06] (03PS6) 10Eileen: Copy handleUpdate back to ChecksFile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030616 [18:18:06] (03PS8) 10Eileen: Minor simplification [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030617 [18:18:06] (03PS9) 10Eileen: Move unpacking of external identifiers to message class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030614 [18:18:07] (03PS5) 10Eileen: Remove last call to handleUpdate from outside the class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 [18:18:10] (03PS3) 10Eileen: Remove legacy-code function is_addess_valid [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031092 [18:18:14] (03PS8) 10Eileen: Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) [18:18:18] (03PS5) 10Eileen: Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) [18:18:22] (03PS4) 10Eileen: Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280501) [18:18:26] (03PS7) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [18:18:30] (03PS1) 10Eileen: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031531 [18:18:34] (03PS1) 10Eileen: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031532 [18:18:45] (03CR) 10Ejegg: [C:03+2] Copy handleUpdate back to ChecksFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030616 (owner: 10Eileen) [18:19:35] (03CR) 10Ejegg: [C:03+2] Minor simplification [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030617 (owner: 10Eileen) [18:19:49] (03PS8) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [18:20:56] (03CR) 10Eileen: "fixed" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [18:21:18] (03PS9) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [18:21:55] (03CR) 10CI reject: [V:04-1] Move the main message import to the DonationQueue handler [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031108 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [18:22:17] (03CR) 10CI reject: [V:04-1] Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [18:22:30] sorry fr-tech, I'd be a bit late for tech-talk by about 15mins [18:22:31] (03CR) 10Eileen: Copy handleUpdate back to ChecksFile (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030616 (owner: 10Eileen) [18:22:34] (03CR) 10CI reject: [V:04-1] Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031531 (owner: 10Eileen) [18:22:55] (03CR) 10CI reject: [V:04-1] Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031532 (owner: 10Eileen) [18:23:25] dang did I trigger an accidental re-base or maybe just pushed up some local only changes then [18:23:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9796683 (10MBeat33) @Damilare, @SBorriello has a Zendesk donor who I think was affected by this - they have multiple recent unsettled donations in ZAR, so just to check, can we... [18:24:54] (03PS2) 10Eileen: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031531 [18:25:26] (03PS2) 10Eileen: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031532 [18:25:42] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9796697 (10MBeat33) And if it's not too big a lift tech-wise, depending on the number of donors affected, scanning to filter out any who may have switched to PayPal or alternate... [18:25:42] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9796696 (10Damilare) We are working on a script to settle all the trxns stuck on Auth. I'd have to search through the export to manually remove the old trxns for this contact. [18:29:11] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031534 [18:29:31] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031534 (owner: 10Eileen) [18:35:17] (03PS1) 10Eileen: Remove a couple more traces of metrics_reporting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031536 [18:36:28] (03PS1) 10Damilare Adedoyin: Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 [18:36:53] (03CR) 10CI reject: [V:04-1] Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 (owner: 10Damilare Adedoyin) [18:39:36] (03Merged) 10jenkins-bot: Move handling of field 'partner' to shared function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030613 (owner: 10Eileen) [18:40:24] (03CR) 10CI reject: [V:04-1] Switch to a slightly less magic way of denoting how address should be updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031093 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [18:40:25] (03CR) 10CI reject: [V:04-1] Finally push the recurring donation to the donation queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031107 (https://phabricator.wikimedia.org/T280501) (owner: 10Eileen) [18:47:57] (03PS6) 10Eileen: Remove last call to handleUpdate from outside the class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 [18:48:02] (03PS2) 10Eileen: Remove a couple more traces of metrics_reporting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031536 [18:48:06] (03CR) 10Ejegg: [C:03+2] Remove a couple more traces of metrics_reporting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031536 (owner: 10Eileen) [18:53:14] (03PS7) 10Eileen: Remove last call to handleUpdate from outside the class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 [18:54:20] (03PS2) 10Damilare Adedoyin: Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 [18:54:45] (03CR) 10CI reject: [V:04-1] Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 (owner: 10Damilare Adedoyin) [18:54:48] (03CR) 10Ejegg: Remove last call to handleUpdate from outside the class (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 (owner: 10Eileen) [18:57:06] (03PS3) 10Damilare Adedoyin: Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 [19:21:23] (03Merged) 10jenkins-bot: Remove a couple more traces of metrics_reporting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031536 (owner: 10Eileen) [19:34:08] (03CR) 10Cstone: [C:03+2] Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031532 (owner: 10Eileen) [19:34:16] 03Fundraising Sprint: ick(), 06Fundraising-Backlog, 10MW-1.43-notes (1.43.0-wmf.5; 2024-05-14): Enable 90 day snooze: Comms Preferences Centre update - https://phabricator.wikimedia.org/T358878#9796998 (10Ejegg) @MSuijkerbuijk_WMF @AKanji-WMF we've got an MVP of this deployed. It's available by clicking on t... [19:45:20] 03Fundraising Sprint: ick(), 06Fundraising-Backlog, 10FR-donorservices, 10FR-email-preferences: Featured Request: Have Civi reflect when an email has been unsubscribed 90 days - https://phabricator.wikimedia.org/T296054#9797044 (10Ejegg) Next thing to do: make snooze date visible in contact summary screen [19:51:44] (03Merged) 10jenkins-bot: Remove unused function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031532 (owner: 10Eileen) [19:54:26] 03Fundraising Sprint: ick(), 06Fundraising-Backlog, 10FR-donorservices, 10FR-email-preferences: Featured Request: Have Civi reflect when an email has been unsubscribed 90 days - https://phabricator.wikimedia.org/T296054#9797070 (10Eileenmcnaughton) SHould create a separate phab for that [19:55:44] 03Fundraising Sprint: ick(), 06Fundraising-Backlog, 10FR-donorservices, 10FR-email-preferences: Show snooze_date on contact summary - https://phabricator.wikimedia.org/T364913 (10Ejegg) 03NEW [19:56:06] (03PS4) 10Damilare Adedoyin: Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 (https://phabricator.wikimedia.org/T364792) [19:58:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Figure out what to do when Adyen autorescues a Chargebacked payment - https://phabricator.wikimedia.org/T364860#9797088 (10XenoRyet) p:05Triage→03Medium [19:58:25] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797089 (10XenoRyet) p:05Triage→03High [20:00:00] (03PS1) 10Eileen: Add fundraise up test check for email update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031552 [20:00:23] (03PS1) 10Eileen: Fix email update, while breaking first name update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031553 [20:04:12] (03PS1) 10Eileen: Possible alternative on timer - [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031555 [20:05:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797104 (10EMartin) @XenoRyet Dylan, might this be impacting markets outside of S.A.? [20:08:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797153 (10XenoRyet) That's a question for @Damilare. Did it look like that filter might have been catching anything else? [20:10:17] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797161 (10Damilare) Yes there were some other currencies in the filter, but only a handful. The fix covers all countries using Dlocal. [20:12:04] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797181 (10XenoRyet) Great, thanks! [20:13:35] (03PS1) 10Eileen: Update timer, for if we don't want to move to inside WMFContact.save [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031557 [20:14:46] ejegg: so 2 alternatives on the timer metrics thing https://gerrit.wikimedia.org/r/1031555 or https://gerrit.wikimedia.org/r/1031557 [20:14:46] the advantage of the first IS the disadvantage - ie wmfContact.save is measuered over all flows as the same metric [20:19:20] (03CR) 10CI reject: [V:04-1] Add fundraise up test check for email update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031552 (owner: 10Eileen) [20:19:52] (03CR) 10Eileen: Remove last call to handleUpdate from outside the class (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031059 (owner: 10Eileen) [20:21:00] (03PS2) 10Eileen: Fix email update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031553 [20:22:53] damilare: ejegg cstone so it seems that I can demonstrate in the test that the email is not updating when it should & it seems that removing setContactID() fixes that https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1031552/1 - so I'm gonna try a test-first approach on the Recur one [20:34:45] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Not capturing transactions in ZAR - https://phabricator.wikimedia.org/T364792#9797324 (10EMartin) Thanks @Damilare If it's not too much trouble, can you tell us the currencies involved? [20:44:09] ah eileen, we still seem to have version 1.0 of omnimail in the extensions dir [20:44:27] ejegg: hmm - I'm confused then [20:44:46] it should be in our vendor dir & we should ignore in the exensions dir [20:44:50] oh sorry, i'm confused too [20:45:00] was mixing up the version of the civi extension [20:45:09] with the version of the composer library [20:51:19] (03PS3) 10Eileen: Add clicked_total & unique to stats fetched [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1028609 (https://phabricator.wikimedia.org/T344383) [20:51:38] (03CR) 10Ejegg: [C:03+2] Add clicked_total & unique to stats fetched [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1028609 (https://phabricator.wikimedia.org/T344383) (owner: 10Eileen) [20:55:34] (03PS10) 10Eileen: Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) [20:55:40] (03CR) 10Ejegg: [C:03+2] Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [21:02:21] eileen: did you mean to include those custom field changes with the deadlock handling in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1031162 ? [21:03:32] ejegg: so the problem was it kinda needs to switch to apiv4 to throw the right exception - but it all got a bit much huh? I need to go back & break it down - but the goal is to do something like that with the custom fields all up & switch contribution & contact to apiv4 [21:04:15] ah ok, i think I can figure it out [21:04:18] - on that - can we just pass through any contribution-y or contact-y custom fields or do we need to filter them with an allow list - my take is that contact has no allow list for 'create' atm but contribution does [21:07:20] huh, i'm not coming up with a lot of reasons not to pass through fields from the message, but I'm sure we had some [21:07:27] so many different cases to consider... [21:07:36] yeah :-( [21:07:52] (03PS1) 10Eileen: Update Recurring ModifyAmountTest to check address updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031570 [21:08:37] ejegg: on this address thing - I just did another (preferred) test to capture it but the way the code is now you can EITHER update name OR email when calling WMFSave - depending which way you contort your face while doing it [21:09:12] currently the fundraise up audits are updating name fields & not email, if I fix to update email I kinda get the reverse [21:10:06] may we just need to pass in new values like 'is_authoritative_name' , is_authoritative_address', 'is_authoritative_email' [21:11:16] ugh [21:11:18] ok [21:11:52] OR we could NOT go through WMFSave to do those updates when we know we want to do them [21:14:53] (03Merged) 10jenkins-bot: Add clicked_total & unique to stats fetched [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1028609 (https://phabricator.wikimedia.org/T344383) (owner: 10Eileen) [21:14:54] (03CR) 10CI reject: [V:04-1] Update OptInQueueConsuer to use a Message class, apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1025472 (https://phabricator.wikimedia.org/T357471) (owner: 10Eileen) [21:19:36] 06Fundraising-Backlog, 10Desktop Improvements (Vector 2022), 10Web-Team-Backlog (FY2023-24 Q4 Sprint 4): Plan for Donate Wiki and Thank You Wiki Rollback and Redesign - https://phabricator.wikimedia.org/T361500#9797565 (10Jdlrobson) p:05Medium→03Low [21:24:56] (03PS2) 10Eileen: Move address DonationQueue tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031126 [21:24:58] (03CR) 10Ejegg: [C:03+2] Move address DonationQueue tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031126 (owner: 10Eileen) [21:27:22] (03PS2) 10Eileen: Move Relationship import test to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031128 [21:27:23] (03PS1) 10Eileen: Directly save the Email, address, contact in Recurring Modify [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031572 [21:27:26] (03CR) 10Ejegg: [C:03+2] Move Relationship import test to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031128 (owner: 10Eileen) [21:28:12] so ejegg this is what is looks like just doing a direct save of that fundraise up data without going through the whole WMFSave routine - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1031572 [21:28:32] - which currently isn't actually saving the data... [21:30:25] (03CR) 10CI reject: [V:04-1] Update Recurring ModifyAmountTest to check address updated [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031570 (owner: 10Eileen) [21:31:34] ^^ test fails cos it demostrates it NOT saving [21:35:18] nice find [21:35:31] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031573 [21:35:55] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1031573 (owner: 10Eileen) [21:37:54] (03Abandoned) 10Eileen: Add fundraise up test check for email update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031552 (owner: 10Eileen) [21:38:08] (03Abandoned) 10Eileen: Fix email update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031553 (owner: 10Eileen) [21:39:07] !log civicrm upgraded from c7b0dfbb to 9268acf3 [21:39:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:47:45] ejegg: so the 2 places in which are trying to update email, address, name on existing contacts are [21:47:45] - RecurringModifiyQueue where txn_type = external_recurring_modification [21:47:45] - RecurringQueue where txn_type = 'subscr_payment' AND it is the first payment against the subscription [21:47:46] IN both cases it seems we assume that the source is more reliable than what we have. I have some misgivings about that assumption - but I guess it's more true than not. [21:47:47] So here is my proposal [21:47:48] 1) those 2 queue consumers can call Contact::save() directly along the lines of https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1031572 [21:47:49] 2) as a follow up - add 2 new address sources - fundraise_up and paypal to the existing option group on the custom field. When we save directly we populate that source + the updated address [21:48:12] hmm, ok [21:48:26] 3) next step is to potentially CHECK the updated date in those saves before doing the update - a bit more heavy on the GET calls but might make sense [21:48:41] (03Merged) 10jenkins-bot: Move address DonationQueue tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031126 (owner: 10Eileen) [21:48:47] wait, but we should also update all of those when someone gives us their updated data on a payment form, shouldn't we? [21:49:12] so currently we don't really update names at all from the looks [21:49:19] whaaa [21:49:27] that sounds like a regression [21:49:42] we should definitely send someone a thank you with the same name they just entered on a form [21:49:47] (03Merged) 10jenkins-bot: Move Relationship import test to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031128 (owner: 10Eileen) [21:49:49] no - it's deliberate - we only do it if the name is empty [21:50:00] but - remember in those cases mostly it will create a new contact [21:50:09] ah right [21:50:11] cos having the contact ID is the exception not the rule [21:51:27] it DOES look like we will overwrite the address on update currently - except ... sometimes ... eg. when we are trying to [21:51:36] lol [21:52:14] but say we are getting an update from Paypal - ie first payment in a new subscription [21:52:32] is paypal address, name, email data really better then ours? [21:52:56] they are sending what they have in their account rather than what was just entered I assume [21:52:57] nah, we should prefer stuff from a form to stuff from PayPal [21:54:17] ok - so for that paypal scenario "- RecurringQueue where txn_type = 'subscr_payment' AND it is the first payment against the subscription" - we are actually fine with the idea of [21:54:17] -create name if not exists, don't change [21:54:17] - create address if not exists, don't change [21:54:17] - leave email well enough along [21:54:23] alone not along [21:54:35] yep yep, that sounds right [21:54:46] if that is what we want I will update test to check for that [21:54:56] & I suspect some code can go [21:55:02] hmm, i guess I wish we could tag the address sources [21:55:08] we can [21:55:12] we have those custom fields now [21:55:16] so we would update paypal addresses if they were from paypal in the first place [21:55:21] oho, really? [21:55:36] then I guess we would just want a ranking of which sources we like best [21:55:44] yeah + update date [21:56:00] but I'm worried you are trying to complicate the code when I haven't finished simplifying :-) [21:56:00] and only update if new_address_source_rank >= old_address_source_rank [21:56:05] haha [21:56:20] - I did add a phab about the address sources [21:56:30] ah ok, i missed that [21:56:39] yeah - it's new-ish [21:57:52] ok - but we won't have that source data yet so I can start pushing it in but to allow me to get past this challenge I can switch to this for paypal? In which case what about FundraiseUp [21:57:52] -create name if not exists, don't change [21:57:52] - create address if not exists, don't change [21:57:53] [21:57:53] - leave email well enough along [22:00:40] that sounds good for paypal [22:00:49] for fundraiseup we always want to update [22:01:02] since we're treating that like our form [22:02:43] ok - so re FundraiseUp - are you OK with this approach - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1031572 [22:03:16] perhaps we need to if-fundraise-up-it [22:04:09] huh, i thought there was another way frup contact updates could come in, not just recurring amount [22:04:25] let me see what the audit does [22:16:54] huh, I guess the recurring changes are the only place we are considering it [22:17:29] but I'd say if we get a new one-time donation from FRUP with the same external ID and a new name or email we should update there too [22:17:42] i guess i'd ask Dami about that [22:19:24] hmm OK - well if we are OK with ^^ in the mean time I can clean up all the craziness & if we want to add address priority as a new feature we will be doing it from a place of sane code [22:19:52] sure, sounds good [22:21:28] (03PS1) 10Eileen: Add fundraise up & paypal as address sources [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031577 (https://phabricator.wikimedia.org/T364700) [22:21:54] (03PS3) 10Eileen: Simplify time handling in (outgoing) tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030752 [22:22:05] (03CR) 10Ejegg: [C:03+2] Simplify time handling in (outgoing) tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030752 (owner: 10Eileen) [22:22:20] ejegg: I just pushed up ^^ so I can add that onto the fundraise up save one - although maybe once both are merged rather than chain them [22:24:37] (03CR) 10Ejegg: [C:04-1] "let's use fundraiseup (no underscore) for consistency" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031577 (https://phabricator.wikimedia.org/T364700) (owner: 10Eileen) [22:25:13] ejegg: but it's not a word!! [22:25:20] haha [22:25:30] paypal totally is a word? [22:26:18] but Fundraise is a word & Up is a word [22:26:34] Pay is a word, and Pal is a word! [22:26:37] so for the label we need to use Fundraiseup or FundraiseUp [22:26:42] or Fundraise Up [22:26:59] pay may be an americanism [22:27:05] sorry pal not pay [22:27:21] oops, kid home, back in a bit [22:31:05] (03CR) 10Eileen: "I'm not sold on that cos PayPal is one (invented) word in their branding where as Fundraise Up is 2 words - which I think is better reflec" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031577 (https://phabricator.wikimedia.org/T364700) (owner: 10Eileen) [22:31:14] (03PS1) 10Eileen: fundraise_up -> fundraiseup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031579 [22:41:11] (03Merged) 10jenkins-bot: Simplify time handling in (outgoing) tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1030752 (owner: 10Eileen) [22:42:19] (03PS1) 10Eileen: Consolidate our test-mailer use [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031582 [22:51:46] naming is hard [23:02:38] (03CR) 10CI reject: [V:04-1] Consolidate our test-mailer use [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1031582 (owner: 10Eileen) [23:03:12] cstone: especially around the small things like punctuation & 's's [23:03:22] - vs _ [23:06:06] 06Fundraising-Backlog: Dark Mode - Codex on Payments Forms - https://phabricator.wikimedia.org/T364434#9797936 (10Cstone) We can also just use the colors from codex separately if we need to implement before we get 1.43 [23:07:11] (03CR) 10Cstone: [C:03+2] Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 (https://phabricator.wikimedia.org/T364792) (owner: 10Damilare Adedoyin) [23:07:37] (03Merged) 10jenkins-bot: Create ApprovePayments maintenance script for Dlocal [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1031537 (https://phabricator.wikimedia.org/T364792) (owner: 10Damilare Adedoyin) [23:24:15] hmmm did not expect my google pay civi contact to have estonia as the country [23:26:45] eileen: would you expect just the country of an address to update anywhere? [23:27:16] cstone: yeah - under very specific but undecipherable circumstances.... [23:27:37] look at the address on this contact of mine 57639257 [23:27:38] (I think the point is they want the country of the last donation for various things) [23:27:53] but it like has the rest of my US address? [23:28:42] although i just donated from the US with that contact and it didnt update back [23:28:50] yeah well the US takes Rome & Paris & sticks them in Georgia so who am I to disagree wiith a bit of own-medicine [23:29:19] - this address stuff is doing my head in [23:29:22] there might be an tallinn, colorado but i dont know about a denver, estonia :P [23:29:46] :-) [23:30:17] how did the donations come in? Paypal? If so I can try to add the scenario to the test [23:30:31] google pay [23:30:42] that contact is only google pay donations im checking the log [23:32:34] whatever happend it lost my state and geolocation and switched the country [23:32:38] i wonder if its just my api code [23:33:29] ooh i think i know whats going on [23:37:14] ok eileen sorry this is on me and the android peeps [23:40:47] ok .... [23:41:25] past christine named things country and donor_country and now I am not sure what is what :P [23:44:12] ok country is the country for contribution tracking and donor_country is the country for donations queue