[00:30:53] (03PS1) 10Wandji collins: Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) [00:32:38] (03CR) 10CI reject: [V:04-1] Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) (owner: 10Wandji collins) [00:43:30] (03PS2) 10Wandji collins: Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) [00:45:05] (03CR) 10CI reject: [V:04-1] Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) (owner: 10Wandji collins) [01:23:44] (03PS3) 10Wandji collins: Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) [01:25:30] (03CR) 10CI reject: [V:04-1] Use ObjectCacheFactory getLocalCluster method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040856 (https://phabricator.wikimedia.org/T363770) (owner: 10Wandji collins) [07:57:59] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040930 (owner: 10L10n-bot) [12:55:34] (03CR) 10Jgleeson: WIP: Add CiviCRM Standalone (031 comment) [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1039235 (owner: 10Jgleeson) [13:17:06] (03CR) 10Raimond Spekking: [C:03+2] "false positive" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040930 (owner: 10L10n-bot) [13:32:20] (03PS4) 10Damilare Adedoyin: Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) [13:34:21] (03CR) 10CI reject: [V:04-1] Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) (owner: 10Damilare Adedoyin) [14:05:33] 10fundraising-tech-ops, 06Infrastructure-Foundations, 10Mail: Update fundraising mail settings to use new production mx hosts - https://phabricator.wikimedia.org/T366740#9875478 (10jhathaway) p:05Triage→03Medium [14:26:11] 06Fundraising-Backlog, 10FR-Alerts: Get group member data from Silverpop failed with code 1 - https://phabricator.wikimedia.org/T223470#9875549 (10Ejegg) Seeing more failmail from deadlocks in this job lately. Perhaps we could requeue them for processing via coworker? [14:59:59] (03PS1) 10Ejegg: Omnigroup load - use API4 internally [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1041148 (https://phabricator.wikimedia.org/T223470) [15:06:07] oh hmm, i had hoped there was a way to create a task from an API4 action object [15:24:33] (03PS3) 10Ejegg: WIP send recurring payments to donations queue [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/556829 (https://phabricator.wikimedia.org/T240581) [15:25:56] (03PS4) 10Aklapper: WIP send recurring payments to donations queue [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/556829 (https://phabricator.wikimedia.org/T240581) (owner: 10Ejegg) [15:27:53] (03PS5) 10Ejegg: Send recurring payments to donations queue [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/556829 (https://phabricator.wikimedia.org/T366526) [15:29:04] 03Fundraising Sprint K (placeholder name), 14Fundraising Sprint: justWork(), 06Fundraising-Backlog, 10FR-donorservices: ask Civi to use PayPal's API when humans cancel recurrings - https://phabricator.wikimedia.org/T361636#9875843 (10MBeat33) hi @Ejegg question for you, I just canceled a recurring that was... [15:33:44] 03Fundraising Sprint K (placeholder name), 14Fundraising Sprint: didAnyoneTryThis(), 14Fundraising Sprint: justWork(), 06Fundraising-Backlog: SEPA direct debit as mainstream payment method in EU campaigns - https://phabricator.wikimedia.org/T324520#9875859 (10Cstone) Test recurring donation: cid=63848894... [15:39:57] 03Fundraising Sprint K (placeholder name), 14Fundraising Sprint: justWork(), 06Fundraising-Backlog, 10FR-donorservices: ask Civi to use PayPal's API when humans cancel recurrings - https://phabricator.wikimedia.org/T361636#9875902 (10Ejegg) >>! In T361636#9875843, @MBeat33 wrote: > hi @Ejegg question for y... [15:53:12] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civi / Adyen mismatch re: recurring retries - https://phabricator.wikimedia.org/T365519#9876010 (10Ejegg) a:03Ejegg [16:06:02] 03Fundraising Sprint K (placeholder name), 14Fundraising Sprint: justWork(), 06Fundraising-Backlog, 10FR-donorservices: ask Civi to use PayPal's API when humans cancel recurrings - https://phabricator.wikimedia.org/T361636#9876058 (10MBeat33) Great, many thanks for confirming @Ejegg I'll add that error cod... [16:07:57] 06Fundraising-Backlog: Add GIACT account verification for ACH - https://phabricator.wikimedia.org/T367078 (10EMartin) 03NEW [16:08:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civi / Adyen mismatch re: recurring retries - https://phabricator.wikimedia.org/T365519#9876077 (10Ejegg) F9R3WD3NK967VXW3 was a retry - looks like we sent it from the IPN listener to Civi as though it was successful when the auth was actually failed. Loo... [17:14:37] 06Fundraising-Backlog, 10FR-Email, 10FR-email-preferences: Email preference center: Adjust export to acoustic to accommodate new data types - https://phabricator.wikimedia.org/T280675#9876462 (10Ejegg) 05Open→03Declined All the things the current prefs center can control are being exported to Acousti... [17:38:09] (03PS19) 10Damilare Adedoyin: Gravy create payment [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1037776 (https://phabricator.wikimedia.org/T364506) [17:41:29] (03PS5) 10Damilare Adedoyin: Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) [17:43:16] (03CR) 10CI reject: [V:04-1] Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) (owner: 10Damilare Adedoyin) [17:46:06] (03PS1) 10Ejegg: Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) [17:46:27] (03CR) 10CI reject: [V:04-1] Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) (owner: 10Ejegg) [17:52:23] (03PS6) 10Damilare Adedoyin: Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) [17:54:13] (03CR) 10CI reject: [V:04-1] Create gravy special page and load components [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040198 (https://phabricator.wikimedia.org/T364508) (owner: 10Damilare Adedoyin) [18:10:37] 03Fundraising Sprint K (placeholder name), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Fr-drupal-upgrade-2021: Spike - do practice standalong installs locally to get our heads around it - https://phabricator.wikimedia.org/T276395#9876866 (10jgleeson) a:03jgleeson [18:25:51] fr-tech does anyone understand why the ended autorescue IPNs were processed in the PaymentCaptureAction ? [18:31:22] phew, tangled! [18:31:31] I'mma split that off into a separate class [18:33:13] wait, what? It's just trying to cancel the auth? [18:33:26] even though the ended autorescue is always a failed auth [18:38:35] (03PS1) 10Ejegg: Don't queue a job for ended autorescue [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041185 [18:38:52] OK, I need to shuffle some changes between ^^^ and its parent patch [18:38:56] (03CR) 10CI reject: [V:04-1] Don't queue a job for ended autorescue [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041185 (owner: 10Ejegg) [18:52:34] (03PS2) 10Ejegg: Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) [18:52:58] (03CR) 10CI reject: [V:04-1] Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) (owner: 10Ejegg) [18:54:03] ejegg: it looks like that behaviour got added last nov https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/958999 [18:54:25] there's some context in the commit and ticket [18:55:48] same with https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/1006995 [18:56:58] yeah, I wish I had gotten in early on the review there to suggest alternatives [18:57:37] Since the ProcessCaptureRequestJob was already kind of overloaded [18:57:44] it would have been easy to add a new job type [18:57:59] specific to autorescues [18:59:28] i guess we could review the way all those messages and actions work, and how to add new ones [18:59:36] perhaps that wasn't clear [18:59:54] I'd actually like to list all the actions in code instead of them being in the config [19:00:17] since I think we never want to turn any of them off [19:00:34] also we should probably remove some configurability we don't need any more [19:00:40] like the multiple capture queues [19:00:55] and the boolean to control capturing from the auth [19:01:50] (03PS3) 10Ejegg: Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) [19:03:19] ok, so ^^^ is the urgent fix [19:04:22] looking [19:05:40] I did have to fix some test data! [19:06:05] the JSON flavor of the refund IPN test depended on that bad parse [19:08:33] (03CR) 10Jgleeson: [C:03+2] "LGTM. I can see how this is painful!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) (owner: 10Ejegg) [19:09:00] (03Merged) 10jenkins-bot: Fix success status mapping on Adyen IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041173 (https://phabricator.wikimedia.org/T365519) (owner: 10Ejegg) [19:15:47] 06Fundraising-Backlog, 07Epic: Gravy backend approve payment - https://phabricator.wikimedia.org/T367112 (10Damilare) 03NEW [19:16:27] 06Fundraising-Backlog, 07Epic: Gravy backend approve payment - https://phabricator.wikimedia.org/T367112#9877117 (10Damilare) a:03Damilare [19:17:19] (03CR) 10DannyS712: [C:03+2] pagers: Use expression builder [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1033107 (https://phabricator.wikimedia.org/T350965) (owner: 10Umherirrender) [19:18:53] thanks jgleeson ! [19:19:51] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1041201 [19:45:09] 03Fundraising Sprint K (placeholder name), 14Fundraising Sprint: justWork(), 06Fundraising-Backlog: Enable Retry logic for ACH transactions - https://phabricator.wikimedia.org/T365100#9877230 (10Cstone) a:03Cstone [19:49:54] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Carte Bancaire losing payment_submethod and recurring token - https://phabricator.wikimedia.org/T365884#9877248 (10Cstone) We added a fix for the submethod check https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface... [19:53:40] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07payments-orchestration: New data flows in support of reconciliation related to the Gr4vy payment orchestration project - https://phabricator.wikimedia.org/T366840#9877254 (10AKanji-WMF) [19:56:44] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Restore automatic handling of DUPLICATE_INVOICE WmfExceptions - https://phabricator.wikimedia.org/T366635#9877261 (10AKanji-WMF) p:05Triage→03Medium [19:59:23] (03Merged) 10jenkins-bot: pagers: Use expression builder [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1033107 (https://phabricator.wikimedia.org/T350965) (owner: 10Umherirrender) [20:21:50] 03Fundraising Sprint K (placeholder name), 06Fundraising-Backlog, 10MW-1.43-notes (1.43.0-wmf.7; 2024-05-28): Send failmail on error in custom filters - https://phabricator.wikimedia.org/T365469#9877341 (10AKanji-WMF) [20:27:36] 06Fundraising-Backlog: Consider moving our mail to a subdomain - https://phabricator.wikimedia.org/T365637#9877388 (10greg) Status update: * There is now a group of ~"folks who care about email servers at the Foundation" made up of representatives from ITS, Production SRE, and FR-Tech. To spur further discussio... [20:29:58] 06Fundraising-Backlog: Scoping: 3-Month Retention Email from Civi for recurring donors - https://phabricator.wikimedia.org/T365486#9877394 (10AKanji-WMF) [20:35:31] (03CR) 10Jgleeson: [C:03+2] "Let's try again!" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1035600 (https://phabricator.wikimedia.org/T363433) (owner: 10Ejegg) [20:36:29] thanks jgleeson ! [20:36:50] did you want to C+2 the parent patch just fixing the tests, too? [20:37:07] i know that SQL is a real bear [20:37:11] cstone: if you were already looking at the ACH stuff feel free to pick 'em up. I was just snooping around stuff earlier and then found the standalone spike to figured I'd go with that for now [20:37:24] but I think it's slightly more correct with the extra level of nesting [20:38:14] since it really should aggregate the values from the joined recur rows before using them in the IF [20:40:04] ejegg: I see [20:40:23] so I think the flappiness was because of that lack of aggregation [20:40:45] 06Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Non Campaign Emails in Database - https://phabricator.wikimedia.org/T361621#9877423 (10AKanji-WMF) p:05Triage→03Medium [20:40:51] (03CR) 10Umherirrender: "Namespaces are from I591628b98d16bfeecd32d157abc263be1036a71b, marked as REL1_39" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1040758 (owner: 10Umherirrender) [20:40:56] the 'IF' checking for cancelled status would sometimes apply to one of the joined rows and sometimes to the other [20:44:08] 06Fundraising-Backlog: Scoping: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#9877432 (10AKanji-WMF) @MSuijkerbuijk_WMF is [[ https://docs.google.com/document/d/15IvOaVsdBNklm9HVMNP9z2oiCLUizj7sOHV5iTcI_Ps/edit | this ]] the copy for this? [20:44:56] nice digging ejegg [20:46:30] (03CR) 10Jgleeson: [C:03+2] "Thanks for the fix!" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1039810 (owner: 10Ejegg) [20:47:01] :) [20:47:31] (03Merged) 10jenkins-bot: Fix flapping test for latest recur_id [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1039810 (owner: 10Ejegg) [20:47:34] (03Merged) 10jenkins-bot: Upgrade eligibility: ignore declines after 1 year [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1035600 (https://phabricator.wikimedia.org/T363433) (owner: 10Ejegg) [20:52:41] (03PS1) 10Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1041225 [20:52:43] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1041225 (owner: 10Ejegg) [20:53:11] (03Merged) 10jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1041225 (owner: 10Ejegg) [20:53:13] (03CR) 10Jgleeson: [C:03+2] "Working for me after rerunning './setup.sh --tools'. thanks!" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1035813 (owner: 10Ejegg) [20:54:13] (03Merged) 10jenkins-bot: Use pynose to provide nosetests [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1035813 (owner: 10Ejegg) [20:54:39] I feel like that tools image is next on the list to move to the new buster docker image [20:57:38] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1041201 (owner: 10Ejegg) [20:58:00] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1041201 (owner: 10Ejegg) [20:58:18] ah yeah [21:17:39] 10fundraising-tech-ops, 10FR-Tech-Analytics: Investigate direct python/pip installs for fundraising analytics tools. - https://phabricator.wikimedia.org/T364124#9877537 (10Jgreen) [21:19:13] !log Standalone SmashPig upgraded from edf573bb to 1d1b770c [21:19:15] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:19:55] !log fundraising python tools upgraded from 8c98b674 to c51f6e62 [21:19:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:31:31] ah man, there are a bunch of donations in there that we shouldn't have recorded :( [21:32:49] still trying to figure out exactly how many [21:33:04] but e.g. CID 62083216 has like 3 it shouldn't have [21:38:27] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civi / Adyen mismatch re: recurring retries - https://phabricator.wikimedia.org/T365519#9877661 (10Ejegg) Oh wow, there are a lot of donations in Civi that shouldn't be there, since I deployed a bug on May 8th that treated failed authorization IPNs as suc... [21:46:27] i wonder ejegg if one of the weird ones i saw was this I dont know if i ever made a phab for it lemme see [21:51:26] gotta head out for now, but I'll try to make more progress on this later today [22:07:18] 10fundraising-tech-ops, 06Infrastructure-Foundations, 10Mail: Update fundraising mail settings to use new production mx hosts - https://phabricator.wikimedia.org/T366740#9877740 (10Dwisehaupt) @jhathaway Thanks. I have shifted our codfw hosts to use the new mx-out hosts. That is the secondary datacenter. We'... [22:12:24] 10fundraising-tech-ops, 06Infrastructure-Foundations, 10Mail: Update fundraising mail settings to use new production mx hosts - https://phabricator.wikimedia.org/T366740#9877747 (10jhathaway) >>! In T366740#9877740, @Dwisehaupt wrote: > @jhathaway Thanks. I have shifted our codfw hosts to use the new mx-out... [22:28:09] ah dang fr-tech i forgot to empty the paypal jobs queue before that deploy [22:28:15] sorry! [22:28:30] will clean it up soon.