[03:43:28] (03PS4) 10Eileen: Update Fidelity Import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136833 (https://phabricator.wikimedia.org/T386017) [03:43:28] (03PS2) 10Eileen: Upstream fix for uploaded file with template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137096 [03:43:28] (03PS6) 10Eileen: WIP - mgd files to support imports [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136834 [03:45:28] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog: Fidelity Mapping Import - Change in gift framework - https://phabricator.wikimedia.org/T386017#10756649 (10Eileenmcnaughton) @MDemosWMF - I've added an additional fix to prevent those extra soft credits - now on stagingEileenMDemosWMF wrote:View Tas... [03:45:29] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog: Fidelity Mapping Import - Change in gift framework - https://phabricator.wikimedia.org/T386017#10756650 (10Eileenmcnaughton) @MDemosWMF - I've added an additional fix to prevent those extra soft credits - now on stagingEileenMDemosWMF wrote:View Tas... [05:29:31] (03CR) 10Eileen: "Hmm - I just tried & it seemed to queue up OK - it's just kinda hard to really see locally how well it will work on prod due to there not " [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1134344 (https://phabricator.wikimedia.org/T383194) (owner: 10Eileen) [06:25:50] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/1137609 (owner: 10L10n-bot) [13:15:49] (03CR) 10Jforrester: [C:03+2] Use namespaced CountryNames class [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1137516 (owner: 10Umherirrender) [13:24:47] (03PS1) 10VriaA: Banner Settings: Fix broken Selector logic link [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1137749 (https://phabricator.wikimedia.org/T153225) [13:26:40] (03PS2) 10VriaA: Banner Settings: Fix broken Selector logic link [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1137749 (https://phabricator.wikimedia.org/T153225) [13:35:13] (03Merged) 10jenkins-bot: Use namespaced CountryNames class [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1137516 (owner: 10Umherirrender) [13:50:51] (03CR) 10Jforrester: [C:03+2] Use namespaced classes [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1137532 (owner: 10Umherirrender) [14:09:15] (03Merged) 10jenkins-bot: Use namespaced classes [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1137532 (owner: 10Umherirrender) [14:22:43] ooh, gravy failstorm yesterday and this morning? [14:22:57] processing recurrings, it looks like? [14:26:08] fr-tech there are a few patches for the donor preference center that are ready for review [15:15:33] 06Fundraising-Backlog, 10MW-1.44-notes (1.44.0-wmf.25; 2025-04-15): Bad UI from payments form when recurring=1 for non recurring methods - https://phabricator.wikimedia.org/T391412#10757139 (10AKanji-WMF) Hi @Pcoombe - this phab deals with an issue where people manage to enter a recurring flow in markets wher... [15:23:32] 06Fundraising-Backlog: Gravy Paypal recurrings failing on smashpig recurring charge - https://phabricator.wikimedia.org/T392369 (10Cstone) 03NEW [15:29:46] 06Fundraising-Backlog: Gravy Paypal recurrings failing on smashpig recurring charge due to no payment method - https://phabricator.wikimedia.org/T392369#10757195 (10Cstone) [15:47:40] (03CR) 10Pppery: "recheck" [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1137749 (https://phabricator.wikimedia.org/T153225) (owner: 10VriaA) [16:23:05] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Transactions stuck in "Authorised" status in Adyen - Urgent - https://phabricator.wikimedia.org/T392000#10757309 (10AnnWF) https://docs.google.com/spreadsheets/d/1aUdvCtviMOxNNiLxHoRizeObxNPZmBBPpj7itH7ZHP8/edit?gid=1099677747#gid=1099677747 refuneded 738... [16:46:25] 06Fundraising-Backlog, 10Recurring-Donations, 07payments-orchestration: Tax ID mapping fails for Gravy donors whose country changes - https://phabricator.wikimedia.org/T392376 (10Ejegg) 03NEW [17:12:06] (03CR) 10Wfan: [C:03+2] "Should to fix gateway_txn_id , order_id, date missing problem in next patch" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1136010 (https://phabricator.wikimedia.org/T381012) (owner: 10Jgleeson) [17:13:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices, 07payments-orchestration: Gr4vy audit file edge cases - https://phabricator.wikimedia.org/T381012#10757390 (10AnnWF) >>! In T381012#10755711, @AMJohnson wrote: > Awesome news. Thank you @AnnWF! > > Apologies if this one is already... [17:39:32] (03PS1) 10Ejegg: Simplify Gravy PayPal request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1137806 [17:39:55] cstone: does that look right? ^^^ [17:43:38] ejegg: i can look in like 15 [17:43:44] thanks! [18:34:16] 06Fundraising-Backlog, 10FR-donorservices: Can donor-facing error #s be made searchable at Gr4vy - https://phabricator.wikimedia.org/T392382 (10MBeat33) 03NEW [18:36:40] 06Fundraising-Backlog, 10FR-donorservices: Can donor-facing error #s be made searchable at Gr4vy - https://phabricator.wikimedia.org/T392382#10757530 (10MBeat33) And, thanks too Amber for the suggestion that a letter prefix to the error code that designated the method, like D12345678.1 for dLocal A12345678.1... [18:38:55] 06Fundraising-Backlog, 10FR-Smashpig, 07payments-orchestration: Gravy API::getTransaction should handle bad response more gracefully - https://phabricator.wikimedia.org/T392383 (10Ejegg) 03NEW [18:42:57] 06Fundraising-Backlog, 10FR-donorservices: Can donor-facing error #s be made searchable at Gr4vy - https://phabricator.wikimedia.org/T392382#10757544 (10Cstone) These are in the gravy console as the External Identifier and look to be searchable under merchant_reference @MBeat33 can you try searching some of t... [18:46:56] 10fundraising-tech-ops, 10FR-Tech-Analytics, 06Privacy Engineering, 06secscrum, 10Security Preview: Security review for python package dbt-metabase - https://phabricator.wikimedia.org/T385337#10757558 (10Mstyles) Hey @Dwisehaupt do you have a particular date for this python package to go into production?... [19:13:08] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog: Update paymentswiki to 1.43.1 - https://phabricator.wikimedia.org/T391910#10757587 (10XenoRyet) 05Open→03Resolved [19:13:37] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10MW-1.44-notes (1.44.0-wmf.26; 2025-04-22): Come up with a better name for PaymentDetailResponse in SmashPig - https://phabricator.wikimedia.org/T383400#10757590 (10XenoRyet) 05Open→03Resolved [19:14:06] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10MW-1.44-notes (1.44.0-wmf.26; 2025-04-22): Switch off CVV fraud filter for non-cc payment methods - https://phabricator.wikimedia.org/T391390#10757592 (10XenoRyet) 05Open→03Resolved [19:14:33] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-email-preferences: Email pref ctr: Set opt_in_source in consumer - https://phabricator.wikimedia.org/T281936#10757596 (10XenoRyet) 05Open→03Resolved [19:14:49] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10BDC-Implementation: Migrate legacy exchange rate API to exchange-rates-api.oanda.com - https://phabricator.wikimedia.org/T389975#10757598 (10XenoRyet) 05Open→03Resolved [19:15:12] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface: Add @covers annotations to all DonationInterface tests - https://phabricator.wikimedia.org/T385539#10757601 (10XenoRyet) 05Open→03Resolved [19:15:33] 06Fundraising-Backlog, 10FR-donorservices: Can donor-facing error #s be made searchable at Gr4vy - https://phabricator.wikimedia.org/T392382#10757603 (10MBeat33) 05Open→03Resolved a:03MBeat33 Christmas in April, thank you @Cstone. I tested this and Gr4vy found my CVV error just like you said. Awesome. [19:15:35] 14Fundraising Sprint Git Rebase Jump, 14Fundraising Sprint Humongous bacteria petting zoo, 14Fundraising Sprint Interstitial ads halfway down the coaster hill, 14fundraising sprint Just in time roller coaster building, and 5 others: Convert large donation... - https://phabricator.wikimedia.org/T270667#10757606 [19:16:31] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 10FR-email-preferences: EmailPreferences form lacks template for user-friendly error display - https://phabricator.wikimedia.org/T340167#10757612 (10XenoRyet) 05Open→03Resolved [19:17:10] 03Fundraising Sprint: Grits & Catfish, 06Fundraising-Backlog, 07payments-orchestration: Gravy Paypal: Update to new integration - https://phabricator.wikimedia.org/T385738#10757619 (10XenoRyet) 05Open→03Resolved [19:19:56] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Amount too low to be accepted by Card Network - https://phabricator.wikimedia.org/T392308#10757630 (10XenoRyet) p:05Triage→03Medium [19:20:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: CiviCRM Message Templates: "Draft" version being sent instead of "Active" - https://phabricator.wikimedia.org/T392300#10757631 (10XenoRyet) p:05Triage→03High [19:21:48] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civi contact summary and contributions page style broken - https://phabricator.wikimedia.org/T392231#10757638 (10XenoRyet) 05Open→03Invalid [19:22:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 13Patch-For-Review: Consolidate error handling in queues - https://phabricator.wikimedia.org/T365418#10757639 (10XenoRyet) p:05Triage→03High [19:23:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Activity Import nothing happening clicking continue on field mapping - https://phabricator.wikimedia.org/T390965#10757640 (10XenoRyet) 05Open→03Resolved [19:24:07] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Activity Import nothing happening clicking continue on field mapping - https://phabricator.wikimedia.org/T390965#10757642 (10XenoRyet) [19:24:23] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe hitting issues on snooze- when contacts have values - https://phabricator.wikimedia.org/T379027#10757643 (10XenoRyet) p:05Triage→03Medium [19:48:55] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email: One CID with an on-hold email address is getting a TY mail activity on every run - https://phabricator.wikimedia.org/T392387 (10Ejegg) 03NEW [19:49:11] ok cstone I just made a phab for that crazy TY mail activity bug ^^^ [19:49:25] will try to recreate locally by putting an email on hold [19:49:32] ok if i find the other contact with it ill add to that ticket [19:56:10] ejegg: that queue for deduping patch didn't give me that error when I tried it yesterday [20:04:19] cstone: I'm taking a look at that deadlock now - it looks like a Fidelity import is happening & is the proximate cause [20:06:22] arg civicrm Error manifest for docker-registry.wikimedia.org/dev/fundraising-civicrm-bookworm-php82-apache2:1.1.0 not found: manifest unknown: manifest unknown2.4s - is that not published? [20:09:19] hmm is it not maybe, eileen i had to do something locally for it at the offiste I think [20:14:25] hmm - well I got past the docker thing cos my container is actually up from before the last git pull - as long as I never reboot I don't see the problem... [20:15:00] I did a git pull on dev-fundraising after I brought up the container - so presumably pulled in someting that caused grumpiness [20:19:15] 03Fundraising Sprint: Grits & Catfish: 2025 Sprint G Priorities/ End of sprint summary - https://phabricator.wikimedia.org/T392392 (10AnnWF) 03NEW [20:22:49] 06Fundraising-Backlog: Cz Republic Online Banking disabled - https://phabricator.wikimedia.org/T392393 (10EMartin) 03NEW [20:24:31] 06Fundraising-Backlog: Spike: Detect any potential issues after we migrate to gravy fully - https://phabricator.wikimedia.org/T392394 (10AnnWF) 03NEW [20:24:44] 06Fundraising-Backlog: Cz Republic Online Banking disabled - https://phabricator.wikimedia.org/T392393#10757804 (10Cstone) @EMartin this could be from the issues Adyen is experiencing today, I know it was loading properly after the offiste [20:27:10] 06Fundraising-Backlog: Cz Republic Online Banking disabled - https://phabricator.wikimedia.org/T392393#10757806 (10EMartin) [20:29:04] 06Fundraising-Backlog: Cz Republic Online Banking disabled - https://phabricator.wikimedia.org/T392393#10757808 (10EMartin) @Cstone Interesting that I don't see any transaction for the last 2 weeks which makes me think something else if afoot: {F59334938} [20:42:02] PHP Deprecated: strtotime(): Passing null to parameter #1 ($datetime) of type string is deprecated in /srv/civi-sites/wmff/vendor/wikimedia/smash-pig/Core/DataStores/PaymentsFraudDatabase.php on line 49 [20:42:54] 06Fundraising-Backlog: Cz Republic Online Banking disabled - https://phabricator.wikimedia.org/T392393#10757820 (10Cstone) @EMartin I'm also getting the timeout error locally from their dev environment which makes me think this particular error is related but that doesn't mean something else isn't going on. We c... [20:47:36] cstone: do you know if that deprecation is as simple as adding a cast to string? ^^ [20:51:47] i think so is that how we fixed the others [20:53:29] ill put up a patch [20:54:34] oh i see its more of a check [20:58:14] or time has been lost [20:59:27] ok time is not always lost [21:02:17] eileen: where did you find that error [21:02:28] cstone: running unit tests [21:02:34] ahh okay [21:02:38] time has been lost locally then [21:02:57] I mean that's how I feel.... [21:03:11] Do you know where to look for all the time I lost? [21:03:58] havent figured out why its lost but when it is lost then null gets passed to strtotime [21:06:20] 06Fundraising-Backlog, 10fundraising-tech-ops: Can we get more MariaDB deadlock tools? - https://phabricator.wikimedia.org/T392396 (10Eileenmcnaughton) 03NEW [21:10:35] 06Fundraising-Backlog, 10fundraising-tech-ops: Can we get more MariaDB deadlock tools? - https://phabricator.wikimedia.org/T392396#10757874 (10Eileenmcnaughton) e.g if we could call something in the Exception construct & log it.... [21:27:20] 03Fundraising Sprint: Grits & Catfish: 2025 Sprint G Priorities/ End of sprint summary - https://phabricator.wikimedia.org/T392392#10757902 (10AnnWF) **__End of sprint report summary__** **Donor Portal and Email Preference Center (In progress) ** - T388744 Create CiviCRM Api call to retrieve information for... [21:31:23] (03CR) 10Cstone: [C:03+2] "thanks this fixes the recurring issue and new donations look good too" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1137806 (owner: 10Ejegg) [21:31:28] (03PS1) 10Eileen: Fix for deadlock crashing queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137823 [21:31:53] (03Merged) 10jenkins-bot: Simplify Gravy PayPal request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1137806 (owner: 10Ejegg) [21:34:03] im gona get that fix out to get those gravy paypals to process ^ [21:34:53] OK - I figured out the fix for deadlocks crashing the queue & it was super simple ^^ [21:35:00] oh nice eileen [21:35:19] yeah - just too aggressive exception throwing [21:35:37] (03CR) 10Cstone: [C:03+2] Fix for deadlock crashing queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137823 (owner: 10Eileen) [21:38:31] (03PS1) 10Cstone: Upgrade smash-pig to v0.9.4.1 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137825 [21:39:29] thanks cstone [21:40:37] ejegg: at the bottom of that civi smashpig patch I just made the plugin version randomly goes down thats not right right ^ [21:40:57] hmm although it looks auto generated [21:42:27] one sec, lemme look [21:42:54] oh, the plugin-api-version [21:43:08] i think that's related to the version of composer you're running locally [21:43:20] ah okay lets see [21:43:26] try a self-upgrade? [21:44:31] (03PS2) 10Cstone: Upgrade smash-pig to v0.9.4.1 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137825 [21:44:39] got it thanks ejegg ! [21:45:23] 06Fundraising-Backlog, 10fundraising-tech-ops: Can we get more MariaDB deadlock tools? - https://phabricator.wikimedia.org/T392396#10757938 (10Eileenmcnaughton) Or this innodb_print_all_deadlocks = ON (I *think* `data_locks` might be mysql only but unsure) [21:49:38] (03CR) 10Ejegg: [C:03+2] Upgrade smash-pig to v0.9.4.1 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137825 (owner: 10Cstone) [21:51:40] 03Fundraising Sprint: Grits & Catfish: 2025 Sprint G Priorities/ End of sprint summary - https://phabricator.wikimedia.org/T392392#10757951 (10AnnWF) 05Open→03Resolved [22:00:29] (03Merged) 10jenkins-bot: Fix for deadlock crashing queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137823 (owner: 10Eileen) [22:10:02] so we've got the errors on those repeated sends in the logs each time, but we're no longer setting no_thank_you it seems like [22:10:18] I swear we used to set no_thank_you=error or something [22:14:44] oh wow, last 'failed' in there is 2016 [22:14:50] what else do we have in there? [22:16:17] (03Merged) 10jenkins-bot: Upgrade smash-pig to v0.9.4.1 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137825 (owner: 10Cstone) [22:17:22] ok, lots more 'failed: BAD_EMAIL', failed: EMAIL_SYSTEM_FAILURE, etc [22:17:26] so when did those stop? [22:21:20] hmm, we have one on 4/7 [22:21:44] (03PS1) 10Cstone: Update smash-pig to v0.9.4.1 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1137828 [22:22:28] ok, it was in thank_you.module [22:22:28] (03CR) 10Cstone: [C:03+2] Update smash-pig to v0.9.4.1 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1137828 (owner: 10Cstone) [22:23:03] err, it still is... [22:23:40] so we are not using thank_you_batch_process now, right? [22:25:06] oh we are - that's still drush [22:32:30] ejegg: yeah - we need to migrate that but... [22:33:21] Hmm - my plan to never restart docker failed - now I have to deal with Error response from daemon: manifest for docker-registry.wikimedia.org/dev/fundraising-civicrm-bookworm-php82-apache2:1.1.0 not found: manifest unknown: manifest unknown [22:33:21] - maybe setup.sh will help [22:34:32] the docker command works - https://docker-registry.wikimedia.org/dev/fundraising-bookworm-php82-apache2/tags/ [22:35:12] eileen: I guess try with the previous version? [22:35:29] edit docker-compose.yml [22:35:55] or just git revert 40276985f91e43e75a37776df5de4b790cea60cc [22:36:22] let me see if the patch for that version of the civi image even got merged to gitlab [22:37:14] ok, looks like it's merged [22:37:27] I'll ask in -releng for someone to publish it [22:37:39] yeah - this process always feels really clunky [22:47:25] (03Merged) 10jenkins-bot: Update smash-pig to v0.9.4.1 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1137828 (owner: 10Cstone) [22:48:43] (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1137831 [22:49:16] (03CR) 10Cstone: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1137831 (owner: 10Cstone) [22:50:13] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1137831 (owner: 10Cstone) [23:12:31] !log civicrm upgraded from d7eefbc4 to b3038510 [23:12:33] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log