[02:01:32] 06Fundraising-Backlog, 10FR-Donation-Processing, 0725 Dec Q3 Planning, 03Civi RoadMap FY 26-27, 07fr-combowiki: Chariot DAFpay button integration - donation form - https://phabricator.wikimedia.org/T405043#12245080 (10AKanji-WMF) [02:38:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.5] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [02:48:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [51.4] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [02:53:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [51.4] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [03:28:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [05:49:16] FIRING: ContextSwitchingSpike: Host context switching high (instance frdb1007.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [05:54:16] RESOLVED: ContextSwitchingSpike: Host context switching high (instance frdb1007.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [08:47:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.1] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [09:01:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [40.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [09:26:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [30.9] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [09:42:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [60.1] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [10:07:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [30.7] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [11:22:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [11.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [12:07:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [54.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [12:13:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [11.4] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [12:27:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [33.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [13:05:44] (03CR) 10Laurabarluzzi: [C:03+2] Make subject more useful for failmail [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1327192 (owner: 10Eileen) [13:13:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [10.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [13:26:19] (03Merged) 10jenkins-bot: Make subject more useful for failmail [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1327192 (owner: 10Eileen) [13:29:46] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [12.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [13:35:18] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint, and 2 others: Send Leadgen emails to Civi then Acoustic - https://phabricator.wikimedia.org/T434150#12246772 (10Pcoombe) Hi @Lars, I'm trying to have [this banner](https://en.wikipedia.org/wiki... [13:42:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, and 2 others: Cancels via donor portal not sticking - https://phabricator.wikimedia.org/T432091#12246798 (10Lars) No more of these have happened in the last two weeks, while previously we had a few p... [13:51:19] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint, and 2 others: Send Leadgen emails to Civi then Acoustic - https://phabricator.wikimedia.org/T434150#12246837 (10Lars) @Pcoombe That's expected, this hasn't been deployed yet on our end. Will le... [14:13:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [61.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [14:48:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [55.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [14:56:45] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10fundraising-tech-ops, 03Civi RoadMap FY 26-27: FR-Tech FY2627Q1 maintenance window - https://phabricator.wikimedia.org/T419557#12247248 (10Dwisehaupt) [14:57:08] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10fundraising-tech-ops, 03Civi RoadMap FY 26-27: FR-Tech FY2627Q1 maintenance window - https://phabricator.wikimedia.org/T419557#12247251 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt [15:05:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [31.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [15:10:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [31.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [15:33:48] (03PS2) 10Lars SG: Add Double Opt-In - Lead Gen message template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) [15:33:48] (03PS3) 10Lars SG: Modify Double Opt In action to ensure contact is opted in [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328276 (https://phabricator.wikimedia.org/T434150) [15:33:48] (03PS3) 10Lars SG: Allow multiple workflows for Double Opt-In send [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328277 (https://phabricator.wikimedia.org/T434150) [15:33:49] (03PS4) 10Lars SG: Add Lead Gen Double Opt in [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328278 (https://phabricator.wikimedia.org/T434150) [15:34:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [13.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [15:35:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [37.8] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [15:36:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, and 2 others: Cancels via donor portal not sticking - https://phabricator.wikimedia.org/T432091#12247608 (10Lars) Now merged upstream. [15:45:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [32.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [15:50:40] (03PS1) 10Lars SG: Specify table /column name for Activity source custom field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328653 (https://phabricator.wikimedia.org/T434150) [16:06:39] FIRING: HighPaymentsMedianRiskScore: Median payments risk_score is running high [48.1] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [16:12:23] (03PS1) 10Damilare Adedoyin: Add more test for combowiki payment flow [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328662 (https://phabricator.wikimedia.org/T434829) [16:14:45] (03CR) 10CI reject: [V:04-1] Add more test for combowiki payment flow [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328662 (https://phabricator.wikimedia.org/T434829) (owner: 10Damilare Adedoyin) [16:16:39] RESOLVED: HighPaymentsMedianRiskScore: Median payments risk_score is running high [32.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentsMedianRiskScore [16:22:03] (03PS2) 10Damilare Adedoyin: Add more tests for combowiki payment flow [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328662 (https://phabricator.wikimedia.org/T434830) [16:29:02] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10FR-Donor-portal, 06FR-donorrelations, 10FR-MGFE: PDF Civi receipts - feature request - https://phabricator.wikimedia.org/T368671#12247998 (10SBurnett-WMF) Checked in with Lars - adding to Sprint+1 [16:29:39] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10FR-Donor-portal, 06FR-donorrelations, and 2 others: PDF Civi receipts - feature request - https://phabricator.wikimedia.org/T368671#12248005 (10SBurnett-WMF) [16:31:15] (03PS3) 10Damilare Adedoyin: Add more tests for combowiki payment flow [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328662 (https://phabricator.wikimedia.org/T434830) [16:42:15] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 13Patch-For-Review: Add tests coverage for the refactored DonationData() code in ComboWiki - https://phabricator.wikimedia.org/T434830#12248078 (10Damilare) a:03Damilare [16:45:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [11.0] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [16:55:22] (03PS4) 10Damilare Adedoyin: Add more tests for combowiki payment flow [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328662 (https://phabricator.wikimedia.org/T434830) [17:11:00] (03PS1) 10Damilare Adedoyin: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1328676 [17:15:24] (03CR) 10Damilare Adedoyin: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1328676 (owner: 10Damilare Adedoyin) [17:17:11] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1328676 (owner: 10Damilare Adedoyin) [17:19:24] !log payments-wiki upgraded from c56b3ad7 to c3ed1fa0 [17:19:25] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [17:26:14] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328681 [17:26:17] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328681 (owner: 10Ejegg) [17:27:30] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328681 (owner: 10Ejegg) [17:27:43] !log disabled payment attempt queue consumer [17:27:45] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [17:28:09] larssandergreen: I'm planning to deploy the stuff in this merge: https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1328681 [17:28:37] including 'populate new gateway custom field' [17:28:51] and the browser / os / version backfill on payment attempts [17:29:03] (03PS2) 10Lars SG: Specify table /column name for Activity source custom field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328653 (https://phabricator.wikimedia.org/T434150) [17:29:07] and the leadgen queue consumer [17:29:11] any concerns? [17:29:31] (03CR) 10Damilare Adedoyin: [C:03+2] Specify table /column name for Activity source custom field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328653 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [17:30:19] ejegg: can we get this in before you deploy? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1328653 [17:30:25] 06Fundraising-Backlog: October 13 2026 Gravy ip changes - https://phabricator.wikimedia.org/T435853 (10Cstone) 03NEW [17:30:41] then it will be created with the right table name instead of the auto-generated one [17:31:20] otherwise no concerns [17:31:46] (03CR) 10Ejegg: [V:03+2] Specify table /column name for Activity source custom field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328653 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [17:32:08] thanks [17:32:29] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328682 [17:32:32] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328682 (owner: 10Ejegg) [17:35:02] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint, and 2 others: Send Leadgen emails to Civi then Acoustic - https://phabricator.wikimedia.org/T434150#12248317 (10SBurnett-WMF) @Lars will we be able to see how many previously dormant or opted-o... [17:38:00] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint, and 2 others: Send Leadgen emails to Civi then Acoustic - https://phabricator.wikimedia.org/T434150#12248318 (10Lars) @SBurnett-WMF Yes, Analytics should be able to answer that question with th... [17:38:50] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint, and 2 others: Send Leadgen emails to Civi then Acoustic - https://phabricator.wikimedia.org/T434150#12248322 (10Lars) silverpop database update needed: ` CREATE TABLE IF NOT EXISTS silverpop_ex... [17:40:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [14.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [17:41:39] (03PS1) 10Lars SG: Export leadgen details to Acoustic [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1328683 (https://phabricator.wikimedia.org/T434150) [17:47:50] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Duplicate banking institution soft credit - https://phabricator.wikimedia.org/T435643#12248371 (10SBurnett-WMF) [17:48:30] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Duplicate banking institution soft credit - https://phabricator.wikimedia.org/T435643#12248376 (10SBurnett-WMF) is this chaos? [17:51:28] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-combowiki: Maybe we can manage legacy Paypal recurrings or provide links? - https://phabricator.wikimedia.org/T435556#12248379 (10SBurnett-WMF) p:05Triage→03Medium [17:56:05] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1328682 (owner: 10Ejegg) [18:01:08] !log civicrm upgraded from 225dc534 to 33fac802 [18:01:09] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [18:01:14] ok, also synced custom fields [18:01:21] let's see if that got the entity type stuff too [18:02:51] yep [18:02:54] running db upgrades [18:03:33] oh there's a logging trigger update for that new custom field [18:07:55] (03PS1) 10Ejegg: Update triggers for civicrm_value_source [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328689 [18:07:58] (03CR) 10Ejegg: [C:03+2] Update triggers for civicrm_value_source [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328689 (owner: 10Ejegg) [18:08:21] Jeff_Green: there are a couple of new triggers associated with that last civicrm deploy [18:08:32] I've updated the full trigger file in that patch ^^^ [18:08:58] but the three new ones are in this paste, if you want to just run those separately so as not to need to stop stuff: https://phabricator.wikimedia.org/P96238 [18:10:25] larssandergreen: does every 10 minutes on the 7s sound good for the leadgen queue consumer? [18:12:05] I guess ideally we'd be sending those DOI emails as fast as possible. Not sure how to balance that against running the queue consumer all the time [18:12:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ACH] Validation error: Payment method has not been successfully stored - https://phabricator.wikimedia.org/T435615#12248482 (10SBurnett-WMF) Mentioned in Gravy slack channel for visibility: https://wikimedia.slack.com/archives/C070F1DVBR... [18:12:37] also ejegg: do we still need to deploy email prefs centre or smashpig or have those already been done? [18:13:06] (03CR) 10Lars SG: [V:03+2 C:03+2] Setup for local lead generation testing [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1324820 (https://phabricator.wikimedia.org/T434150) (owner: 10Ejegg) [18:13:32] larssandergreen: I deployed epc last week [18:14:29] let me double check on the smashpig settings for the new queue [18:15:04] ah nope, I need that [18:16:52] ejegg: Jeff_Green: I'm fine if we catch those trigger updates next time we update without doing anything now for this specifically. Those shouldn't ever change, so it doesn't really matter if we miss some log lines for those sources [18:18:35] larssandergreen & ejegg, there's a mariadb update that came out today that has some security fixes. maybe the thing to do is combine the two things and pick a time for a brief civi outage [18:19:01] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 0725 Dec Q3 Planning: Due by Feb 15: Braintree - check if we are using this - https://phabricator.wikimedia.org/T412175#12248490 (10SBurnett-WMF) 05Open→03Resolved a:03SBurnett-WMF Checked with @Dwisehaupt and he reviewed cert status. There was not... [18:20:24] Jeff_Green: Can I ask you for one more thing if you've got a minute? - a proposed puppet patch for the new lead-generation queue in my homedir on frpm1002 ~/puppet/0001-Add-config-for-lead-generation-queue.patch [18:20:45] oh the mariadb updates - sure, I guess a brief outage is the thing to do [18:21:06] and it sounds like those triggers are low priority, so we can do it whenever is least disruptive for an outage [18:21:44] whatever makes sense for the mariadb is fine with me, triggers can definitely wait but could be done if we're doing downtime anyways [18:22:28] ejegg: looking re. patch. did you see the think above re. the envrionment variable? I think we need to set it somewhere for wsgi? I don't think putting it in the apache config works [18:23:03] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Maybe we can manage legacy Paypal recurrings or provide links? - https://phabricator.wikimedia.org/T435556#12248506 (10Cstone) [18:23:13] huh, that's too bad. I'd prefer not to have to set it in code since I think python would already compile the code that's setting it [18:23:47] I guess if it's in the wsgi launcher script it's maybe only going to compile that one [18:24:18] right, I think that's where it would go [18:26:03] ejegg: puppet patch is deploying [18:26:57] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 03Civi RoadMap FY 26-27, 07fr-current-sprint: Implement new audience framework in CiviCRM and export to Acoustic - https://phabricator.wikimedia.org/T426818#12248535 (10Lars) Also @Eileenmcnaughton we can celebra... [18:33:24] (03Merged) 10jenkins-bot: Update triggers for civicrm_value_source [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328689 (owner: 10Ejegg) [18:42:55] oh shoot, that backfill script failed ? [18:51:42] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ACH] Validation error: Payment method has not been successfully stored - https://phabricator.wikimedia.org/T435615#12248600 (10SBurnett-WMF) Matheus Cruz  [2:24 PM] Hello @stacy.burnett, I'm checking this one and will keep you posted [18:58:45] !log re-enabled payment attempts queue consumer [18:58:52] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [19:03:23] (03PS1) 10Wfan: fix replace $1 from translation update [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328695 (https://phabricator.wikimedia.org/T435431) [19:04:18] (03CR) 10Dmiranda: [C:03+2] "Thanks @wfan@wikimedia.org!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328695 (https://phabricator.wikimedia.org/T435431) (owner: 10Wfan) [19:04:21] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team: CI build slow and timing out sometimes - https://phabricator.wikimedia.org/T434024#12248635 (10Andrew) I'm adjusted the supermicro servers (1077-10... [19:09:53] (03Merged) 10jenkins-bot: fix replace $1 from translation update [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328695 (https://phabricator.wikimedia.org/T435431) (owner: 10Wfan) [19:18:20] (03PS1) 10Wfan: fix the translation replace update [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328697 (https://phabricator.wikimedia.org/T433143) [19:25:54] (03CR) 10Dmiranda: [C:03+2] "Thank you @wfan@wikimedia.org!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328697 (https://phabricator.wikimedia.org/T433143) (owner: 10Wfan) [19:30:47] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.17; 2026-08-25), 13Patch-For-Review: Recurring convert in Combowiki - pre payment suggestion - https://phabricator.wikimedia.org/T435431#12248696 (10XenoRyet) 05Open→03Resolved [19:31:03] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Remove un-needed separate endowment/foundation calculated fields from Civi & Acoustic - https://phabricator.wikimedia.org/T418885#12248698 (10XenoRyet) 05Open→03Resolved [19:31:25] 03Fundraising Sprint: Ottoman-o-war, 07Fundraising-ComboWiki-Dependency, 03Civi RoadMap FY 26-27, 07Epic, and 3 others: Add Gravy Venmo Vue Component - https://phabricator.wikimedia.org/T432231#12248702 (10XenoRyet) 05Open→03Resolved [19:31:38] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.17; 2026-08-25), 13Patch-For-Review: Recurring convert in Combowiki - post payment - https://phabricator.wikimedia.org/T433143#12248707 (10XenoRyet) 05Open→03Resolved [19:31:50] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 10FR-Tech onboarding task, 05MW-1.47-notes (1.47.0-wmf.17; 2026-08-25): Combowiki: employer field explain text - https://phabricator.wikimedia.org/T435132#12248711 (10XenoRyet) 05In progress→03Resolved [19:32:12] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.17; 2026-08-25), 13Patch-For-Review: Important: prevent duplicate contribution tracking message in ComboWiki - https://phabricator.wikimedia.org/T434829#12248714 (10XenoRyet) 05Open→03Resolved [19:32:16] FIRING: ContextSwitchingSpike: Host context switching high (instance frpm1002.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [19:32:24] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-MGFE, and 2 others: Add daf_contact_id to Acoustic export - https://phabricator.wikimedia.org/T432185#12248717 (10XenoRyet) 05Open→03Resolved [19:32:37] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-MGFE, and 2 others: Add MGFE requested fields from audience framework refresh - https://phabricator.wikimedia.org/T430012#12248720 (10XenoRyet) 05Open→03Resolved [19:32:53] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Update is_eligible_for_donor_portal and getDonorPortalUrl to use new segments - https://phabricator.wikimedia.org/T430369#12248723 (10XenoRyet) 05Open→03Resolved [19:33:09] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Increase mailing name field to varchar(255) in Civi upstream - https://phabricator.wikimedia.org/T432997#12248725 (10XenoRyet) 05Open→03Resolved [19:33:31] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 03Civi RoadMap FY 26-27, 07fr-current-sprint: Implement new audience framework in CiviCRM and export to Acoustic - https://phabricator.wikimedia.org/T426818#12248729 (10XenoRyet) 05Open→03Resolved [19:33:48] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10FR-payments-general, 07Epic, 07fr-current-sprint: Larger checkout.com testing - https://phabricator.wikimedia.org/T430710#12248731 (10XenoRyet) 05Open→03Resolved [19:34:04] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10FR-Donor-portal, 10FR-email-preferences, and 3 others: Preference Centre - Make unsubscribed status clear for user - https://phabricator.wikimedia.org/T427288#12248734 (10XenoRyet) 05In progress→03Resolved [19:34:41] (03Merged) 10jenkins-bot: fix the translation replace update [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1328697 (https://phabricator.wikimedia.org/T433143) (owner: 10Wfan) [19:36:18] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10FR-payments-general, 07fr-current-sprint, 13Patch-For-Review: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#12248753 (10XenoRyet) 05Open→03Resolved [19:36:29] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog: Create dedicated app_install_id field in contribution_tracking - https://phabricator.wikimedia.org/T433883#12248755 (10XenoRyet) 05Open→03Resolved [19:37:20] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Content update for 1st? and 2nd recurring failure email - https://phabricator.wikimedia.org/T416435#12248767 (10XenoRyet) [19:37:29] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Content update for 1st? and 2nd recurring failure email - https://phabricator.wikimedia.org/T416435#12248768 (10XenoRyet) 05Open→03Resolved [19:39:15] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-matching-gifts, and 2 others: Explore DoubleTheDonation integration - https://phabricator.wikimedia.org/T433406#12248773 (10Lars) a:03Lars [19:39:20] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.16; 2026-08-18): Populate fields from URL/Session values in ComboWiki - https://phabricator.wikimedia.org/T432271#12248774 (10XenoRyet) 05In progress→03Resolved [19:39:36] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 07fr-current-sprint: Send intial contribution tracking row from ComboWiki - https://phabricator.wikimedia.org/T433539#12248788 (10XenoRyet) 05In progress→03Resolved [19:39:44] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 07fr-combowiki, 07fr-current-sprint: For ComboWiki, add a helper class to integrate data from get, post, and session - https://phabricator.wikimedia.org/T433537#12248791 (10XenoRyet) 05In progress→... [19:40:14] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 07fr-current-sprint, and 2 others: Connect DonationInterface to fraud scoring service - https://phabricator.wikimedia.org/T429605#12248797 (10XenoRyet) 05Open→03Resolved [19:40:30] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.16; 2026-08-18): combowiki: UI change after hitting submit / waiting for payment processor response - https://phabricator.wikimedia.org/T434110#12248799 (10XenoRyet) 05Open→03Resolved [19:40:46] 03Fundraising Sprint: Ottoman-o-war, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.17; 2026-08-25): eslint for combowiki - https://phabricator.wikimedia.org/T434820#12248803 (10XenoRyet) 05Open→03Resolved [19:41:05] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Add last otg amount change fields to Acoustic export - https://phabricator.wikimedia.org/T431854#12248806 (10XenoRyet) 05Open→03Resolved [19:42:24] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-current-sprint, 13Patch-For-Review: Pinless debit card routing to local netwroks in the US - https://phabricator.wikimedia.org/T433862#12248808 (10XenoRyet) 05Open→03Resolved [19:42:35] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-combowiki, 05MW-1.47-notes (1.47.0-wmf.16; 2026-08-18): Employer field in combowiki - https://phabricator.wikimedia.org/T433142#12248811 (10XenoRyet) 05Open→03Resolved [19:43:22] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-payments-general: Import errors for prefix and suffix - https://phabricator.wikimedia.org/T434727#12248817 (10XenoRyet) 05Open→03Resolved [19:43:38] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Dedupe interface missing the last donation date - https://phabricator.wikimedia.org/T434612#12248819 (10XenoRyet) 05Open→03Resolved [19:44:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Google Pay recurring with different name (from separate issue with token import test) changed contact name - https://phabricator.wikimedia.org/T434715#12248825 (10XenoRyet) 05Open→03Resolved [19:44:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Omnigroupmember.load gets stuck if csv download is borked - https://phabricator.wikimedia.org/T435128#12248832 (10XenoRyet) 05Open→03Resolved [19:44:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Invalid PaymentMethod passed in for Gravy Dlocal Pix in CiviCRM - https://phabricator.wikimedia.org/T435081#12248834 (10XenoRyet) 05Open→03Resolved [19:44:59] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: first failure email template syntax issue - https://phabricator.wikimedia.org/T434965#12248836 (10XenoRyet) 05Open→03Resolved [19:45:52] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-WMF-Audit: Adyen - new unknown payment method 'accel' & 'maestro_usa' - https://phabricator.wikimedia.org/T434269#12248839 (10XenoRyet) 05Open→03Resolved [19:46:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: API Explorer Settings don't show any actions - https://phabricator.wikimedia.org/T434618#12248841 (10XenoRyet) 05Open→03Resolved [19:46:25] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donor-portal, and 2 others: Cancels via donor portal not sticking - https://phabricator.wikimedia.org/T432091#12248843 (10XenoRyet) 05Open→03Resolved [19:46:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Tech onboarding task: Add catch to adyen audit to not set processor_contact_id when its a gravy adyen transaction - https://phabricator.wikimedia.org/T432484#12248845 (10XenoRyet) 05Open→03Resolved [19:47:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Donation-Processing: Chariot data issue - DAFpay payment method - https://phabricator.wikimedia.org/T434317#12248861 (10XenoRyet) 05Open→03Resolved [19:47:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Smarty templates failing to render when donor name has an apostrophe - https://phabricator.wikimedia.org/T421166#12248864 (10XenoRyet) 05Open→03Resolved [19:48:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: [Aug 14th] Fail Mail (civi1002) run-job: Generate finance batch failed with code 1 - https://phabricator.wikimedia.org/T434914#12248868 (10XenoRyet) 05In progress→03Resolved [19:50:09] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Failmail alert Validation error during recurring charge, in field: payment_submethod - https://phabricator.wikimedia.org/T435133#12248873 (10Laurabarluzzi) [19:52:44] 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap: FR-Tech FY2627Q1 Analytics Updates - https://phabricator.wikimedia.org/T433888#12248887 (10AStein-WMF) 05In progress→03Resolved [19:57:16] RESOLVED: ContextSwitchingSpike: Host context switching high (instance frpm1002.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [20:06:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations, 10Recurring-Donations: CZK recurring donation amount conversion variability - https://phabricator.wikimedia.org/T435144#12248932 (10Lars) 05Open→03Invalid I'll close this in favour of the above phab. [20:11:09] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 10FR-payments-general: Create dedicated app_install_id field in contribution_tracking - https://phabricator.wikimedia.org/T433883#12248942 (10AKanji-WMF) [20:11:36] 03Fundraising Sprint: Ottoman-o-war, 06Fundraising-Backlog, 07fr-current-sprint, 13Patch-For-Review, 07payments-orchestration: Pinless debit card routing to local netwroks in the US - https://phabricator.wikimedia.org/T433862#12248944 (10AKanji-WMF) [20:16:08] 06Fundraising-Backlog, 07Fundraising-ComboWiki-Dependency, 06Fundraising-Tech-Roadmap, 07fr-combowiki: Figure out the use case for how we want to use testkitch for Combo wiki - https://phabricator.wikimedia.org/T433307#12248988 (10AnnWF) [20:55:02] (03PS6) 10Ejegg: Update fundraising-ml image to add Jupyter [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1321209 (https://phabricator.wikimedia.org/T434031) [20:55:06] (03CR) 10Ejegg: [V:03+2 C:03+2] Update fundraising-ml image to add Jupyter [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1321209 (https://phabricator.wikimedia.org/T434031) (owner: 10Ejegg) [21:07:31] (03CR) 10Ejegg: "Looks great except for the link on the 'yes' button - should we point that to the donorpreferences page straight away?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [21:18:21] (03PS3) 10Lars SG: Add Double Opt-In - Lead Gen message template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) [21:19:08] (03CR) 10Lars SG: Add Double Opt-In - Lead Gen message template (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [21:40:32] (03PS4) 10Lars SG: Add Double Opt-In - Lead Gen message template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) [21:40:34] (03CR) 10Ejegg: [C:03+2] Add Double Opt-In - Lead Gen message template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [21:46:16] 06Fundraising-Backlog, 06Wikimedia-Fundraising, 10Wikimedia-Fundraising-CiviCRM: Show donor's address in TY email when donors is in NY State - https://phabricator.wikimedia.org/T425992#12249192 (10ppenloglou) 05Open→03Resolved a:03ppenloglou This went live as part of the wider Civi Mails refresh we... [21:48:46] 06Wikimedia-Fundraising: New component for image + text side by side - https://phabricator.wikimedia.org/T435876 (10ppenloglou) 03NEW [21:49:02] 06Wikimedia-Fundraising: New component for image + text side by side - https://phabricator.wikimedia.org/T435876#12249215 (10ppenloglou) a:03RLopez-WMF [21:50:27] 06Wikimedia-Fundraising: Improve donate CTA in Footer - https://phabricator.wikimedia.org/T435877 (10ppenloglou) 03NEW [21:58:01] (03Merged) 10jenkins-bot: Add Double Opt-In - Lead Gen message template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328275 (https://phabricator.wikimedia.org/T434150) (owner: 10Lars SG) [22:13:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ PayPal ] Validation error during recurring charge, in field: email - https://phabricator.wikimedia.org/T435881 (10Laurabarluzzi) 03NEW [22:21:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ PayPal ] Validation error during recurring charge, in field: email - https://phabricator.wikimedia.org/T435881#12249426 (10Laurabarluzzi) [23:00:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [14.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [23:00:58] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ PayPal ] Validation error during recurring charge, in field: email - https://phabricator.wikimedia.org/T435881#12249534 (10Lars) I think we have occasionally seen cases where this happens (contact created from donation with no name or e... [23:01:58] (03PS1) 10Mstyles: CentralNotice: Require reauth to enable campaigns [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1328729 (https://phabricator.wikimedia.org/T208278) [23:04:18] (03CR) 10CI reject: [V:04-1] CentralNotice: Require reauth to enable campaigns [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1328729 (https://phabricator.wikimedia.org/T208278) (owner: 10Mstyles) [23:15:31] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ACH] Validation error: Payment method has not been successfully stored - https://phabricator.wikimedia.org/T435615#12249577 (10SBurnett-WMF) Response from Gravy: We confirmed that payment method 2bdccec0-e4ee-4397-9792-1897fe5b070c cha... [23:17:00] (03PS1) 10Eileen: Fix StripeMg validation issue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328735 [23:18:01] (03CR) 10CI reject: [V:04-1] Fix StripeMg validation issue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328735 (owner: 10Eileen) [23:26:14] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328735 (owner: 10Eileen) [23:34:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-failmail: [ PayPal ] Validation error during recurring charge, in field: email - https://phabricator.wikimedia.org/T435881#12249609 (10Lars) [[ https://civicrm.wikimedia.org/civicrm/admin/search#/create/ContributionRecur?params=%7B%22version%22:4,%... [23:40:51] (03CR) 10Eileen: "just noting this fix is one of 2 ways to go here - the other is to set the gateway to stripemg on contribution_extra & trxn_id - trying to" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328735 (owner: 10Eileen) [23:41:04] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Annual recurring one off issues and edge cases - https://phabricator.wikimedia.org/T435888 (10Cstone) 03NEW [23:41:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Annual recurring one off issues and edge cases - https://phabricator.wikimedia.org/T435888#12249630 (10Cstone) [23:42:49] (03PS2) 10Eileen: Fix StripeMg validation issue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1328735 (https://phabricator.wikimedia.org/T432813)