[00:28:09] PROBLEM - check_load on franio2002 is CRITICAL: LOAD CRITICAL - total load average: 34.09, 23.96, 13.32 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [00:43:09] RECOVERY - check_load on franio2002 is OK: LOAD OK - total load average: 7.12, 5.68, 8.67 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2002&service=check_load [00:52:44] (03CR) 10Eileen: "I think the second patch wound up cleaner than the first" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235355 (https://phabricator.wikimedia.org/T416014) (owner: 10Ejegg) [00:52:47] (03CR) 10Eileen: [C:03+2] Don't uncancel when messages processed out of order [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235355 (https://phabricator.wikimedia.org/T416014) (owner: 10Ejegg) [01:53:52] 06Fundraising-Backlog: Add Dontate link in hamburger menu on mobile for logged in users - https://phabricator.wikimedia.org/T416101 (10Commander_Keane) 03NEW [07:51:46] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 0725 Dec Q3 Planning, and 2 others: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#11572369 (10Eileenmcnaughton) We have `channel` reliably populated in our Database now thank... [10:11:28] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 0725 Dec Q3 Planning, and 2 others: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#11573063 (10MSuijkerbuijk_WMF) Yes that sounds good - Please can you look at implementing th... [14:42:56] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 [14:43:39] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 (owner: 10Ejegg) [14:46:50] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 (owner: 10Ejegg) [14:47:32] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 (owner: 10Ejegg) [14:56:43] hmm [14:57:16] oh, silverpop library update? [14:57:21] let's update vendor [14:59:56] (03PS1) 10Ejegg: Update omnimail / silverpop packages [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1235815 [14:59:59] (03CR) 10Ejegg: [C:03+2] Update omnimail / silverpop packages [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1235815 (owner: 10Ejegg) [15:00:32] (03PS2) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 [15:14:38] (03Merged) 10jenkins-bot: Update omnimail / silverpop packages [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1235815 (owner: 10Ejegg) [15:23:22] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 (owner: 10Ejegg) [15:24:27] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235809 (owner: 10Ejegg) [15:26:48] !log fundraising civicrm upgraded from 611d18de to f91f955b [15:26:49] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [15:27:34] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 0725 Dec Q3 Planning: Save screen height and width in their own contribution tracking fields - https://phabricator.wikimedia.org/T411901#11574143 (10Damilare) a:03Damilare [15:28:53] well, looks like the 'acoustic data' query is still working with the new headers, so that's good [15:30:25] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Donor cancelled recurring donation at PayPal. Donation still shows as active in Civi. - https://phabricator.wikimedia.org/T416014#11574153 (10Ejegg) OK, the fix to check dates is deployed, and the previously-reactivated ones have be... [15:47:18] (03PS1) 10Damilare Adedoyin: Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) [15:47:43] (03CR) 10CI reject: [V:04-1] Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) (owner: 10Damilare Adedoyin) [16:03:16] (03PS2) 10Damilare Adedoyin: Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) [16:04:36] (03PS3) 10Damilare Adedoyin: Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) [16:05:36] (03CR) 10CI reject: [V:04-1] Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) (owner: 10Damilare Adedoyin) [16:06:03] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, and 3 others: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#11574414 (10Lars) @MSuijkerbuijk_WMF It looks like this is already being filled in... [16:08:12] (03PS1) 10Ejegg: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1235832 [16:08:17] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1235832 (owner: 10Ejegg) [16:12:30] (03Merged) 10jenkins-bot: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1235832 (owner: 10Ejegg) [16:12:34] Jeff_Green: can you remind me when the payments SSL cert is scheduled to be updated? Apple just sent all the WMF accountholders a warning about the Mar 3 expiry and I'd like to reply to the rest of the group [16:12:58] I see the current cert was installed on Feb 20 of last year, so about the same this year? [16:13:19] (03PS4) 10Damilare Adedoyin: Save screen height and width in their own contribution tracking fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) [16:14:21] (03CR) 10Jgleeson: [C:03+2] "LGTM" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235444 (owner: 10Ejegg) [16:14:27] ejegg: here's the task https://phabricator.wikimedia.org/T411785, given rob's last update I think it'll happen soon [16:14:57] (03Merged) 10jenkins-bot: Fix failmail on Braintree report generation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235444 (owner: 10Ejegg) [16:17:51] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 07Epic: Send double opt-in email template after donation for donors in Italy and Austria - https://phabricator.wikimedia.org/T416198 (10Lars) 03NEW [16:18:07] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 07Epic: Send double opt-in email template after donation for donors in Italy and Austria - https://phabricator.wikimedia.org/T416198#11574509 (10Lars) [16:18:38] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 07Epic: Send double opt-in email template after donation for donors in Italy and Austria - https://phabricator.wikimedia.org/T416198#11574512 (10Lars) a:05Lars→03None [16:25:30] 14Fundraising Sprint Michelin Star, 03Fundraising Sprint: Ball pit, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, and 2 others: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#11574601 (10Lars) @MSuijkerbuijk_WMF I wonder if it makes sense to include the FY in the sour... [16:27:09] thanks Jeff_Green ! [16:30:56] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 07fr-current-sprint, 07payments-orchestration: Recurring token migration - Paypal - https://phabricator.wikimedia.org/T405483#11574656 (10AKanji-WMF) For reference @Damilare has opened Case Number:- 17252781 with Paypal cc: @EMartin - this task is curr... [16:58:21] (03PS1) 10Wfan: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235842 [16:58:43] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Epic: Export double opt-in status to Acoustic - https://phabricator.wikimedia.org/T407019#11574814 (10Ejegg) @Lars I think you did this already, right? Shall we move it into a sprint and call it done? related gerrit change: Ib6e99219f51fc72eff1b7f99c... [16:58:48] (03CR) 10Wfan: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235842 (owner: 10Wfan) [16:58:50] thanks for the CR jgleeson ! [17:01:46] (03PS1) 10Ejegg: Update composer packages [wikimedia/fundraising/SmashPig/vendor] - 10https://gerrit.wikimedia.org/r/1235843 [17:01:49] (03CR) 10Ejegg: [C:03+2] Update composer packages [wikimedia/fundraising/SmashPig/vendor] - 10https://gerrit.wikimedia.org/r/1235843 (owner: 10Ejegg) [17:01:50] !log payments-wiki upgraded from c5cadd72 to c035aa84 [17:01:51] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [17:02:11] 06Fundraising-Backlog: testing this again - https://phabricator.wikimedia.org/T416211 (10AKanji-WMF) 03NEW p:05Triage→03Medium [17:02:18] we should see less client side error, I am testing apple pay now~ [17:02:33] 06Fundraising-Backlog: testing this again - https://phabricator.wikimedia.org/T416211#11574860 (10AKanji-WMF) 05Open→03Declined [17:03:23] and apple pay works, this time not blocking~ [17:04:39] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Epic, 07fr-current-sprint: Export double opt-in status to Acoustic - https://phabricator.wikimedia.org/T407019#11574866 (10Lars) @Ejegg Yes, have moved, thanks. [17:05:08] PROBLEM - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [17:05:52] huh, how many procs SHOULD coworker have? ^^^ [17:06:20] that's me. it's down for queue server maintenance. [17:06:27] i'll ack it. [17:06:30] ah right, maintenance week! [17:06:59] ACKNOWLEDGEMENT - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker Dwisehaupt known. maint week. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [17:08:19] once this last job finishes, i'll put payments in maint mode and reboot frqueue1003 [17:16:40] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Donations failed with "expired card", but in Gr4vy the expiration date appears to be valid? - https://phabricator.wikimedia.org/T413665#11574922 (10Ejegg) @AMJohnson My search results suggest that some issuers will show 'expired car... [17:35:08] RECOVERY - check_coworker on civi1002 is OK: PROCS OK: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [17:36:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Donations failed with "expired card", but in Gr4vy the expiration date appears to be valid? - https://phabricator.wikimedia.org/T413665#11575025 (10AMJohnson) Interesting thanks @Ejegg! Luckily we have a workaround for now. If we no... [17:37:07] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 10fundraising-tech-ops, 13Patch-For-Review: FR-Tech FY2526Q3 maintenance window - https://phabricator.wikimedia.org/T364185#11575027 (10Dwisehaupt) [17:38:14] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1235847 [17:38:36] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1235847 (owner: 10Ejegg) [17:38:48] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1235847 (owner: 10Ejegg) [17:41:00] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for khill - https://phabricator.wikimedia.org/T415969#11575037 (10Dwisehaupt) certificate renewed and sent via email. password sent via sms. [17:43:08] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for jcuriel - https://phabricator.wikimedia.org/T415735#11575043 (10Dwisehaupt) certificate renewed and sent via email. password sent via sms. [17:43:19] (03PS1) 10Ejegg: Update text for opt-in success [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235848 (https://phabricator.wikimedia.org/T405047) [17:47:30] (03CR) 10Lars SG: [C:03+2] Update text for opt-in success [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235848 (https://phabricator.wikimedia.org/T405047) (owner: 10Ejegg) [17:48:06] thanks larssandergreen [18:01:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: test - https://phabricator.wikimedia.org/T416221 (10AKanji-WMF) 03NEW p:05Triage→03Low [18:01:33] (03Merged) 10jenkins-bot: Update text for opt-in success [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235848 (https://phabricator.wikimedia.org/T405047) (owner: 10Ejegg) [18:01:40] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: test - https://phabricator.wikimedia.org/T416221#11575172 (10AKanji-WMF) 05Open→03Invalid [18:26:40] ok, so what's the issue with smashpig vendor here [18:28:34] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 10fundraising-tech-ops, 13Patch-For-Review: FR-Tech FY2526Q3 maintenance window - https://phabricator.wikimedia.org/T364185#11575301 (10Dwisehaupt) [18:35:26] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, and 3 others: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#11575362 (10Lars) @MSuijkerbuijk_WMF We may want to think about backfilling this d... [18:37:34] (03PS2) 10Ejegg: Update composer packages [wikimedia/fundraising/SmashPig/vendor] - 10https://gerrit.wikimedia.org/r/1235843 [18:37:58] (03PS2) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1235847 [18:38:14] (03CR) 10Ejegg: [C:03+2] Update composer packages [wikimedia/fundraising/SmashPig/vendor] - 10https://gerrit.wikimedia.org/r/1235843 (owner: 10Ejegg) [18:38:18] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1235847 (owner: 10Ejegg) [18:40:20] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Donations failed with "expired card", but in Gr4vy the expiration date appears to be valid? - https://phabricator.wikimedia.org/T413665#11575400 (10Ejegg) Sounds good @AMJohnson, I'll move this ticket to 'Done' on our tracking board. [18:42:36] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: Lao form appears to be broken - https://phabricator.wikimedia.org/T414490#11575403 (10Ejegg) [18:42:41] 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for spatton - https://phabricator.wikimedia.org/T415116#11575406 (10Dwisehaupt) certificate renewed and sent via email. password sent via sms. [18:48:12] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations: Add link in Civi UI to re-send send 'Monthly Convert' thank you email - https://phabricator.wikimedia.org/T416228 (10Ejegg) 03NEW [18:50:17] (03PS6) 10Ejegg: Refactor sendSuccessThankYouMail paramters [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235110 (https://phabricator.wikimedia.org/T416228) [18:50:20] (03PS2) 10Ejegg: Resend send monthly convert Thank You via UI [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235399 (https://phabricator.wikimedia.org/T416228) [18:51:17] (03CR) 10Ejegg: [V:03+2 C:03+2] Update composer packages [wikimedia/fundraising/SmashPig/vendor] - 10https://gerrit.wikimedia.org/r/1235843 (owner: 10Ejegg) [18:52:00] (03PS1) 10Cstone: Try two with less link breaking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235864 (https://phabricator.wikimedia.org/T365488) [18:55:53] !log standalone (IPN listener) SmashPig upgraded from b42f2de4 to 937d6e40 [18:55:54] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [18:57:15] that looks right cstone, lemme just pull it down for a local smoke test [18:57:27] i refactored yours a bit to let any workflow through [18:57:44] er the change to the send/render [19:07:33] (03PS2) 10Cstone: Try two with less link breaking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235864 (https://phabricator.wikimedia.org/T365488) [19:07:36] (03CR) 10Ejegg: [C:03+2] Try two with less link breaking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235864 (https://phabricator.wikimedia.org/T365488) (owner: 10Cstone) [19:07:55] yep, looks like both of them work! [19:08:31] thanks ejegg [19:14:00] 06Fundraising-Backlog, 10FR-Donation-Processing: USD Benevity import Duplicate Error - https://phabricator.wikimedia.org/T416233 (10MDemosWMF) 03NEW [19:23:28] 03Fundraising Sprint: rich, the, 06Fundraising-Backlog, 07fr-current-sprint: Benevity - moving possible duplicate errors to group for dedupe - https://phabricator.wikimedia.org/T396535#11575618 (10Lars) @Eileenmcnaughton Do you know where the patch is for this phab? It seems like it may not be working ({... [19:25:21] (03Merged) 10jenkins-bot: Try two with less link breaking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235864 (https://phabricator.wikimedia.org/T365488) (owner: 10Cstone) [19:33:33] (03PS1) 10Wfan: Avoid to log the cross-origin error to logfile [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235868 (https://phabricator.wikimedia.org/T382623) [19:36:16] (03CR) 10CI reject: [V:04-1] Avoid to log the cross-origin error to logfile [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235868 (https://phabricator.wikimedia.org/T382623) (owner: 10Wfan) [19:36:36] (03PS2) 10Wfan: Avoid to log the cross-origin error to logfile [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235868 (https://phabricator.wikimedia.org/T382623) [19:38:36] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Transactions missing contribution_tracking_id - https://phabricator.wikimedia.org/T409237#11575720 (10Ejegg) The following query suggests that a bunch of these actually DO have contribution tracking IDs, and the rest are manually entered (nulls in all the... [20:00:04] (03PS1) 10Eileen: Add handling for user initiated conversion [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235871 [20:19:00] (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235875 [20:19:20] (03CR) 10Cstone: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235875 (owner: 10Cstone) [20:20:15] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1235875 (owner: 10Cstone) [20:22:22] !log civicrm upgraded from f91f955b to 5b9f3cd4 [20:22:23] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [20:32:54] (03PS1) 10Ejegg: WIP show inputs for missing Apple Pay name fields [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235876 (https://phabricator.wikimedia.org/T407033) [20:40:06] (03PS1) 10Wfan: Avoid refund lookup for ach trustly [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) [20:48:29] 14Fundraising Sprint: ARMchair, 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, and 2 others: Set up the dlocal -> Gravy settlement report - https://phabricator.wikimedia.org/T396146#11575971 (10Cstone) [20:53:24] (03PS1) 10Wfan: add log for recurring queue failed when donation not in civi [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235881 (https://phabricator.wikimedia.org/T415430) [21:02:28] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10MediaWiki-extensions-CentralNotice, 0725 Dec Q3 Planning: Special:HideBanners request from Thank You page shows warnings in console - https://phabricator.wikimedia.org/T411903#11576019 (10Ejegg) [21:04:35] 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-acoustic, 07fr-current-sprint: Export phone numbers to Acoustic - https://phabricator.wikimedia.org/T380946#11576038 (10Damilare) [21:05:07] 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 07fr-acoustic, 07fr-current-sprint, 13Patch-For-Review: Develop process for tracking & handling SMS consent records - https://phabricator.wikimedia.org/T379702#11576039 (10Damilare) [21:06:51] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 07fr-current-sprint, 07payments-orchestration: Recurring token migration - Paypal - https://phabricator.wikimedia.org/T405483#11576071 (10EMartin) @Damilare I have included our new account reps on the case to see if they can assist. [21:10:14] PROBLEM - check_ipsec on frban2002 is CRITICAL: Strongswan CRITICAL - ok: 0 not-conn: fran2001_v4 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2002&service=check_ipsec [21:13:07] ^^^ fixed up after reboot. should clear in a second. [21:14:35] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap: Mr Anonymous Anonymous - ADYEN Transaction Fees January 16 2026 adyen_1147_USD - https://phabricator.wikimedia.org/T415693#11576117 (10XenoRyet) [21:15:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Email: Message template preview gives invalidChecksum and doesn't fill in email - https://phabricator.wikimedia.org/T415329#11576129 (10XenoRyet) p:05Triage→03Low [21:15:16] RECOVERY - check_ipsec on frban2002 is OK: Strongswan OK - 1 ESP OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2002&service=check_ipsec [21:16:06] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 07Epic: Send double opt-in email template after donation for donors in Italy and Austria - https://phabricator.wikimedia.org/T416198#11576141 (10Lars) Just summarizing what we discussed tod... [21:18:09] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Add text to payments form on badly formatted links - https://phabricator.wikimedia.org/T414497#11576150 (10XenoRyet) p:05Triage→03Low [21:21:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Fail Mail (civi1002) run-job: Silverpop emails - Check main database file uploade completed (DatabaseUpdate) failed with code 1 - https://phabricator.wikimedia.org/T406485#11576181 (10Ejegg) We've added some documentation to the failmail zoo, and I change... [21:21:32] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Email: Message template preview gives invalidChecksum and doesn't fill in email - https://phabricator.wikimedia.org/T415329#11576183 (10AnnWF) {$unsubscribe_link} could works, fyi https://civicrm.wikimedia.org/civ... [21:24:39] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad, 06SRE: Q3:rack/setup/install frdb1008 - https://phabricator.wikimedia.org/T414374#11576218 (10Jclark-ctr) a:03VRiley-WMF [21:25:08] 10Fundraising analytics stack, 10fundraising-tech-ops: reconfigure codfw minio cluster after additional devices added - https://phabricator.wikimedia.org/T415609#11576222 (10Dwisehaupt) a:03Dwisehaupt one last data copy was done. config was pushed out. data wiped. cluster reconfigured with 4 hosts and 4 driv... [21:26:01] 03Fundraising Sprint: Ball pit, 06Fundraising-Backlog, 10fundraising-tech-ops, 13Patch-For-Review: FR-Tech FY2526Q3 maintenance window - https://phabricator.wikimedia.org/T364185#11576233 (10Dwisehaupt) [21:43:10] PROBLEM - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 17.84, 20.68, 15.79 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [21:43:37] ACKNOWLEDGEMENT - check_load on franio2001 is CRITICAL: LOAD CRITICAL - total load average: 17.84, 20.68, 15.79 Dwisehaupt known. data copy https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [21:44:24] man, that was quick on an alert. i had just removed the downtime. :) [22:00:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Email: Message template preview gives invalidChecksum and doesn't fill in email - https://phabricator.wikimedia.org/T415329#11576454 (10Lars) @AnnWF This is more of a general problem rather than specific to the un... [22:08:55] (03PS1) 10Lars SG: Fix delete repeated snooze activities [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235886 (https://phabricator.wikimedia.org/T415642) [22:11:42] (03PS2) 10Lars SG: Fix delete repeated snooze activities [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235886 (https://phabricator.wikimedia.org/T415642) [22:18:35] (03PS1) 10Eileen: Add refund handling for dlocal audit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235887 (https://phabricator.wikimedia.org/T414531) [22:19:59] I think ^^ will deal with the dlocal failmail.... [22:48:52] (03CR) 10Eileen: Avoid refund lookup for ach trustly (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) (owner: 10Wfan) [22:50:49] thanks eileen~ [22:52:44] (03PS2) 10Wfan: Avoid refund lookup for ach trustly [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) [22:52:48] updated~ [22:53:08] RECOVERY - check_load on franio2001 is OK: LOAD OK - total load average: 0.06, 1.29, 7.58 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=franio2001&service=check_load [23:04:21] (03CR) 10Wfan: Avoid refund lookup for ach trustly (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) (owner: 10Wfan) [23:10:34] (03PS1) 10Eileen: Actually I think this is the missing bit on braintree report [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235890 [23:13:05] (03CR) 10Eileen: [C:03+2] Avoid refund lookup for ach trustly [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) (owner: 10Wfan) [23:13:25] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: gravy MISSING_PREDECESSOR - https://phabricator.wikimedia.org/T415430#11576753 (10Eileenmcnaughton) I think the dlocal one is this https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/1235887?usp=dashboard [23:18:11] (03CR) 10Eileen: [C:03+1] "Go ahead & up to +2 when you are ready to deploy Lars - this is a virtual +2" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235886 (https://phabricator.wikimedia.org/T415642) (owner: 10Lars SG) [23:28:31] (03Merged) 10jenkins-bot: Avoid refund lookup for ach trustly [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235878 (https://phabricator.wikimedia.org/T415430) (owner: 10Wfan) [23:30:55] (03CR) 10Cstone: [C:03+2] Avoid to log the cross-origin error to logfile [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235868 (https://phabricator.wikimedia.org/T382623) (owner: 10Wfan) [23:31:09] (03CR) 10Wfan: [C:03+2] Add handling for user initiated conversion [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235871 (owner: 10Eileen) [23:31:15] (03CR) 10Cstone: [C:03+2] Add handling for user initiated conversion [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235871 (owner: 10Eileen) [23:31:39] (03Merged) 10jenkins-bot: Add handling for user initiated conversion [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235871 (owner: 10Eileen) [23:31:55] Thanks cstone [23:33:09] (03Merged) 10jenkins-bot: Avoid to log the cross-origin error to logfile [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1235868 (https://phabricator.wikimedia.org/T382623) (owner: 10Wfan) [23:33:14] (03CR) 10Cstone: [C:03+2] Actually I think this is the missing bit on braintree report [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235890 (owner: 10Eileen) [23:33:35] thanks from me too [23:33:45] (03Merged) 10jenkins-bot: Actually I think this is the missing bit on braintree report [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1235890 (owner: 10Eileen) [23:34:04] nps [23:38:03] (03CR) 10Eileen: [C:03+2] "I compared this line by line against the v3 version & they are the same. I thought that apiv4 might unpack that json in subsidiaries but I" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [23:43:24] (03PS1) 10Wfan: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235894 [23:44:40] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235894 (owner: 10Wfan) [23:45:10] (03CR) 10Wfan: "recheck" [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235894 (owner: 10Wfan) [23:45:42] (03CR) 10Wfan: "recheck" [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235894 (owner: 10Wfan) [23:45:49] (03CR) 10Eileen: [C:03+1] "I took a good look at this & it looks like all the pieces make sense. I haven't got away of running it but it looks low risk so I'm prepar" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204993 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [23:49:35] (03CR) 10Wfan: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1235894 (owner: 10Wfan) [23:50:36] !log payments-wiki upgraded from c035aa84 to 53bfb05b [23:50:36] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [23:50:40] (03CR) 10Eileen: "THis was surprisingly quick to run on staging but the syntax is not quite right" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1235828 (https://phabricator.wikimedia.org/T411901) (owner: 10Damilare Adedoyin) [23:54:06] (03Merged) 10jenkins-bot: Add API4 MatchingGiftPolicies.export [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204920 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg) [23:55:02] 10Fundraising analytics stack, 10fundraising-tech-ops: reconfigure codfw minio cluster after additional devices added - https://phabricator.wikimedia.org/T415609#11576835 (10Dwisehaupt) 05Open→03Resolved For reference, the data copy this last time took 78 minutes as compared to the previous runs that w... [23:57:05] (03Merged) 10jenkins-bot: More type hints for matching gifts classes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1204925 (https://phabricator.wikimedia.org/T410052) (owner: 10Ejegg)