[00:07:18] PROBLEM - check_payments_wiki on payments2004 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - string OK not found on https://payments2004.frack.codfw.wmnet:443https://payments.wikimedia.org/index.php/Special:SystemStatus - 639 bytes in 0.041 second response time https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_payments_wiki [00:12:17] sorry ^^^ is from me recloning payments1008. payments2004 is back up now [00:12:18] RECOVERY - check_payments_wiki on payments2004 is OK: HTTP OK: HTTP/1.1 200 OK - 529 bytes in 0.047 second response time https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2004&service=check_payments_wiki [04:50:40] 06Fundraising-Backlog: Direct mail returns not linked to Acoustic - https://phabricator.wikimedia.org/T378717#10291538 (10MSuijkerbuijk_WMF) Thanks @MDemosWMF From Faircom: We receive Donor ID on the file from your team. It is a unique identifier for each donor. The appeal will change with each campaign, as... [06:57:02] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/1086730 (owner: 10L10n-bot) [13:30:11] (03PS3) 10Damilare Adedoyin: Update smashpig to v0.8.22 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1087254 [13:38:29] (03CR) 10CI reject: [V:04-1] Update smashpig to v0.8.22 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1087254 (owner: 10Damilare Adedoyin) [13:43:00] (03PS4) 10Damilare Adedoyin: Update smashpig to v0.8.22 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1087254 [14:19:04] (03CR) 10Ejegg: [C:04-1] "I think we need to add the paymentMethod property to the Refund message class - currently it only exists on the Authorisation class and it" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [14:24:59] (03CR) 10Ejegg: "We should probably find and delete any matching message in the pending table, right?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1087248 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [14:30:51] (03PS4) 10Eileen: Also fetch 'non-campaign' mailings from Acoustic [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1084933 (https://phabricator.wikimedia.org/T361621) [14:30:54] (03CR) 10Ejegg: [C:03+2] Also fetch 'non-campaign' mailings from Acoustic [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1084933 (https://phabricator.wikimedia.org/T361621) (owner: 10Eileen) [14:32:16] (03PS4) 10Wfan: make the message clearer that we have ach and sepa will not have recurring token no need to warn [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1085602 [14:32:45] (03CR) 10Ejegg: [C:03+2] "Re-applying Dami's C+2 to trigger Zuul" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1085602 (owner: 10Wfan) [14:34:34] (03Merged) 10jenkins-bot: make the message clearer that we have ach and sepa will not have recurring token no need to warn [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1085602 (owner: 10Wfan) [14:53:36] (03Merged) 10jenkins-bot: Also fetch 'non-campaign' mailings from Acoustic [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1084933 (https://phabricator.wikimedia.org/T361621) (owner: 10Eileen) [15:01:47] (03PS1) 10Ejegg: Jazz up mailcheck text [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087491 (https://phabricator.wikimedia.org/T363473) [15:02:52] jgleeson|trickor: I couldn't find your old patch for mailcheck, so I just made that one ^^^ [15:13:18] 06Fundraising-Backlog: Update import code (inc upstream) to permit importing last 4 digits of CC number (to financial_trxn.pan_truncation) - https://phabricator.wikimedia.org/T373061#10292890 (10AKanji-WMF) November 4 - confirmed that this is still a requirement with @MDemosWMF [15:21:09] 06Fundraising-Backlog: Fundraiseup - Map new fundraiseup recurring status to Contribution recur status - https://phabricator.wikimedia.org/T379081 (10Damilare) 03NEW [15:24:17] (03CR) 10Ejegg: [C:03+2] "Oops, thought I had to remove + replace the C+2 after seeing the postmerge fail" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087228 (owner: 10Damilare Adedoyin) [15:25:59] (03CR) 10CI reject: [V:04-1] Jazz up mailcheck text [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087491 (https://phabricator.wikimedia.org/T363473) (owner: 10Ejegg) [15:27:16] (03CR) 10Ejegg: "vscode file checked in by mistake" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [15:27:54] thanks ejegg [15:33:10] (03PS1) 10Damilare Adedoyin: remove vscode debug files from git [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087496 [15:34:44] (03CR) 10Damilare Adedoyin: [C:03+2] Update smashpig to v0.8.22 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1087254 (owner: 10Damilare Adedoyin) [15:36:52] (03CR) 10Damilare Adedoyin: [C:03+2] remove vscode debug files from git [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087496 (owner: 10Damilare Adedoyin) [15:43:41] (03Merged) 10jenkins-bot: Update smashpig to v0.8.22 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1087254 (owner: 10Damilare Adedoyin) [15:55:29] (03Merged) 10jenkins-bot: remove vscode debug files from git [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087496 (owner: 10Damilare Adedoyin) [15:56:16] (03CR) 10Ejegg: "Maybe put the whole .vscode directory in your personal global gitignore so as not to have to put all the editor-specific ones in each repo" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087496 (owner: 10Damilare Adedoyin) [15:58:03] (03PS1) 10Damilare Adedoyin: remove entire vscode config files from git [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087506 [16:00:11] damilare: how about just putting that in your personal global gitignore rather than the .gitignore checked into the repo? [16:00:31] oh [16:00:52] I didn't know that was possible, that would come in really handy to [16:00:56] too* [16:08:26] (03PS2) 10Damilare Adedoyin: Undo update to git ignore [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087506 [16:15:32] yeah, /home/$user/.gitignore is great [16:32:18] (03PS3) 10Eileen: Decommission legacy function from legacy code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082293 [16:32:20] (03CR) 10Ejegg: [C:03+2] Decommission legacy function from legacy code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082293 (owner: 10Eileen) [16:51:47] (03PS2) 10Eileen: Migrate matching gifts notify email to a setting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082299 [16:52:02] (03CR) 10Ejegg: [C:03+2] Migrate matching gifts notify email to a setting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082299 (owner: 10Eileen) [16:55:30] (03Merged) 10jenkins-bot: Decommission legacy function from legacy code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082293 (owner: 10Eileen) [16:55:32] thanks greg-g! [16:55:48] (03CR) 10Damilare Adedoyin: [C:03+2] Undo update to git ignore [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087506 (owner: 10Damilare Adedoyin) [17:07:57] (03PS4) 10Wfan: Add payment method to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) [17:08:23] (03CR) 10CI reject: [V:04-1] Add payment method to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [17:10:27] (03PS5) 10Wfan: Add payment method to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) [17:11:41] 06Fundraising-Backlog, 10fundraising-tech-ops, 10SRE Observability (FY2024/2025-Q2): Shift frack alerting to use alertmanager instead of icinga - https://phabricator.wikimedia.org/T367370#10293590 (10lmata) [17:15:50] (03Merged) 10jenkins-bot: Undo update to git ignore [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087506 (owner: 10Damilare Adedoyin) [17:16:00] (03Merged) 10jenkins-bot: Migrate matching gifts notify email to a setting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1082299 (owner: 10Eileen) [17:27:56] (03PS6) 10Wfan: Add payment method to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) [17:48:02] (03CR) 10Ejegg: "Looking pretty good! Let's just declare the property to avoid dynamic property warnings" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [17:56:30] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293821 (10Jgreen) [17:59:06] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293830 (10Jgreen) [17:59:58] (03CR) 10Wfan: "Since we have ach and sepa all auto rescuable, this succeeded refund message not means the end of retryable, so we should keep the record " [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1087248 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [18:00:17] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Possible issue with the civi snooze - https://phabricator.wikimedia.org/T376959#10293834 (10bsisolak) For CID 64598559, I see the Snooze on October 23rd by the user on the unsubscribe page. RML Signup Oct 6 Sent email Oct 8 RML Sign... [18:08:20] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293884 (10Jgreen) [18:12:22] (03PS7) 10Wfan: Add payment method and order id to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) [18:13:08] (03CR) 10CI reject: [V:04-1] Add payment method and order id to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [18:17:44] (03PS1) 10Wfan: Add payment method and order id to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087543 (https://phabricator.wikimedia.org/T378860) [18:19:51] (03PS8) 10Wfan: Add payment method and order id to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087237 (https://phabricator.wikimedia.org/T378860) [18:20:10] (03Abandoned) 10Wfan: Add payment method and order id to refund queue message [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1087543 (https://phabricator.wikimedia.org/T378860) (owner: 10Wfan) [18:20:40] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293903 (10Jgreen) [18:22:13] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293913 (10Jgreen) [18:22:34] (03PS4) 10Wfan: For pending transactions like sepa and ach do not failmail [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1087248 (https://phabricator.wikimedia.org/T378860) [18:22:47] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293918 (10Jgreen) 05Open→03In progress p:05Triage→03Medium a:03Jgreen [18:23:43] (03PS5) 10Wfan: For pending transactions like sepa and ach do not failmail [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1087248 (https://phabricator.wikimedia.org/T378860) [18:24:39] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293924 (10Jgreen) [18:44:09] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293970 (10Jgreen) 05In progress→03Open [18:46:53] 10fundraising-tech-ops, 10BDC-Implementation: [Trino/Hive] Setup and configuration - https://phabricator.wikimedia.org/T377364#10293975 (10Jgreen) 05In progress→03Open [18:47:22] 10fundraising-tech-ops, 10BDC-Implementation, 07Epic: EPIC: Metabase Implementation - https://phabricator.wikimedia.org/T377363#10293979 (10Jgreen) 05In progress→03Open [18:47:39] 10fundraising-tech-ops, 06Infrastructure-Foundations, 10netops: Test prototype fundraising pybal replacement based on haproxy + anycast-healthchecker. - https://phabricator.wikimedia.org/T373942#10293980 (10Jgreen) 05In progress→03Open [18:49:45] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293983 (10Jgreen) [18:50:01] 10fundraising-tech-ops: Deactivate fundraising accounts for nnichols - https://phabricator.wikimedia.org/T377995#10293986 (10Jgreen) 05Open→03Resolved [19:16:09] (03PS1) 10Eileen: Documentation update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1087555 [19:17:09] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1087556 [19:17:47] that docs update should be an easy +2 (I would possibly +2 it myself as a non-code change) [19:18:21] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1087556 (owner: 10Eileen) [19:20:15] !log civicrm upgraded from 26d8013c to 65a8de90 [19:20:17] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:36:26] 03Fundraising Sprint: UnusedFunctionButDontDeleteItIncaseItsUsed(), 06Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Non Campaign Emails in Database - https://phabricator.wikimedia.org/T361621#10294130 (10Eileenmcnaughton) @Jmando I just deployed this - we should have the last 4 weeks... [19:37:39] 03Fundraising Sprint: UnusedFunctionButDontDeleteItIncaseItsUsed(), 06Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Non Campaign Emails in Database - https://phabricator.wikimedia.org/T361621#10294136 (10Eileenmcnaughton) Also ouch - is that link to a google doc your current process? [19:39:59] My floods of gerrit spam & phab spam & nagios spam I've started getting floods of Asana spam - is this because I have been added to some new recipient group (& I should probably filter it all out) or because there is current high activity in Asana (& I can wait it out) [19:40:31] 10fundraising-tech-ops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 2 others: Frack eqiad network upgrade: design, installation and configuration - https://phabricator.wikimedia.org/T377381#10294144 (10cmooney) >>! In T377381#10274577, @Jclark-ctr wrote: > @cmooney fyi i have 10x of the 100g gre... [19:42:02] on that note - do people have tactics re phab & gerrit spam - for github & gitlab I have all push notifications turned off because they have good ways of checking them in app - but I don't know I've found on phab or gerrit (or slack) good ways for that [19:45:56] 03Fundraising Sprint: UnusedFunctionButDontDeleteItIncaseItsUsed(), 06Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Non Campaign Emails in Database - https://phabricator.wikimedia.org/T361621#10294153 (10Eileenmcnaughton) ` sh -c "echo '{\"start_date\":\"8 weeks ago\", \"end_date\":\"... [19:55:16] (03PS8) 10Ejegg: API to request new link to RecurUpgrade or EmailPreferences [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1015406 (https://phabricator.wikimedia.org/T352337) [19:55:16] (03PS1) 10Ejegg: WIP new checksum link JS and template [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087562 (https://phabricator.wikimedia.org/T352337) [19:57:01] (03CR) 10CI reject: [V:04-1] WIP new checksum link JS and template [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087562 (https://phabricator.wikimedia.org/T352337) (owner: 10Ejegg) [20:00:57] 06Fundraising-Backlog: Superset set up access for Bree Ryan - https://phabricator.wikimedia.org/T379112 (10RMurthy) 03NEW [20:11:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Civi and acoustic odd merging behavior - https://phabricator.wikimedia.org/T377904#10294219 (10AKanji-WMF) p:05Triage→03Low [20:19:43] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civicrm server /srv over 80% full, it's time to purge unnecessary data. - https://phabricator.wikimedia.org/T377946#10294240 (10AKanji-WMF) - in triage discussed compressing audit files - @EMartin - question: how long should we be keeping Ingenico audit f... [20:21:40] 06Fundraising-Backlog, 10FR-donorservices, 07payments-orchestration: Gr4vy console setting: enable viewing of name in Buyers tab - https://phabricator.wikimedia.org/T378007#10294255 (10AKanji-WMF) Want to chat about this @MBeat33 - the team suggests we aren't using Buyers/don't plan to use this tab. [20:22:18] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Civicrm server /srv over 80% full, it's time to purge unnecessary data. - https://phabricator.wikimedia.org/T377946#10294258 (10EMartin) @AKanji-WMF three years. [20:22:19] 06Fundraising-Backlog, 07Unplanned-Sprint-Work: By Nov 5: Changes to export templates: Donation and Recurring Status fields - https://phabricator.wikimedia.org/T378361#10294257 (10AKanji-WMF) [20:34:06] 06Fundraising-Backlog, 10fundraising-tech-ops: Superset set up access for Bree Ryan - https://phabricator.wikimedia.org/T379112#10294296 (10AKanji-WMF) [20:39:29] 06Fundraising-Backlog, 07payments-orchestration: Setup Gravy Adyen AutoRescue - https://phabricator.wikimedia.org/T378630#10294302 (10AKanji-WMF) The team outlined several reasons as to why we should look to implement this in Q3 vs Q2. - risk around interference with current processor flows in BE - easier to i... [20:47:17] 06Fundraising-Backlog: Add a 'recurring cancellation reason' field in Acoustic - https://phabricator.wikimedia.org/T378974#10294316 (10AKanji-WMF) [21:29:30] 06Fundraising-Backlog: Coding changes to offline gifts - https://phabricator.wikimedia.org/T372467#10294466 (10AKanji-WMF) discussed with @MDemosWMF - decided to revisit in FY2425 Q3 [21:43:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Slow thank you email - https://phabricator.wikimedia.org/T377906#10294519 (10greg) a:03Eileenmcnaughton (We missed this in sprint/chaos review yesterday) https://civicrm.org/blog/eileen/developer-content-down-rabbit-h... [21:47:48] 06Fundraising-Backlog: Investigate how deceased donors are handled between Civi - Acoustic - https://phabricator.wikimedia.org/T379114#10294540 (10Peachey88) [21:53:43] anilk: I think that meeting was helpful - yay for Melanie being back [21:54:18] totally, thanks for all the thinking [21:54:42] I will think about the slack thread - interesting that we were just talking to Rosie about preferences....off to pickup now, be back later [21:58:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Slow thank you email - https://phabricator.wikimedia.org/T377906#10294575 (10Eileenmcnaughton) Just noting this is the upstream patch - I'm just gonna try to tweak up the description & then push for a merge https://githu...