[07:13:36] (03CR) 10CI reject: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_35) - 10https://gerrit.wikimedia.org/r/969567 (owner: 10L10n-bot) [07:48:24] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_35) - 10https://gerrit.wikimedia.org/r/969567 (owner: 10L10n-bot) [11:13:00] (03CR) 10Damilare Adedoyin: [C: 03+2] Add api to remove recuring payment token then delete customer from venmo vault for one-time donor (032 comments) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606) (owner: 10Wfan) [11:14:01] (03CR) 10Damilare Adedoyin: [C: 03+2] "Recheck" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606) (owner: 10Wfan) [11:14:32] (03PS10) 10Damilare Adedoyin: Add api to remove recuring payment token then delete customer from venmo vault for one-time donor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606) (owner: 10Wfan) [11:26:14] 10Fundraising Sprint Uptown cURL: Enable all payment methods for LATAM pre-test in Q2 - https://phabricator.wikimedia.org/T349857 (10RKumar_WMF) All banner links given do not show recurring option, can you please provide? Also, please confirm that recurring is only available for credit cards for all LATAM countr... [11:44:03] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: "Typed card number does not match card brand" errors - https://phabricator.wikimedia.org/T349600 (10RKumar_WMF) From Adyen: Adyen Support 5:12 PM (0 minutes ago) to me, emartin@wikimedia.org, mbeattie@wikimedia.org, jake.leininger@... [12:59:46] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Adyen checkout/v68/paymentMethods returning 404 for iOS app payment sessions - https://phabricator.wikimedia.org/T350023 (10jgleeson) [14:01:10] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: "Typed card number does not match card brand" errors - https://phabricator.wikimedia.org/T349600 (10RKumar_WMF) Further from Adyen: Adyen Support 7:22 PM (7 minutes ago) to me, mbeattie@wikimedia.org, jake.leininger@adyen.com, emar... [14:19:54] (03CR) 10Ejegg: "recheck" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/963853 (https://phabricator.wikimedia.org/T347606) (owner: 10Wfan) [14:23:45] weird, zuul isn't listening to the recheck request [14:29:34] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [14:30:07] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [14:30:11] I feel like that's happened before ejegg [14:31:11] yeah, I'm sure there is are a lot of moving pieces in that stack [14:31:19] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Jgreen) [14:31:20] I vaguely remember either some depends-on weirdness or one of the other magic CI commands fixing it [14:31:51] is it recheck/resubmit ? [14:32:55] i think so [14:33:00] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Jgreen) [14:33:10] brb [14:42:28] back [14:49:52] back soon! [15:05:01] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [15:26:15] 10Fundraising-Backlog, 10FR-donorservices, 10Recurring-Donations: Get CiviProxy -> CiviCRM auth working in production for API4 calls - https://phabricator.wikimedia.org/T349946 (10AKanji-WMF) @Ejegg should I pull this into sprint based on desired recurring upgrade work? Or wait until post maint-window? [15:30:11] 10Fundraising-Backlog, 10FR-donorservices, 10Recurring-Donations: Get CiviProxy -> CiviCRM auth working in production for API4 calls - https://phabricator.wikimedia.org/T349946 (10Ejegg) sure @AKanji-WMF , this would be great to have in sprint. [15:42:30] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [15:44:26] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [15:50:20] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [15:51:02] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [16:32:51] PROBLEM - Host frdb1006 is DOWN: PING CRITICAL - Packet loss = 100% [16:36:15] RECOVERY - Host frdb1006 is UP: PING OK - Packet loss = 0%, RTA = 0.40 ms [16:39:23] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [16:56:59] PROBLEM - Host frdb1005 is DOWN: PING CRITICAL - Packet loss = 100% [17:00:25] RECOVERY - Host frdb1005 is UP: PING OK - Packet loss = 0%, RTA = 0.40 ms [17:00:52] 10Fundraising Tech - Chaos Crew: Fundraiseup new recurring missing utm_* fields in Contribution Tracking table - https://phabricator.wikimedia.org/T350063 (10Damilare) [17:02:31] (03PS4) 10Damilare Adedoyin: Upgrade recur amount when donor upgrades their recurring in Fundraiseup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/968725 (https://phabricator.wikimedia.org/T348544) [17:15:43] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Fundraiseup new recurring missing utm_* fields in Contribution Tracking table - https://phabricator.wikimedia.org/T350063 (10greg) [17:18:21] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [17:18:58] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [17:24:05] PROBLEM - Host pay-lvs1004 is DOWN: PING CRITICAL - Packet loss = 100% [17:24:37] ^ known for reboot [17:26:15] RECOVERY - Host pay-lvs1004 is UP: PING OK - Packet loss = 0%, RTA = 1.02 ms [17:40:51] (03CR) 10Jgleeson: [C: 03+2] "LGTM" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966579 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [17:40:53] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [17:45:56] 10fundraising-tech-ops: Modify fundstatgen.py to use mysql connection encryption. - https://phabricator.wikimedia.org/T350067 (10Jgreen) [17:46:28] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [17:51:46] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations: Reduce db writes when saving a contact with a fiscal_number - https://phabricator.wikimedia.org/T350068 (10Ejegg) [17:52:40] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [17:56:45] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [18:01:00] (03Merged) 10jenkins-bot: Copy invoice_id to processor_contact_id [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966579 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [18:17:11] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [18:22:17] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Dwisehaupt) [18:24:01] (03PS4) 10Ejegg: Store and retrieve ShopperReference from new field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) [18:26:33] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Dwisehaupt) First mailing column sql query successful. ` MariaDB [civicrm]> ALTER TABLE civicrm_mailing_event_queue MODIFY job_id int unsigned nu... [18:28:02] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Dwisehaupt) Second mailing column sql query successful. ` MariaDB [civicrm]> ALTER TABLE `civicrm_mailing_event_queue` -> ADD CONSTRAINT `FK_... [18:31:32] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Dwisehaupt) Third mailing column sql query successful. ` MariaDB [civicrm]> UPDATE civicrm_mailing_event_queue q -> INNER JOIN civicrm_mail... [18:31:58] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Dwisehaupt) [18:36:51] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [18:39:45] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [18:42:19] (03CR) 10CI reject: [V: 04-1] Store and retrieve ShopperReference from new field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [18:42:57] (03CR) 10Jgleeson: [C: 03+2] "Tested locally and working as expected. thanks!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [18:53:19] (03PS5) 10Ejegg: Store and retrieve ShopperReference from new field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) [18:53:20] (03PS1) 10Ejegg: Disable all eval(cv(...)) lines to debug tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969950 [18:53:38] oops, ^^^ is not to be merged [18:53:51] ok, set as WIP [18:55:58] anyhow jgleeson PS5 above should have fixed the one test [18:57:01] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [18:59:33] 10Fundraising Sprint Uptown cURL, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices, 10Patch-For-Review: Deduper and legacy merging overwriting issue - https://phabricator.wikimedia.org/T345206 (10AnnWF) [19:12:03] (03PS14) 10Wfan: Add declineMonthlyConvert as param for post Monthly Convert donation api [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/963970 (https://phabricator.wikimedia.org/T347606) [19:13:40] ok jgleeson, PS5 is passing tests: https://gerrit.wikimedia.org/r/966590 [19:14:04] (03CR) 10Jgleeson: [C: 03+2] "LGTM take two" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [19:15:49] thanks! [19:24:33] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/969957 [19:24:35] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/969957 (owner: 10Ejegg) [19:32:19] (03Merged) 10jenkins-bot: Store and retrieve ShopperReference from new field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966590 (https://phabricator.wikimedia.org/T349107) (owner: 10Ejegg) [19:32:21] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/969957 (owner: 10Ejegg) [19:35:48] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Ejegg) [19:36:58] 10fundraising-tech-ops: encrypt fundraising database client->server communication - https://phabricator.wikimedia.org/T170321 (10Jgreen) [19:42:06] interesting use of civi - https://civicrm.stackexchange.com/questions/45795/csv-download-from-a-custom-pop-up [19:45:53] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10FR-donorservices, 10Recurring-Donations: Get CiviProxy -> CiviCRM auth working in production for API4 calls - https://phabricator.wikimedia.org/T349946 (10AKanji-WMF) [19:45:57] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10Patch-For-Review: Suppress Paypal Grant CID to avoid duplicate db recording - https://phabricator.wikimedia.org/T349392 (10Ejegg) a:03Ejegg [19:48:30] 10Fundraising-Backlog, 10FR-Smashpig, 10Recurring-Donations, 10Patch-For-Review: Use auto-capture when authorizing recurring charges - https://phabricator.wikimedia.org/T348858 (10XenoRyet) [19:50:33] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Fundraiseup new recurring missing utm_* fields in Contribution Tracking table - https://phabricator.wikimedia.org/T350063 (10Damilare) [19:52:11] 10Fundraising-Backlog: Frontend Import of Fundraiseup Chargeback - https://phabricator.wikimedia.org/T347881 (10XenoRyet) [19:55:11] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog: Sprint Uptown cURL - Priorities: - https://phabricator.wikimedia.org/T349655 (10AKanji-WMF) [20:01:08] dwisehaupt: where is the list of strings for activity type cleanup? [20:03:24] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: "Typed card number does not match card brand" errors - https://phabricator.wikimedia.org/T349600 (10greg) p:05Triage→03High [20:03:35] looking to see. i thought it was in a script but i didn't find it on first grep. [20:06:18] 10Fundraising Tech - Chaos Crew: Adyen checkout/v68/paymentMethods returning 404 for iOS app payment sessions - https://phabricator.wikimedia.org/T349945 (10greg) p:05Triage→03Medium [20:06:39] 10Fundraising Tech - Chaos Crew: Adyen checkout/v68/paymentMethods returning 404 for iOS app payment sessions - https://phabricator.wikimedia.org/T349945 (10greg) [20:06:41] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Adyen checkout/v68/paymentMethods returning 404 for iOS app payment sessions - https://phabricator.wikimedia.org/T350023 (10greg) [20:07:55] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops: Do a prune on detail data for activity records of type thank you - https://phabricator.wikimedia.org/T325123 (10Eileenmcnaughton) Some strings | string |e.g actvity ID| |

Köszönjük, hogy|15889668| [20:08:05] i think this one, but i need to find the script that i know i used in the past: https://collab.wikimedia.org/wiki/Fundraising/Engineering#Clear_full-text_records_of_old_Thank_You_emails [20:08:44] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Fundraiseup new recurring missing utm_* fields in Contribution Tracking table - https://phabricator.wikimedia.org/T350063 (10AKanji-WMF) p:05Triage→03High [20:09:30] but that isn't using strings, so it may be wrong. [20:10:04] 10Fundraising Tech - Chaos Crew: Adyen checkout/v68/paymentMethods returning 404 for iOS app payment sessions - https://phabricator.wikimedia.org/T349945 (10greg) p:05Medium→03High [20:10:20] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Fundraiseup new recurring missing utm_* fields in Contribution Tracking table - https://phabricator.wikimedia.org/T350063 (10Cstone) a:05Damilare→03Cstone [20:12:49] eileen: oh. i think i ran the p-c job: process-control/civi1002/wmf_data_management_archive_thankyou.yaml [20:13:20] yeah, ArchiveThankYou.php has the list of strings it was searching for. [20:19:11] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-DonationInterface: Run Phan on CentralNotice and DonationInterface - https://phabricator.wikimedia.org/T182714 (10AKanji-WMF) 05Open→03Declined Closing in favour of individual separate tasks [20:22:33] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Docker: Fix "max execution timeout" failures locally when applying CiviCRM Database update - https://phabricator.wikimedia.org/T349201 (10greg) p:05Triage→03Medium [20:22:57] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10fundraising-tech-ops: Tools for Analyzing MySQL Slow Query Log - https://phabricator.wikimedia.org/T349565 (10greg) p:05Triage→03Medium [20:25:03] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Coworker create (& upstream) Searchkit over view of queue status - https://phabricator.wikimedia.org/T349752 (10Dwisehaupt) [20:27:17] 10Fundraising-Backlog: Matching Check imports - not creating employer/employee relationship - https://phabricator.wikimedia.org/T349858 (10AKanji-WMF) [20:27:19] 10Fundraising Sprint Uptown cURL, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Matching EFT imports - not creating employer/employee relationship - https://phabricator.wikimedia.org/T349550 (10AKanji-WMF) [20:28:28] 10Fundraising-Backlog: Creating an new import for Overflow.co for Stock and DAF gifts - https://phabricator.wikimedia.org/T349930 (10AKanji-WMF) Thanks @EMartin and @MDemosWMF - in talking to the team it sounds like the next step should be a convo between Melanie and Eileen - I'll look to set that up. [20:29:05] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Coworker - send fail mail when a queue aborts - https://phabricator.wikimedia.org/T349751 (10Dwisehaupt) [20:36:49] 10Fundraising-Backlog, 10FR-donorservices: Venmo Security Check Loading Window - https://phabricator.wikimedia.org/T347503 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @AMJohnson checked again on October 30, issue seems to have been resolved. [20:37:01] 10Fundraising-Backlog: Update contact on CiviCRM when FRUP donor updates their email through donor portal - https://phabricator.wikimedia.org/T349663 (10Dwisehaupt) [20:38:36] 10Fundraising-Backlog: Update contact on CiviCRM when FRUP donor updates their email through donor portal - https://phabricator.wikimedia.org/T349663 (10Dwisehaupt) [20:39:10] 10Fundraising-Backlog: Update contact on CiviCRM when FRUP donor updates their email through donor portal - https://phabricator.wikimedia.org/T349663 (10Damilare) [20:39:36] thanks Dallas, I added an update to the task as well. [20:40:06] !log disable fundraising scheduled jobs for deployment [20:40:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:40:58] ok, just the data export running right now [20:41:03] I think I can push the new code [20:41:15] ha. :) [20:41:48] !log fundraising civicrm upgraded from 2c79475e to 71d26d3b [20:41:51] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:42:03] ok, going to do a system.flush [20:43:03] ok, new column is there [20:43:07] going to run extension.upgrade [20:44:34] 10Fundraising-Backlog: Investigate Recurring on Credit/Debit for Brazil - https://phabricator.wikimedia.org/T349656 (10HNordeenWMF) Yes! confirming we will change the text on the Banner/donate.wiki button to "Credit card" for brazil. Recurring will be enabled for that button. [20:45:07] ok, numbers match for the new column vs the old column where payment processor is Adyen [20:45:20] let's see if we can find an Adyen donation that's soon to be due for a charge [20:51:11] 10Fundraising-Backlog: Investigate Recurring on Credit/Debit for Brazil - https://phabricator.wikimedia.org/T349656 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF Perfect, thank you - resolving. [20:58:26] !log re-enabled fundraising scheduled jobs after deployment [20:58:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:58:39] k, gonna check that off and move the task [20:59:28] (03CR) 10Wfan: "no table created for me and" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969388 (https://phabricator.wikimedia.org/T349158) (owner: 10Ejegg) [20:59:32] 10Fundraising Sprint Uptown cURL, 10Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2324Q2 maintenance window - https://phabricator.wikimedia.org/T337579 (10Ejegg) [21:02:09] (03CR) 10Ejegg: "Thanks for the review wfan. Instructions to get the table created included in reply inline." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969388 (https://phabricator.wikimedia.org/T349158) (owner: 10Ejegg) [21:02:23] ok folks, I need to go pick up my kid [21:02:36] i will try to get back online for a bit later [21:09:54] (03CR) 10Cstone: [C: 03+1] "It ran okay for me but seeing if wfan can get it to behave too" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969388 (https://phabricator.wikimedia.org/T349158) (owner: 10Ejegg) [21:10:29] we should aim to deploy this with the field rename - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/966942 [21:18:49] (03CR) 10Jgleeson: [C: 03+1] "Code looks good but not got my head in the DataAxle space so will let someone else +2" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/966942 (https://phabricator.wikimedia.org/T336890) (owner: 10Eileen) [21:32:18] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: Snoozed donor received fundraising email - https://phabricator.wikimedia.org/T349277 (10SHust) Hi team, it appears that the Civi 'snooze' feature isn’t working! I tried to pause Adri's email, CID 56939616, for 60 days, and when I che... [21:41:30] (03PS1) 10Eileen: Add some more strings for deleting activities [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969972 (https://phabricator.wikimedia.org/T325123) [21:45:02] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: Snoozed donor received fundraising email - https://phabricator.wikimedia.org/T349277 (10SHust) Additionally, I tested my CID 61216138 on the old 90-day '[[ https://www.pages04.net/wikimedia/WMFUnsubscribe/Unsubscribe?vs=NzlkMTkzZWMtY... [21:58:16] hi eileen, looks like there's another queue item clogging the snooze queue [21:58:31] damilare: I'll look [21:58:49] I can't remember how you found the last error in the logs [21:58:57] thanks [22:01:27] damilare: so the contact has the scheduled activity here https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=37646335&selectedChild=activity [22:02:38] yep, but looks like there's something causing it to retry [22:04:24] the relevant line in syslog starts with Oct 30 22:02:43 civi1002 coworker[30329] - just trying to see if I can see the full line [22:14:33] ok looks like that contact has left [22:14:37] damilare: so they are all failing on the snooze - perhaps Acoustic isn't responding? but just wondering when we deployed the last change [22:15:26] I hadn't deployed since you merged the PR as I was out on friday eileen [22:15:44] but it has been deployed now? [22:16:01] checking [22:16:07] i deployed civi just now and it was just the two recently merged things [22:16:09] it looks like cstone might have deployed it on friday 74781efdc2bf87ce971ec26764 [22:16:14] so I think damilare's is out [22:17:06] thanks ejegg|afk [22:17:16] the most recent is 71d26d3b eileen [22:19:37] (03PS1) 10Eileen: Log omnimail update fails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969973 [22:19:55] maybe if we push ^^ out we can get more info ? (& refine that as we go) [22:19:57] eileen, looks like in addition to the queue weirdness, the searchkit may not be scheduling the contacts according to Sandra here: https://wikimedia.slack.com/archives/C045WH0QYS2/p1698704162377309?thread_ts=1698702563.210169&cid=C045WH0QYS2 [22:24:01] damilare: hmm - we had better test that code is getting triggered locally huh [22:26:21] yepyep [22:34:04] damilare: should we try that extra logging above as a first step or roll back the Friday update as a first step? [22:34:58] I'm trying to test the logging locally, it's taking a while to load [22:35:11] in a minute [22:37:44] weird, email snoozed activity is showing up locally for me [22:37:55] hmm - test on production :-) [22:40:26] (03PS1) 10Jgleeson: WIP: Send recurring payment instalments to the donations queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969975 (https://phabricator.wikimedia.org/T240581) [22:40:28] (03PS1) 10Jgleeson: WIP: Send recurring payment instalments to the donations queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969976 (https://phabricator.wikimedia.org/T240581) [22:43:38] (03CR) 10Damilare Adedoyin: [C: 03+2] Log omnimail update fails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969973 (owner: 10Eileen) [22:46:58] I'm logging off now, I'd check back on this in the morning... bye for now [22:52:21] ok cool - I'll see what the logging show [22:52:22] s [22:57:17] 10Fundraising-Backlog: Please read: Async Triage Oct 25/2023 - https://phabricator.wikimedia.org/T349783 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF [22:59:55] (03Merged) 10jenkins-bot: Log omnimail update fails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/969973 (owner: 10Eileen) [23:23:28] huh, ok, I am getting the same error locally on the recurring upgrade form [23:23:34] as I get in production [23:24:18] so... how the heck was I running just the other day when I took a screenshot for that google doc? [23:24:32] well, at least I have something to debug through now [23:40:42] wat, civiproxy just not responding at all now? [23:41:48] ohh, we should update the civiproxy image to new php [23:42:42] whee, more yak shaving