[00:01:43] (03CR) 10Ejegg: [C:03+2] Clean up thank you exception handling [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137852 (https://phabricator.wikimedia.org/T392387) (owner: 10Eileen) [00:02:03] (03CR) 10Ejegg: [C:03+2] Include previous exception as part of construct [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138117 (owner: 10Eileen) [00:27:50] (03Merged) 10jenkins-bot: Clean up thank you exception handling [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137852 (https://phabricator.wikimedia.org/T392387) (owner: 10Eileen) [00:27:52] (03Merged) 10jenkins-bot: Include previous exception as part of construct [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138117 (owner: 10Eileen) [00:42:30] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138502 [00:42:46] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138502 (owner: 10Eileen) [00:43:34] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138502 (owner: 10Eileen) [00:45:55] !log config revision changed from c635ed3c to bfbce54f [00:45:55] - disabling process control while I deploy the thank you patches [00:45:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:46:53] !log civicrm upgraded from b3038510 to c8946ea5 [00:46:55] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:10:50] !log config revision changed from bfbce54f to 47a5d384 [01:10:52] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:19:02] ejegg: cstone wfan I just deployed the Thank you fix & tested it & it is mostly good [01:19:02] working - the failure reason is being set [01:19:03] not working - the parameters from drush are not getting through - specifically messageLimit and days - the [01:19:03] defaults for these are 1450 & 60 respectively. In the short term that's OK - I have already retried the last 60 [01:19:03] days & 37 contacts got updated to bad mail - [01:19:03] https://civicrm.wikimedia.org/civicrm/admin/search#/edit/4993 [01:19:04] I've tried to engage DR to check if they want to follow up at all since almost all of them have obvious typos [01:19:06] next steps [01:19:07] - I can get messageLimit back now by switching the jobs to use cv [01:19:07] - numberOfDays won't work until https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1138131/3/drupal/sites/default/civicrm/extensions/wmf-thankyou/Civi/Api4/Action/ThankYou/BatchSend.php is deployed - but it's probably not a super rush as I don't think the 60 days default is hurting us much [01:19:08] - activity clean up? (I've done a couple of contacts manually through sk) [01:21:13] how many of them are there for clean up? search kit should be helpful for all of them right? [01:21:22] \Civi::settings()->get('thank_you_days'); haha [01:21:38] we can get this update if we need, but your update patch looks nice! [01:22:03] do we want any unit test for that? [01:22:57] wfan: unfortunately that patch of mine is in the middle of a chain I think [01:23:20] cstone: I have a list of 37 IDs - they are all within 60 days but I guess it goes back further [01:24:02] - there are some pretty easy ones in that chain though - ie the first one https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1138119/4 [01:24:34] but I think we don't need to rush - ie if we didn't fix 'days' today the world would not end [02:01:13] ok, back from what turned out to be hours of kid time. Glad to hear the main issue is fixed. I'll keep reviewing along that chain [02:03:13] (03CR) 10Ejegg: [C:03+2] Remove unused variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138119 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [02:05:43] (03PS9) 10Wfan: Replace use api3 to api4 for epc [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1135845 (https://phabricator.wikimedia.org/T386871) [02:05:56] (03CR) 10CI reject: [V:04-1] Replace use api3 to api4 for epc [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1135845 (https://phabricator.wikimedia.org/T386871) (owner: 10Wfan) [02:13:44] eileen I'm reviewing https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1138118 [02:14:23] cool cool [02:15:17] and trying to figure out where in Send or Render it would get the mailing data if we don't set it [02:16:06] i see it pulling some basic contact data [02:16:21] and financial_type_id.name [02:16:25] in getContact [02:16:28] The ThankYou workflow template would load it - but in this case we just did a query & then we are checking that we did that query.... [02:16:32] but don't we need a lot? [02:16:34] ohh [02:17:14] got it, that's how the individual send button is working then [02:17:20] yep [02:17:22] & preview [02:17:42] right, I reviewed all this a while ago, it's just faded [02:18:48] :-) [02:20:55] so somehow we've still got some things from this month marked as no_thank_you=' failed: MISSING_MANDATORY_DATA ' [02:21:01] let's see [02:21:10] hmm ok [02:21:12] contribution id 122519642 [02:21:19] on cid 63220371 [02:21:32] ah, looks like Engage DAF is the gateway [02:21:35] so what's it missing? [02:23:18] - also a thankyou WAS send? [02:23:55] - could this be a case of the thank you starting before the contribution is fully saved? [02:24:05] hmm, not sure [02:24:26] the previous 4 with that reason are also DAF [02:25:22] though paypal not engage [02:27:52] the row was written at 12:40:27, then updated with the no_thank_you at 12:41:33 so a whole minute had passed [02:27:56] so perhaps those imports sometimes run at low traffic times such that the import is sooo up to date they are processing [02:28:15] Missing stuff from the TY params: array["amount","currency"] [02:28:15] (03Merged) 10jenkins-bot: Remove unused variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138119 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [02:28:17] when was contribution_extra written? [02:28:41] that's the row I'm looking at [02:28:50] both of those timestamps on 2025-04-04 [02:29:09] is there a gap between contribution & contribution_extra being written? [02:29:12] ah, is it expecting the original_* bits to be populated? [02:30:12] no lag, contribution was written at the same second as contribution_extra [02:30:38] Yah - I think I'm looking at the wrong one - 1 April [02:30:54] must be because original_currency and original_amount are null [02:31:30] the TY mail was sent on the 8th, 4 days later - probably manually? [02:33:07] so we either make sure original_currency & amount are set or do COALESCE(original_currency, currency) [02:35:24] - I just checked the email they would have received & it is missing currency & amount info [02:37:37] So [02:37:37] a) having that code there did not result in us preventing the contact from getting an invalid thank you [02:37:37] b) we should possibly have some sort of data job to check for rows with these values missing (like our verifyDeletedContacts) [02:37:37] c) I could make the ThankYou workflow template default back to 'currency' & 'total_amount' if the wmf_contribution_extra fields are missing (I'd do this with a test on the top of the chain) [02:39:25] i like the coalesce() solution for now [02:41:16] ejegg: yeah - we could put that in - but it would only help through BatchSend - I'd need to fix the ThankYou template to fix for all flows [02:41:32] ah yeah, I guess the template then [02:42:07] if you try to send an email for that donation you can see what they received & it's missing info [02:42:13] right, and data hygiene [02:43:00] hmm, so I think we DO want to go to removing the getMailingData from batchSend [02:43:17] so as to have a single way to get things [02:43:24] yeah - the chain doesn't quite go that far as it stands [02:43:41] I forget how much we tested the speed of the individual queries vs the single joined one [02:43:53] I think that was my last worry about it [02:44:30] yeah - in general simple queries are not where we lose speed - where we had issues before was with caches & serialization [02:44:49] ah right [02:44:50] ie cache misses leading to arrays being serialized & unserialized a lot [02:46:44] (03CR) 10Ejegg: [C:03+2] "Agreed this is out of place here. Maybe we could still do some checking in the Render function or the workflow template? And as discussed " [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138118 (owner: 10Eileen) [02:47:07] once the chain is merged I'll look at it at the top of the chain [02:48:01] (03CR) 10Ejegg: [C:03+2] "Yep, also no longer needed now that this logic is in a private fn" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138124 (owner: 10Eileen) [02:49:18] hmm, IS EMPTY in API4 means null, '', or 0 ? [02:50:03] I think I rendered the query in SK & got NULL or '' [02:50:06] ejegg: to be honest I suspect we should only retrieve out contributions to send one by one in a while loop - small queries are quick & we could get of our freshness checks - eg. [02:50:07] `SELECT id FROM civicrm_contribution WHERE thank_you_date IS NULL AND no_thank_you empty and receive_date > x LIMIT 1` [02:50:07] Is probably super fast [02:50:11] oh i need to rebuild again, i've still got my local db on the version from 2 yrs ago [02:50:24] huh? [02:50:27] 2 yrs? [02:50:43] looking for a version with the 'draft' functionality working! [02:51:38] ejegg: it could be longer ago - because we never really used it [02:51:46] or maybe it never did properly work [02:51:57] yeah, i want to dig into that translations code [02:52:26] there were some changes made last year I think - no year before - at Tim's house - or maybe the year before [03:03:27] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog: Rearrange adyen audit columns in the settlement report to match the default - https://phabricator.wikimedia.org/T392528#10763480 (10Cstone) Okay it looks like the column we aren't using is in the payment details report and not the settlement report. Our co... [03:03:28] gaslighting myself with these report columns [03:04:51] OH [03:04:55] is it the parent account [03:05:12] ok, I think I can get most of the IDs affected by the multi-ty-activity bug out of the logs for the TY sender [03:07:40] ejegg: oh - I've deleted a lot of them via sk [03:08:38] if we can get 2 more patches up that chain I can switch the drush job over & fix the numberOfDays thing... [03:09:00] right, I'll get back to review [03:09:53] ejegg: do you remember why we have two adyen production accounts, was it just how it was set up? [03:10:00] or like ones the parent of the other [03:10:19] one is the parent account [03:10:46] we could have multiple merchant accounts if we needed to [03:10:59] so we would always have one parent and at least one merchant but could have 10 merchant? [03:12:02] (03Merged) 10jenkins-bot: Remove missing data cruft [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138118 (owner: 10Eileen) [03:12:39] yeah cstone [03:12:48] (03Merged) 10jenkins-bot: Do not check the message age - we only just loaded it [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138124 (owner: 10Eileen) [03:12:55] ok thanks [03:13:59] hmm, css for dropdowns is a bit odd for me locally [03:14:08] probs something cached from that old version I had installed [03:19:41] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog: Setting up consolidated settlement reporting for all PSPs via Gr4vy for new recon process - https://phabricator.wikimedia.org/T391496#10763482 (10Cstone) @EMartin Okay I just double checked everything and our columns match the default ones Gravy is expecti... [03:19:47] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog: Setting up consolidated settlement reporting for all PSPs via Gr4vy for new recon process - https://phabricator.wikimedia.org/T391496#10763483 (10Cstone) a:03Cstone [03:22:37] (03CR) 10Ejegg: "Just a lil uncertain about the effect of the slightly different join conditions" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [03:23:32] sorry eileen I need to head to bed - was trying to figure out if that left join rather than a full join could end up marking some things as no_thank_you prematurely [03:23:43] I don't THINK so [03:23:59] ejegg: yeah valid point - I'll dig - I guess the job is OK as is until tomorrow [03:24:37] i mean, we always write the contact + email rows before the contribution + extra rows [03:25:05] so it shouldn't be likely to erroneously mark a ton of contribs because they were written before the email [03:25:23] I think I can make the API use INNER though [03:30:48] ejegg: I'm kinda inclined to use a BETWEEN - ie receive_date >$earliest & - 20 seconds [03:30:48] - I would have to the actively filter no-email from the next query though [03:33:41] 06Fundraising-Backlog: Please set a default rule in donation_rules.yaml in logs for gravy transactions - https://phabricator.wikimedia.org/T392544 (10Cstone) 03NEW [03:34:23] ok, so long as they match [03:36:22] 06Fundraising-Backlog: There is no active payment service for this transaction. response from gravy, do we want failmail? - https://phabricator.wikimedia.org/T392545 (10Cstone) 03NEW [03:46:08] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Attempt to recover emails - https://phabricator.wikimedia.org/T392546 (10Eileenmcnaughton) 03NEW [04:37:37] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: We are trying to send thank yous to 2 contacts with unusable emails - https://phabricator.wikimedia.org/T392548 (10Eileenmcnaughton) 03NEW [04:54:54] (03PS5) 10Eileen: Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) [04:57:00] (03CR) 10Eileen: "After thinking about it these are really 2 separate processes" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [04:59:55] (03PS4) 10Eileen: Extract days & earliest contribution days [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 [04:59:55] (03PS4) 10Eileen: Extract getMesssageLimit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138132 [04:59:55] (03PS4) 10Eileen: Extract update contributions without email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138133 [04:59:55] (03PS4) 10Eileen: Proposal - remove in-app disabling of thank you emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138134 [05:01:48] wfan: your patch is back!! but it's conflicted :-( [05:19:46] (03CR) 10CI reject: [V:04-1] Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [05:26:03] (03CR) 10CI reject: [V:04-1] Extract update contributions without email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138133 (owner: 10Eileen) [05:26:21] (03CR) 10CI reject: [V:04-1] Extract days & earliest contribution days [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 (owner: 10Eileen) [05:26:29] (03CR) 10CI reject: [V:04-1] Proposal - remove in-app disabling of thank you emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138134 (owner: 10Eileen) [05:28:18] (03CR) 10CI reject: [V:04-1] Extract getMesssageLimit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138132 (owner: 10Eileen) [05:29:52] (03PS6) 10Eileen: Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) [05:29:52] (03PS5) 10Eileen: Extract days & earliest contribution days [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 [05:29:52] (03PS5) 10Eileen: Extract getMesssageLimit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138132 [05:29:52] (03PS5) 10Eileen: Extract update contributions without email [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138133 [05:29:53] (03PS5) 10Eileen: Proposal - remove in-app disabling of thank you emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138134 [05:57:25] (03CR) 10CI reject: [V:04-1] Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [05:58:07] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [06:33:11] (03PS8) 10Eileen: Update Fidelity Import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136833 (https://phabricator.wikimedia.org/T386017) [06:33:11] (03PS6) 10Eileen: Upstream fix for uploaded file with template [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137096 [06:33:11] (03PS9) 10Eileen: mgd file for Fidelity import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136834 [06:33:11] (03PS5) 10Eileen: Superficial cleanup in test class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138140 [06:33:12] (03PS5) 10Eileen: Switch contact creates to apiv4 in Benevity Test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138141 (https://phabricator.wikimedia.org/T359219) [06:33:13] (03PS2) 10Eileen: port date patch [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138477 (https://phabricator.wikimedia.org/T359219) [06:33:17] (03PS1) 10Eileen: Extract getUserJob [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138597 [06:33:21] (03PS1) 10Eileen: Start to make Import hook class non-static [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138598 (https://phabricator.wikimedia.org/T359219) [07:00:49] (03CR) 10CI reject: [V:04-1] Update Fidelity Import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136833 (https://phabricator.wikimedia.org/T386017) (owner: 10Eileen) [07:56:32] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136833 (https://phabricator.wikimedia.org/T386017) (owner: 10Eileen) [08:19:17] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1136833 (https://phabricator.wikimedia.org/T386017) (owner: 10Eileen) [10:57:50] PROBLEM - Host frdb2004 is DOWN: PING CRITICAL - Packet loss = 100% [12:21:30] 10fundraising-tech-ops, 06DC-Ops: Possible frdb2004 hardware failure. - https://phabricator.wikimedia.org/T392579 (10Jgreen) 03NEW [12:22:29] 10fundraising-tech-ops, 06DC-Ops: Possible frdb2004 hardware failure. - https://phabricator.wikimedia.org/T392579#10764535 (10Jgreen) [12:22:44] 10fundraising-tech-ops, 06DC-Ops: Possible frdb2004 hardware failure. - https://phabricator.wikimedia.org/T392579#10764536 (10Jgreen) p:05Triage→03Unbreak! [12:28:20] PROBLEM - Host frdev1002 is DOWN: PING CRITICAL - Packet loss = 100% [12:29:55] ACKNOWLEDGEMENT - Host frdb2004 is DOWN: PING CRITICAL - Packet loss = 100% Jeff_Green Hardware failure: T392579 [12:32:12] RECOVERY - Host frdev1002 is UP: PING OK - Packet loss = 0%, RTA = 0.46 ms [13:00:05] 10fundraising-tech-ops: Q1:rack/setup/install fran1002 - https://phabricator.wikimedia.org/T369940#10764717 (10Jgreen) We should be using the first two NICs, the 10G's, so dhcp gets 04:32:01:DB:D2:30 `racadm>> racadm nicstatistics NIC.Integrated.1-1-1:Broadcom Adv. Dual 10Gb Ethernet - 04:32:01:DB:D2:30 Partiti... [13:11:22] 10fundraising-tech-ops: Q1:rack/setup/install fran1002 - https://phabricator.wikimedia.org/T369940#10764767 (10Jgreen) PXEboot config needs fixing: ` Booting from PXE Device 1: Embedded NIC1 Port 1 Partition 1 PXE: No media detected. Boot Failed: PXE Device 1: Embedded NIC1 Port 1 Partition 1 ` [13:17:54] 10fundraising-tech-ops: Q1:rack/setup/install fran1002 - https://phabricator.wikimedia.org/T369940#10764772 (10Jgreen) ` Booting from PXE Device 1: Integrated NIC 1 Port 1 Partition 1 >>Start PXE over IPv4. Station IP address is 10.64.40.99 Server IP address is 10.64.40.71 NBP filename is /pxelinux.0 N... [13:25:49] (03CR) 10Ejegg: [C:03+2] Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [13:44:10] (03CR) 10Ejegg: [C:03+2] Extract days & earliest contribution days [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 (owner: 10Eileen) [13:47:36] 10fundraising-tech-ops: Q1:rack/setup/install fran1002 - https://phabricator.wikimedia.org/T369940#10764859 (10Jgreen) Required fixing "Legacy Boot Protocol" from "none" to "PXE" in the device settings for "Integrated NIC 1 Port 1" [13:56:10] (03CR) 10CI reject: [V:04-1] Do separate update for no-email emails [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [13:56:11] (03CR) 10CI reject: [V:04-1] Extract days & earliest contribution days [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 (owner: 10Eileen) [14:59:08] PROBLEM - Host frban1001 is DOWN: PING CRITICAL - Packet loss = 100% [15:00:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Transactions stuck in "Authorised" status in Adyen - Urgent - https://phabricator.wikimedia.org/T392000#10765232 (10MBeat33) @AMJohnson had a donor reach out in Zendesk where it looked like we refunded both of the donations. I [[ https://docs.google.com/s... [15:02:14] RECOVERY - Host frban1001 is UP: PING OK - Packet loss = 0%, RTA = 0.34 ms [15:09:32] 10fundraising-tech-ops: Q1:rack/setup/install fran1002 - https://phabricator.wikimedia.org/T369940#10765320 (10Jgreen) 05Open→03In progress a:03Jgreen [15:30:44] 10fundraising-tech-ops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 2 others: Migrate non-fundraising hosts out of eqiad D6 - https://phabricator.wikimedia.org/T390240#10765458 (10Jgreen) 05Invalid→03Resolved [16:41:28] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: We are trying to send thank yous to 2 contacts with unusable emails - https://phabricator.wikimedia.org/T392548#10765740 (10MDemosWMF) @Eileenmcnaughton For the Citibank record CID 31398010 on my end I don't see the need for an email on the record since w... [17:42:56] (03PS1) 10Wfan: ext:upgrade-db" is deprecated. Use the main "updb" command instead [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138861 [17:44:23] (03PS2) 10Wfan: ext:upgrade-db" is deprecated. Use the main "updb" command instead [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138861 [18:42:05] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog: Fidelity Mapping Import - Change in gift framework - https://phabricator.wikimedia.org/T386017#10766131 (10MDemosWMF) Did another test and the relationship link is working as well as no anonymous soft credit! Will wait to test again on production once it's... [18:49:00] (03CR) 10Jgleeson: "Even though it makes sense to upgrade this due to the warning, when running it, I now get a new error that I don't get when I run the old " [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138861 (owner: 10Wfan) [19:09:46] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1138896 [19:12:46] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1138896 (owner: 10Jgleeson) [19:13:11] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1138896 (owner: 10Jgleeson) [19:14:39] (03PS1) 10Jgleeson: Update SmashPig to v0.9.5 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138898 [19:30:36] 14Fundraising Sprint: didAnyoneTryThis(), 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog, 13Patch-For-Review: Create new Benevity import - https://phabricator.wikimedia.org/T359219#10766298 (10Elbar53) The current Benevity files have not been imported yet. You can see them here: Fundraising/Develop... [19:35:56] (03PS1) 10Jgleeson: Update composer.lock [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1138908 [19:37:11] (03CR) 10Jgleeson: [C:03+2] Update SmashPig to v0.9.5 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138898 (owner: 10Jgleeson) [19:40:04] (03Merged) 10jenkins-bot: Update SmashPig to v0.9.5 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138898 (owner: 10Jgleeson) [19:41:47] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Add ability to attach pdfs to end of year emails - https://phabricator.wikimedia.org/T378677#10766331 (10AKanji-WMF) According to @SHust this is still a requirement [19:43:01] (03PS2) 10Jgleeson: Update composer.lock [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1138908 [19:44:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Transactions stuck in "Authorised" status in Adyen - Urgent - https://phabricator.wikimedia.org/T392000#10766340 (10AKanji-WMF) @AnnWF anything remaining to do on this task? [19:45:19] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1138910 [19:46:33] 06Fundraising Tech - Chaos Crew, 06Fundraising Tech Chaos Holding Pen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Civi merge is not saving the state when a mailing address is present. - https://phabricator.wikimedia.org/T379684#10766342 (10AKanji-WMF) [19:48:33] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Civi: Advanced Search is slow for email addresses with underscores - https://phabricator.wikimedia.org/T147156#10766353 (10AKanji-WMF) clearing old tasks @MBeat33 - can we please resolve this? [19:50:00] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1138910 (owner: 10Jgleeson) [19:53:05] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: "Zendesk TIckets" tab not showing related tickets on for some donors - https://phabricator.wikimedia.org/T374154#10766362 (10AKanji-WMF) Reprioritizing after speaking with Sandra - this would help quality of life to work on! [19:54:07] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1138910 (owner: 10Jgleeson) [19:54:45] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: "Zendesk TIckets" tab not showing related tickets on for some donors - https://phabricator.wikimedia.org/T374154#10766371 (10jgleeson) This one is no longer blocked. We eventually got the WMF sandbox Zendesk instance up and running,... [19:55:01] (03PS3) 10Jgleeson: Update composer.lock [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1138908 [20:06:47] (03CR) 10Jgleeson: [C:03+2] Update composer.lock [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1138908 (owner: 10Jgleeson) [20:07:05] jeez [20:07:22] the dance to deploy an update to smashpig takes nearly an hour with all the CI checks [20:07:47] yeahh jgleeson its a lot! [20:08:11] yeah a one-hour deployment period is no joke [20:09:15] civi is just as long except the last step [20:09:25] so i gave it 20 mins then came back to relalize it had merged in like 1 haha [20:09:32] ha [20:21:49] (03Merged) 10jenkins-bot: Update composer.lock [core] (fundraising/REL1_43) - 10https://gerrit.wikimedia.org/r/1138908 (owner: 10Jgleeson) [20:23:04] (03PS1) 10Wfan: Clean up AR with gravy dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138918 (https://phabricator.wikimedia.org/T389547) [20:23:35] (03PS2) 10Wfan: Clean up AR with gravy dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138918 (https://phabricator.wikimedia.org/T389547) [20:26:34] (03PS1) 10Wfan: Add rapipago for Gravy dlocal AR [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1138920 (https://phabricator.wikimedia.org/T389547) [20:33:29] wfan: I don't see AR on that link in the commit [20:33:39] should I? [20:34:05] ah there is another patch hold on [20:35:18] !log payments-wiki upgraded from d250a3b8 to c6ba1f35 [20:35:20] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [20:36:21] (03PS1) 10Wfan: Add ar cash link [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138923 (https://phabricator.wikimedia.org/T389547) [20:37:11] wfan: I mean the spreadsheet link [20:37:19] on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/1138918 [20:37:57] the "per doc here" made me think that would contain the rules used to update the payment submethods file [20:38:11] that assumption could be wrong! [20:39:36] https://docs.google.com/spreadsheets/d/117qd1q2wEGjjP1FtkOkO4RHGu6cYN_Eh9l0kVtAC2qo/edit?gid=434193234#gid=434193234 [20:40:07] hmm, https://docs.google.com/spreadsheets/d/117qd1q2wEGjjP1FtkOkO4RHGu6cYN_Eh9l0kVtAC2qo/edit?gid=434193234#gid=434193234 [20:40:27] interesting it;s on dLocal tab [20:40:38] seems like the link is not linking the tab... and line 213 [20:40:39] 312 [20:41:16] (03PS3) 10Wfan: Clean up AR with gravy dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138918 (https://phabricator.wikimedia.org/T389547) [20:41:23] (03CR) 10Jgleeson: [C:03+2] "LGTM!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1138920 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [20:42:50] (03CR) 10Jgleeson: [C:03+2] "LGTM!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138918 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [20:43:03] (03CR) 10Jgleeson: [C:03+2] "LGTM!" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138923 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [20:43:23] (03PS2) 10Wfan: Add rapipago for Gravy dlocal AR [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1138920 (https://phabricator.wikimedia.org/T389547) [20:44:57] oooh good spot wfan [20:44:57] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi: underscore in email address slows search - https://phabricator.wikimedia.org/T234100#10766521 (10SHust) 05Open→03Resolved [20:44:58] I missed that [20:45:07] ha haha, thanks jack! [20:45:27] (03Merged) 10jenkins-bot: Clean up AR with gravy dlocal [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1138918 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [20:45:43] https://sandbox.wikimedia.gr4vy.app/merchants/default/transactions just tested two ar one in cash one in cc [20:46:21] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Civi: Advanced Search is slow for email addresses with underscores - https://phabricator.wikimedia.org/T147156#10766523 (10SHust) 05Open→03Resolved [20:57:38] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [21:17:23] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email, 06FR-donorrelations: Civi: harmonize currency & amount in TY emails - https://phabricator.wikimedia.org/T220636#10766611 (10MBeat33) 05Open→03Resolved a:03MBeat33 Issues resolved since Task was active [21:25:23] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 06FR-donorrelations: "Zendesk TIckets" tab not showing related tickets on for some donors - https://phabricator.wikimedia.org/T374154#10766700 (10SHust) No worries, @jgleeson, and ty for the update! [21:28:34] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [21:36:59] hey all - this might help with the unit test speed issues ... https://github.com/civicrm/civicrm-core/pull/32345 [21:56:38] nice eileen [21:57:11] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Wikimedia-production-error: BannerExistenceException: Banner by that name already exists! - https://phabricator.wikimedia.org/T286673#10766796 (10Reedy) [21:57:18] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Wikimedia-production-error: BannerExistenceException: Banner by that name already exists! - https://phabricator.wikimedia.org/T286673#10766797 (10Reedy) ==== Error ==== * mwversion: 1.44.0-wmf.25 * reqId: `d8dbf3bc-5242-43fb-88fe-2afcc297d9da`... [22:02:48] (03CR) 10Eileen: [V:03+2] "Putting back the +2 as this did pass & some known but unrelated test slow down is causing the tests to time out & fail" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138130 (https://phabricator.wikimedia.org/T270624) (owner: 10Eileen) [22:03:38] (03CR) 10Eileen: [V:03+2] "Returning the + 2 as the test did pass & an unrelated issue is currently causing problems" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138131 (owner: 10Eileen) [22:06:18] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138930 [22:06:32] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138930 (owner: 10Eileen) [22:07:20] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1138930 (owner: 10Eileen) [22:10:13] !log update CIviCRM civicrm: revision 3ca2db06 [22:10:16] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:11:12] if someone is able to check my commit on process control I will push it out & disable the thank you module [22:12:27] looking [22:14:10] i think that looks ok compared to the others in p-c land. [22:16:07] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Fix thank you to default to total_amount & currency when original_amount & original_currency empty, - https://phabricator.wikimedia.org/T392646 (10Eileenmcnaughton) 03NEW [22:17:32] !log config revision changed from 47a5d384 to 7bf2c087 [22:17:35] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:17:41] thanks dwisehaupt I pushed it out - will see what the logs say [22:22:20] (03CR) 10Wfan: [V:03+2] Add ar cash link [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1138923 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [22:22:42] (03CR) 10Wfan: [C:03+2] "reapply the +2 from jack" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1138920 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [22:23:13] (03Merged) 10jenkins-bot: Add rapipago for Gravy dlocal AR [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1138920 (https://phabricator.wikimedia.org/T389547) (owner: 10Wfan) [22:23:17] (03PS1) 10Eileen: Port patch to speed up managed reconcile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138934 [22:24:01] (03CR) 10CI reject: [V:04-1] Port patch to speed up managed reconcile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138934 (owner: 10Eileen) [22:35:13] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Thank you code cleanup - load data consistently - https://phabricator.wikimedia.org/T392648 (10Eileenmcnaughton) 03NEW [22:47:59] (03PS3) 10Eileen: CiviCRM upstream updates to 6.1.2 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1137097 [22:48:03] (03PS3) 10Eileen: port date patch [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138477 (https://phabricator.wikimedia.org/T359219) [22:48:07] (03PS2) 10Eileen: Port patch to speed up managed reconcile [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1138934 [22:53:27] so I think that numberOfDays=1 doesn't seem to be catching many - gonna update to 2 - I assume the time calc must need that [23:42:55] !log config revision changed from 7bf2c087 to 1c84d1a7 [23:42:58] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [23:44:48] on a sample size of 1 run - the reconcile patch was 4 mins faster - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1138934/2 [23:57:13] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog, 07payments-orchestration: How will we expose payment processor errors/issues with gravy - https://phabricator.wikimedia.org/T387411#10767137 (10Cstone) We started adding some failmail https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterfac... [23:57:29] 03Fundraising Sprint: Hot Pot!, 06Fundraising-Backlog, 07payments-orchestration: How will we expose payment processor errors/issues with gravy - https://phabricator.wikimedia.org/T387411#10767138 (10Cstone) [23:57:30] 06Fundraising-Backlog: There is no active payment service for this transaction. response from gravy, do we want failmail? - https://phabricator.wikimedia.org/T392545#10767139 (10Cstone)