[08:30:24] (03PS1) 10Jforrester: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1052668 [08:31:18] (03PS2) 10Jforrester: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1052668 (https://phabricator.wikimedia.org/T287405) [08:32:33] (03CR) 10Jforrester: "Last merge was Ie57b4a53bab9ea3b5fb5749ee116151bc0b20d73 back in March(!)." [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/1052668 (https://phabricator.wikimedia.org/T287405) (owner: 10Jforrester) [12:10:12] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9960597 (10jgleeson) I'll take a peek at the logs on our side [12:12:39] (03PS6) 10Jgleeson: Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) [12:31:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9960637 (10jgleeson) The logs look good on our side regarding the export files getting generated and uploaded. I can manually push up... [12:55:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9960697 (10jgleeson) Ok, I can see some invalid data in that export. Most rows consist of the following format: | Email | ID | | ema... [13:07:45] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9960739 (10jgleeson) Running the following pattern match search shows me 1670 non-valid email addresses: `grep -E '[^[:space:]][[:spa... [13:13:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9960775 (10jgleeson) And `grep -E '(^|,)[^,]*([0-9]{4}[[:space:]-]?){3}[0-9]{4}([[:space:],]|$)' Unsubscribes-20240708064226.csv` sho... [13:25:17] 06Fundraising-Backlog: Credit Card numbers showing up in Acoustic Export - https://phabricator.wikimedia.org/T369503 (10jgleeson) 03NEW [15:19:14] (03CR) 10Jgleeson: "On the latest PS, you can check out different SmashPig Gerrit patches without restoring them each time." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) (owner: 10Jgleeson) [15:19:58] (03PS4) 10Jgleeson: WIP: Add CiviCRM Standalone [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1039235 [15:19:58] (03PS7) 10Jgleeson: Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) [15:21:01] (03PS8) 10Jgleeson: Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) [15:21:14] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Fr-tech chores list - https://phabricator.wikimedia.org/T258527#9961503 (10Damilare) [15:23:58] (03PS9) 10Jgleeson: Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) [15:27:11] (03PS10) 10Jgleeson: Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) [15:35:00] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9961559 (10Ejegg) Could the jobs be late because the export is so big? We spent a few days recalculating everyone's segment and statu... [15:38:35] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9961586 (10jgleeson) The size could be a factor, but I don't think they are late unless I missed that. [15:41:36] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Email: Civi to Acoustic Export: Job "MASTER SUPPRESSION LIST" failing. - https://phabricator.wikimedia.org/T369457#9961616 (10Cstone) This phab was last time this happened {T366059} [15:49:32] 06Fundraising-Backlog, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog, 13Patch-For-Review: Update donation links in apps to use wmf_ parameters - https://phabricator.wikimedia.org/T367360#9961677 (10HNordeenWMF) [15:59:52] (03PS1) 10Ejegg: Tag PayPal Giving Fund donations with contact_id [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1052773 (https://phabricator.wikimedia.org/T363768) [16:28:33] (03CR) 10Wfan: [C:03+2] "Both scripts works great with no error for me, thanks Jack~" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) (owner: 10Jgleeson) [16:28:53] thanks wfan !!! [16:29:32] ty for those helpful scripts~ [17:13:54] 06Fundraising-Backlog, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog, 13Patch-For-Review: Update donation links in apps to use wmf_ parameters - https://phabricator.wikimedia.org/T367360#9962096 (10Tsevener) @Cstone @JMando To be clear, I'm only talking about changing a `banner` parameter... [17:26:33] (03PS1) 10Wfan: Do not pass billingAddress for ach [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1052790 (https://phabricator.wikimedia.org/T360825) [17:34:45] 10fundraising-tech-ops, 10DNS, 06SRE, 06Traffic, 13Patch-For-Review: Cleanup unused DNS subdomains - https://phabricator.wikimedia.org/T367012#9962213 (10Dzahn) Thanks @AKanji-WMF Are you still using http://mandrillapp.com/ / MailChimp for fundraising emails with benefactors.wikimedia.org ? [18:13:46] (03PS1) 10Ejegg: Tag Giving Fund donations with contact_id [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1052796 (https://phabricator.wikimedia.org/T363768) [18:14:01] (03PS2) 10Ejegg: Tag Giving Fund donations with contact_id [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1052796 (https://phabricator.wikimedia.org/T363768) [19:07:46] (03CR) 10Ejegg: "Hi Dami, do you think it's worth considering hiding the buyer record creation behind the createPayment call? If the calling code needs to " [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1041627 (owner: 10Damilare Adedoyin) [19:20:03] 14Fundraising Sprint: justWork(), 03Fundraising Sprint: meow(), 06Fundraising-Backlog, 13Patch-For-Review: ACH on PaymentsWiki - Form Changes - https://phabricator.wikimedia.org/T360825#9962700 (10AnnWF) @AKanji-WMF @EMartin Hi all, since we are removing the billing address, seems like this show image inst... [19:44:06] (03CR) 10Wfan: [V:03+2 C:03+2] Add script to safely checkout smashpig patch to payments [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1046738 (https://phabricator.wikimedia.org/T367798) (owner: 10Jgleeson) [19:44:12] ty! [19:44:48] 03Fundraising Sprint: meow(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Update segments & statuses for 2024-25 - https://phabricator.wikimedia.org/T368974#9962787 (10Eileenmcnaughton) a:03Eileenmcnaughton These should all be done now @KHaggard - not sure if @JMando wants to do some checks [19:45:01] 03Fundraising Sprint: meow(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Update segments & statuses for 2024-25 - https://phabricator.wikimedia.org/T368974#9962790 (10Eileenmcnaughton) [19:47:13] 03Fundraising Sprint: meow(), 06Fundraising-Backlog: Remove CVV Placeholder for Japanese donors - https://phabricator.wikimedia.org/T368647#9962807 (10Ejegg) a:03Ejegg [19:48:45] 14Fundraising Sprint: justWork(), 03Fundraising Sprint: meow(), 06Fundraising-Backlog: Update RML donors to have status of non-donor in CiviCRM - https://phabricator.wikimedia.org/T365534#9962811 (10Eileenmcnaughton) @KHaggard we worked around this at the Acoustic end didn't we? [19:49:31] (03PS1) 10Ejegg: Blank placeholder for CVV in Japanese [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052810 (https://phabricator.wikimedia.org/T368647) [19:51:25] (03CR) 10CI reject: [V:04-1] Blank placeholder for CVV in Japanese [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052810 (https://phabricator.wikimedia.org/T368647) (owner: 10Ejegg) [19:53:43] 06Fundraising-Backlog: Update app submit payment endpoint to handle wmf_ and utm_ fields - https://phabricator.wikimedia.org/T367916#9962830 (10Cstone) [19:57:20] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Sepa oddity - https://phabricator.wikimedia.org/T368080#9962846 (10greg) p:05Triage→03High [20:07:53] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Civi-Dedupe: New Segmentation Framework not quite usable yet - https://phabricator.wikimedia.org/T363527#9962875 (10Eileenmcnaughton) @KHaggard we have addressed a couple of issues since this was logged - what is the status on this phab? [20:10:06] 06Fundraising-Backlog: run PayPal refund script for June - https://phabricator.wikimedia.org/T369383#9962899 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @Damilare has done this as part of today's chaos tasks. [20:15:27] 06Fundraising-Backlog: run PayPal refund script for June - https://phabricator.wikimedia.org/T369383#9962917 (10MBeat33) Thanks Anil and Damilare. [20:16:32] 06Fundraising-Backlog: Give RLewis access to https://civicrm.wikimedia.org/admin/config/large_donation/configure - https://phabricator.wikimedia.org/T368152#9962927 (10AKanji-WMF) May be addressed by SearchKit update/functionality - will wait and see a few cycles [20:20:33] 06Fundraising-Backlog: Drupal admin civi contact getting activites for all contributions - https://phabricator.wikimedia.org/T367612#9962940 (10Eileenmcnaughton) It creates civicrm_activity_contact records in the DB - not sure they are extra - it might be instead of the target contact having the record - ie ther... [20:24:47] 06Fundraising-Backlog: Drupal admin civi contact getting activites for all contributions - https://phabricator.wikimedia.org/T367612#9962952 (10AKanji-WMF) * core code * was probably coming before from the source contact [20:30:47] 06Fundraising-Backlog, 10FR-Civi-Dedupe: Create button to merge "Unknown: Shares contact information" - https://phabricator.wikimedia.org/T367234#9962957 (10AKanji-WMF) [20:36:14] 06Fundraising-Backlog, 10fundraising-tech-ops: Investigate pulling in page view data to the fr-tech version of superset - https://phabricator.wikimedia.org/T252049#9962992 (10AKanji-WMF) I've added to our next FR Analytics/FR-Tech meeting time on the 9th. [20:43:38] ejegg: re your review on this patch: https://gerrit.wikimedia.org/r/1046738, the logic you described in your comment was how it was implemented in DI [20:43:55] check if donor record exists using email and if it doesn't create a new one [20:44:38] I thought it would be good to separate the logic for gr4vy donor data interactions into its own class [20:46:31] damilare: yep, I was just wondering if we should keep that logic 'under the hood' in Smashpig rather than making the calling code need to be aware of it [20:47:12] for example, I'd rather we don't have to write special-case code in the recurring donation charge job to handle the buyer record [20:47:41] if we keep that all inside the createPayment call we can just use the existing code, I think [20:48:24] for the recurring payment charge, we can use the gravy buyer id saved in the external identifer for subsequeunt payments [20:48:44] there shouldn't be a need to fetch the donor data for every charge [20:50:06] hmm, ok. In that case, it would be treated just like the Adyen shopperReference? [20:50:28] yea something like that, I do agree with keeping it simple on DI too [20:51:42] so it should be processor_contact_id if we want to make it compatible with that Adyen createPayment [20:52:04] exactly, I just checked that too [20:52:12] the description matches the case here [20:53:15] I'm just wondering why we had to use the custom external_identifier table for frup [20:53:51] oh sorry this is a custom field on the recurring table [20:55:50] ah yeah, since we can have a donor create two recurrings under different processor_contact_id and different donor records, then later have their records merged, we had to put the Adyen shopperReference on the recurring record [20:57:09] cstone: don't forget the mouse https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1037205 [20:58:13] COVID has kept the mouse alive hah and then no car ac...but I will look when I get back eileen !! [20:58:19] damilare: did you mean to link to that patch. it doesnt seem related to the chat [20:58:20] (03CR) 10Ejegg: "Still looking, just some initial feedback on the try/catch" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1043745 (https://phabricator.wikimedia.org/T367112) (owner: 10Damilare Adedoyin) [20:58:32] cstone: oh yeah - you still have covid? [20:58:34] (the patch you linked was fun-dev payments-smashpig checkout patch) [20:58:43] the mouse lives to squeak another day [20:59:34] ahh must have shifted when I did the copy, it's this patch: https://gerrit.wikimedia.org/r/1041627 jgleeson [21:00:27] thanks. I was really confused lol [21:00:48] (03PS1) 10Eileen: Civi-stock 5.76-beta [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1052815 (https://phabricator.wikimedia.org/T368999) [21:00:56] ejegg: makes sense and since we don't have a donor portal not managed by us we wouldn't need the external identifier to link the records [21:02:05] sorry jgleeson, I guess I don't have good ctrl c-v skills :( [21:02:23] ejegg: I think php let's you use vars declared in the try, in the catch [21:02:32] (drive-by review review) [21:02:51] albeit terribly unobvious [21:03:05] ahh 😂 [21:03:28] (03CR) 10CI reject: [V:04-1] Civi-stock 5.76-beta [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1052815 (https://phabricator.wikimedia.org/T368999) (owner: 10Eileen) [21:13:14] that's pretty cool/weird. I just got an email from zendesk saying I have a voicemail from a US caller [21:13:22] which it saved as a ticket [21:13:40] Time of call: 2024-07-08 20:09:31 UTC [21:13:41] Location: Orlando, Florida, United States [21:13:43] Length of phone call: 28 seconds [21:14:12] (this is my own trial account that I added during the offiste) [21:22:29] dang, i'm unable to load much of anything in fr-dev docker this week [21:22:42] containers seem to die quietly [21:22:45] not logging anything [21:23:26] while they're still alive I try to load the :9001 site and get a PR_END_OF_FILE_ERROR [21:27:32] let's try a rebuild [21:32:55] ugh, still PR_END_OF_FILE_ERROR on :9001 and wmff [21:34:54] what version of docker-compose is the rest of fr-tech using? [21:34:58] I'm on 1.29.2 [21:37:05] huh, connection reset even on the http url [21:37:13] so it's not just the cert [21:44:31] in host syslog I see lots of spam on start then like a minute later a couple of 'docker-sdkas[...]jda21u312.scope Deactivated successfully.' [21:44:43] then a ton of other shutdown-related log lines [21:45:01] lots of 'cleaning up dead shim' [21:45:52] hmm, let's try running in a fresh checkout [22:03:47] (03PS2) 10Eileen: Civi-stock 5.76-beta [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1052815 (https://phabricator.wikimedia.org/T368999) [22:03:47] (03PS1) 10Eileen: Fix PSR0 declaration & Increment min version on ExchangeRates [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1052822 [22:03:47] (03PS1) 10Eileen: Port query consolidation work [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1052823 [22:24:31] dang, the setup fails because the containers die midway [22:24:54] (03PS1) 10Wfan: ACH on PaymentsWiki - Form Changes [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052824 (https://phabricator.wikimedia.org/T360825) [22:26:36] (03CR) 10CI reject: [V:04-1] ACH on PaymentsWiki - Form Changes [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052824 (https://phabricator.wikimedia.org/T360825) (owner: 10Wfan) [22:27:32] ok, so the non-apache containers seem to stay up [22:28:24] did my amache image get busted somehow? [22:29:31] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002 - https://phabricator.wikimedia.org/T369565 (10RobH) 03NEW [22:30:18] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002 - https://phabricator.wikimedia.org/T369565#9963351 (10RobH) [22:30:50] 10fundraising-tech-ops, 06DC-Ops, 10ops-eqiad: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002 - https://phabricator.wikimedia.org/T369565#9963359 (10RobH) [22:31:39] (03PS2) 10Wfan: ACH on PaymentsWiki - Form Changes [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052824 (https://phabricator.wikimedia.org/T360825) [22:32:29] 10fundraising-tech-ops, 06DC-Ops, 10ops-codfw: Q1:rack/setup/install frqueue2003, pay-lb2001, pay-lb2002 - https://phabricator.wikimedia.org/T369566 (10RobH) 03NEW [22:32:45] 10fundraising-tech-ops, 06DC-Ops, 10ops-codfw: Q1:rack/setup/install frqueue2003, pay-lb2001, pay-lb2002 - https://phabricator.wikimedia.org/T369566#9963385 (10RobH) [22:39:24] ok, apache2 seems not to have written any logs inside the container [22:40:29] ahh, what's this: rsyslogd: imfile: no working or state file directory set, imfile will create state files in the current working directory (probably the root dir). Use global(workDirectory="/some/path") to set the working directory [v8.2302.0 try https://www.rsyslog.com/e/2450 [22:40:54] if I get into the container as root and run the entrypoint script directly [22:55:19] (03PS2) 10Ejegg: Blank placeholder for CVV in Japanese [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1052810 (https://phabricator.wikimedia.org/T368647) [23:02:08] good luck ejegg I have not broken my local in that way (yet) [23:11:30] thanks cstone [23:11:41] i edited out the imfile part and still no dice [23:12:11] it must be having trouble connecting out to the logging container [23:14:09] we're using (EOL) buster for the logger container [23:14:48] Let's see if we can swap out a newer image [23:15:10] i always forget how to query the docker registry [23:16:59] ah it's this secret URL: https://docker-registry.wikimedia.org/v2/_catalog [23:17:27] ah dang, and only has buster-rsyslog [23:22:05] well, I can get the site up if I hack out rsyslog completely [23:22:10] but that's suboptimal [23:22:28] at least I can smoke test the JP placeholder [23:26:43] wonder what it takes to update that so there is a bookworm one. [23:34:18] (03CR) 10Ejegg: [C:03+2] Merge branch 'REL1_39' into fundraising/REL1_39 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1050423 (owner: 10Ejegg) [23:36:13] ok. can't go down that rabbit hole any further. too many things on the list to check off. :) [23:37:02] (03PS2) 10Ejegg: RecordCaptureJob implements Runnable [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1048569 (https://phabricator.wikimedia.org/T366367) [23:37:02] (03PS3) 10Ejegg: Delete RunnableJob abstract class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1048570 (https://phabricator.wikimedia.org/T366367) [23:42:34] (03Merged) 10jenkins-bot: Merge branch 'REL1_39' into fundraising/REL1_39 [core] (fundraising/REL1_39) - 10https://gerrit.wikimedia.org/r/1050423 (owner: 10Ejegg)