[00:03:00] AndyRussG: oh hah [00:03:15] I thought for some reason I had put it in the SmashPig extension [00:03:17] but no [00:03:46] I guess I started with it there [00:04:39] sorry, I've been so dependent on the IDE search-navigation and git tab-completion that I forget where in the tree things are [00:05:19] anyway, the logic about it being there, and not in the forgetme extension, is that the forgetme extension is something that people in the wider civi community can use, even if they don't use omnimail or smashpig [00:05:42] (and they most certainly do not use our wmf_contribution_extra table) [00:07:03] So it would theoretically be nice to move the code to the smashpig extension, but the way we record donations, the only link between a one-time contribution and the processor we used to charge it, is in the wmf_contribution_extra table. [00:08:07] If we were taking contributions via the Civi pages (and if we wrote out all the code in the smashpig extension to handle one-time contributions, right now it only handles recurring), there would be a different set of joins we could use to find the payment processor, using just core tables [00:08:42] and in that case it would make sense to put this code into the smashpig extension, since it's another way to take advantage of code in smashpig [00:10:16] right, i see I moved the action to wmf-civicrm between PS1 and PS2 [00:21:06] 10Fundraising-Backlog, 10FR-Docker, 10Patch-For-Review: Add a test version of a main-cluster wiki to fundraising-dev, for dev work on on Donate Wiki, CentralNotice and banners (CentralDonut) - https://phabricator.wikimedia.org/T317498 (10Ejegg) Thanks @Pcoombe ! That gets the XML export down to 2.7MB from al... [00:43:27] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Name parser breaks Irish last names - https://phabricator.wikimedia.org/T321343 (10Ejegg) Looks like there is a PR upstream https://github.com/theiconic/name-parser/pull/39 that was last touched almost 3 years ago and is... [01:11:49] (03PS1) 10Ejegg: Work around name parser bug on multibyte initials [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/845102 (https://phabricator.wikimedia.org/T321343) [01:13:45] ejegg: There was payment method we handn't seen before when running the audit Unknown Payment Method jcbprepaidanonymous , I know we added them to ReferenceData before but is there another place they need to go? [01:14:03] oh fun cstone [01:14:16] and are the submethods on that file ones we choose? [01:14:22] just not sure how to categorize it [01:14:34] 'jcbprepaidanonymous' => [ [01:14:34] 'method' => 'cc', [01:14:34] 'submethod' => 'jcb-debit', [01:14:34] ], maybe [01:14:34] I guess just map it to jcb [01:14:38] ah okay [01:14:52] is that what the variants are doing? [01:15:02] ah right, I think so [01:15:10] lemme refresh my memory [01:16:13] ok, so 'variant' is actually another field they give us on the audit for some small number of cards [01:16:49] ahh okay [01:17:01] Ah okay i see at the top [01:17:04] but in this case it looks like the jcbprepaidanonymous is coming in on the main Payment Method column [01:17:05] this is the same for both [01:17:18] yeah [01:17:50] so the question is just do we care whether it's broken out from other JCB cards in Civi [01:17:58] and I'm guessing we probably don't [01:18:03] yeah I agree [01:18:37] ah yeah, visadankort (?) gets mapped to just 'visa', for example [01:18:54] only for those with the dankest credit [01:18:54] yeah I was looking at that one hah [01:19:01] theres one in this same audit file [01:19:53] so yeah, if we're mapping it to just-plain jcb in the ReferenceData we shouldn't have to add it anyplace else [01:20:10] have we ever like put an example data point into one of these files explaining what each part means, like in a comment at the top? [01:20:46] not yet, but we could! [01:22:26] is the method in there from the civi database? [01:23:36] nahh, it's just our weird intermediate normalized form [01:23:53] used in donationinterface config files and in queue messages [01:24:10] okay [01:24:10] but then when we get to civi we map that to a payment_instrument [01:24:33] which just have human readable names like 'Credit Card: Visa' and then a database ID [01:24:42] is ithe submethod our name too? [01:24:53] yep yep, the submethod is our name [01:25:07] oh right, it's the submethod that maps to the payment_instrument [01:25:34] the method is just like a high-level grouping of submethods that share a bunch of behaviors [01:25:53] (and in smashpig, correspond to different PaymentProvider implementations) [01:26:48] lessee, I should describe that someplace in our docs [01:27:59] Just at the top of the 'Data and flow' section maybe? [01:32:08] (03PS1) 10Cstone: Add jcbprepaidanonymous and an example of how to add new methods. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845124 (https://phabricator.wikimedia.org/T314753) [01:33:17] chaos whitespace on the comment haha [01:34:05] (03CR) 10Ejegg: [C: 03+2] "Thanks for the example!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845124 (https://phabricator.wikimedia.org/T314753) (owner: 10Cstone) [01:34:13] thanks for explaining ejegg [01:34:43] (03Merged) 10jenkins-bot: Add jcbprepaidanonymous and an example of how to add new methods. [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845124 (https://phabricator.wikimedia.org/T314753) (owner: 10Cstone) [01:34:56] so I'm dithering over whether to split the whole 'Data and Flow' chunk of our main doc page into a subpage [01:35:12] not sure how many links it would break [01:35:29] i think we should or at least move those big text chunks to their own page [01:38:55] oh wait, there already is a https://wikitech.wikimedia.org/wiki/Fundraising/Data_and_flow [01:40:51] ejegg: ah yeah I do think all those intermediate topics should have subpages! [01:41:09] the main page should ideally just be a short overview of each subtopic (just my 2 cents) [01:41:25] ejegg: also did u see my question earlier ^ ? [01:41:41] ah yeah now I see ur replies! [01:43:09] ejegg: ah I see... hmmm that makes sense as to why it should go in wmf-civcrm :) [01:43:11] (03PS1) 10Cstone: Update SmashPig to v0.8.3.4 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/845133 [01:43:23] I feel my ask in that case would be to add a comment explaining that? [01:43:25] most smash pig deploys in one 24 hour period?? [01:44:33] and then also if there are bits and pieces of org.wikimedia.forgetme that do reference our custom table (I thought I saw some? maybe I'm wrong) we could in a separate patch flag them with TODOs to move them over, too? how does that sound [01:44:53] cstone: a new Guinees world recotd? [01:45:01] *Guiness [01:45:16] AndyRussG: the existing org.wikimedia.forgetme doesn't reference any of our custom tables, no [01:48:34] (03CR) 10Cstone: [C: 03+2] Update SmashPig to v0.8.3.4 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/845133 (owner: 10Cstone) [01:50:55] (03Merged) 10jenkins-bot: Update SmashPig to v0.8.3.4 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/845133 (owner: 10Cstone) [02:00:08] (03PS1) 10Cstone: Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/845151 [02:14:24] (03CR) 10Cstone: [C: 03+2] Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/845151 (owner: 10Cstone) [02:21:27] (03PS8) 10Ejegg: Add Adyen data deletion requests to forgetme [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/832580 (https://phabricator.wikimedia.org/T311741) [02:21:44] ok AndyRussG I added a comment explaining why it is where it is [02:21:52] and where it may someday graduate to being [02:26:47] (03Merged) 10jenkins-bot: Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/845151 (owner: 10Cstone) [02:29:46] (03PS1) 10Cstone: Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/845176 [02:30:40] (03CR) 10Cstone: [C: 03+2] Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/845176 (owner: 10Cstone) [02:36:48] ah thanks for doing that ejegg! [02:37:14] ejegg: also if you still are around let me point out where I thought I'd found custom stuff in the forgetme ext [02:40:23] these last two methods seem to reference custom stuff like wmf_donor something? I can't say I actually understand in detail what that code's doing tho... https://gerrit.wikimedia.org/r/plugins/gitiles/wikimedia/fundraising/crm/+/refs/heads/master/drupal/sites/default/civicrm/extensions/org.wikimedia.forgetme/CRM/Forgetme/Metadata.php#266 [02:41:50] ejegg: comment looks great btw, thanks for that!! :) [02:42:52] (03Merged) 10jenkins-bot: Update SmashPig to v0.8.3.4 and DonationInterface to v2.5.8.4 [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/845176 (owner: 10Cstone) [02:43:06] AndyRussG: oh huh, I hadn't noticed that [02:43:21] they're explicitly filtering out that table rather than depending on it [02:43:35] so it won't break by running on anyone else's install [02:43:39] (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/845179 [02:47:02] (03CR) 10Cstone: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/845179 (owner: 10Cstone) [02:48:39] !log civicrm upgraded from 3e24d6f7 to 89a46665 [02:48:42] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:54:15] noooo I SWORE i checked all these files for missing types of course theres another now [02:54:18] just Externally [02:54:56] awwwwww [02:55:17] i must have checked 15 16 and 18 and skipped 17 [02:56:53] oooooh [02:56:57] it RefundedExternally [02:56:58] amex [02:58:38] oh so special amex [02:59:17] might as well add chargebackexternally too [03:00:24] im curious how it shows up in the big audit [03:12:48] (03PS1) 10Cstone: Add types for Amex [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845193 (https://phabricator.wikimedia.org/T314753) [03:13:02] (03PS4) 10Ejegg: WIP add donatewiki service [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/841965 (https://phabricator.wikimedia.org/T317498) [03:13:33] getting really close to working ^^^ [03:14:24] (03CR) 10Ejegg: [C: 03+2] Add types for Amex [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845193 (https://phabricator.wikimedia.org/T314753) (owner: 10Cstone) [03:14:57] (03Merged) 10jenkins-bot: Add types for Amex [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845193 (https://phabricator.wikimedia.org/T314753) (owner: 10Cstone) [03:15:14] haha thanks ejegg [03:15:19] :) [03:15:24] i fixed the error message too so now it actually shows the type [03:15:28] instead of the column title [03:15:43] oh nice [03:15:55] im gona wait for todays file and big audit files to see if theres any more rogue data [03:16:28] and thats exciting ejegg to the docker! [03:17:06] im gona head out have a good weekend ejegg [03:17:28] you too cstone, thanks! [03:28:13] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10fr-donorservices: Civi showing completed donations when they were rejected at Paypal - https://phabricator.wikimedia.org/T320989 (10Ejegg) @MBeat On Monday I'll take a stab at doing it automatically. Only about half of the IDs from IPNs with 'Denied'... [03:32:11] (03CR) 10Ejegg: "Let's be consistent at least within one payment provider as to where we map from the 'normalized' param names to the processor-side param " [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) (owner: 10Jgleeson) [09:44:40] 10Fundraising-Backlog, 10fundraising-tech-ops, 10fr-donorservices: Fundraising access request for Donor Relations agent Salvatore Borriello - https://phabricator.wikimedia.org/T321100 (10krobinson) @Dwisehaupt I think Salvatore was having some issues finding the attachment. He has pinged you, but I wanted to... [11:32:02] (03CR) 10Jgleeson: WIP: Implement PayPal EC DoExpressCheckoutPayment API call (031 comment) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) (owner: 10Jgleeson) [18:31:35] 10Fundraising-Backlog, 10fundraising-tech-ops, 10fr-donorservices: Fundraising access request for Donor Relations agent Salvatore Borriello - https://phabricator.wikimedia.org/T321100 (10Dwisehaupt) @krobinson Thanks. I don't see a ping but I'll reach out. [20:22:53] (03PS2) 10Jgleeson: WIP: Implement PayPal EC DoExpressCheckoutPayment API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) [20:24:04] (03PS1) 10Jgleeson: WIP: Update Smashig README [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845699 [20:24:22] (03PS3) 10Jgleeson: WIP: Implement PayPal EC DoExpressCheckoutPayment API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) [20:24:41] n/ [20:28:58] (03PS4) 10Jgleeson: WIP: Implement PayPal EC DoExpressCheckoutPayment API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) [20:29:39] (03PS5) 10Jgleeson: WIP: Implement PayPal EC DoExpressCheckoutPayment API call [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/845045 (https://phabricator.wikimedia.org/T318756) [20:33:58] intersting looks like the big audit file doesn't have the amex standalone stuff in it [20:34:17] explains why it had no issue with externally in the name [20:50:13] PROBLEM - check_mysql on frdb1005 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_mysql [20:51:56] looks like that ^^ is a downtime expiration. checking since i know there was work being done. [21:45:12] (03CR) 10Wfan: [C: 03+1] "Tested with https://wmff.localhost:32353/civicrm/api4#/explorer/Name/parse?names=%22%5B'something%20%C3%B4%20someone'%5D%22&_format=php, l" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/845102 (https://phabricator.wikimedia.org/T321343) (owner: 10Ejegg) [22:10:17] RECOVERY - check_mysql on frdb1005 is OK: Uptime: 204968 Threads: 6 Questions: 31916459 Slow queries: 227 Opens: 1233 Open tables: 1162 Queries per second avg: 155.714 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_mysql [22:37:06] (03CR) 10Wfan: [C: 03+2] "related with https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/SmashPig/+/315754/ todo, so we can drop the support for 'raw' dumps" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/841987 (owner: 10Ejegg) [22:38:54] (03Merged) 10jenkins-bot: Drop support for imports from STOMP queues [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/841987 (owner: 10Ejegg)