[06:29:30] (03CR) 10CI reject: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/821989 (owner: 10L10n-bot) [06:53:13] (03CR) 10Raimond Spekking: [C: 03+2] "false positive" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/821989 (owner: 10L10n-bot) [08:52:13] hi damilare [08:52:28] hey jgleeson [08:53:03] I can see the failmail wave. did you say you've reached out to adyen? [08:53:14] no I haven't [08:53:25] I'm looking for the right email to send to [08:54:18] lemme find that for you. if you wanna email adyen saying we're seeing timeouts, I'll email advancement to see if we can disable any active campaigns [08:54:45] Ok, I was going to email support, but wasn't sure if there was a faster route [08:55:09] I need leave to make preperations to get to the airport in about an hour so I'll be around until then [08:56:41] yeah first port of call would be support@adyen.com and then maybe cc in matt.heisser@adyen.com who is our account manasger [08:56:44] manager* [08:58:27] done [09:05:32] great damilare, just writing about switch off the campaign and adyen [09:06:14] ccool [09:08:14] cool - you are on it - was just wondering [09:10:40] ok damilare let's disable the active campaign [09:11:21] I'll do that. do you wanna disable adyen locally via LocalSettings and just confirm the GatewayChooser routes you to Ingenico? [09:11:26] then we can also push that up [09:11:35] sure [09:11:59] fr-tech. I'm switching off 'en6C Banner: weekly test' so we can shut down Adyen, switch over to Ingenico and confirm it works [09:13:41] thanks jgleeson! [09:15:38] gateway chooser routes to ingenico, so I'd be pushing [09:16:16] great. push that config change up and I'll quickly review and you can deploy that [09:18:05] done jgleeson [09:18:26] jgleeson: I disabled the 6C and bundle campaigns as they are both on Adyen. We probably can't re-route the bundle to Ingenico because of Israel [09:19:27] pcoombe: I just disabled C2223_en6C_dsk_FR and C2223_en6C_m_FR [09:20:01] jgleeson: I don't think it worked? Not in the logs, so I went ahead and did it myself https://meta.wikimedia.org/wiki/Special:CentralNoticeLogs [09:21:01] ah I wonder if I did it at the same time [09:21:12] anyway thanks for switching them off [09:22:03] damilare: just checking your config change [09:24:23] Thanks a lot jgleeson [09:24:33] I'm just wondering if we should also update the config in extension.json [09:26:51] You mean this property: DonationInterfaceGatewayPriorityRules? [09:27:31] yeah we added Israel to the rules for adyen and ingenico [09:27:46] pcoombe: does Ingenico not support Israel / [09:27:48] ? [09:28:13] damilare: let's push out the current change and we can follow up on the priority rules [09:28:20] wanna deploy payments? [09:28:42] IL is currently in the list of countries for Adyen/Ingenico [09:28:49] sure [09:30:27] yeah damilare but pcoombe's comment makes me wonder if we need to take it out [09:31:44] well I see Israel on the list of supported countries for ingenico https://support.direct.ingenico.com/en/payment-methods/visa/countries-and-currencies [09:33:24] That was for Visa cards only, this is a more descriptive list by country https://support.direct.ingenico.com/en/payment-methods/visa/countries-and-currencies [09:35:05] ok let's leave that as it is. Kristie is also mentioning on slack that a french email just wen tout [09:35:08] went out* [09:40:56] I think Israel was maybe a translations issue rather than actually being able to process payments? [09:40:59] fwiw I just made a French donation through adyen a few minutes ago and it worked fine [09:42:02] that's confusing [09:42:06] we just disabled adyen [09:44:22] my donation was at 9:26 so probably before you disabled it [09:45:12] Does France work on Ingenico then? I thought it was somewhere we had to use Adyen, as we've been doing for so long [09:48:19] it seems to be loading but there's something odd [09:48:27] when I click through this link https://donate.wikimedia.org/w/index.php?title=Special:LandingPage&country=FR&uselang=en-gb&utm_medium=spontaneous&utm_source=fr-redir&utm_campaign=spontaneous [09:48:41] I get sent to an Ingenico form [09:48:50] but there is a query param which says gateway=adyen [09:49:25] I'm just checking locally the impact of that [09:50:51] yeah donatewiki is setting that. If it still ends up at the right gateway though I guess that's fine? [10:03:09] Ingenico doesn't have any EUR test card details!?!?1 https://epayments.developer-ingenico.com/documentation/testcases/detail/visa [10:03:15] surely that can't be right [10:04:02] ah they have a multi currency card [10:04:55] hah, looks like it's a choice of US, Canada, or... Russia?! [10:06:17] indeed [10:06:34] ok that trxn worked fine regardless of the gateway=adyen param [10:06:47] phew [10:06:51] I just wanted to make sure it didn't get picked up anywhere we write stuff [10:09:36] Thanks jgleeson, I was afraid it’d be picked up in the js. But the form updates the #gateway field correctly. Not from the url parameter [10:54:23] looks like the failmail backlog has caught up. [10:54:59] I'm gonna drop off now, have a good week damilare|brb and fr-tech. Catch you all when I get back [10:55:06] also thanks pcoombe ! [10:55:09] bye for now [10:57:22] Thanks again Jack! [11:00:17] I double checked the bundle banners, and they are all successfully pointing to Ingenico now. Not putting them back up just yet though [11:00:39] damilare|brb: Did we hear anything back from Adyen? [11:03:38] No response yet pcoombe [11:37:01] damilare: Rakhi pointed out a now resolved Carte Bancaire issue mentioned in https://www.adyen.com/status. Does that seem like our issue i.e. were we only seeing timeouts on Carte Bancaire? [12:27:28] fr-tech: How do we feel about switching processing back to Adyen? [13:22:06] sorry pcoombe, I didn't get a notification for this [13:26:01] I just sent a reply to your email, I can't tell what the donor card type was from the logs. I can only see that the failed Donations were from FR and it was due to the checkout request timing out [13:35:00] Hi fr-tech [13:35:31] damilare: it looks like adyen did respond [13:36:26] ohh hmm, that's weird I can't find it in my mail [13:37:38] Gmail has it stacked for me on the ayden support thread [13:40:15] ohh I just saw the response, thanks cstone [13:40:23] had to sign into Adyen to find it [13:40:32] Not sure why it didn't make it to my email though [13:41:08] Yeah hmm I see it addressed to you but maybe it stacked weirdly 🤔 [14:41:09] heads up fr-tech, I'd be enabling Adyen shortly [14:41:21] sounds good damilare [14:41:33] damilare: thx! [14:41:51] just out of curiosity, this is via gateway chooser settings, correct? [14:51:53] Hey fr-tech: I might miss standup this morning. Got a potential COVID situation here, and I'll need to do some running around to get everyone tested. I don't have any symptoms myself, so I should be back at the keyboard a little later today. [14:52:20] XenoRyet: good luck, take care, hope everyone's well!!!!! [14:55:56] Sorrry to hear and hope everyone's safe XenoRyet. [15:52:04] (03CR) 10Cstone: "recheck" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/810394 (https://phabricator.wikimedia.org/T303425) (owner: 10Cstone) [16:02:08] (03PS6) 10Wfan: Add Braintree recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/810394 (https://phabricator.wikimedia.org/T303425) (owner: 10Cstone) [16:03:33] (03CR) 10Wfan: [C: 03+2] "Tested fine~" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/810394 (https://phabricator.wikimedia.org/T303425) (owner: 10Cstone) [16:05:41] (03Merged) 10jenkins-bot: Add Braintree recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/810394 (https://phabricator.wikimedia.org/T303425) (owner: 10Cstone) [16:07:40] damilare: thanks so much for your clear and effective communication re adyen today. [16:09:15] My pleasure greg-g [17:07:59] Ditto ^ thank you Dami and Jack too. [17:33:20] is Tech Talk got cancelled or postponed for Braintree meeting? [17:33:38] nm, I see Christine haha [17:42:27] 10Fundraising Sprint NaN is a Number, 10fundraising-tech-ops, 10FR-Braintree-Integration: Add the Braintree audit dirs to production via puppet - https://phabricator.wikimedia.org/T314521 (10Dwisehaupt) @AnnWF I have standardized the puppet config to create the audit paths needed for civi. You can use the fo... [17:47:34] 10fundraising-tech-ops: New remote log files for payments-wiki - https://phabricator.wikimedia.org/T314819 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt This has been deployed. ` [frack::puppet] a62920ea New central log files for braintree_gateway and client errors ` [17:52:19] PROBLEM - check_puppetrun on frdev1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/spool/audit] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [17:57:19] PROBLEM - check_puppetrun on frdev1002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/spool/audit] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [17:58:22] ^^^ that was related to the push i just did. fixed for the frdev role and pushed out. [18:02:16] RECOVERY - check_puppetrun on frdev1002 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [18:11:22] 10Fundraising Sprint NaN is a Number, 10fundraising-tech-ops, 10FR-Braintree-Integration: Add the Braintree audit dirs to production via puppet - https://phabricator.wikimedia.org/T314521 (10Dwisehaupt) I have also added the queue consumer dir and deployed that. ` /var/log/fundraising/braintree ` [18:23:00] (03PS1) 10Damilare Adedoyin: Add missing supported countries in Paypal present in Braintree [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/822131 (https://phabricator.wikimedia.org/T314500) [19:40:46] (03CR) 10Cstone: [C: 03+2] "More islands!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/822131 (https://phabricator.wikimedia.org/T314500) (owner: 10Damilare Adedoyin) [19:43:34] (03Merged) 10jenkins-bot: Add missing supported countries in Paypal present in Braintree [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/822131 (https://phabricator.wikimedia.org/T314500) (owner: 10Damilare Adedoyin) [19:58:29] 10Fundraising Tech - Chaos Crew, 10fr-donorservices, 10Patch-For-Review: Adyen donors think they are recurring July 2022 - https://phabricator.wikimedia.org/T313854 (10greg) a:03Ejegg [20:14:44] 10Fundraising-Backlog: Review Cadence of Audit Files for Manually settled Adyen Donations - https://phabricator.wikimedia.org/T314753 (10EMartin) @ejegg. @krobinson As expected, Finance wants to stay with weekly settlement. I did inquire with Adyen and they advise the following as it relates to the manually set... [20:16:14] Fr-tech: Thea just let me know that Mariana will be the point person for monitoring the France email send tomorrow. Not that we expect anything to go sideways again, but if it does ping her on slack, and she may check in here on our channel as well to see if things are going smoothly. [20:16:35] thanks XenoRyet [20:24:10] greg-g: I'm assuming the meeting in a few mins w civi is on? I can't see Tim online & I know it wasn't on last he tried to attend so I'm not sure if he thinks it is - I just tried to ping him [20:26:03] Either way, it was mostly a check-in regarding contract status [20:29:32] ah ok - well I'll jump in & see who shows up [20:55:48] 10Fundraising Sprint Machine Unlearning, 10Fundraising Sprint NaN is a Number, 10Fundraising-Backlog, 10FR-Braintree-Integration: add braintree audit parse cron job - https://phabricator.wikimedia.org/T314317 (10AnnWF) https://civicrm.wikimedia.org/admin/config/wmf_audit/braintree_audit now have /var/spool... [21:27:21] 10Fundraising Sprint Machine Unlearning, 10Fundraising Sprint NaN is a Number, 10Fundraising-Backlog, 10FR-Braintree-Integration: add braintree audit parse cron job - https://phabricator.wikimedia.org/T314317 (10AnnWF) wfan@frpm1001:/srv/localsettings/process-control/civi1001$ cat braintree_report.yaml ---... [21:34:55] (03PS1) 10Ejegg: DO NOT MERGE: commit vendor/bin typechanges [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/822176 (https://phabricator.wikimedia.org/T314826) [21:39:29] XenoRyet: oops, totally spaced on our meeting [21:39:50] hopping into the call right now [21:40:33] Oh, no worries. I was figuring we'd forego check-ins this week with respect for meeting light week. [21:40:45] ah ok, sounds good [21:41:00] Though while I've got you here, did you see my question in the other channel, if it's a quick one to answer. [21:41:14] yeah, the applePay check is in the banners and I think on donatewiki [21:41:34] Ah, that makes sense. [21:41:45] I think it just looks for some JS things that are built into Safari and no other browser [21:41:58] then for GPay there is no device specificity [21:42:05] it's all just connected to your google account [21:42:08] like with Amazon [21:42:39] Good times, thanks [21:44:04] 10Fundraising Tech - Chaos Crew, 10Patch-For-Review: Adjust drush wrapper path for bin/symlink behavior of recent Composer versions - https://phabricator.wikimedia.org/T314826 (10Ejegg) OK, it looks like it's deleting a symlink from vendor/bin/drush to vendor/drush/drush. It is replacing that symlink with a PH... [21:44:20] dwisehaupt: this is the actual change that breaks things on prod: https://gerrit.wikimedia.org/r/822176 [21:44:35] I guess I'll temporarily pull it onto staging to see what the fix would be [21:44:42] fr-tech: don [21:44:54] 't be surprised if drush is broken on staging for a bit [21:55:06] oops, gotta run an errand. I won't break staging drush just yet [22:13:39] ejegg|afk: interesting. i don't see anywhere in our config that we manage those drush symlinks. digging further in case it's masked in some abstraction. [22:40:40] ejegg|afk: i think it's rsync that's getting in our way here. trying to figure out how to test this on staging. [22:41:37] (making the assumption that this is happening during the rsync_blasting phase and not the code_update stage.)