[00:05:15] (03CR) 10jerkins-bot: [V: 04-1] Remove extra space [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708607 (owner: 10Eileen) [00:08:45] (03Merged) 10jenkins-bot: Re-apply WMF patches [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/706183 (owner: 10Eileen) [00:08:47] (03Merged) 10jenkins-bot: Move archive thank you test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/701221 (https://phabricator.wikimedia.org/T285446) (owner: 10Eileen) [00:10:47] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708607 (owner: 10Eileen) [00:24:02] (03Merged) 10jenkins-bot: Remove extra space [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708607 (owner: 10Eileen) [00:24:04] (03Merged) 10jenkins-bot: Remove code to install extensions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/705787 (https://phabricator.wikimedia.org/T280595) (owner: 10Eileen) [00:50:27] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/708613 [01:03:34] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/708613 (owner: 10Eileen) [01:08:39] !log civicrm revision changed from 739c936298 to 158ed65e00, config revision is 6011d9c471 [01:08:47] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:42:39] (03PS1) 10Eileen: Add CA recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708614 (https://phabricator.wikimedia.org/T285375) [01:43:13] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [01:54:04] (03PS1) 10Eileen: Add Danish recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708615 (https://phabricator.wikimedia.org/T285375) [01:55:45] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [02:24:23] (03PS1) 10Eileen: Add German recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708620 (https://phabricator.wikimedia.org/T285375) [02:24:56] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [02:37:32] (03PS1) 10Eileen: Add french recurring translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708621 (https://phabricator.wikimedia.org/T285375) [02:38:14] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [03:08:08] (03PS2) 10Eileen: Add German recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708620 (https://phabricator.wikimedia.org/T285375) [03:09:41] (03PS2) 10Eileen: Add CA recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708614 (https://phabricator.wikimedia.org/T285375) [03:16:08] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) @CDenes_WMF the first couple of these (es & ro) are on live - you can tes... [03:17:55] (03PS2) 10Eileen: Add Danish recurring fail translation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708615 (https://phabricator.wikimedia.org/T285375) [03:39:38] (03PS1) 10Eileen: Add Latin American recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708623 (https://phabricator.wikimedia.org/T285375) [03:42:43] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [03:58:30] (03PS1) 10Eileen: Add fr_CA recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708624 (https://phabricator.wikimedia.org/T285375) [03:59:01] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [04:22:00] (03PS1) 10Eileen: Add hu recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708625 (https://phabricator.wikimedia.org/T285375) [04:22:32] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [06:43:49] 10fundraising-tech-ops: fix data snapshots from fundraising SEIM server to logger/archive servers - https://phabricator.wikimedia.org/T287503 (10ayounsi) [06:43:51] 10fundraising-tech-ops, 10DC-Ops, 10SRE, 10ops-codfw: (Need By: TBD) rack/setup/install frdb2003.frack.codfw.wmnet - https://phabricator.wikimedia.org/T281177 (10ayounsi) [06:43:54] 10fundraising-tech-ops: deploy Let's Encrypt certificates for additional fundraising services - https://phabricator.wikimedia.org/T280034 (10ayounsi) [07:04:12] (03PS1) 10AndyRussG: Add test-only config $wgDonationInterfaceMockRedisForCoreTests [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708715 (https://phabricator.wikimedia.org/T287599) [07:04:53] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10Patch-For-Review: Fix or disable DonationInterface test - https://phabricator.wikimedia.org/T287599 (10AndyRussG) [07:06:57] (03CR) 10jerkins-bot: [V: 04-1] Add test-only config $wgDonationInterfaceMockRedisForCoreTests [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708715 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [07:31:12] (03Abandoned) 10AndyRussG: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/706923 (owner: 10Ejegg) [07:31:52] (03Abandoned) 10AndyRussG: DO NOT MERGE Testing CI stuff [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708345 (owner: 10AndyRussG) [07:32:50] (03Abandoned) 10AndyRussG: Merge feature branch 'adyen' into master [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708333 (owner: 10AndyRussG) [07:33:28] (03Abandoned) 10AndyRussG: Merge feature branch 'adyen' into master [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/706905 (owner: 10Ejegg) [07:34:42] (03Abandoned) 10AndyRussG: Update SmashPig version requirement [extensions/DonationInterface] (adyen) - 10https://gerrit.wikimedia.org/r/706904 (owner: 10Ejegg) [07:50:19] (03Abandoned) 10AndyRussG: Add test-only config $wgDonationInterfaceMockRedisForCoreTests [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708715 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [07:54:45] (03PS1) 10AndyRussG: Use testing SmashPig context for non-DonationInterface tests, too [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) [07:57:44] (03CR) 10jerkins-bot: [V: 04-1] Use testing SmashPig context for non-DonationInterface tests, too [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [08:00:39] (03PS2) 10AndyRussG: Use testing SmashPig context for non-DonationInterface tests, too [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) [08:02:17] (03Abandoned) 10AndyRussG: Update composer.lock and vendor for SmashPig 0.7.0 [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/706931 (owner: 10Ejegg) [09:49:06] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 10 others: Some of our CI tests are not running - https://phabricator.wikimedia.org/T285446 (10Eileenmcnaughton) The buildkit main... [11:34:10] 10Fundraising-Backlog: Amount mis-match Ingenico vs Civi in Stop Fraud report - https://phabricator.wikimedia.org/T287668 (10RKumar_WMF) [12:12:59] (03CR) 10Jgleeson: "Real nice find! This looks like a perfect way to fix the issue." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [13:22:12] howdy fr-tech! [13:24:12] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 10 others: Some of our CI tests are not running - https://phabricator.wikimedia.org/T285446 (10jgleeson) Thanks for updating that... [14:00:06] Hi jgleeson [14:00:13] Woo for tests passing ! [14:03:57] yep yep [14:06:55] I'm gonna xdebug through a test run with that set to see what it does around the redis stuff [14:07:07] but great a find by AndyRussG eitherway cstone [15:00:26] 10fundraising-tech-ops: fix data snapshots from fundraising SEIM server to logger/archive servers - https://phabricator.wikimedia.org/T287503 (10Jgreen) 05Open→03Resolved p:05Triage→03Medium [15:45:44] hi fr-tech! France pre-test going live in 15 minutes :) [15:45:54] thanks haley_ ! [15:47:46] thanks haley_ ! [15:51:55] (03CR) 10Jgleeson: [C: 03+2] "This works great. I followed the code through and can see the SequenceGenerater object is now using SmashPig\Core\SequenceGenerators\SqlSe" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [15:55:19] (03Merged) 10jenkins-bot: Use testing SmashPig context for non-DonationInterface tests, too [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [15:56:11] (03CR) 10Jgleeson: "Just for reference, it looks like this is the test config which is used when setting that test context https://github.com/wikimedia/wikim" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708720 (https://phabricator.wikimedia.org/T287599) (owner: 10AndyRussG) [16:04:46] 10fundraising-tech-ops: install/configure frdev1002 - https://phabricator.wikimedia.org/T285892 (10Jgreen) [16:05:46] 10fundraising-tech-ops: install/configure frdev1002 - https://phabricator.wikimedia.org/T285892 (10Jgreen) [16:05:48] 10fundraising-tech-ops: Fundraising environment support for Debian 11 (Bullseye) - https://phabricator.wikimedia.org/T277184 (10Jgreen) [16:26:51] 10Wikimedia-Fundraising-Banners: Mobile small nag RML doesn't depend on frb-rml-enabled/frb-rml-disabled - https://phabricator.wikimedia.org/T287626 (10Pcoombe) @jbolorinos-ctr Can you double check this fix looks fine? https://en.m.wikipedia.org/wiki/NASA?banner=pcoombe_T287626 Expected behaviour: - nag and ma... [16:28:18] 10Fundraising-Backlog: Amount mis-match Ingenico vs Civi in Stop Fraud report - https://phabricator.wikimedia.org/T287668 (10DStrine) We don't usually get reports like this. We hav access to the ingenico admin panels and reports but we usually don't investigate this stuff. Adding @EMartin as she might know how... [16:38:09] XenoRyet: you around? [16:38:55] oh nm [16:49:44] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart: Adyen Checkout: Investigate possible changes to allow simultaneous deployment of old and new integrations - https://phabricator.wikimedia.org/T287604 (10DStrine) [17:40:32] ejegg: the new code is merged and deployed but old Adyen is rolled back to via config [17:41:05] ok [17:41:37] ahh, so we needed to update the allowlist of pages for the adyen API [17:41:42] d'oh [17:42:00] OK, so that's another thing we should replicate locally I guess - lock down all access except for that allowlist [17:43:07] ejegg: it didn't go live due to network traffic blocks [17:43:25] a bunch of issues came up during the deploy mostly unknown unknowns and we ran outta time [17:43:30] ahhh darn [17:45:42] ok, catching up on email [17:45:51] and I see a couple other bumps on the way there [17:46:13] interesting that only the Checkout gateway was hitting the contribution tracking sequence issue in the core test [17:46:16] we had an etherpad somewhere [17:46:37] https://etherpad.wikimedia.org/p/adyen-checkout-deploy-notes [17:47:28] ejegg: no CI was running on the adyen branch, only on the master branch [17:47:40] ohhhh lawd [17:47:45] yerp [17:47:56] well fun [17:48:05] yeah that was a surprise! [17:48:10] good find by AndyRussG on that one [17:48:14] 10Fundraising-Backlog, 10fundraising-tech-ops: fill out fr-tech analytics requirements spreadsheet. - https://phabricator.wikimedia.org/T287515 (10Jgreen) [17:48:26] ahhhhhhh thanks good find on the queues issue with the tests jgleeson [17:48:27] weird, I was almost certain i'd seen the new DI/adyen tests fail in CI [17:48:49] it could have been a recent CI change too, I didn't dig too deeply there [17:49:24] so the three blockers were: 1) figure our why CI was erroring out 2) once we got CI working, fix the tests (including core tests) that had not been running, and 3) network rules preventing payments boxes from calling Adyen [17:49:28] in that order [17:49:36] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/702028 [17:49:51] also, you missed the best longest 8-hour video call ever on Tuesday [17:50:09] well, the full jenkins logs are now gone, but ^^^ def had some composer test failures in the first patch sets [17:50:16] whoa, what a marathon! [17:50:25] AndyRussG: we also forgot that we needed a new API key for live and there was the fun with the new Adyen API URLs [17:50:31] arr [17:51:16] but to be fair, we wouldn't have hit any of those issues until we tried to deploy so they were always gonna trip us up [17:51:30] the tuesday call was all about this deploy? [17:51:33] on the whole it was a great team effort to work through them [17:53:04] yeah monday we got SmashPig out just with the missing keys and URL issues then when I +2d the first DI patch it got confusing [17:53:06] ejegg: adyen now has three different data centres that you can point to when using their API via different URLs [17:53:15] oh interesting [17:53:26] US/Europe/Australia [17:53:49] hmm, i'm trying to figure out why the payments server would have been trying to hit the WSDL endpoint [17:53:51] and in the console it defaults to Australia kinda without telling you [17:54:10] ohhh man [17:54:15] that's a pretty silly default [17:54:21] yerp [17:56:02] ejegg: so we missed the window for testing and they're using old Adyen, and the e-mail schedule is now super-squisehd and hard to find breaks in [17:56:17] oh damn [17:56:26] so one idea now is to make both old and new work simultaneously on prod, selectable via URL [17:56:47] so that the new one can be tested at its own pace as needed while campaigns can continue to reach the old one [17:56:59] so the banners will steer people directly to Special:AdyenCheckoutGateway? [17:57:43] I guess, or e-mails, and then when we're ready will switch some default for upstream in the pipeline, something like that [17:57:46] hmm, that'll require a bit of hackiness around the wgDonationInterfaceGatewayAdapters [17:58:03] ejegg: sure, so here's the task to figure that out: https://phabricator.wikimedia.org/T287604 [17:58:41] bummer, more test code in the deployed path [17:58:41] ejegg: WRT to the WSDL. That was getting requested at the point of the class being instantiated. The silly soapClient thingy pulls in down right away [17:59:09] ohhhhh right [17:59:19] well shoot, we should lazy-instatiate that [18:00:42] i'mma just do that patch now [18:05:47] fr-tech I have to miss the office hours meeting, or most of it, hopefully it'll be recorded? [18:06:35] (03PS1) 10Ejegg: Lazy-instantiate Adyen SOAP client [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/708806 [18:47:35] fr-tech should we discuss the Applepay requirement to have mobile native payments? We're kinda blocked until we make a decision on that. [18:47:43] either on irc or a call? [18:47:57] appreciate it's short notice! we could do it tomorrow [18:50:07] just checking the docs to see what ACTUAL changes we'd need if we chose the "our certificate" route. We might be able to reuse most of what we have and carry on as planned. [18:50:21] choose* [18:50:30] if we chose to choose [18:51:16] jgleeson: and ejegg : so Elliott was in meetings with me and the IOS team like a year ago. when we were doing investigations I said quite often this needs to work in the app. That is still the case. whatever path you take needs to work for the iOS team [18:51:42] if that mean taking another approach, then that's what has to happen [18:55:55] thanks dstrine [19:00:44] ok fr-tech so it looks like at least from the docs that there is no actual changes to the code that implements Apple Pay when you use either Adyen's certificate or our own. The main differences that come with pursuing the "own certificate" route are mainly in the set up and will probably mean we either need to get full access to the existing WMF Apple developer account (currently the iOS folks [19:00:46] have done stuff for us on request) or set up our own, so we can carry out the set up steps needed. [19:01:56] the steps for doing stuff on our own are listed here, I'll also put these on the ticket https://docs.adyen.com/payment-methods/apple-pay/enable-apple-pay#page-introduction [19:03:26] 10Fundraising Sprint Ketchup Flume Ride, 10Fundraising Sprint Mandatory corn dogs, 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, and 5 others: Handle Apple Pay submission in Adyen Components - https://phabricator.wikimedia.org/T282596 (10jgleeson) Instructions on how to set up Apple P... [19:05:28] catch you all tomorrow o/ [19:07:32] fr-tech I'm checking out early to nap then go get some real food, see everyone tomorrow [19:08:48] cstone: cya, enjoy :) [19:08:50] jgleeson|away: cya! [19:18:19] looks like an expensive query is running on frdb right now. [19:18:38] by User : 1100 and has been running 14981 seconds. [19:19:03] it has the load elevated on frdb1004 pretty significantly. [19:22:18] i've captured the query and am going to kill the process now. [19:26:21] killed off a few more queries that were backed up behind it which had been running for 1200+ seconds too. [19:26:34] they were dupes of a similar query and i've captured that one too. [19:58:50] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart: Adyen Checkout: Investigate possible changes to allow simultaneous deployment of old and new integrations - https://phabricator.wikimedia.org/T287604 (10Ejegg) Blockers: * We use a single gateway code 'adyen' for old and new. $wgDonationInerfa... [20:59:45] (03PS1) 10AndyRussG: PHPUnit tests: reset globals and use salt in API tests [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/708645 (https://phabricator.wikimedia.org/T266088) [21:01:21] (03CR) 10AndyRussG: "> Patch Set 2:" [extensions/DonationInterface] (adyen) - 10https://gerrit.wikimedia.org/r/703258 (https://phabricator.wikimedia.org/T266088) (owner: 10AndyRussG) [21:01:57] 10Wikimedia-Fundraising-Banners: Mobile small nag RML doesn't depend on frb-rml-enabled/frb-rml-disabled - https://phabricator.wikimedia.org/T287626 (10jbolorinos-ctr) Screenshot of fix applied country=US Main {F34569795} Nag {F34569799} Screenshot of fix applied country=ES Main {F34569793} Nag {F34569801} [21:09:02] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 10 others: Some of our CI tests are not running - https://phabricator.wikimedia.org/T285446 (10Eileenmcnaughton) @jgleeson we stil... [21:35:59] (03PS1) 10Eileen: Add italian recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708841 (https://phabricator.wikimedia.org/T285375) [21:40:38] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [21:48:48] (03PS1) 10Eileen: Add lv recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708843 (https://phabricator.wikimedia.org/T285375) [21:58:20] (03PS1) 10Eileen: Add Japanese recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708845 (https://phabricator.wikimedia.org/T285375) [21:58:51] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [21:59:33] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) [22:00:22] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10FR-AutoTY-Email, 10Patch-For-Review: Production of new Auto Recurring Failure Email (multilingual) - https://phabricator.wikimedia.org/T285375 (10Eileenmcnaughton) cstone - I pushed up a few more translations - we may as well get them ou...