[00:04:31] ejegg: I can still try to finish the CR in a bit, it looked pretty simple on first read-through [00:04:59] if you're proposing deploying it today, until what time would you have space to do so? [00:05:02] thanks AndyRussG, that would be great! [00:05:15] another couple of hours, i guess [00:06:30] ejegg: ok [00:07:19] ohhhh, i just saw an email from Haley yesterday talking about test scheduling 'November 6C Pre-testing Schedule' [00:07:35] and she;s saying a 1 hr en6C Apple Pay test on Nov 2 [00:07:48] so I don't know if that's the first they'd want to do [00:08:09] I remember someone saying they hoped to have it for the end of the france campaign [00:09:08] and the france campaign ENDS nov 2 [00:09:16] so I assumed they would want to try it there ASAP [00:10:02] ejegg: heheh prol'ly a corrrect assumption eh [00:10:50] oh, I'll deploy those spanish monthly convert thankyous [00:11:37] oh wait, they're already out [00:11:38] cool [00:12:14] 10Fundraising Sprint Technical debt house of horrors, 10Fundraising-Backlog: Production of Upsell Thank You email in Spanish-LATAM - https://phabricator.wikimedia.org/T292542 (10Ejegg) @CDenes_WMF these are live in production now. [00:16:07] (03PS2) 10Ejegg: Form chooser: only count submethods for selected method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732468 (https://phabricator.wikimedia.org/T293935) [00:16:09] (03PS1) 10Ejegg: Revert "Revert FormSettings changes" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732841 [00:28:28] (03PS1) 10Ejegg: Apple Pay: Validate the validation_url [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732843 [00:30:42] (03PS1) 10Ejegg: Apple Pay: allow rate-limiting session start API [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732845 [00:33:45] ejegg: oh yeah - I pushed that out [00:33:51] thanks eileen ! [00:33:59] it seemed safe to include it [00:36:16] yep, totally. I just pushed the ticket into done and pinged cdenes [00:43:48] (03PS2) 10Ejegg: Apple Pay: Validate the validation_url [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732843 [00:43:50] (03PS2) 10Ejegg: Apple Pay: allow rate-limiting session start API [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732845 [01:26:20] wow there are a ton of DI patches in review [01:27:07] ejegg: so sitting down now to look at this... I guess we're talking about https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/732841 and https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/732468, right? [01:31:45] ok so https://localhost:9001/index.php?title=Special:GatewayFormChooser&country=US¤cy=USD&payment_method=cc&amount=3.14 does now take me to Ingenico [01:33:34] also same on master, but that's expected because the form settings got reverted [01:43:34] ok confirmed that reverting the form chooser revert on top of master takes us to ingenico [01:43:44] and the fix fixes it [01:44:00] *reverting the form settings revert [01:46:39] hi AndyRussG [01:46:50] hellosie-ohs [01:46:53] almost there eh [01:47:02] yep yep, reverting the revert [01:47:53] then there were these two lil followups, one super trivial to center a button: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/732100 [01:48:13] and one to show a message instead of just a broken form when the donor is in an unsupported browser: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/732104 [01:49:10] ejegg: yeah I did see those... really don't think I can get to those additional ones tonight, I'm afraid [01:49:17] k [02:11:19] ejegg: I assume you're pretty confident this won't produce any unexpected effects on links currently in the wild, either in banners or relatively recent e-mails? [02:12:03] pretty confident, yeah [02:13:19] ok cool [02:13:28] right, apple is not a submethod but a whole method unto itself [02:14:02] so when the payment_method is set to cc, we're no longer even looking at the added apply bits [02:14:54] yep, that's the idea here [02:16:06] so adyen-us still wins, or ties for first place with cc-vmad, whatever that is [02:16:51] ah old ingenico looks like [02:17:14] I guess if it is, we should set its selection_weight to 0 [02:17:25] ties for first in # of payment methods, but has a lower weight [02:17:30] no need to zero it out [02:17:47] ah no rather new ingenico [02:18:32] I guess cc-ingenico is the new one [02:18:34] ? [02:18:57] ah no, we should get rid of that [02:19:17] well, we should get rid of this whole file [02:19:26] but the cc-ingenico is not at all used [02:20:00] (03CR) 10AndyRussG: [C: 03+2] "cleaning up those apply apply bits ;)" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732468 (https://phabricator.wikimedia.org/T293935) (owner: 10Ejegg) [02:20:28] (03CR) 10AndyRussG: [C: 03+2] "oky now we have the apply crumbs scooper, this can stayyyyyy ;)" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732841 (owner: 10Ejegg) [02:20:38] thanks AndyRussG ! [02:20:42] yw :) [02:21:34] k now to walk the dog, pick up Sofi from her boyfriend's place, have "the talk" level 2 with her, finish helping Ceci study for math, cook empanadas for lunch tomorrow, and cook a few more things that will go bad if I don't [02:21:35] (03PS1) 10Ejegg: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/732849 [02:21:37] I kid u not [02:21:38] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/732849 (owner: 10Ejegg) [02:21:54] thanks 4 the patches and 4 deploying! [02:21:56] whew, hope you get some sleep tonight! [02:22:03] (03Merged) 10jenkins-bot: Form chooser: only count submethods for selected method [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732468 (https://phabricator.wikimedia.org/T293935) (owner: 10Ejegg) [02:22:41] (03Merged) 10jenkins-bot: Revert "Revert FormSettings changes" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/732841 (owner: 10Ejegg) [02:22:43] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/732849 (owner: 10Ejegg) [02:23:45] (03PS1) 10Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 [02:24:05] (03CR) 10Ejegg: [C: 03+2] Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 (owner: 10Ejegg) [02:24:48] (03CR) 10jerkins-bot: [V: 04-1] Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 (owner: 10Ejegg) [02:25:44] thanks!! yeah it'll be okok [02:36:04] (03PS2) 10Ejegg: Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 [02:43:27] (03CR) 10Ejegg: [C: 03+2] Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 (owner: 10Ejegg) [02:46:29] (03Abandoned) 10Ejegg: Set exchange rate for bitpay test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/702779 (owner: 10Ejegg) [02:53:41] (03Merged) 10jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/732850 (owner: 10Ejegg) [02:59:52] !log updated payments-wiki from 088a8cda1e to 6e810fb401 [02:59:57] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:20:40] 10Fundraising Sprint Technical debt house of horrors, 10Fundraising-Backlog, 10Fr-endowment, 10MediaWiki-extensions-DonationInterface, 10MW-1.38-notes (1.38.0-wmf.6; 2021-10-26): Update "Other ways to give" link on Endowment form - https://phabricator.wikimedia.org/T293877 (10Ejegg) Thanks for the patch... [08:45:32] 10Fundraising Sprint Technical debt house of horrors, 10Fundraising-Backlog, 10Fr-endowment, 10MediaWiki-extensions-DonationInterface, 10MW-1.38-notes (1.38.0-wmf.6; 2021-10-26): Update "Other ways to give" link on Endowment form - https://phabricator.wikimedia.org/T293877 (10Pcoombe) 05Open→03Resolve... [14:53:03] 10Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: Upgrade Superset to 1.3.1 - following Product install - https://phabricator.wikimedia.org/T294130 (10EYener) [16:17:05] 10fundraising-tech-ops: upgrade fundraising Superset instance to 1.3.1+ - https://phabricator.wikimedia.org/T292294 (10Jgreen) [16:17:07] 10Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: Upgrade Superset to 1.3.1 - following Product install - https://phabricator.wikimedia.org/T294130 (10Jgreen) [16:17:33] 10Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: Upgrade Superset to 1.3.1 - following Product install - https://phabricator.wikimedia.org/T294130 (10Jgreen) 05duplicate→03Resolved [16:18:54] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops: server error seen when attempting to export long search kit list - https://phabricator.wikimedia.org/T287995 (10Jgreen) 05Open→03Resolved [17:55:36] 10fundraising-tech-ops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:(Need By: TBD) rack/setup (4) fundraising hosts - https://phabricator.wikimedia.org/T289812 (10Dwisehaupt) 05Resolved→03Open @Cmjohnson I believe the network config was swapped for all the hosts. When attempting to build them I see that the pay-l... [17:59:58] 10Fundraising-Backlog: Declines due to API settings - https://phabricator.wikimedia.org/T294059 (10EMartin) //From Ingenico: The logs I received from my team were not very clear so I couldn’t distinguish if you are submitting the “hosted checkout” API or the “create payment” API. So based on your teams comments... [18:31:03] damilare: remember when you said "low hanging fruit" tickets, well the work implied in the ticket above doesn't get any lower, it's practically walking!!! :) [18:31:31] this task https://phabricator.wikimedia.org/T294059 [18:44:38] 10Fundraising-Backlog, 10FR-Tech-Analytics: Low Traffic Test Difference in impressions between event_sanitized.centralnoticebannerhistory and pgehres.bannerimpressions - https://phabricator.wikimedia.org/T294145 (10JMando) [19:30:20] jgleeson|afk: haha thanks, taking a look [20:12:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 25 [=5],ipset_error 6 [=1],minFraud_endpoint_critical 8 [=1] [20:17:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 18 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:20:08] hmm fr-tech looks like a minor comms blip? ^ [20:22:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 27 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:22:35] jgleeson: there's an ongoing network incident [20:23:52] oh thanks Spookreeeno [20:24:15] You might wanna watch -operations [20:24:29] I don't really understand the issue [20:24:29] thanks for the heads up Spookreeeno [20:26:56] jgleeson: yeah it's a fiber cut. we should be ok to ride it out. [20:27:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 19 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:29:14] jgleeson, dwisehaupt: if you have traceroutes, they can be passed on [20:31:52] looks like they are recovering already for us. no need. [20:32:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 25 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:32:28] the failmail is manageable at this point [20:32:48] we'll still see this alert for a little since it pulls log lines over a period of time so it takes a little for the to age out. [20:37:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 16 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:42:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 4 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 11 [=5],ipset_error 10 [=1],minFraud_endpoint_critical 8 [=1] [20:42:19] Should be back [20:46:51] thx so much Spookreeeno jgleeson dwisehaupt [20:47:02] Actually not [20:47:07] Change rolled itself back [20:47:07] oh hmmm [20:47:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 1 [=1],Yubico_endpoint_critical 1 [=1],cURL_error 5 [=5],ipset_error 3 [=1],minFraud_endpoint_critical 4 [=1] [20:47:15] fr-tech last call for observations about Adyen meeting time next week [20:48:20] ahh I just noticed ej egg is busy at the last proposed time [20:48:27] I guess I'll e-mail rather with a bunch of options [20:48:57] I'm ok with any time. I'm not strictly necessary for that one anyway so don't schedule around me. [20:52:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Astropay-DLocal_endpoint_critical 1 [=1],cURL_error 8 [=5],minFraud_endpoint_critical 2 [=1] [20:53:54] Should be fine now [20:54:08] XenoRyet: oki thanks [20:54:24] I think ej egg should be there since he's been looking at those 905 errors [20:57:11] RECOVERY - check_log_messages on frav1002 is OK: OK [22:29:13] (03PS8) 10Cstone: Handle RECURRING_CONTRACT ipn notifcations for iDEAL recurring [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/731188 (https://phabricator.wikimedia.org/T284606)