[00:18:44] 10fundraising-tech-ops: reimage fundraising_database role servers to Bullseye - https://phabricator.wikimedia.org/T315601 (10Dwisehaupt) [14:39:41] 10fundraising-tech-ops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install frbast1002, frmon1002, frpig1002 - https://phabricator.wikimedia.org/T319460 (10Jclark-ctr) [14:40:54] 10fundraising-tech-ops, 10DC-Ops, 10SRE, 10ops-eqiad: Q1:rack/setup/install frbast1002, frmon1002, frpig1002 - https://phabricator.wikimedia.org/T319460 (10Jclark-ctr) @Jgreen these have been received is this urgent or could I wait till after fundraising to rack and cable these? [16:29:38] 10Fundraising-Backlog: Investigate why contribution tracking has utm_medium of NULL - https://phabricator.wikimedia.org/T324169 (10JMando) Not sure if it helps, but it does look like many of these do have a utm_source (from drupal.contribution_tracking): {F35848027} The data above is for the december english ca... [16:33:12] (03PS7) 10Damilare Adedoyin: [WIP] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:35:33] (03CR) 10CI reject: [V: 04-1] [WIP] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:45:54] (03PS8) 10Damilare Adedoyin: [WIP] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:51:01] (03PS9) 10Damilare Adedoyin: [WIP] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:51:44] (03CR) 10Damilare Adedoyin: "Thanks jgleeson for the initial CR. I've fixed the refactor suggestions" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:51:53] (03PS10) 10Damilare Adedoyin: IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:54:17] (03CR) 10CI reject: [V: 04-1] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:56:13] (03PS11) 10Damilare Adedoyin: [WIP] IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [16:58:42] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Bug: duplicate Adyen transactions in short sequence - https://phabricator.wikimedia.org/T324860 (10AndyRussG) [17:00:36] (03PS12) 10Damilare Adedoyin: IP Velocity Filter: Increase count after first run [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/859168 (https://phabricator.wikimedia.org/T323544) (owner: 10AndyRussG) [17:53:42] 10Fundraising-Backlog: Update post-payments monthly convert threshold - https://phabricator.wikimedia.org/T324863 (10Pcoombe) [18:10:00] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Handle the GatewayChooser error handling nicer - https://phabricator.wikimedia.org/T324706 (10greg) p:05Triage→03High [18:10:02] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Handle the GatewayChooser error handling nicer - https://phabricator.wikimedia.org/T324706 (10greg) p:05High→03Medium [18:10:04] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Bug: duplicate Adyen transactions in short sequence - https://phabricator.wikimedia.org/T324860 (10greg) p:05Triage→03Medium [18:59:55] 10Fundraising-Backlog, 10fr-donorservices: CIVI error displaying contribs for cid=11355636 - https://phabricator.wikimedia.org/T324787 (10MBeat33) This issue persists today. We're unable to help the donor in ZD #1245007 increase the amount of their recurring donation. If this could be pulled into a sprint or C... [19:08:51] 10Fundraising-Backlog, 10fr-donorservices: CIVI error displaying contribs for cid=11355636 - https://phabricator.wikimedia.org/T324787 (10greg) Weird. Adding to Chaos Crew. Luckily still just one donor, right? [19:09:06] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10fr-donorservices: CIVI error displaying contribs for cid=11355636 - https://phabricator.wikimedia.org/T324787 (10greg) [19:20:11] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10fr-donorservices: CIVI error displaying contribs for cid=11355636 - https://phabricator.wikimedia.org/T324787 (10MBeat33) Thank you, and yes just this one odd duck. [20:16:15] (03PS1) 10AndyRussG: Update Monthly Convert amounts for FY2223 en6C [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/866645 (https://phabricator.wikimedia.org/T324863) [20:16:28] cstone: ^ [20:16:37] p.s. have not smoke tested yet ............ ;) [20:20:24] cstone: oki now smoke tested locally, nothing bad deteted........ [20:20:28] *detected [20:27:59] Nice AndyRussG I have one more stop before I get back [20:30:50] cstone: ah oki no rush, I'm around, thx! [21:15:00] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10Jdlrobson) This looks like a fundraising banner! (frb is in the stack trace!) [21:16:43] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10Jdlrobson) 72,716 errors in the last 7 days - that's a huge amount! For comparison our next error has a rate of 4,621 in last... [21:16:50] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10Jdlrobson) [21:24:44] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10spatton) Hi @Jdlrobson I'll try to look into this but want to quickly state that this isn't a fundraising banner; I think some... [21:34:33] (03CR) 10Cstone: [C: 03+2] "Looks good!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/866645 (https://phabricator.wikimedia.org/T324863) (owner: 10AndyRussG) [21:36:12] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10Jdlrobson) Got it. That's very confusing but glad to hear this doesn't impact fundraising - note this banner does seem to be r... [21:36:25] (03Merged) 10jenkins-bot: Update Monthly Convert amounts for FY2223 en6C [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/866645 (https://phabricator.wikimedia.org/T324863) (owner: 10AndyRussG) [21:46:24] (03CR) 10Damilare Adedoyin: [C: 03+1] "Hi wfan, thanks for working on this. Looks good to me, just one small requested change and I'd be good to +2." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/865833 (https://phabricator.wikimedia.org/T324270) (owner: 10Wfan) [22:01:26] (03PS1) 10Cstone: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/866651 [22:02:10] (03CR) 10Cstone: [C: 03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/866651 (owner: 10Cstone) [22:02:22] thx cstone !! [22:02:28] np AndyRussG [22:02:52] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/866651 (owner: 10Cstone) [22:04:29] (03PS1) 10Cstone: Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/866652 [22:13:30] (03CR) 10Cstone: [C: 03+2] Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/866652 (owner: 10Cstone) [22:21:15] (03Merged) 10jenkins-bot: Update DonationInterface submodule [core] (fundraising/REL1_35) - 10https://gerrit.wikimedia.org/r/866652 (owner: 10Cstone) [22:26:29] 10Wikimedia-Fundraising-Banners, 10Wikimedia-production-error: Logstash Client error: "RangeError: Incorrect locale information provided" - https://phabricator.wikimedia.org/T324867 (10Pcoombe) Looks like it was https://meta.wikimedia.org/wiki/Special:CentralNoticeBanners/edit/POTY_2022_P2 which included a bun... [22:27:44] !log payments-wiki upgraded from 35555b67 to 77297c12 [22:27:48] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:28:58] 10Fundraising-Backlog, 10MW-1.40-notes (1.40.0-wmf.14; 2022-12-12), 10Patch-For-Review: Update post-payments monthly convert threshold - https://phabricator.wikimedia.org/T324863 (10Cstone) I just deployed these changes [22:37:33] (03CR) 10Wfan: Add Skeleton for Dlocal (031 comment) [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/865833 (https://phabricator.wikimedia.org/T324270) (owner: 10Wfan) [23:34:05] cstone: should we mention somewhere in Slack that that's deployed? (or maybe you already have?) [23:40:21] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10MW-1.40-notes (1.40.0-wmf.14; 2022-12-12): Update post-payments monthly convert threshold - https://phabricator.wikimedia.org/T324863 (10AndyRussG) [23:51:26] I did AndyRussG [23:51:52] cstone: ah yay thx so much! [23:54:44] ahh so many channels