[00:05:38] (03Merged) 10jenkins-bot: Fix Phan error related to message parameters [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1091401 (owner: 10Bartosz Dziewoński) [01:01:08] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: should we retry if Insufficient funds/over credit limit - https://phabricator.wikimedia.org/T379987 (10AnnWF) 03NEW [01:06:52] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: should we retry if Insufficient funds/over credit limit - https://phabricator.wikimedia.org/T379987#10325030 (10AnnWF) [01:08:06] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: auto rescue for one time payment - https://phabricator.wikimedia.org/T379987#10325035 (10AnnWF) [01:16:07] (03CR) 10Umherirrender: [C:03+2] Use explicit nullable type on parameter arguments (for PHP 8.4) [extensions/CentralNotice] (REL1_43) - 10https://gerrit.wikimedia.org/r/1091441 (https://phabricator.wikimedia.org/T376276) (owner: 10Umherirrender) [02:11:11] (03Merged) 10jenkins-bot: Use explicit nullable type on parameter arguments (for PHP 8.4) [extensions/CentralNotice] (REL1_43) - 10https://gerrit.wikimedia.org/r/1091441 (https://phabricator.wikimedia.org/T376276) (owner: 10Umherirrender) [02:13:55] (03CR) 10Umherirrender: [V:03+2 C:03+2] Use explicit nullable type on parameter arguments (for PHP 8.4) [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1091418 (https://phabricator.wikimedia.org/T376276) (owner: 10Umherirrender) [03:32:38] (03PS1) 10Ejegg: Limit cancel_reason insert to silverpop_update_world [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1091479 [03:32:53] fr-tech is anyone about to review that for me? ^^^ [03:33:32] Should address the failmail that we just got, and let me do the full 180 day contact change sync tonight [03:33:36] If this time is right I can ejegg [03:33:41] yay! [03:33:46] thanks cstone [03:34:03] Hah wasn't sure if my irc was off [03:35:02] so you can just compare it with the delete / insert above, into the silverpop_has_recur table [03:35:32] instead of blanking that whole table, we just delete the ones with an email in silverpop_update_world: [03:35:42] DELETE recur FROM silverpop_update_world t INNER JOIN silverpop_has_recur recur ON t.email = recur.email; [03:36:18] and then when we insert into that table we join on silverpop_update_world to just insert the data for emails with recent changes [03:36:53] so the DELETE for the cancel_reason table has the matching join (and thus leaves the data for previously computed entries) [03:37:17] Thanks ejegg [03:38:07] but the INSERT was missing it, so it's trying to insert cancel_reason for anyone we had previously computed silverpop_has_recur stuff for - in this case, a row from an export a few days ago which wasn't in today's silverpop_export_world [03:38:07] (03CR) 10Cstone: [C:03+2] "looks good thanks for explaining !" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1091479 (owner: 10Ejegg) [03:38:14] thanks cstone! [03:38:43] I'll kick off the 180 day export and then push a change to revert it back to 7 days [03:38:49] well, once that merges [03:39:44] (03Merged) 10jenkins-bot: Limit cancel_reason insert to silverpop_update_world [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1091479 (owner: 10Ejegg) [03:41:45] (03PS1) 10Ejegg: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1091482 [03:41:47] (03CR) 10Ejegg: [C:03+2] Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1091482 (owner: 10Ejegg) [03:42:48] (03Merged) 10jenkins-bot: Merge branch 'master' into deploy [wikimedia/fundraising/tools] (deploy) - 10https://gerrit.wikimedia.org/r/1091482 (owner: 10Ejegg) [03:44:12] !log fundraising python tools upgraded from c6e2dbcc to b230f718 [03:44:13] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:46:42] ok, the 180 day job is running [04:08:48] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Paypal Giving Fund coding - Channel - https://phabricator.wikimedia.org/T379013#10325326 (10Ejegg) OK @MDemosWMF , the next time one comes in it should be correctly tagged. [04:17:40] 06Fundraising-Backlog: Acoustic SMS: Domain needed for short links - https://phabricator.wikimedia.org/T379318#10325327 (10Ejegg) I believe the code to handle passing along query string params would need to be in https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/UrlShortener/+/refs/heads/maste... [05:32:21] 06Fundraising-Backlog, 10MediaWiki-extensions-WikimediaMaintenance, 10Web-Team, 10WikimediaMessages, and 2 others: Donate sidebar link consistency (sitesupport-url) - https://phabricator.wikimedia.org/T379205#10325377 (10tstarling) I mostly need someone from fundraising to sign off on the change to `utm_ca... [06:36:53] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10325432 (10MSuijkerbuijk_WMF) Hi all @AKanji-WMF I'm adding below a few use cases, hope it helps. - Exclude email donors from second gift asks - Identify... [08:00:45] 06Fundraising-Backlog, 10FR-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935#10325500 (10TomaszGorski) Hi @AKanji-WMF. I believe that the double wording error on the existing unsub page has been fixed since then as we ran LATAM campaigns and the un... [08:29:16] (03CR) 10Wfan: "let's fix the format error" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [08:57:12] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Add ability to attach pdfs to end of year emails - https://phabricator.wikimedia.org/T378677#10325614 (10krobinson) @Eileenmcnaughton last year it took about 5 days to deploy and I would vote not to slow that down too much as we pref... [09:20:30] (03CR) 10Wfan: "failed for recurring: cannot store payment method of type: googlepay" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [09:28:57] 03Fundraising Sprint: void(), 06Fundraising-Backlog: 2nd Recurring Failure Email in Civi - https://phabricator.wikimedia.org/T365488#10325764 (10AnnWF) a:03AnnWF [10:54:39] (03CR) 10Jgleeson: [V:03+2 C:03+2] "I also didn't know about this! thanks for bringing it up to date." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1091345 (owner: 10Ejegg) [11:44:38] (03CR) 10Jgleeson: [C:03+2] "Working great! thanks" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1015406 (https://phabricator.wikimedia.org/T352337) (owner: 10Ejegg) [11:48:45] (03CR) 10Jgleeson: [C:03+2] "Thanks for this!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087562 (https://phabricator.wikimedia.org/T352337) (owner: 10Ejegg) [11:56:12] (03Merged) 10jenkins-bot: API to request new link to RecurUpgrade or EmailPreferences [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1015406 (https://phabricator.wikimedia.org/T352337) (owner: 10Ejegg) [11:56:13] (03Merged) 10jenkins-bot: Client side for requesting new checksum links [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1087562 (https://phabricator.wikimedia.org/T352337) (owner: 10Ejegg) [13:39:50] (03PS8) 10Damilare Adedoyin: Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) [13:43:40] (03CR) 10CI reject: [V:04-1] Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [14:01:22] (03CR) 10Damilare Adedoyin: "Thanks wfan for the CR, I've updated the allowed params in the gravy api as suggested." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [14:04:49] (03CR) 10Damilare Adedoyin: "Thanks wfan for the CR." [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [15:40:01] (03PS1) 10Damilare Adedoyin: Set backend processor and transaction id after capture [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1091750 (https://phabricator.wikimedia.org/T379943) [15:45:45] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10Cloud-VPS (Debian Buster Deprecation): Upgrade centralnotice-staging to get off debian buster - https://phabricator.wikimedia.org/T360949#10327171 (10Andrew) 05Open→03Resolved a:03Andrew I have deleted remaining Buster VMs. [15:46:41] (03PS1) 10Damilare Adedoyin: Populate backend processor and transaction id in approve payment result to ensure capture id is saved on Civi. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091752 (https://phabricator.wikimedia.org/T379943) [15:47:05] 10fundraising-tech-ops, 06Infrastructure-Foundations, 10netops, 06SRE: Manage frack switches with Netbox - https://phabricator.wikimedia.org/T268802#10327190 (10cmooney) [15:47:26] (03PS2) 10Damilare Adedoyin: Populate backend processor and transaction id in approve payment result to ensure capture id is saved on Civi. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091752 (https://phabricator.wikimedia.org/T379943) [15:49:22] (03CR) 10CI reject: [V:04-1] Populate backend processor and transaction id in approve payment result to ensure capture id is saved on Civi. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091752 (https://phabricator.wikimedia.org/T379943) (owner: 10Damilare Adedoyin) [16:06:15] thanks for all the review jgleeson|skrun ! [16:08:57] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Paypal Giving Fund coding - Channel - https://phabricator.wikimedia.org/T379013#10327352 (10MDemosWMF) Thanks @Ejegg ! FYI @BJordan-WMF [16:22:32] np ejegg [16:29:40] (03PS1) 10Ejegg: Alphabetize lists in DonationData [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091760 (https://phabricator.wikimedia.org/T379680) [16:31:39] that word makes me think of ceral made of letters [16:32:12] or letters made or cereal... [16:32:24] oh lol, already found a duplicate [16:32:52] https://www.tesco.com/groceries/en-GB/products/314932920?srsltid=AfmBOorHifM7uZ5wrFa2mTCEic75xrXvk8Fa6--Qx9-bDWitvo3MCPIz [16:32:56] it's a thing! [16:33:27] in the US too https://www.usafoodstore.co.uk/post-alpha-bits-cereal-12-oz-340g-american-breakfast-cereal.html [16:33:59] £7.25 a box is pretty pricy. You'd have to really like US cereal to pay that [16:34:28] bring a couple of boxes home from the US next time you visit. [16:35:01] good idea [16:35:13] Chicago cereal raid [16:36:15] (03PS1) 10Ejegg: Remove duplicate key in DonationData, update comments [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091765 [16:47:25] (03PS2) 10Ejegg: Alphabetize lists in DonationData [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091760 (https://phabricator.wikimedia.org/T379680) [16:47:25] (03PS1) 10Ejegg: Add recipient_id to DonationData [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1091768 (https://phabricator.wikimedia.org/T379680) [16:49:18] ]recurringrererrrerrecurring [16:51:02] very meta [16:51:03] ack [16:51:25] something wacky with cursor focus, was trying to search on Civi gitlab [16:55:08] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 13Patch-For-Review: Update donation interface such that recipient_id reaches donationn queue - https://phabricator.wikimedia.org/T379680#10327564 (10Ejegg) [17:18:15] (03PS9) 10Damilare Adedoyin: Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) [17:20:19] (03CR) 10CI reject: [V:04-1] Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [17:39:58] (03CR) 10Wfan: "https://etherpad.wikimedia.org/p/google-pay-debug maybe their sandbox testing not stable, but since I am still seeing that, please see if " [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [18:18:32] 10fundraising-tech-ops: Fundraising access request for skivlehan - https://phabricator.wikimedia.org/T378541#10327899 (10Dwisehaupt) [18:19:12] 10fundraising-tech-ops: Fundraising access request for skivlehan - https://phabricator.wikimedia.org/T378541#10327901 (10Dwisehaupt) @SKivlehan-WMF Thanks. I have added them to puppet and they will be deployed shortly. I will be sending along instructions on how to ssh in. [18:51:11] RECOVERY - check_memory on fran1001 is OK: OK Memory 35% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_memory [18:59:17] 06Fundraising-Backlog, 10FR-donorservices: Unsubscribe footer copy error in ES + email question - https://phabricator.wikimedia.org/T259935#10328032 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF Perfect, thank you @TomaszGorski - resolving [19:35:41] fr-tech I'm going to try uploading a little logging thing directly to the vendor dir on civi deployment branch, then will revert [19:37:17] after trying the data file upload again [19:39:29] (03PS1) 10Ejegg: temp logging [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091807 [19:40:49] (03CR) 10Ejegg: [C:03+2] temp logging [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091807 (owner: 10Ejegg) [19:41:13] (03PS1) 10Ejegg: Update vendor for temp logging fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091808 [19:43:22] ejegg: was that the csv upload failing [19:43:35] eileen: yeah, same as you logged in that phab [19:43:41] had you been looking into it? [19:43:55] I just wanted to get logs similar to the ones we get from the python-based ones [19:44:01] hmm - how far did it get ? I assumed 180 days would get the file down around the 5m it was doing [19:44:03] and see what was different [19:44:31] i'm not sure how many rows, but it did get that Connection closed (by server) prematurely [19:47:34] oh - we can check in Acoustic for the number of rows [19:48:52] (03CR) 10Ejegg: [C:03+2] Update vendor for temp logging fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091808 (owner: 10Ejegg) [19:49:17] ah, i perhaps can't see the data jobs as my own user? [19:49:26] so i guess I need to log in as the service user eileen ? [19:54:48] I'm checking it [19:55:11] you need to do that as the service user, or am I perhaps looking in the wrong place? [20:02:12] (03Merged) 10jenkins-bot: temp logging [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091807 (owner: 10Ejegg) [20:02:13] (03Merged) 10jenkins-bot: Update vendor for temp logging fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091808 (owner: 10Ejegg) [20:08:34] ah dang, typo [20:12:41] (03PS1) 10Ejegg: Fix typo [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091812 [20:12:44] (03CR) 10Ejegg: [C:03+2] Fix typo [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091812 (owner: 10Ejegg) [20:13:05] (03PS1) 10Ejegg: Update vendor for typo fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091813 [20:13:11] (03CR) 10Ejegg: [C:03+2] Update vendor for typo fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091813 (owner: 10Ejegg) [20:36:36] (03Merged) 10jenkins-bot: Fix typo [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1091812 (owner: 10Ejegg) [20:36:37] (03Merged) 10jenkins-bot: Update vendor for typo fix [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091813 (owner: 10Ejegg) [20:40:33] ah dang, is that complex logging including the whole contents of the file?? [20:40:40] hmm, that's a bit overkill [20:40:45] I just wanted the cyphers [20:41:24] well, I'll relocate while it does the rest of that upload and check back [21:33:05] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Add ability to attach pdfs to end of year emails - https://phabricator.wikimedia.org/T378677#10328427 (10Eileenmcnaughton) @krobinson it will take some fr-tech work to split it into 2 runs - so maybe open a new phab to track that req... [21:40:25] (03PS1) 10Ejegg: Update vendor for simple logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091820 [21:42:01] (03CR) 10Ejegg: [C:03+2] Update vendor for simple logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091820 (owner: 10Ejegg) [21:42:54] (03Merged) 10jenkins-bot: Update vendor for simple logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091820 (owner: 10Ejegg) [22:04:02] hmm, even the 'complex' logging seems to be just the SFTP commands, not stuff about the underlying ssh transport [22:17:51] oh fun, new webhook type from gravy deployed on a friday? [22:18:33] So I don't actually see any failure / premature close in that one with the complex logging [22:18:42] i wonder if it actually got through that time [22:18:50] and why the logging would make any difference [22:18:56] it certainly made things slower [22:20:19] ok, just trying one more run with the SIMPLE logging, then I'll revert all the log stuff [22:22:03] huh, with the simple logging it got the premature disconnect very quickly [22:37:05] 06Fundraising-Backlog: U.S. Payments Wiki form variant with Phone Number and SMS opt-in - https://phabricator.wikimedia.org/T380106 (10spatton) 03NEW [22:38:18] (03PS1) 10Ejegg: Update vendor pointer to revert all temp logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091825 [22:38:22] (03CR) 10Ejegg: [C:03+2] Update vendor pointer to revert all temp logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091825 (owner: 10Ejegg) [22:39:06] (03Merged) 10jenkins-bot: Update vendor pointer to revert all temp logging [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1091825 (owner: 10Ejegg) [23:04:06] ugh that failmail is going to get pretty annoying [23:05:41] 10fundraising-tech-ops, 10BDC-Implementation, 06Data-Platform-SRE, 06Privacy Engineering, and 2 others: [minIO] Investigate packaging, install, security monitoring. - https://phabricator.wikimedia.org/T367123#10328637 (10Dwisehaupt) Initial puppet manifest and templates made for minio and deployed. Verifie... [23:19:45] PROBLEM - Host fran1001 is DOWN: PING CRITICAL - Packet loss = 100% [23:20:07] PROBLEM - Host frdb1003 is DOWN: PING CRITICAL - Packet loss = 100% [23:20:51] ^^ those are for kernel upgrade/reboots. will be back online in a sec. [23:21:11] RECOVERY - Host fran1001 is UP: PING OK - Packet loss = 0%, RTA = 0.44 ms [23:22:11] RECOVERY - Host frdb1003 is UP: PING OK - Packet loss = 0%, RTA = 0.49 ms [23:42:10] 06Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2425Q2 maintenance window (Nov 11-15th, 2024) - https://phabricator.wikimedia.org/T337583#10328683 (10Dwisehaupt) [23:53:09] 06Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2425Q2 maintenance window (Nov 11-15th, 2024) - https://phabricator.wikimedia.org/T337583#10328693 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt [23:55:55] 06Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: swap analytics db origin from frdb1003 to frdb2003 - https://phabricator.wikimedia.org/T378859#10328699 (10Dwisehaupt) [23:56:24] 06Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: swap analytics db origin from frdb1003 to frdb2003 - https://phabricator.wikimedia.org/T378859#10328700 (10Dwisehaupt) 05Open→03Resolved a:03Dwisehaupt