[02:56:55] 10Fundraising-Backlog, 10FR-donorservices: Civi Snooze feature stopped working! - https://phabricator.wikimedia.org/T358259#9570937 (10Cstone) [12:21:00] hey jgleeson, how's it going? [12:21:20] any chance we can test the jobs container today [12:25:09] hey damilare. not too bad. Yep it shouldn't much longer, I'm working on that now and had hoped to have it done yesterday but I got sidetracked digging into that problem with the auto-rescue issue with cstone and wfan. It looks like the code isn't closing down refused retries but we couldn't work out why. [12:25:16] be* [12:25:54] ohh that's interesting [12:26:16] is there a ticket for the issue so I can get more context [12:26:37] (03CR) 10Damilare Adedoyin: [C: 03+2] "Mac folks are thankful for the lookout." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1003453 (owner: 10Jgleeson) [12:30:04] I'm not sure about a ticket. That spreadsheet we're both in contains all of cstone's live test responses we got back from Adyen and I think we were digging into row 5 which should in theory result in a cancelled recurring donation but for some reason it was ending up as pending. There's a few unit tests which cover that scenario and the tests work so we were trying to work out the difference [12:30:06] between the real raw response and the mocked response it the test to see why it wasn't being cancelled [12:30:16] thanks for the review! [12:31:12] ahh looking [12:31:19] np re:review [13:03:32] damilare: there is a task in the backlog it's tied to the testing phab but not more context than what Jack said [13:05:31] thanks cstone, I found the task and started looking into it. Nothing has come across yet [13:29:11] ok looks like the reason for the rescue not being cancelled is because we are trying to cast from a string to bool [13:29:34] As shown in the test in the patch below [13:29:52] I'd push up an update when I get back shortly [13:31:13] (03PS1) 10Damilare Adedoyin: WIP: Fix autorescue not cancelling [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005981 [13:32:24] we looked at the string/bool falsey difference for the returned flags but found in the mapping code setting the retryScheduled flag was being cast with (bool) so we moved away. we might have not looked hard enough! [13:49:09] (03CR) 10CI reject: [V: 04-1] WIP: Fix autorescue not cancelling [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005981 (owner: 10Damilare Adedoyin) [14:14:38] damilare|brb: did you hit this error when running django banner stats? [14:14:40] File "/srv/django-banner-stats/packages/django/db/models/sql/query.py", line 11, in [14:14:42] from collections import Counter, Iterator, Mapping, OrderedDict [14:14:44] ImportError: cannot import name 'Iterator' from 'collections' (/usr/lib/python3.11/collections/__init__.py) [14:15:25] it looks like the old django code is trying to import Iterator from a module that has been renamed in python 3.11 [14:16:03] I don't really want to be editing the django framework code to make it work. are you running python 3.11 on your local image you're testing with? [14:49:50] (03PS1) 10Jgleeson: WIP: add ./setup --django [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1006002 [14:51:53] fr-tech gotta go grab the kids but that patch ^^^ sets up DjangoBannerStats on the new fundraising-bookworm-python Docker image on fundraising-dev. Check out commit message for info. I need to fix that error mentioned above so will continue with that when I get back. [15:45:26] 10Fundraising Sprint: canOfWorms(🪱), 10Fundraising-Backlog: Reattempt failed Ingencio transactions from Oct, Nov, Dec 2023 - https://phabricator.wikimedia.org/T353677#9572368 (10Ejegg) We should also probably exclude * anyone who we canceled because the charge failed due to of one of the reasons that they can... [15:48:26] gonna go start dinner before meetings back soon [15:49:19] ahh jgleeson sorry just got back, I'd check that, but I didn't have to install any package [15:49:56] just used the bundled package [15:49:58] I think the issue is with bookworm dami. it's installing a new version of python3 [15:50:42] ohh we've updated since that last patch [15:51:09] hmm we might have ti reinstall the packages as a last resort. [15:53:11] so the issue is that the other jobs image and patch was using an older version of debian but we're now building images for a later version which pulls in a newer version of python when we do apt install python3. the newer version changes one of the modules that the old django code depends on. we can fix it but it means tweaking the build to either install the older version of python or updating [15:53:13] the code to use the new collectons.abc package [15:54:04] I confirmed this by looking at the fundraising-tool image. python3 on that using the same install command is python3.8 but python3 is bumped up to 3.11 (affected version) on debian bookworm [15:54:48] so you won't have run into it before [15:55:39] a real lazy quick fix would be to build our new image from bullseye and not bookworm. We could see what's running in production [15:56:55] I'll do that for today to get it working for your call and then look into the longer term fix [15:58:03] 10Fundraising Sprint: canOfWorms(🪱), 10Fundraising-Backlog: Reattempt failed Ingencio transactions from Oct, Nov, Dec 2023 - https://phabricator.wikimedia.org/T353677#9572402 (10Ejegg) @EMartin When I limit the cancel reason to just 'auto (maximum failures reached)' for October, November, and December, I actua... [16:05:37] sounds good to me [16:17:35] I going to a doctor appointment this morning at 8:30, might miss the standup. Or join later. [16:23:51] (03PS1) 10Damilare Adedoyin: Auto Rescue - Not actually cancelling when retry.rescueScheduled:false [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1006018 (https://phabricator.wikimedia.org/T358264) [16:24:17] (03CR) 10CI reject: [V: 04-1] Auto Rescue - Not actually cancelling when retry.rescueScheduled:false [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1006018 (https://phabricator.wikimedia.org/T358264) (owner: 10Damilare Adedoyin) [16:32:35] (03PS2) 10Damilare Adedoyin: Auto Rescue - Not actually cancelling when retry.rescueScheduled:false [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1006018 (https://phabricator.wikimedia.org/T358264) [16:32:59] (03CR) 10CI reject: [V: 04-1] Auto Rescue - Not actually cancelling when retry.rescueScheduled:false [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1006018 (https://phabricator.wikimedia.org/T358264) (owner: 10Damilare Adedoyin) [16:34:10] (03PS3) 10Damilare Adedoyin: Auto Rescue - Not actually cancelling when retry.rescueScheduled:false [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1006018 (https://phabricator.wikimedia.org/T358264) [16:34:41] (03CR) 10Ejegg: [V: 03+2] Fix PHP8.2 notice [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005188 (owner: 10Ejegg) [16:40:30] (03PS2) 10Damilare Adedoyin: Update test for failed autorescue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005981 (https://phabricator.wikimedia.org/T358264) [16:42:17] jgleeson, seems the issue was in the casting of string to boolean, I demonstrated and fixed it in the tests on the smashpig patch [16:44:54] (03PS3) 10Damilare Adedoyin: Update test for failed autorescue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005981 (https://phabricator.wikimedia.org/T358264) [17:01:28] 10Fundraising Sprint: canOfWorms(🪱), 10Fundraising-Backlog, 10Patch-For-Review: Auto Rescue - Not actually cancelling when retry.rescueScheduled:false - https://phabricator.wikimedia.org/T358264#9572574 (10Damilare) [17:16:59] 10Fundraising Sprint: canOfWorms(🪱), 10Fundraising-Backlog, 10Patch-For-Review: Auto Rescue - Not actually cancelling when retry.rescueScheduled:false - https://phabricator.wikimedia.org/T358264#9572634 (10Damilare) a:03Damilare [17:21:02] fr-tech ^ [17:21:26] wb AndyRussG! we're just on standup [17:22:38] jgleeson: thx!!!!! :) have a fun standup [18:19:51] welcome back AndyRussG! [18:20:18] damilare: thx!!!! ;) [18:45:42] (03PS2) 10Ejegg: Can the worm (remove more contribution tracking refs) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005642 (https://phabricator.wikimedia.org/T354708) (owner: 10Eileen) [18:45:46] (03CR) 10Ejegg: [C: 03+2] Can the worm (remove more contribution tracking refs) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005642 (https://phabricator.wikimedia.org/T354708) (owner: 10Eileen) [18:47:21] (03PS8) 10Ejegg: Remove contribution tracking dependenciy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005588 (owner: 10Eileen) [18:47:25] (03CR) 10Ejegg: [C: 03+2] Remove contribution tracking dependenciy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005588 (owner: 10Eileen) [18:50:44] Congrats on the new WMF gig AndyRussG ! [18:54:22] ejegg ¡¡¡mil gracias!!! ;D [19:05:08] (03Merged) 10jenkins-bot: Can the worm (remove more contribution tracking refs) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005642 (https://phabricator.wikimedia.org/T354708) (owner: 10Eileen) [19:06:56] (03Merged) 10jenkins-bot: Remove contribution tracking dependenciy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1005588 (owner: 10Eileen) [19:30:01] grats AndyRussG ! [19:45:37] cstone: thanks so much aaaah :) [19:51:57] (03PS2) 10Jgleeson: WIP: add ./setup --django [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1006002 [19:52:44] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1006068 [19:52:54] AndyRussG: how long is the contract? [19:53:19] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1006068 (owner: 10Eileen) [19:53:32] 10Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for rmurthy - https://phabricator.wikimedia.org/T357191#9573081 (10Dwisehaupt) Renewed certificate and sent via email. Password sent via SMS. [19:54:44] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1006068 (owner: 10Eileen) [19:58:05] (03PS3) 10Jgleeson: WIP: add ./setup --django [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1006002 [20:03:48] !log civicrm upgraded from c50fcae3 to a65551c7 [20:04:20] eileen: Failed to log message to wiki. Somebody should check the error logs. [20:04:20] ejegg: I just deployed that - I could disable contribution_tracking for real now - but perhaps it should wait for Monday [20:06:34] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops: for next maintenance window - Remove disabled custom fields - https://phabricator.wikimedia.org/T356443#9573148 (10Eileenmcnaughton) [20:15:30] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Delete blank contacts - https://phabricator.wikimedia.org/T322887#9573202 (10Ejegg) Looks like maybe a couple of mistaken imports? Anyway, they do all seem delete-able from the spot checks I've done so far. [20:29:23] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Delete blank contacts - https://phabricator.wikimedia.org/T322887#9573274 (10Ejegg) OK, they are deleted [20:30:44] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: Civi Snooze feature stopped working! - https://phabricator.wikimedia.org/T358259#9573278 (10Ejegg) [20:36:46] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-donorservices: Civi Snooze feature stopped working! - https://phabricator.wikimedia.org/T358259#9573336 (10Ejegg) Hmm, looks like maybe some of these are piled up behind the data axle import job. I'll see if I can increase the priority for the snoo... [21:14:59] (03PS1) 10Ejegg: Create snooze jobs with higher priority [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1006080 (https://phabricator.wikimedia.org/T358259) [21:29:18] 10fundraising-tech-ops: Request access to CIVI Staging - https://phabricator.wikimedia.org/T358042#9573532 (10RLopez-WMF) Thanks [21:39:27] huh, upping the weight doesn't seem to get those snooze jobs running more frequently [22:31:26] eileen: longer than a cat's whisker and shorter than a comet's tail ;p [23:24:03] (03PS1) 10Amire80: Remove angle brackets from donate_interface-email-unsub-delay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1006090 (https://phabricator.wikimedia.org/T358397)