[00:00:47] (03CR) 10CI reject: [V: 04-1] Set time to noon for stock imports [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994335 (https://phabricator.wikimedia.org/T355452) (owner: 10Ejegg) [00:04:07] odd [00:04:42] how is that possibly related? [00:27:20] ejegg: if you have a chance I'm trying to figure out why segmentation aging process control job isn't working [00:27:26] if I run [00:27:27] echo '{"where":[["wmf_donor.last_donation_date","<=","1 month ago"],["wmf_donor.donor_status_id","=",80]],"version":4,"values":{"donor_segment_id":"","donor_status_id":""},"limit":10000}' | drush @wmff cvapi WMFDonor.update --in=json [00:27:30] it works [00:27:39] but in the job it isn't [00:28:37] probably missing slashes I guess [00:29:55] hmm I think I see [00:30:03] sh -c "echo '{\"mailing_prefix\":\"sp\", \"options\":{\"limit\":50000},\"thottle_number\":3000, \"throttle_seconds\":120, \"mail [00:30:03] _provider\":\"Silverpop\"}' | /usr/local/bin/drush @wmff -vv --in=json cvapi Omnirecipient.load" [00:39:49] (03PS1) 10Ejegg: Fix email pref ctr DB name in config [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/994859 [00:42:47] oops, just saw your message [00:42:56] but I guess you just needed a duck? [00:43:44] I'm just thoroughly puzzled by that venmo TY test [00:44:06] and what on earth it could have to do with the import hook [00:44:37] i couldn't replicate locally, so I just rebuilt while on that same patch [00:45:25] running tests again [00:48:35] ejegg: were you running fundraising dev v2 locally or just tested it out when you did it? [00:54:07] running it locally [00:54:18] and after the fresh rebuild, it is failing just like in CI [00:54:35] so that's good anyway, I can check out what's happening [00:54:38] er sorry not the test thing I just have two broken set ups now hah [00:54:51] is it cause something isnt getting cleaned up? [00:55:07] i wish I knew! [00:55:13] only just got it to fail [00:57:56] I just put a process control update onto frpm if someone can check [00:58:44] looking eileen [01:00:36] yep eileen that looks like the way we have to do it [01:01:41] ok - trying it [01:13:02] well, the issue with my patch isn't in the actual code that runs [01:13:06] it's something in the setup [01:13:24] if I rebuild on my patch then go back to master, the venmo test still breaks [01:13:52] I'll try a patch that JUST adds the Stock payment instrument then [01:16:33] (03PS3) 10Ejegg: Move Venmo-specific TY stuff to own test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994849 [01:16:35] (03PS1) 10Ejegg: Add Stock payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994865 [01:29:56] (03CR) 10Ejegg: "This is working quite well for me! While running a full setup distracted, I almost accidentally deleted my local tools src, because the pr" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/953984 (owner: 10Jgleeson) [01:31:44] (03CR) 10Ejegg: "Oops, scratch that, I guess the reclone step is indeed at the beginning of each separate step, I had just let my eyes glaze over while civ" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/953984 (owner: 10Jgleeson) [01:33:19] ok, so just adding the 'Stock' payment method does it [01:33:22] WHAAAAA [01:34:02] (03CR) 10CI reject: [V: 04-1] Add Stock payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994865 (owner: 10Ejegg) [01:44:37] ok, so somewhere in WorkflowMessage::exportAll the venmoUserName from the $model is NOT getting exported to the venmo_user_name of the output $params [01:52:06] AAARGH, hard-coded payment_instrument_id! [01:52:08] noooo [01:52:16] there's got to be a better way to do that [01:57:05] (03PS4) 10Ejegg: Move Venmo-specific TY stuff to own test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994849 [01:57:07] (03PS2) 10Ejegg: Add Stock payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994865 [02:04:22] (03Abandoned) 10Ejegg: Add Stock payment instrument [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994865 (owner: 10Ejegg) [02:04:43] (03PS8) 10Ejegg: Set time to noon for stock imports [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994335 (https://phabricator.wikimedia.org/T355452) [02:05:50] wfan (or cstone) could you take a quick look at this test fix please? https://gerrit.wikimedia.org/r/994849 [02:18:39] (03CR) 10Eileen: [C: 03+2] Move Venmo-specific TY stuff to own test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994849 (owner: 10Ejegg) [02:19:31] thanks eileen ! [02:22:02] 10Fundraising Sprint: brb(), 10Fundraising-Backlog: Scheduled job to age recurring statuses not running correctly. - https://phabricator.wikimedia.org/T355850 (10Eileenmcnaughton) [02:35:51] (03Merged) 10jenkins-bot: Move Venmo-specific TY stuff to own test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994849 (owner: 10Ejegg) [02:37:41] yay [02:40:49] (03CR) 10CI reject: [V: 04-1] Set time to noon for stock imports [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994335 (https://phabricator.wikimedia.org/T355452) (owner: 10Ejegg) [02:41:07] d'oh [02:41:34] oh man, php execution timeout [02:41:35] PHP Fatal error: Maximum execution time of 180 seconds exceeded in /src/wikimedia/fundraising/crm/drupal/sites/all/modules/civicrm/Civi/Api4/Generic/BasicGetFieldsAction.php on line 165 [02:42:04] I'mma just V+2 it for now [02:42:13] but I guess we need to speed up some tests! [02:42:30] (03CR) 10Ejegg: [V: 03+2] Set time to noon for stock imports [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994335 (https://phabricator.wikimedia.org/T355452) (owner: 10Ejegg) [02:44:58] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Release-Engineering-Team: CiviCRM tests timing out in CI - https://phabricator.wikimedia.org/T356361 (10Ejegg) [02:45:34] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review: Recent lapsed recurring segment not being calculated - https://phabricator.wikimedia.org/T355851 (10Eileenmcnaughton) [02:45:58] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/994868 [02:46:01] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/994868 (owner: 10Ejegg) [02:46:29] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops: Switch to Smarty3 - https://phabricator.wikimedia.org/T354707 (10Eileenmcnaughton) [02:46:50] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/994868 (owner: 10Ejegg) [02:49:22] (03PS9) 10Eileen: Start to add WMFMessage as class for getting message information [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 [02:55:26] !log civicrm upgraded from 6e1e0d21 to 21bf2138 [02:55:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:58:34] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Gift date in Civi thank you for stock gift imported from Overflow is wrong - https://phabricator.wikimedia.org/T355452 (10Ejegg) Hi @MDemosWMF , I've just deployed a change that will set any Stock gift imported via that import UI (th... [03:00:59] (03PS10) 10Eileen: Start to add WMFMessage as class for getting message information [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 [03:06:02] (03PS11) 10Eileen: Start to add WMFMessage as class for getting message information [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 [03:08:24] (03PS12) 10Ejegg: Start to add WMFMessage as class for getting message information [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 (owner: 10Eileen) [03:08:39] (03CR) 10Ejegg: [C: 03+2] "And we're off! Let's see where this takes us." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 (owner: 10Eileen) [03:10:03] (03PS1) 10Eileen: Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 [03:10:12] ejegg: thanks - I just wondered looking at it about ^^ [03:11:09] The WmfQueueMessage namespace might be a good idea [03:11:24] but I would keep the 'Donation' in the names of those classes [03:11:30] (maybe I will rename those other folders to be singular now cstone has pointed out the inconsistency - although I think I'd rather merge this first https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/994353 [03:11:44] to separate them from e.g. an EmailPreferencesQueueMessage [03:12:22] k, i've got to call it quits for today [03:12:28] have a good rest of yours! [03:12:43] (03PS2) 10Eileen: Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 [03:12:57] thanks ejegg|away [03:14:13] Thanks Eileen and Elliott for that Venmo unit test >~< [03:14:47] lol + I just clicked a button [03:30:08] (03CR) 10CI reject: [V: 04-1] Start to add WMFMessage as class for getting message information [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 (owner: 10Eileen) [03:30:47] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review, 10Unplanned-Sprint-Work: FY2324 Q3: Need to map new Civi fields over to Acoustic - https://phabricator.wikimedia.org/T353386 (10Eileenmcnaughton) I think a small reduction in the checksum emails could be due to some deduping but I need... [03:33:11] (03CR) 10CI reject: [V: 04-1] Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 (owner: 10Eileen) [03:35:17] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review, 10Unplanned-Sprint-Work: FY2324 Q3: Need to map new Civi fields over to Acoustic - https://phabricator.wikimedia.org/T353386 (10Eileenmcnaughton) Aha - I see it - it looks like it likely just needs rebuilding - but I can't recall why w... [03:38:26] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/976850 (owner: 10Eileen) [03:52:40] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Unplanned-Sprint-Work: Deploy & potentially enable 'The Island' theme - https://phabricator.wikimedia.org/T356365 (10Eileenmcnaughton) [04:01:18] (03PS1) 10Eileen: Add TheIsland theme [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994871 (https://phabricator.wikimedia.org/T356365) [04:16:53] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Unplanned-Sprint-Work: Fix no-thank-you reason default - https://phabricator.wikimedia.org/T356366 (10Eileenmcnaughton) [04:17:48] (03PS1) 10Eileen: Fix no-thank you reason code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994872 (https://phabricator.wikimedia.org/T356366) [04:33:21] (03PS1) 10Eileen: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 [04:33:57] cstone: if you are still about can you +2 the IDE one ^^ - I keep fighting my IDE if I don't get that merged [04:36:24] (03PS2) 10Eileen: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 [04:39:11] (03PS3) 10Eileen: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 [04:39:20] (03CR) 10CI reject: [V: 04-1] IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 (owner: 10Eileen) [04:41:26] (03PS4) 10Eileen: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 [04:43:17] (03PS5) 10Eileen: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 [04:46:18] (03PS3) 10Eileen: Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 [04:49:38] (03PS1) 10Eileen: Rename WMFStatistics to WMFStatistics [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994874 [04:49:40] (03PS1) 10Eileen: Renmae WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 [04:49:42] (03PS1) 10Eileen: Rename WMFHelpers to WMFHelper [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994876 [04:52:24] (03CR) 10CI reject: [V: 04-1] Renmae WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (owner: 10Eileen) [04:52:48] (03CR) 10CI reject: [V: 04-1] Rename WMFHelpers to WMFHelper [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994876 (owner: 10Eileen) [05:03:35] (03CR) 10CI reject: [V: 04-1] Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 (owner: 10Eileen) [05:06:03] (03CR) 10CI reject: [V: 04-1] Rename WMFStatistics to WMFStatistics [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994874 (owner: 10Eileen) [05:07:21] (03PS4) 10Eileen: Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 [05:11:07] (03PS2) 10Eileen: Rename WMFStatistics to WMFStatistic [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994874 [05:14:49] (03PS2) 10Eileen: Renmae WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 [05:16:30] (03PS3) 10Eileen: Rename WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (https://phabricator.wikimedia.org/T356115) [05:19:21] (03CR) 10CI reject: [V: 04-1] Rename WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (https://phabricator.wikimedia.org/T356115) (owner: 10Eileen) [05:24:21] (03PS2) 10Eileen: Rename WMFHelpers to WMFHelper [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994876 (https://phabricator.wikimedia.org/T356115) [05:55:08] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review, 10Unplanned-Sprint-Work: FY2324 Q3: Need to map new Civi fields over to Acoustic - https://phabricator.wikimedia.org/T353386 (10Eileenmcnaughton) ok @KHaggard - the next file to export - ie in about 18 hours time - should have a fulle... [06:01:13] (03PS4) 10Eileen: Rename WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (https://phabricator.wikimedia.org/T356115) [06:01:32] (03PS3) 10Eileen: Rename WMFHelpers to WMFHelper [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994876 (https://phabricator.wikimedia.org/T356115) [06:27:57] 10Fundraising-Backlog: Upgrading project Landing pages code - https://phabricator.wikimedia.org/T356370 (10MSuijkerbuijk_WMF) [06:31:11] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_40) - 10https://gerrit.wikimedia.org/r/993898 (owner: 10L10n-bot) [06:34:23] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_41) - 10https://gerrit.wikimedia.org/r/994467 (owner: 10L10n-bot) [06:46:39] 10Fundraising-Backlog: Upgrading project Confirmation email - https://phabricator.wikimedia.org/T356373 (10MSuijkerbuijk_WMF) [13:15:56] 10Fundraising Sprint: brb(), 10Fundraising-Backlog: Scheduled job to age recurring statuses not running correctly. - https://phabricator.wikimedia.org/T355850 (10jgleeson) I updated this job file today as it failed. The run user, which I think is Jenkins, doesn't have drush in its $PATH. I've switched it over... [13:22:58] (03PS1) 10Damilare Adedoyin: Remove auto rescue message from pending queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995034 (https://phabricator.wikimedia.org/T356224) [13:42:28] (03CR) 10CI reject: [V: 04-1] Remove auto rescue message from pending queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995034 (https://phabricator.wikimedia.org/T356224) (owner: 10Damilare Adedoyin) [13:51:03] (03PS2) 10Damilare Adedoyin: Remove auto rescue message from pending queue [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995034 (https://phabricator.wikimedia.org/T356224) [13:55:34] (03PS1) 10Damilare Adedoyin: WIP: Move successful autorescue to recurring queue without reading from Pending db table [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/995038 (https://phabricator.wikimedia.org/T356224) [13:55:59] (03CR) 10CI reject: [V: 04-1] WIP: Move successful autorescue to recurring queue without reading from Pending db table [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/995038 (https://phabricator.wikimedia.org/T356224) (owner: 10Damilare Adedoyin) [13:58:33] (03CR) 10Ejegg: [C: 03+2] IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 (owner: 10Eileen) [13:59:53] (03CR) 10Ejegg: [C: 03+2] Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 (owner: 10Eileen) [14:03:41] (03CR) 10Ejegg: [C: 03+2] "Seeing 'statistic' in the singular reminds me of that quote - "The import of one donation is a joy, the import of a million donations is a" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994874 (owner: 10Eileen) [14:04:58] (03CR) 10Ejegg: [C: 03+2] Rename WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (https://phabricator.wikimedia.org/T356115) (owner: 10Eileen) [14:16:14] (03Merged) 10jenkins-bot: IDE reformatting [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994873 (owner: 10Eileen) [14:17:51] (03Merged) 10jenkins-bot: Perhaps this naming is better [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994869 (owner: 10Eileen) [14:22:17] (03Merged) 10jenkins-bot: Rename WMFStatistics to WMFStatistic [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994874 (owner: 10Eileen) [14:22:19] (03Merged) 10jenkins-bot: Rename WMFHooks to WMFHook [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994875 (https://phabricator.wikimedia.org/T356115) (owner: 10Eileen) [15:02:48] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, and 2 others: Fix no-thank-you reason default - https://phabricator.wikimedia.org/T356366 (10MDemosWMF) Major gifts recently changed their rules to have no threshold for auto TY's from Civi so I'm n... [15:57:45] 10Fundraising Sprint Uptown cURL, 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review: Data Axle Project - Re-import to Civi - https://phabricator.wikimedia.org/T341902 (10MDemosWMF) @Eileenmcnaughton I looked into why you couldn't find the Single: Y/N field in the files and I think wire... [16:02:16] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10Patch-For-Review: Gift date in Civi thank you for stock gift imported from Overflow is wrong - https://phabricator.wikimedia.org/T355452 (10MDemosWMF) That's great, thank you! [16:42:12] (03CR) 10Ejegg: "Should we take this opportunity to start correcting Wmf -> WMF like we have in other places?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994353 (owner: 10Eileen) [16:54:31] fr-tech I'm feeling like we can +2 jgleeson|away's fr-dev patch now, and just fix any issues as we find them [16:55:59] (03PS45) 10Ejegg: fundraising-dev v2 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/953984 (owner: 10Jgleeson) [16:56:12] (03CR) 10Ejegg: [C: 03+2] fundraising-dev v2 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/953984 (owner: 10Jgleeson) [16:56:30] I just fixed the copypasta that cstone mentioned in her last comment [16:57:38] (03CR) 10Ejegg: [V: 03+2 C: 03+2] fundraising-dev v2 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/953984 (owner: 10Jgleeson) [16:58:04] (03PS2) 10Ejegg: Fix email pref ctr DB name in config [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/994859 [17:00:09] 10Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for sdaniels - https://phabricator.wikimedia.org/T356134 (10Dwisehaupt) Renewed certificate and sent via email. Password sent via SMS. [17:22:50] ejegg: was a fresh install working for you locally with version2 I was struggling last night but haven't tried yet today [17:22:57] I could just be independently cursed though [17:23:54] oh no! [17:24:08] OK, maybe that was premature [17:24:19] Trying again [17:24:32] I think large likelihood it's only breaking for me [17:26:47] what was the breakage? [17:27:12] hmm, so it's not asking for all the ports like the old one did [17:27:23] i guess the idea is we just do that in the .env file manually? [17:29:56] that part was working fine it was with civi [17:30:08] i put it in the etherpad lets see [17:30:36] +++ git clone --recursive /srv/civicrm-buildkit/app/tmp/git-cache/wikimedia/fundraising/crm.git /srv/civi-sites/wmff [17:30:37] fatal: could not create work tree dir '/srv/civi-sites/wmff': Permission denied [17:30:37] [+] Restarting 1/1 [17:30:37] ✔ Container dockertest-civicrm-1 Started 10.7s [17:30:37] CiviCRM WMFF Installed! [17:30:48] mainly more that it failed but then said it worked hah [17:31:07] 10Fundraising Sprint: brb(), 10Fundraising-Backlog: Add ACH payment type for US donation processing - https://phabricator.wikimedia.org/T324517 (10AnnWF) a:03AnnWF [17:31:11] ah ok, it's not checking for the exit value [17:50:25] 10Fundraising-Backlog: Enable Post-Payment Monthly Convert for CL, CO, MX, PE, UY, BR - https://phabricator.wikimedia.org/T356277 (10HNordeenWMF) [17:50:41] 10Fundraising-Backlog: Enable Post-Payment Monthly Convert for CL, CO, MX, PE, UY, BR - https://phabricator.wikimedia.org/T356277 (10HNordeenWMF) [18:11:25] 10Fundraising-Backlog, 10Design-System-Team: CentralNotice Banner Sequence Feature: we need a drag/drop icon - https://phabricator.wikimedia.org/T162756 (10Volker_E) [18:46:24] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, 10FR-Tech-Analytics: Some Recent Email Send have No Clicks Or Donations - https://phabricator.wikimedia.org/T355974 (10Cstone) I updated the 61 contribution trackings from that utm_campaign with 40 donations. Since donations can still come in on the li... [18:59:34] (03PS1) 10Ejegg: Add --skip-reclone option for setup [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 [19:00:03] 10Fundraising-Backlog: Venmo non-native browser flow posing checkout issues for donors - https://phabricator.wikimedia.org/T348699 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF In talking to @EMartin it sounds like @Pcoombe has addressed this on donatewiki. [19:00:47] (03PS1) 10Ejegg: Default exposed port for mariadb to 3307 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 [19:12:38] cstone: ah i just got a problem on civiproxy [19:12:39] fatal: destination path 'src/civiproxy' already exists and is not an empty directory. [19:12:57] because of the overlay mounts, i think [19:12:59] (03PS2) 10Eileen: Remove code that sets UI default for no thank you [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994872 (https://phabricator.wikimedia.org/T356366) [19:13:23] those overlay mounts are totally silly [19:13:43] I'm going to submit another pull request upstream to allow the self-signed cert when in dev mode [19:14:19] but my civi install did go well [19:15:52] (03CR) 10Ejegg: [C: 03+2] Remove code that sets UI default for no thank you [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994872 (https://phabricator.wikimedia.org/T356366) (owner: 10Eileen) [19:16:31] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Unplanned-Sprint-Work: Remove engage UI code - https://phabricator.wikimedia.org/T356433 (10Eileenmcnaughton) [19:30:47] so on a fresh install I think we can omit a lot of those prompts too [19:31:05] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Unplanned-Sprint-Work: Remove engage UI code - https://phabricator.wikimedia.org/T356433 (10MDemosWMF) That's right, Engage is no longer entering data into Civi so we wouldn't need this user role. [19:31:06] e.g. if the payments db doesn't exist, always install the wiki [19:31:28] if the vendor dir doesn't exist, always composer install [19:34:27] (03Merged) 10jenkins-bot: Remove code that sets UI default for no thank you [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994872 (https://phabricator.wikimedia.org/T356366) (owner: 10Eileen) [19:38:22] ejegg: is the bug with civiproxy on the new build patch [19:38:51] oh wait cstone I had the same problem with civicrm cloning [19:38:58] sorry I spoke too soon [19:39:15] so it looks like src/civi-sites got created as root/root [19:39:39] oh wow also civiproxy, privatebin, and tools [19:39:51] jgleeson: yeah, this is a build in a fresh directory [19:41:02] so let's see, why would those 4 directories get created as root/root and the others not [19:41:38] there are some dirs that i noticed do get created by the civicrm image by root [19:41:47] which i think i commented out [19:42:58] ah it might be that ejegg because tools was one of them [19:44:46] I'll try rebuilding from a new dir [19:45:55] so civiproxy has those overlay mounts [19:47:59] jgleeson: should we just create those dirs as the setup user before bringing the containers up [19:48:02] ? [19:49:08] just looking [19:49:17] orrr right, I think we need to clone all the dirs before we bring up any containers [19:49:29] since bringing the containers up will do some overlay mounting [19:49:48] and then the directories won't be empty to clone stuff into [19:50:14] ejegg: when you say overlay, is that when a directory is mounted and then an additional mount with a targeted file in that dir? [19:50:20] if so, that should be fine [19:51:09] so git clone unfortunately just dies when it sees there are already files in e.g. the civiproxy dir [19:51:25] since we are trying to do the clone after we bring the image up [19:52:33] 10Fundraising Spring - anybodyKnowWhatThisDoes(), 10Fundraising-Backlog: Correcting Gift Source Coding on Benevity import - https://phabricator.wikimedia.org/T354911 (10NNichols) @Eileenmcnaughton Everything looks good with my test! We are going to work on fixing the other months and will come back if we see... [19:52:36] ok I see the issue [19:54:08] lets see how easy it is to change the order of things [19:54:19] so should we move the 'docker compose up' bits into the individual setup_ functions? [19:54:28] just looking at that [19:54:34] and out of the top of the --full case? [19:54:36] cool cool [19:54:37] although I think it does [19:54:47] ontop of the initial [19:56:52] I think there's also a way to stop it creating dirs as root [19:57:14] it should be running as the defined user [19:59:56] if we create them first it should be OK i think [20:01:39] yeah although that initial bootup for full might be trickier [20:02:42] I guess it has to run as root to create a bind mount [20:02:50] if the host folder doesn't exist [20:07:56] is there every a time when we don't want the code under /src/ not to be owned by the host user [20:08:12] it might be easier to run a post-run chmod -R script to ensure that [20:08:20] chown * [20:09:12] I'll try that first [20:09:18] saves updating each file [20:10:39] (03PS1) 10Eileen: Remove legacy engage role code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995092 (https://phabricator.wikimedia.org/T356433) [20:11:46] huh that might not work [20:13:48] the docker daemon creates the bind mount source dirs as root due to them not existing at the time of the docker compose up cmd. I just noticed this is also happening on the -depends: containers too, which means the user running the script won't be able to change the perms post-run [20:13:51] grrr [20:14:16] ok I guess we can just ensure the bind mount source dirs exist prior to bring anything up [20:17:24] (03PS1) 10Eileen: Replace is single field with marital status [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 [20:17:37] ok, and maybe we stop the civiproxy container when we want to clone that one [20:18:12] (03CR) 10CI reject: [V: 04-1] Remove legacy engage role code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995092 (https://phabricator.wikimedia.org/T356433) (owner: 10Eileen) [20:20:18] Can I get + 2 on https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/995093 - I need that field to do the data axle import [20:23:40] oh we can do this another way. a pre-run event [20:29:30] (03CR) 10Ejegg: [C: 03+2] Replace is single field with marital status [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 (owner: 10Eileen) [20:30:27] (03CR) 10Ejegg: [C: 03+2] Replace is single field with marital status (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 (owner: 10Eileen) [20:37:08] (03PS1) 10Jgleeson: Add init script to ensure bind-mount source directories exist before build. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995096 [20:37:21] ejegg: cstone that should fix the root perms bugs ^^ [20:38:09] gonna test it out in a fresh dir now [20:38:14] (03CR) 10CI reject: [V: 04-1] Replace is single field with marital status [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 (owner: 10Eileen) [20:38:40] thanks jgleeson, i'll test it too [20:40:52] ha [20:40:59] oh right i need to stop the other one first to avoid port clashes [20:41:11] now I get this on a fresh build which feels a little confusing [20:41:13] **** Clone CiviCRM Buildkit [20:41:15] src/civicrm-buildkit exists. Remove and re-clone CiviCRM Buildkit repo? [yN] y [20:41:17] Are you sure? This will delete src/civicrm-buildkit, including local branches. [yN] [20:42:28] jgleeson ah right, I make it check for the .git dir in this one: https://gerrit.wikimedia.org/r/995084 [20:42:45] instead of just the empty folder [20:43:25] eileen: we need to speed up tests, or extend the php execution timeout on CI ! [20:43:48] ejegg: yeah - it is possible to cheat & reset the timer in code? [20:44:15] not sure why it slowed - wasn't immediate after the upgrade was merged [20:44:15] with an ini_set or something? [20:44:19] yep [20:44:25] worth a shot [20:44:51] ejegg: how about skip-clone [20:45:07] are they all reclones [20:45:13] just feels like an odd word [20:45:30] :S [20:45:47] So i only want it to skip when the dir is already there and it's deleting and cloning again [20:46:02] (03PS2) 10Eileen: Replace is single field with marital status [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 [20:46:06] if the .git dir is missing, it should clone fresh even with skip-reclone [20:46:18] hence the 're' [20:46:44] I think that's what it ends up doing in the fn, right? only checking SKIP_RECLONE if the dir exists? [20:47:36] ok jgleeson that does stop making the dirs as root [20:47:39] so on the speed thing - I wonder if it is cumulative slow down or a slow test - it says 180 sec - but when is that 'reset' [20:47:56] but we still need to have the civiproxy dir empty when we try to clone into it [20:48:14] eileen: cumulative, so one of the last tests we added was the last straw [20:48:26] i think it's the process that starts when you run phpunit [20:48:46] hmm - but the tests total take longer than 180 sec don't they? [20:49:16] the full install process takes a lot of the CI time [20:49:38] but that's a lot of separate php executions [20:49:56] then phpunit runs all the tests, and I think that phase has just started taking 3 min [20:50:33] jgleeson: I'm still getting the civiproxy/proxy folder created as root [20:51:02] and if we pre-create that then it'll refuse to clone [20:51:19] i think we do need to do all the clones before bringing up any containers [20:51:46] or at least that one [20:52:01] what's the issue with that one? [20:52:14] I'm just looking at a way to not have to pass --mac first [20:52:27] I don't like the assumed order [20:52:36] we need to clone into an empty directory [20:53:14] and when the civiproxy container is brought up it creates src/civiproxy/proxy (as root) and then overlay mounts proxy.php in there [20:53:36] ah [20:53:36] so it seems like even when we bring the container down, the root-owned folder stays there [20:54:12] i'm going to try just removing the ' docker compose -f $DOCKER_COMPOSE_FILE up -d [20:54:15] ' [20:54:21] line from the --all case [20:54:35] hmm [20:54:41] (03PS1) 10Eileen: Try re-setting time out somewhere in the middle [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995098 [20:54:50] ejegg: well we can see what impact ^^ has [20:55:29] sure eileen , I hope that works! [20:55:37] Oh i made a phab for it and tagged releng [20:55:44] let's see if they said anything [20:56:14] nope nothing yet [20:56:23] but if your patch works we can close https://phabricator.wikimedia.org/T356361 [20:56:30] ejegg: so is the issue your having due to another container bring civiproxy up before the script [20:57:09] oh i think it was just that docker compose -f $DOCKER_COMPOSE_FILE up -d [20:57:11] 10Fundraising Spring - anybodyKnowWhatThisDoes(), 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10fundraising-tech-ops: Remove disabled custom fields - https://phabricator.wikimedia.org/T356443 (10Eileenmcnaughton) [20:57:24] which runs before any of the project-specific functions [20:58:04] 10Fundraising Spring - anybodyKnowWhatThisDoes(), 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10fundraising-tech-ops: Remove disabled custom fields - https://phabricator.wikimedia.org/T356443 (10Eileenmcnaughton) [20:58:13] hmm I'm surprised none of the depends: are not bringing that up [20:58:22] email-prefs probably runs after it [20:58:34] any thoughts on what confirmation, if any, we need to do for this https://phabricator.wikimedia.org/T356443 - perahps just get Nora to confirm? [20:58:35] yeah it does [20:58:36] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review, 10Unplanned-Sprint-Work: FY2324 Q3: Need to map new Civi fields over to Acoustic - https://phabricator.wikimedia.org/T353386 (10KHaggard) I think we exported the full Unsubscribes list because it wasn't causing any slowdowns to do it a... [21:00:02] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review, 10Unplanned-Sprint-Work: FY2324 Q3: Need to map new Civi fields over to Acoustic - https://phabricator.wikimedia.org/T353386 (10Eileenmcnaughton) @KHaggard - there is no push to not do the full list on our part - I think I just need to... [21:00:22] oops, i left that 'export' in [21:00:55] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "Thanks for this! I'm gonna look at getting rid of the need for the strict ordering of flags as a follow on." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 (owner: 10Ejegg) [21:01:12] oh crap I saw that and still +2ed it [21:01:24] hah, no worries, I'll remove it in a follow-on [21:01:24] (03CR) 10Jgleeson: [V: 04-1 C: 04-1] Add --skip-reclone option for setup [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 (owner: 10Ejegg) [21:01:34] it doesn't auto merge right? [21:01:38] I think we're ok [21:01:38] ah nope [21:01:45] (03CR) 10Jgleeson: Add --skip-reclone option for setup [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 (owner: 10Ejegg) [21:01:50] ejegg: was this one OK too? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/994876 - if so I'll recheck that Queue one over the top of it [21:02:54] 10Fundraising Spring - anybodyKnowWhatThisDoes(), 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10fundraising-tech-ops: Remove disabled custom fields - https://phabricator.wikimedia.org/T356443 (10Eileenmcnaughton) @NNichols can you eyeball the above list & confirm you still want to wave your magic w... [21:03:54] i think so eileen, i just wanted to pull it down first [21:03:58] oh cool [21:04:02] only now i'm in a bit of a docker muddle [21:04:47] (03CR) 10Eileen: [C: 03+2] Replace is single field with marital status [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 (owner: 10Eileen) [21:05:15] grr, suddenly can't pull from gerrit or push to review [21:05:32] (03CR) 10Eileen: [C: 03+2] "I put the +2 back because there was no material change - just removed an extra highlighted comment (& I didn't want to lose the v+2 given " [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995093 (owner: 10Eileen) [21:05:49] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995092 (https://phabricator.wikimedia.org/T356433) (owner: 10Eileen) [21:07:30] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/995100 [21:07:50] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/995100 (owner: 10Eileen) [21:08:04] jgleeson: want to C+2 the parent of that patch, just fixing the email pref ctr db name in public config? [21:08:37] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/995100 (owner: 10Eileen) [21:10:28] !log civicrm upgraded from 21bf2138 to f89f3a58 [21:10:30] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:11:10] (03PS2) 10Ejegg: Add --skip-reclone option for setup [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 [21:11:21] ok jgleeson I fixed it with the webUI ^^ [21:11:41] lookin [21:12:34] I just realised how bad of a suggestion my skip-clone was.... everything else is called reclone! ha [21:12:46] brain fog tonight [21:13:34] ok, i need to fix my stunnel config I guess [21:13:40] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "Take Two: Thanks for this! I'm gonna look at getting rid of the need for the strict ordering of flags as a follow on." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995084 (owner: 10Ejegg) [21:13:57] ejegg: I wanted to chat about that [21:14:02] I had some backup way to tunnel to gerrit over https but I guess that's not working any more either [21:14:09] it looks like that used to be used to scp down the githooks [21:14:15] but now gerrit doesn't use scp [21:14:30] oho, are the githooks not being installed then? [21:15:19] I think it might have used to use scp but it doesn't anymore [21:15:53] checkout the clone command here ejegg https://gerrit.wikimedia.org/r/admin/repos/wikimedia/fundraising/dev,general [21:16:00] it uses mkdir and other stuff [21:16:14] I think it might have used scp in the past? but I don't see it anymore [21:16:26] ah curl [21:16:35] right, that sounds familiar [21:17:09] that was one of the changes I made because it looked like it wasn't needed anymore [21:17:18] you can compared with build/old/setup.sh [21:17:22] -d [21:19:21] gonna reboot the router and see if that helps my gerrit connection [21:19:34] huh [21:20:16] 10Fundraising-Backlog: Matching gift import - https://phabricator.wikimedia.org/T356445 (10Elbar53) [21:21:02] ejegg: wanna +2 this one https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/dev/+/995096 [21:22:47] ejegg: it passed - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/995098 - not sure if it was the change or the fluke but if we merge it we can see if other test runs improve [21:23:01] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "LGTM" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/994859 (owner: 10Ejegg) [21:23:43] (03PS2) 10Eileen: Try re-setting time out somewhere in the middle [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995098 (https://phabricator.wikimedia.org/T356361) [21:24:01] - I just added the bug to the commit message [21:26:20] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "fr-tech bat signal. this will require an update to mysql workbench. port 3306 now needs to be 3007." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 (owner: 10Ejegg) [21:27:13] (03CR) 10Ejegg: [C: 03+2] "Thanks, this fixes cloning civi!" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995096 (owner: 10Jgleeson) [21:27:25] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Add init script to ensure bind-mount source directories exist before build. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995096 (owner: 10Jgleeson) [21:27:28] (03CR) 10Eileen: "Ug - that's gonna take a bit of adjusting to for me" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 (owner: 10Ejegg) [21:27:39] (03PS2) 10Ejegg: Add init script to ensure bind-mount source directories exist before build. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995096 (owner: 10Jgleeson) [21:28:05] eileen: I think the other convention I've seen is 33066:3306 [21:28:14] (03CR) 10Ejegg: "eileen if you already have a .env file with 3306 you don't need to change. This is just the default for new setups." [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 (owner: 10Ejegg) [21:28:32] yeah - it's just that I have it in multiple idea workspaces [21:28:38] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Add init script to ensure bind-mount source directories exist before build. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995096 (owner: 10Jgleeson) [21:29:14] so that .env file - not sure I know too much about that one [21:29:15] eileen you can set it still to 3306 and keep it as it is [21:29:19] also - can we merge https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/dev/+/986201 [21:29:55] oh that's nice eileen [21:30:08] (03PS3) 10Ejegg: Add path to docker-composer.override.samples [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/986201 (owner: 10Eileen) [21:30:27] (03CR) 10Ejegg: [C: 03+2] Add path to docker-composer.override.samples [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/986201 (owner: 10Eileen) [21:30:29] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Add path to docker-composer.override.samples [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/986201 (owner: 10Eileen) [21:30:31] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] Add path to docker-composer.override.samples [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/986201 (owner: 10Eileen) [21:30:35] ha [21:31:01] eileen: either in .env or build/ports.sh [21:31:15] so would .env go under Config/civicrm ? [21:31:18] .env is better as it will persist between fundraising-dev updates [21:31:27] eileen: .env is in the top-level [21:31:31] no it's in the fundraising-dev root [21:31:32] it's got ports for everything [21:31:37] ah - yeah I see it [21:31:45] ./setup.sh env [21:31:51] will show you the content [21:31:53] s [21:32:18] ok - so that is not in git & won't change unless I choose to? [21:32:26] right [21:32:37] ok [21:34:08] (03PS3) 10Ejegg: Try re-setting time out somewhere in the middle [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995098 (https://phabricator.wikimedia.org/T356361) (owner: 10Eileen) [21:34:11] (03CR) 10Ejegg: [C: 03+2] Try re-setting time out somewhere in the middle [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995098 (https://phabricator.wikimedia.org/T356361) (owner: 10Eileen) [21:34:18] there's a fn in ./setup.sh called init_env that looks at builds/ports.sh and creates a .env file from it [21:34:46] ejegg: want me to kill that docker-compose up in --full [21:35:03] oh right, i'm testing that now [21:35:08] i think it's redundant [21:35:14] yep yep, i think so too [21:35:18] or should be if the individual builds work correctly [21:36:12] also I noticed the docker_compose_up convenience method I added passed `--force-recreate` and now I'm wondering why I did that [21:36:21] not using it will likely make things faster [21:37:00] and I can't think why force recreating a container, and potentially destroying any app generated files not mounts, is a good idea [21:37:09] not mounted* [21:37:35] FYI - notes on 1-click from Tim - https://lab.civicrm.org/dev/core/-/wikis/CiviMail-One-Click-Unsubscribe [21:38:15] ejegg: got a fix for the --mac --reclone ordering [21:38:19] will push up [21:38:54] if it helps jgleeson these are the aliases I use alias bkb='docker-compose --file ~/dev/fundraising-dev/docker-compose.yml exec - [21:38:54] w /srv/civi-sites civicrm bash' [21:38:54] alias bkr='docker-compose --file /home/eileen/dev/fundraising-dev/docker-compose [21:38:55] .yml exec -u root civicrm bash' [21:38:55] alias bkd='docker-compose --file /home/eileen/dev/fundraising-dev/docker-compose [21:38:55] .yml down' [21:38:55] alias bku='docker-compose --file /home/eileen/dev/fundraising-dev/docker-compose [21:38:55] .yml --file /home/eileen/dev/fundraising-dev/docker-compose.override.yml --env- [21:38:56] file /home/eileen/dev/fundraising-dev/.env up -d' [21:38:56] alias redis-monitor='docker-compose --file /home/eileen/dev/fundraising-dev/dock [21:38:57] er-compose.yml exec queues redis-cli monitor' [21:38:57] alias redis-cli='docker-compose --file /home/eileen/dev/fundraising-dev/docker-c [21:39:24] I have a bunch too [21:39:33] are you suggesting we put them in the project? [21:39:43] maybe a utils dir [21:40:06] well - if they are in the project then what would people do to add them to aliases? [21:41:00] it would still require them to be added to the usual $SHELLrc file [21:41:09] e.g. .bashrc [21:41:19] zshrc on mac I think [21:42:05] mine are more generic [21:42:06] # docker shortcuts [21:42:08] alias dps="docker ps" [21:42:10] alias dc="docker compose" [21:42:12] alias dcu="docker compose up -d" [21:42:14] alias dcd="docker compose down" [21:42:16] alias dce="docker compose exec" [21:42:18] alias dcb="docker compose build" [21:42:42] I forgot about this one alias cma='cmatrix -b' [21:42:53] jgleeson: wel I suppose if they were in a utils that we added to the patch [21:43:03] path - then we could share them [21:44:28] now I think about it I could add an alias for civibuild so it doesn't matter if I'm logged into the shell or not [21:44:47] yep [21:46:13] 10Fundraising-Backlog, 10FR-Amazon, 10Recurring-Donations, 10Epic: Amazon recurring (Epic) - https://phabricator.wikimedia.org/T107391 (10AKanji-WMF) After speaking with colleagues, we would like to officially implement Amazon Recurring as a payment method. [21:47:21] 10Fundraising Sprint Uptown cURL, 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review: Data Axle Project - Re-import to Civi - https://phabricator.wikimedia.org/T341902 (10Eileenmcnaughton) OK - that seems to have worked - https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=19... [21:47:46] 10Fundraising Sprint: brb(), 10Fundraising-Backlog: Add ACH payment type for US donation processing - https://phabricator.wikimedia.org/T324517 (10AKanji-WMF) Flagging for awareness that our next direct-debit payment priority is SEPA: {T324520} [21:48:08] ejegg: I'm ready to start that Data Axle import - any reservations about me kicking it off - we should be able to pause the queue if need be [21:48:28] dwisehaupt: is away right now but that feels non-blocking to me [21:50:30] (03PS1) 10Jgleeson: Detect --mac and --skip-reclone flags first so strict ordering isn't needed. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995105 [21:50:53] ejegg: ^^ that gets rid of the need for --mac and --skip-reclone strict ordering [21:51:04] (03Merged) 10jenkins-bot: Try re-setting time out somewhere in the middle [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/995098 (https://phabricator.wikimedia.org/T356361) (owner: 10Eileen) [21:51:15] and it restores the order in the help text :> [21:55:31] hmm maybe we don't need --mac [21:56:22] I could just set an .env var based on looking at the `uname` [21:56:33] that way we don't even need to remember a flag [21:57:34] uname will show Linux/Darwin(MacOS) [21:59:48] damilare: cstone wfan. is it annoying to have to remember to use `--mac` before other flags? [22:00:02] for me no [22:00:05] or is it safe to assume mac users would just like the mac config [22:00:31] It's ok to add --mac [22:01:10] ok cool I'll leave it alone [22:01:50] fr-tech any name suggestions for a command that drops all contains, volume mounts, dbs and queues along with deleting ./src [22:02:06] the obvious ./setup.sh reset sounds too casual [22:02:07] eileen: i'm back now. [22:02:12] maybe ./setup.sh nuke [22:02:19] maybe ./setup.sh destroy [22:02:21] go for it on the import [22:02:25] dropItLikeItsHot [22:02:30] LOL [22:02:34] Wait, we aren't in sprint naming... [22:02:38] ;-) [22:02:48] haha flush or destroy all sounds good to me [22:03:06] containers*** [22:04:26] does it just delete or delete and restart? [22:05:37] dwisehaupt: kicking it off [22:06:32] 10Fundraising Sprint Uptown cURL, 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10Patch-For-Review: Data Axle Project - Re-import to Civi - https://phabricator.wikimedia.org/T341902 (10Eileenmcnaughton) It is running https://civicrm.wikimedia.org/civicrm/queue/monitor?name=user_job_1101 [22:08:18] destroy though [22:08:53] cstone: kinda like a clean slate command [22:09:15] deletes all src, .env, containers and volumes including database and redis [22:09:29] ./setup.sh clean? [22:09:30] probably only useful for testing new builds [22:09:44] yeah I like clean also but that can sometimes mean clean code up [22:09:49] and not destroyer of worlds [22:10:08] echo "I AM DEATH" [22:10:29] ./setup.sh rip [22:10:33] lol [22:10:45] I actually like that one [22:11:10] any objections to ./setup.sh rip fr-tech [22:11:34] funny enough to not get called by accident [22:11:55] haha i guess not called by accident but might cause future confusion vs destroy? [22:12:13] ok destory it is [22:15:13] ok - the data import has started - a bit more than 100k rows & so far 850 rows are in [22:20:03] jgleeson: catching up on all the work you and elliott did earlier, is current docker testing state master dev + the patch you are working on right now? [22:20:56] (03CR) 10Ejegg: [C: 03+2] Detect --mac and --skip-reclone flags first so strict ordering isn't needed. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995105 (owner: 10Jgleeson) [22:20:58] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Detect --mac and --skip-reclone flags first so strict ordering isn't needed. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995105 (owner: 10Jgleeson) [22:21:05] problem sovled to my question haha [22:21:36] cstone: try a new build in a fresh dir [22:21:42] yep yep doing that now [22:21:43] docker compose down -v any others [22:21:53] jgleeson: how about removing that extra d c up? [22:22:01] ohhh doing it [22:22:02] or did you already merge that? [22:22:06] cstone: hang fire [22:22:13] too late but its ok ahha [22:22:16] cstone: just one more fix for the civiproxy [22:22:23] civicrm should work this time though [22:22:25] nice [22:22:47] ah is this the permissions fix? [22:22:57] yeah, basically [22:23:06] ok cool [22:23:09] just creating the dirs first so root doesn't create them [22:24:59] bah i forgot i need to run the ups store ill be back in like 10 mins [22:25:28] 10Fundraising Sprint: brb(), 10Fundraising-Backlog, 10fundraising-tech-ops: Create way to used shared version of logdog on logging server - https://phabricator.wikimedia.org/T355713 (10Dwisehaupt) Added to the fundraising module as `fundraising::logdog`. Deployed to the civi and logger roles. Waiting for ver... [22:25:56] ejegg: my IDE is telling me SKIP_RECLONE is missing a $ [22:26:06] (03PS1) 10Wfan: add more queue for opt-in and unsubscribe [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995017 [22:26:10] in ask_reclone [22:27:00] ooh probably jgleeson [22:27:05] lemme fix that [22:28:51] (03PS1) 10Jgleeson: Remove redundant `docker compose up` from --all build [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995111 [22:28:55] I think I also need to use the bash constants instead of the strings [22:29:12] (03CR) 10Ejegg: [C: 03+2] Remove redundant `docker compose up` from --all build [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995111 (owner: 10Jgleeson) [22:29:14] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Remove redundant `docker compose up` from --all build [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995111 (owner: 10Jgleeson) [22:30:47] ty ejegg [22:30:51] trying a new build now [22:31:02] core takes a while to download [22:31:16] actually just gonna try a --mac --civicrm [22:31:29] I still think there is a bug with `--mac --civicrm-core` [22:31:51] I was getting some "can't find driver` exception when buildkit was setting up the DB [22:31:55] only on --mac [22:32:09] I broke something because it was working a day earlier [22:34:35] (03PS2) 10Ejegg: Default exposed port for mariadb to 3307 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 [22:34:37] (03PS1) 10Ejegg: Fix SKIP_RECLONE [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995112 [22:34:53] ok jgleeson ^^^ should fix it [22:36:10] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "LGTM!" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995112 (owner: 10Ejegg) [22:36:26] (03CR) 10Jgleeson: [V: 03+2] Default exposed port for mariadb to 3307 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 (owner: 10Ejegg) [22:36:33] (03CR) 10Ejegg: [V: 03+2] Default exposed port for mariadb to 3307 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995085 (owner: 10Ejegg) [22:36:48] thanks! [22:36:55] ok, i'll try a fresh install too [22:38:20] I think it's asking for the test address multiple times [22:39:07] ooh, it's recreating the civicrm container before installing buildkit? [22:39:10] odd [22:39:16] but i'm not sure if that causes problems [22:39:17] the proxy id [22:39:18] ? [22:39:56] buildkit gets put on the civicrm container [22:40:13] oh right, and it comes first, so rebuild won't clobber any other civi stuff [22:42:36] eileen: from the db side, looks like minimal to no impact from the data load. [22:42:40] **** MacOS Setup: sync container source code to local to retain generated build config [22:42:42] container => local: sync complete - ./src/civi-sites/wmff:/srv/civi-sites/wmff [22:42:44] container => local: sync complete - ./src/civi-sites/wmff.sh:/srv/civi-sites/wmff.sh [22:42:46] CiviCRM WMFF Installed! [22:42:48] CiviCRM WMFF URL: https://wmff.localhost:32353/civicrm [22:42:50] Civicrm WMFF user/password: admin/admin [22:42:52] real 4m39.334s [22:42:54] user 0m16.460s [22:42:56] sys 0m5.692s [22:42:58] Unknown flag: --mac [22:43:00] unknown flag?!?! [22:43:29] ah, because you moved --mac into the first loop [22:43:47] need to explicitly skip that and ==skip-reclone in the second loop now [22:44:23] oh right [22:44:26] ah [22:44:29] thanks [22:44:58] jgleeson: so the old setup would save a container prefix (default fundraising-dev) to the env file [22:45:06] but the new one always seems to use the pwd [22:45:25] which is a problem for me because I have ~/src/docker/fundraising-dev symlinked to ~/fd [22:45:27] oh right [22:45:32] and if I set up in ~/fd [22:45:37] fr-container [22:45:41] fd* [22:45:42] then try to stop or something in the full path [22:45:54] it doesn't have the right container name [22:46:03] ok I'll bring that bit back [22:46:08] thanks! [22:46:12] that was another one I never knew why we had it [22:46:37] ah so it's a docker compose default to use the dirname! [22:46:39] as I thought it was intended for two distinct builds of the stack but then you'd have to use different ports for all services [22:46:46] i was looking for `pwd` or something [22:47:39] well, i have working civi again, i'm going to review a few more eileen patches [22:49:10] cool. cstone it looks like --mac --civicrm-core is broken still. --mac --civicrm works! [22:49:17] not sure what's causing this [22:49:21] [[Install dmaster/default (type 'drupal-demo' in '/srv/civicrm-buildkit/build/dmaster')]] [22:49:23] [[Execute /srv/civicrm-buildkit/app/config/drupal-demo/install.sh]] [22:49:25] [[Setup MySQL and HTTP for CMS]] [22:49:27] PHP Fatal error: Uncaught Error: Call to a member function getDriver() on null in phar:///srv/civicrm-buildkit/bin/amp/src/Amp/Database/MySQL.php:59 [22:49:29] Stack trace: [22:49:31] #0 phar:///srv/civicrm-buildkit/bin/amp/src/Amp/InstanceRepository.php(56): Amp\Database\MySQL->createDatasource() [22:49:33] #1 phar:///srv/civicrm-buildkit/bin/amp/src/Amp/Command/CreateCommand.php(107): Amp\InstanceRepository->create() [22:49:35] #2 phar:///srv/civicrm-buildkit/bin/amp/vendor/symfony/console/Command/Command.php(255): Amp\Command\CreateCommand->execute() [22:49:37] #3 phar:///srv/civicrm-buildkit/bin/amp/vendor/symfony/console/Application.php(1009): Symfony\Component\Console\Command\Command->run() [22:49:39] #4 phar:///srv/civicrm-buildkit/bin/amp/vendor/symfony/console/Application.php(273): Symfony\Component\Console\Application->doRunCommand() [22:49:41] #5 phar:///srv/civicrm-buildkit/bin/amp/vendor/symfony/console/Application.php(149): Symfony\Component\Console\Application->doRun() [22:49:43] #6 phar:///srv/civicrm-buildkit/bin/amp/src/Amp/Application.php(62): Symfony\Component\Console\Application->run() [22:49:46] #7 ph in phar:///srv/civicrm-buildkit/bin/amp/src/Amp/Database/MySQL.php on line 59 [22:49:47] only happens with the --mac (no local files mounted into container) [22:49:49] some missing config maybe? [22:50:43] could be. we don't send any code in when building civicrm-core. builtkit picks the code I think [22:51:06] I'll see what it creates without --mac [22:51:11] didnt account for traffic back now though [22:51:16] as that works [22:51:33] oh also [22:51:44] there's pretty colours if you run `./setup.sh urls` [22:52:09] heh [22:57:00] (03PS1) 10Jgleeson: Add ./setup.sh destroy command [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995113 [22:57:54] (03PS2) 10Ejegg: Add TheIsland theme [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994871 (https://phabricator.wikimedia.org/T356365) (owner: 10Eileen) [22:58:18] (03CR) 10Ejegg: [C: 03+2] "Confirmed, this looks better!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994871 (https://phabricator.wikimedia.org/T356365) (owner: 10Eileen) [23:03:04] eileen: do you want to fix the casing of Wmf along with the move in that top queue object move patch? [23:03:12] Seems like a good time to do it [23:03:20] so we standardize on WMF [23:05:56] 10Fundraising Sprint: brb(), 10Fundraising-Backlog: Scheduled job to age recurring statuses not running correctly. - https://phabricator.wikimedia.org/T355850 (10Ejegg) For the nitty-gritty of why the 'drush' command wasn't found: process-control runs its commands via python's subprocess.Popen, which doesn't a... [23:07:43] ah good to know [23:13:41] re:misisng config. exactly that ejegg [23:14:06] pushing up a fix and then you should be good to go cstone [23:14:11] fingers...crossed [23:14:57] ah im halfway through an earlier attempt but it hasnt broken yet although just got to civi [23:16:02] (03Merged) 10jenkins-bot: Add TheIsland theme [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/994871 (https://phabricator.wikimedia.org/T356365) (owner: 10Eileen) [23:19:03] wmff will install but --core will fail [23:19:18] you should be able to install it afterwards individually though [23:19:23] once I test this patch [23:21:56] ok cool just getting to that step [23:23:24] although mac or linux jgleeson [23:23:33] --mac [23:23:47] linux is fine. I broke --mac --civicrm-core [23:23:51] ok hah im on linux with what im talking about [23:23:54] ok cool [23:24:09] jgleeson: is there any reason ppl on a mac wouldn't want to use the --mac switch sometimes? [23:24:12] oh cool. that should all work then if it was just the permissions issue from earlier [23:24:17] We could probably detect OS [23:24:20] yeah so far so good [23:24:30] ejegg: I mentioned that a bit earlier, with uname [23:24:36] oh i missed it [23:24:52] I figured it was maybe glitter we didn't need [23:25:14] I asked folks and they were ok to keep it [23:25:19] k cool [23:25:44] I can't see a reason why someone on a mac would want to use the bind-mount compose config though [23:26:02] oh wow, queue consume output went minimal [23:26:12] let's see how we can get back a little bit of that [23:26:25] i have a civi on linux! [23:26:26] i don't like all the dumb 'box requirements' boilerplate [23:26:37] but i do want to see if it's consuming messages or not [23:27:34] nice cstone [23:27:57] cstone when it completes. checkout `./setup.sh urls` [23:28:03] prettyiful [23:28:05] wait, service queues is not running [23:28:29] ooh colors [23:28:37] oh derp, am in the full path and not the symlink [23:28:38] although not as impactful on a whitebackground :P [23:29:44] errr, what's happening here [23:29:53] :) [23:30:05] ejegg: I'm confused [23:30:10] did I break the queues> [23:30:12] ? [23:30:13] cstone: can you run scripts/civicrm-consume-queues.sh and see what output you get? [23:30:22] I just get all the 'Running... ' lines [23:30:25] (with pretty colors) [23:30:27] oh right [23:30:29] lets see [23:30:35] but no actual output of the consumer scripts [23:30:53] i get usual spam [23:31:02] huh, ok, lemme keep digging [23:31:05] looks liek they are running [23:31:17] processed a contribution tracking one i did [23:31:23] ohhh it still refers to docker-compose locally [23:31:24] i did get instant error though when trying to donate looking into that [23:31:28] not docker compose [23:32:03] the logs dont seem to exist [23:32:39] ok, replacing hyphen with space is all i needed [23:32:48] do you have any logs ejegg [23:32:53] looking [23:33:18] yep, but i'm in my old dir now [23:33:30] i blew away the fresh install [23:33:44] hmm my fresh install does not seem to have them [23:34:05] (03PS1) 10Ejegg: Use compose v2 for queue consume script [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995116 [23:34:13] jgleeson: ^^^ fixed it for me [23:34:22] ejegg: why do you think that worked for me though [23:34:28] cause i still have the old one too ? [23:34:34] ohhhh [23:34:40] errr no idea cstone [23:34:41] I missed one [23:34:55] oh wait I think we updated the colours recently on that [23:35:05] maybe the dash came back with that patch [23:35:13] lemme try the no -s [23:35:19] (03CR) 10Jgleeson: [V: 03+2 C: 03+2] "Good spot!" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995116 (owner: 10Ejegg) [23:36:13] hmm [23:37:04] how did i have a contribution tracking entry [23:37:39] i guess that was my only 1.1 I get hah [23:37:48] there we go [23:38:24] ok yeah works without the - too (but also worked with the - haha) [23:38:32] jgleeson: do you have logs? [23:38:56] I have nothing at the moment I ran destroy [23:39:06] aren't you getting any? [23:39:07] haha [23:39:09] no [23:39:20] i have empty folder [23:39:20] hmm [23:39:30] well wieth a gitignore [23:42:27] did config private step get completely separated too (looking for it) [23:43:12] bah i need to go grab food, thanks for working so late on this jgleeson I'll add my questions to etherpad [23:43:39] cstone: yeah the config-private should come up st the start [23:43:52] checkout init() in the setup.sh [23:43:59] thanks for notes and testing also [23:46:04] (03PS1) 10Ejegg: Strip some spam off the start of queue consumer logs [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/995118 [23:46:14] gotta run! [23:59:27] oh I think I broke the config-private setup cstone so that's why you didn't see it. I'll push up a fix now