[00:07:31] (03Merged) 10jenkins-bot: Match new RML group members with orphan data [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/763617 (https://phabricator.wikimedia.org/T222854) (owner: 10Ejegg) [02:00:52] ok fr-tech I'm going to deploy Civi [02:02:49] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/767265 [02:11:40] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/767265 (owner: 10Ejegg) [02:13:06] !log Fundraising CiviCRM updated from 2874d623 to e9f0eff5 [02:13:07] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:34:19] eileen: what do you think of simply adding the RML match job to the omnimail_groupmember_load job so it always tries to match 1000 rows after a new import? [02:34:58] ejegg: is that actually easier? Feels less visible - but you are thinking for long term keeping up reasons? [02:35:42] eileen: just easier to add one line to the other yaml than to work out exactly when to schedule it so it doesn't conflict time-wise with the job that write new rows to the table [02:35:54] or do you think that's not a worry? [02:36:22] If it's easier I think it's OK - I don't have a strong feeling either way [02:36:54] hmm, I guess it would muddy the output a bit [02:37:34] I do want to run it quickly after the RML load job just to try to catch the contacts before they're merged to a lower number [02:38:20] ohh, do merges update civicrm_mailing_provider_data contact_ids? [02:48:05] eileen: OK, I pushed a change to the process-control yaml [02:48:09] do you want to take a look? [02:49:59] ejegg: that looks OK - it's a bit confusing with the 2 action rows but that will fail hard if it is an issue so yep looks OK [02:50:11] cool cool [02:50:28] thanks for checking it out! [02:53:06] hmm, that only runs 56 times a day, so it's still 100+ days to run through all the contacts [02:54:26] I might up the batch size - it gets through the default 1000 batch in 7 seconds or so [02:54:34] I'll go with 10000 [02:55:07] right, that matches the limit on the load call [02:55:18] argh, though I gave the param a different name :P [02:58:46] ejegg: did it just cause a fail mail? [02:58:46] welp, that failed [02:58:53] :-) [02:59:06] yep, maybe I need to filter for contacts w/o email? [02:59:47] hmm, seems I have no limit on email is primary either [03:00:22] so let's see, I replace that left join with a full join and add the is_primary [03:00:28] might as well fix the batch param name [03:04:44] eileen: do you think I need to add is_deleted = 0 as well? [03:05:08] (and a join to the contact table) [03:05:15] ejegg: yeah I guess there is no point updating to a deleted contact [03:07:13] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Amazon_endpoint_critical 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages [03:07:44] man you stop doom-scrolling for a minute & it all goes off... https://www.stuff.co.nz/national/health/coronavirus/300529966/live-riot-police-rush-into-parliament-grounds-as-protesters-ignore-calls-to-leave [03:10:52] oh jeez [03:12:11] RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages [03:16:19] (03PS1) 10Ejegg: Fixes for RML match api action [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/767269 (https://phabricator.wikimedia.org/T222854) [03:16:30] does that look better eileen ? ^^^ [03:16:48] lemme just smoke test that [03:17:04] ejegg: I thought there was a default batch size? [03:17:48] oh so the 10k overrides the 1k [03:18:34] yep, I figured it should keep up with the load call [03:18:43] and it was tearing through the 1k pretty fast [03:18:56] like 10 sec [03:20:09] so the gerrit patch looks right - did you test it? [03:20:21] about to test on staging [03:20:52] actually, i might as well run a big batch on staging to reproduce the crash first [03:21:21] cool [03:23:16] huh, no crash on a 10k batch [03:24:24] guessing a contact whose email was deleted after the last staging refresh? [03:25:12] ok, the counter in prod got up to 3613470, so the problem should be with the next contact_id in that group [03:26:50] right, the next contact_id is 3615471 and has no matching email, so that's going to be fixed with the inner join [03:27:57] well, just to make sure it doesn't break or slow down like crazy I'll pull the patch and run a batch on staging [03:29:18] yeah, seems as fast as ever [03:29:24] eileen: patch checks out [03:29:47] ejegg: ok [03:29:55] (03CR) 10Eileen: [C: 03+2] Fixes for RML match api action [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/767269 (https://phabricator.wikimedia.org/T222854) (owner: 10Ejegg) [03:30:14] ohh I managed to click submit before it removed the v+2 :-) [03:42:04] thanks! [03:42:37] (03PS1) 10Ejegg: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/767273 [03:42:42] (03CR) 10Ejegg: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/767273 (owner: 10Ejegg) [03:42:50] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/767273 (owner: 10Ejegg) [03:43:59] !log updated CiviCRM from e9f0eff5 to cb0605ed [03:44:00] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:46:45] k, deployed (plus an update to the yaml to use 'limit') [03:46:53] and running [03:48:18] cool, that worked! [03:48:25] ok, signing off for now [03:51:46] 10Fundraising Sprint Cheese String Theory, 10Fundraising Sprint Discworld reformatted as ntfs, 10Fundraising-Backlog, 10FR-form-chooser-refactor: break out form chooser tasks - https://phabricator.wikimedia.org/T301799 (10Ejegg) https://etherpad.wikimedia.org/p/formchooser [04:27:28] (03PS5) 10Cstone: Filter media wiki country with civicrm countries [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/763274 (https://phabricator.wikimedia.org/T301493) (owner: 10Wfan) [04:32:41] (03CR) 10Cstone: [C: 03+2] "Looks good, thanks!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/763274 (https://phabricator.wikimedia.org/T301493) (owner: 10Wfan) [04:34:34] (03Merged) 10jenkins-bot: Filter media wiki country with civicrm countries [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/763274 (https://phabricator.wikimedia.org/T301493) (owner: 10Wfan) [04:51:09] (03PS4) 10Cstone: Email preference center's language list filter with civi [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/765324 (https://phabricator.wikimedia.org/T301911) (owner: 10Wfan) [04:58:08] (03CR) 10Cstone: [C: 03+2] "Thanks for all the hard work on this! Less language mess!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/765324 (https://phabricator.wikimedia.org/T301911) (owner: 10Wfan) [05:00:01] (03Merged) 10jenkins-bot: Email preference center's language list filter with civi [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/765324 (https://phabricator.wikimedia.org/T301911) (owner: 10Wfan) [05:00:21] Thanks Christine :) [05:53:12] (03CR) 10jerkins-bot: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_37) - 10https://gerrit.wikimedia.org/r/767329 (owner: 10L10n-bot) [12:56:54] (03PS5) 10Damilare Adedoyin: WIP: Fix civicrm contact preferred languages [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [13:38:03] (03PS6) 10Damilare Adedoyin: WIP: Fix civicrm contact preferred languages [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [16:11:06] (03CR) 10Jgleeson: [C: 03+1] "After today's successful test on staging I feel like it's ready to be taken out of WIP. Just a few minor points inline." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [16:14:47] hi fr-tech! [16:15:16] I think those failmails for the omnimail unsubscribe must be related to the new RML match job that's running [16:15:32] it hits the same tables, and isn't throttling at all [16:16:23] I'm going to see if that unsubscribe job really has a lot of work to do [16:16:27] hey ejegg [16:16:31] hi jgleeson [16:16:32] &fr-tech [16:19:10] hmm, seems like not really. I think I'm just going to up the time limit on that job [16:20:44] So the RML job is running through the whole existing RML group now, but once it catches up to the end of that group it will only have to process new rows [16:20:56] let's see how long that should take [16:23:09] 10-12 days probably [16:26:27] that's a dedicated job ejegg! [16:27:35] heh jgleeson we could have made it review fewer RML group members at the cost of a pricier query each run [16:28:03] it seemed better to burn a bunch of cycles when fundraising was at a low ebb and then be able to just use the cheap query to stay up to date. [16:31:12] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: failed recurring email should send user directly to recurring area of ways to give - https://phabricator.wikimedia.org/T302445 (10TomaszGorski) If it is not that easy to get a link to a specific part of the Ways to Give page, discussed with Sandra as an O... [16:32:07] sounds good [16:37:32] (03PS7) 10Damilare Adedoyin: Fix civicrm contact preferred languages [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [16:45:05] (03PS8) 10Damilare Adedoyin: Fix civicrm contact preferred languages [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [16:58:32] (03CR) 10Damilare Adedoyin: "Thanks Jack, the patch has been updated with your recommendations and some other refactoring." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/765305 (https://phabricator.wikimedia.org/T279389) (owner: 10Jgleeson) [17:37:11] PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Amazon_endpoint_critical 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages [17:42:11] RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages [17:50:51] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Unit tests for Form Chooser - https://phabricator.wikimedia.org/T302891 (10Ejegg) [17:53:46] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Move PayPal and Adyen to folders based on gatway code - https://phabricator.wikimedia.org/T302892 (10Ejegg) [19:23:02] 10Fundraising Sprint Discworld reformatted as ntfs, 10Fundraising-Backlog, 10FR-Adyen, 10FR-Gpay-Integration: Add Google Pay JS to Donation Interface - https://phabricator.wikimedia.org/T300686 (10AnnWF) a:03AnnWF [20:46:23] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Move PayPal and Adyen to folders based on gateway code - https://phabricator.wikimedia.org/T302892 (10Ejegg) [20:53:31] 10Fundraising Sprint Anti-matter doesn't matter, 10Fundraising Sprint Cheese String Theory, 10Fundraising Sprint Discworld reformatted as ntfs, 10Fundraising-Backlog, and 3 others: Cleanup our language mess - https://phabricator.wikimedia.org/T279389 (10jgleeson) a:05jgleeson→03Damilare [20:53:57] 10Fundraising Sprint Anti-matter doesn't matter, 10Fundraising Sprint Cheese String Theory, 10Fundraising Sprint Discworld reformatted as ntfs, 10Fundraising-Backlog, and 3 others: Cleanup our language mess - https://phabricator.wikimedia.org/T279389 (10jgleeson) Manual language lookups [[ https://docs.goo... [20:54:19] fr-tech thanks for all the help finding all those default languages! [21:04:25] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Refactor base GatewayAdapter to take config as constructor arg - https://phabricator.wikimedia.org/T302922 (10Ejegg) [21:08:09] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Add supported country lists to gateway config - https://phabricator.wikimedia.org/T302923 (10Ejegg) [21:14:03] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Add supported country lists to gateway config - https://phabricator.wikimedia.org/T302923 (10Ejegg) [21:34:33] (03PS3) 10Jgleeson: Update preferred language logic to use default locale lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/766844 (https://phabricator.wikimedia.org/T298400) [21:37:47] (03PS4) 10Jgleeson: Update preferred language logic to use default locale lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/766844 (https://phabricator.wikimedia.org/T298400) [21:41:31] (03PS1) 10Jgleeson: DO NOT MERGE: fix xdebug & unit tests on crm [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/767589 [21:52:45] (03CR) 10jerkins-bot: [V: 04-1] DO NOT MERGE: fix xdebug & unit tests on crm [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/767589 (owner: 10Jgleeson) [21:59:14] have a good rest of your day fr-tech. bye for now o/ [22:50:02] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Create rules for gateway tiebreaking based on querystring - https://phabricator.wikimedia.org/T302935 (10Ejegg) [23:04:24] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Refactor & rename FormChooser::getOneValidForm - https://phabricator.wikimedia.org/T302936 (10Ejegg) [23:18:15] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Refactor fail page logic - https://phabricator.wikimedia.org/T302938 (10Ejegg) [23:24:05] 10Fundraising-Backlog, 10FR-form-chooser-refactor, 10MediaWiki-extensions-DonationInterface: Clean up remaing ffname use - https://phabricator.wikimedia.org/T302939 (10Ejegg)