[00:07:31] i have to start making dinner. i'll check back later on to see if the triggers are ready. [00:27:49] (03PS4) 10Cstone: WIP Hebrew recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708968 (https://phabricator.wikimedia.org/T285375) (owner: 10Eileen) [00:28:15] eileen: I think this is behaving ^^ the html looks weird but it works when I click the links when testing [00:29:06] i added a wrapping the whole thing just to help testing locally we can remove that if needed although I don't think itll cause any issues [01:05:29] cstone: I'm inclined to merge it & push it out & get them to check [01:18:04] cstone: if you remove the WIP I will [01:44:50] eileen: I'm back looking at the code btw [03:31:33] (03CR) 10AndyRussG: [C: 03+2] "Coooool!!!! Thanks!!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/710135 (https://phabricator.wikimedia.org/T280595) (owner: 10Eileen) [03:32:00] yay - thanks AndyRussG [03:32:23] eileen: yw, thank u! staff meeting just ended, just gonna get another snack quickly, brb [03:42:07] (03Merged) 10jenkins-bot: Trigger update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/710135 (https://phabricator.wikimedia.org/T280595) (owner: 10Eileen) [03:44:06] eileen: hey I'm back in the call btw if you're still about! [03:44:13] AndyRussG: yep I am [03:44:49] ah cool [03:54:14] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/710412 [03:54:27] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/710412 (owner: 10Eileen) [04:00:04] !log civicrm revision changed from e52f569991 to 931b3defbe, config revision is 3696499932 [04:00:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [04:41:52] (03CR) 10AndyRussG: [C: 03+2] "Yaaaayyyyyy!!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/710137 (https://phabricator.wikimedia.org/T281268) (owner: 10Eileen) [04:52:00] (03Merged) 10jenkins-bot: Follow up settings fixes on custom update fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/710137 (https://phabricator.wikimedia.org/T281268) (owner: 10Eileen) [05:41:56] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/710421 [05:42:28] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/710421 (owner: 10Eileen) [05:44:10] !log civicrm revision changed from 931b3defbe to c132d2f943, config revision is 3696499932 [05:44:16] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [05:52:28] 10Fundraising Sprint Ketchup Flume Ride, 10Fundraising Sprint Mandatory corn dogs, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 5 others: Benefactor Page Listing Custom field - https://phabricator.wikimedia.org/T281268 (10Eileenmcnaughton) @rlewis - can you test this works now [06:40:47] (03CR) 10AndyRussG: "Heyyy fantastic work, many congratulations on this!!!!! :) :)" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710364 (https://phabricator.wikimedia.org/T286880) (owner: 10Damilare Adedoyin) [08:12:29] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint onion pit, and 2 others: Recurring donors segmentation criteria Civi/Acoustic - https://phabricator.wikimedia.org/T283798 (10MSuijkerbuijk_WMF) Thanks @Eileenmcnaughton for explaining o... [08:54:07] (03PS2) 10Damilare Adedoyin: Add tax disambiguation clause to Payments Wiki in France [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710364 (https://phabricator.wikimedia.org/T286880) [10:27:50] 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint onion pit, 10FR-AutoTY-Email: All non english soft descriptors should be the DR email address: donate@wikimedia.org - https://phabricator.wikimedia.org/T285499 (10Damilare) a:03Damilare [12:53:57] (03PS1) 10Damilare Adedoyin: All non english soft descriptors should be the DR email address [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710535 (https://phabricator.wikimedia.org/T285499) [12:57:18] (03CR) 10jerkins-bot: [V: 04-1] All non english soft descriptors should be the DR email address [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710535 (https://phabricator.wikimedia.org/T285499) (owner: 10Damilare Adedoyin) [13:01:21] (03PS2) 10Damilare Adedoyin: All non english soft descriptors should be the DR email address [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710535 (https://phabricator.wikimedia.org/T285499) [14:14:15] (03PS5) 10Cstone: Hebrew recurring copy [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/708968 (https://phabricator.wikimedia.org/T285375) (owner: 10Eileen) [15:00:35] (03PS1) 10Jgleeson: Added Adyen Checkout quick links to Payments-Wiki main page. [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/710574 [15:05:00] 10Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-Email: Automated Email Import to Civi - https://phabricator.wikimedia.org/T256315 (10EYener) Pinging this thread again, as this has come up as an Analytics request to report on RML data: For added clarity, there are different API calls that gather data from A... [15:16:38] 10Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: Fundraising Access for Joseph Mando - https://phabricator.wikimedia.org/T286556 (10JMando) Public yubikey: ccccccvkhvbn Public ssh: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoq0qAiQMUEYAWZSUFmkrmJQXaxJQVUvw6K5Us4M9zmJZ4TRN5WB+CO1H/bONohaTy/W7K... [15:33:19] 10Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: Fundraising Access for Joseph Mando - https://phabricator.wikimedia.org/T286556 (10Jgreen) Walked JMando through creating a separate key for fundraising, ~/.ssh/fr_id_rsa and configuring his ssh client to specify this key for FR hosts. Updat... [16:37:29] fr-tech I need to pop out to pick up the kids so I'll be a little late joining the call. back soon! [16:48:22] AndyRussG: i have a question about the updated triggers. i noticed there is a chunk that are removing the utf8mb4 collation or replacing it with the associated utf8 collation. is that desired? [16:49:24] i'm not certain (yet) if it is specified in the trigger file or pulled in via some other means. [16:51:04] oh. never mind. i think i see what happened here. [16:52:36] PROBLEM - check_mysql on frdb1002 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) [16:53:04] ^^ that's me and by desin [16:53:08] design even. [16:54:49] so the "COLLATE utf8mb4_bin" has been removed from a chunk of triggers but at least the utf8bin wasn't added in it's place. [16:57:36] PROBLEM - check_mysql on frdb1002 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) [17:02:36] RECOVERY - check_mysql on frdb1002 is OK: Uptime: 3282779 Threads: 12 Questions: 133528285 Slow queries: 1631 Opens: 910535630 Flush tables: 1 Open tables: 200 Queries per second avg: 40.675 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [17:19:48] fr-tech. i noticed 2 failmails from smashpig during the 3 minutes when the trigger update was being done. not sure if anything is needed to address those or if they will retry. [17:22:46] dwisehaupt: so the bin thing was just during update? [17:25:08] utf8bin? yeah. i accidentally ran a previous trigger file since the docs i was using were incorrect with rsync commands. [17:25:30] okie yeah theres like ~15 damaged messages from it but i can requeue them [17:25:36] so there was a 3 minute period where collate was set to utf8bin vs utf8mb4bin [17:25:56] sorry about that. i'm fixing the docs now. [17:26:02] no worries [17:48:34] hey fr-tech any news on the acoustic data breakage? [17:50:01] I was just looking at that [17:50:09] Just finishing up eating then I can take a look too [17:50:47] cstone: I think we could add a try/catch block around the bit of the code that handles the creation of new mail records so that it skips the problematic ones and processes the others [17:51:07] although that won't actually fix the core problem which I think there's already a ticket for [17:52:57] the bit that's breaking is here https://github.com/wikimedia/wikimedia-fundraising-crm/blob/8a783367ef21fa8f6b486e75e386bb3625955dd9/drupal/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnimailing/Load.php#L61 [17:53:30] sorry not that line, this one https://github.com/wikimedia/wikimedia-fundraising-crm/blob/8a783367ef21fa8f6b486e75e386bb3625955dd9/drupal/sites/default/civicrm/extensions/org.wikimedia.omnimail/api/v3/Omnimailing/Load.php#L90 [17:54:02] during the INSERT INTO `civicrm_mailing` [17:54:25] do you know if all of them are bad data or just the original [17:54:27] hmm I guess it's a case of weighing up that sticky plaster fix vs the actual fix [17:54:53] cstone: I can't tell sadly. It looks like the error is due to an encoding mismatch [17:54:58] so it might just be a few [17:55:11] although when it bombs out, the others in the stack don't get processed either I think [17:55:13] yeah its chopping off the last of a japanese caracter [17:55:20] can we see the source file [17:56:39] I'm not sure it's a file [17:56:43] hmm [17:56:50] I think it might be an api [17:57:41] i guess i assumed nothing was an api with this hah [17:59:48] I think the request that gets the mail items to be processed is here https://github.com/eileenmcnaughton/omnimail-silverpop/blob/master/src/Mailer.php#L69 [18:01:01] the client underneath that is using Guzzle [18:02:01] although I guess that means that we could actually make the same HTTP request from the civi box using something like cURL [18:02:20] there are some filter params like date ranges and stuff [18:02:28] to determine how many are affected [18:02:44] ah okay yeah in that cause i think actual fix route? i was thinking if it was some like csv with a bad line or something [18:02:57] ah yeah that would have been nice [18:05:15] it does look like just one bad character though [18:05:35] and the subject is fine in acoustic [18:06:03] %%DC::firstname::lastname-SL%%昨年、私たちからの支援依頼に応えてくださり、ありがとうございました [18:20:29] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Silverpop fail mail - https://phabricator.wikimedia.org/T288187 (10Cstone) For this email's case the subject is 130 bytes long when the db column is set to 128 `subject` varchar(128) COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT 'Subject of mailing',... [18:21:08] dstrine: would you be able to jump into todays standup real quick [18:27:02] dstrine: I updated the task but we think it'll be a quick fix for eileen [18:29:37] uff. that would be a table alter. [18:30:58] its a civi core table so we were thinking just truncate on input or something [18:31:08] on insert [18:31:39] as long as we aren't doing some exact comparison on data for something later, yeah. [18:31:47] damilare: I figured out why exactly you were getting "No such special page" for Adyen Checkout locally before! [18:33:01] yeah just more of a quick fix for now dwisehaupt [18:33:06] that said, the table really isn't that big so an alter would be doable for us. [18:33:07] it's that your patch (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DonationInterface/+/710364) doesn't have as part of its history the changes since we merged the adyen barnch [18:33:25] more so not sure how changing a civi core table like that works [18:33:36] totally [18:33:38] this can be fixed easily either by rebasing locally or often just clicking the "rebase" button on the Gerrit interface, which I'll try now... [18:33:52] (03PS3) 10AndyRussG: Add tax disambiguation clause to Payments Wiki in France [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710364 (https://phabricator.wikimedia.org/T286880) (owner: 10Damilare Adedoyin) [18:34:00] guess we need to be more concise in our subject lines and get to the point :) [18:34:11] cstone: sory just saw your message. Im in the call [18:36:39] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Silverpop fail mail - https://phabricator.wikimedia.org/T288187 (10DStrine) [18:37:09] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Japanese subject line too long for omnimailing load - https://phabricator.wikimedia.org/T288187 (10Cstone) [18:50:55] heading out. have a great long weekend fr-tech folks. see you tuesday! [18:51:53] (03CR) 10AndyRussG: "Hi! Looks like this is almost ready!!!!!! Thanks so much!!!!" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/710364 (https://phabricator.wikimedia.org/T286880) (owner: 10Damilare Adedoyin) [18:52:43] jgleeson|away: cya! [19:56:17] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint onion pit, and 2 others: Recurring donors segmentation criteria Civi/Acoustic - https://phabricator.wikimedia.org/T283798 (10KHaggard) Cool, thanks @Eileenmcnaughton - here's a sheet wh... [20:10:17] well Adyen got back to us but I'm guessing no one can check at the moment? It's all on Jack's machine right now? [20:10:28] I don't want to actually ping him this late on a friday [20:22:21] yeah I don't have a way to test dstrine [20:22:33] yeah I figured [23:15:29] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Japanese subject line too long for omnimailing load - https://phabricator.wikimedia.org/T288187 (10Eileenmcnaughton) @cstone we could probably trim it I think - & add an elipsis