[04:10:15] (03CR) 10AndyRussG: Adyen Checkout: Don't add (hidden) submethod button elements (031 comment) [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/718941 (https://phabricator.wikimedia.org/T290103) (owner: 10AndyRussG) [04:15:14] (03PS4) 10AndyRussG: Adyen Checkout: Don't add (hidden) submethod button elements [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/718941 (https://phabricator.wikimedia.org/T290103) [04:16:13] (03PS5) 10AndyRussG: Adyen Checkout: Don't add (hidden) submethod button elements [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/718941 (https://phabricator.wikimedia.org/T290103) [04:16:39] (03PS6) 10AndyRussG: Adyen Checkout: Don't add (hidden) submethod button elements [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/718941 (https://phabricator.wikimedia.org/T290103) [04:33:38] (03PS1) 10Eileen: 5.43.alpha [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/723806 [05:44:43] (03PS2) 10Eileen: 5.43.alpha [wikimedia/fundraising/crm/civicrm] - 10https://gerrit.wikimedia.org/r/723806 [11:44:06] 10Fundraising-Backlog, 10FR-Tech-Analytics: Collaborate on WikipediaPortal migration - https://phabricator.wikimedia.org/T291432 (10EYener) Hey XenoRyet checking in - would Tuesday or Wednesday this week be a good time to bring this to Tech Talks? [15:17:39] 10Fundraising-Backlog: Creating shortened URL for Major Gifts Annual Appeal 2021 - https://phabricator.wikimedia.org/T291847 (10RLewis) [15:40:12] 10fundraising-tech-ops: Creating shortened URL for Major Gifts Annual Appeal 2021 - https://phabricator.wikimedia.org/T291847 (10Jgreen) 05Open→03Resolved This is done! [15:40:18] (03CR) 10Ejegg: [V: 03+2] "Manually submitting b/c repo doesn't have CI" [wikimedia/fundraising/privatebin] - 10https://gerrit.wikimedia.org/r/720832 (https://phabricator.wikimedia.org/T285881) (owner: 10Ejegg) [15:40:27] (03CR) 10Ejegg: [V: 03+2] "Manually submitting b/c repo doesn't have CI" [wikimedia/fundraising/privatebin] - 10https://gerrit.wikimedia.org/r/720839 (https://phabricator.wikimedia.org/T285881) (owner: 10Ejegg) [15:42:37] PROBLEM - check_puppetrun on frmx1001 is CRITICAL: CRITICAL: Puppet has 17 failures. Last run 8 minutes ago with 17 failures. Failed resources (up to 3 shown) [15:47:37] RECOVERY - check_puppetrun on frmx1001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [17:48:29] woot, got CentralNotice staging up and running on bullseye: https://cn-stage-1.wmcloud.org/wiki/Special:CentralNotice [17:48:50] new Vector looks super-clean [17:50:24] looks good [17:56:55] (03CR) 10Jgleeson: [C: 03+2] "Thanks the tweak! Looks good to go!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/717712 (https://phabricator.wikimedia.org/T290106) (owner: 10Cstone) [17:57:55] (03Merged) 10jenkins-bot: Add Apple Pay recurring [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/717712 (https://phabricator.wikimedia.org/T290106) (owner: 10Cstone) [18:05:54] fr-tech should we use tonight's call to chat a bit about how we plan to set up the new fr-tech dev server? [18:06:20] and maybe add some subtasks to https://phabricator.wikimedia.org/T291577 [18:06:33] guessing we'll need a vm, a bunch of urls, some dns magic [18:07:02] bunch of ssh config [18:07:26] ejegg's magic port mapping nginx spaghetti [18:08:56] served over SSL/TLS [18:09:31] jgleeson: yeah, i think it should be pretty straightforward! [18:09:44] technically once we decide on our URLs we can hand them over to Matt to do his thing [18:10:18] as long as we can get our own cert for a wmcloud subdomain via letsencrypt [18:24:01] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 13 others: Fix civicrm repo to be non-symlinked - https://phabricator.wikimedia.org/T289100 (10jgleeson) Thanks for the explanatio... [18:36:31] PROBLEM - check_mysql on frdb2002 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) [18:37:20] that's expected with the testing. i'll set a downtime. [18:38:41] ACKNOWLEDGEMENT - check_mysql on frdb2002 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) Dwisehaupt known - testing alters. [18:44:02] ejegg: I'm looking at your Resolver patch trying to find a good ticket to pick up around it [18:44:11] https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/722703 [18:45:34] cstone: are you already working on the SmashPig bit to normalize the Ingenico getHostedCheckoutStatus response? [18:45:53] jgleeson: that normalization would be the most pressing bit [18:46:12] i havent gotten to anything today yet ejegg [18:46:53] jgleeson then after that, maybe figuring out the right queries to get https://phabricator.wikimedia.org/T291226 [18:47:19] but the 226 patch would be built on top of the existing very-WIP patch, so might be a pain to rebase later [18:47:48] also, good point on deciding URLs jgleeson [18:48:03] lemme see what we can claim [18:49:53] ejegg: that's a bunch of nice discrete conditions [18:50:05] maybe I could WIP something on that [18:51:39] re: domains. looks like we can add cnames to a handful of DNS zones [18:51:59] e.g. fr-tech-dev.wmflabs.org [18:53:01] 10Fundraising-Backlog, 10FR-Tech-Analytics: Collaborate on WikipediaPortal migration - https://phabricator.wikimedia.org/T291432 (10XenoRyet) Let's go ahead and plan for the Wednesday Tech Talk this week. [18:54:19] 10Fundraising-Backlog, 10FR-Tech-Analytics: Collaborate on WikipediaPortal migration - https://phabricator.wikimedia.org/T291432 (10EYener) Great! Thank you. I'll be there for the second half of it (I have a conflict for the first 30 minutes). [18:56:46] for the jackgleeson.co.uk stuff I just added a bunch of A records pointing to different IPs but in our case I think we'll have one IP address right? [18:58:12] jgleeson: ooh, maybe the 'web proxy' feature can actually handle it all for us! [18:58:22] lemme try that quick over on cn-stage-1 [18:59:07] nice! [19:07:48] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster: Add existing donor matching to PendingTransaction::resolve - https://phabricator.wikimedia.org/T291226 (10jgleeson) Currently thinking about how to take this cookie out of the jar. [19:20:40] hmm, it looks like it SHOULD be working on https://cn-stage-testfwd.wmcloud.org/ but I'm still getting 502 bad gateway [19:21:13] so that supposedly forwards to cn-staging-1's port 8099 [19:23:03] 10Fundraising-Backlog, 10fundraising-tech-ops: Document Ingenico integration - https://phabricator.wikimedia.org/T291874 (10AndyRussG) [19:27:55] 10Fundraising-Backlog: Documentation: Confirm and document decision about where to put stuff - https://phabricator.wikimedia.org/T291878 (10AndyRussG) [19:31:42] 10Fundraising-Backlog, 10fundraising-tech-ops: Documentation: Complete review and coding of existing documentation - https://phabricator.wikimedia.org/T291879 (10AndyRussG) [19:31:53] and if I ssh in there with -R 8099:localhost:9001 I can curl localhost:8099 (on the cn box) and get the stuff from my docker-served 9001 port [19:32:53] 10Fundraising-Backlog, 10fundraising-tech-ops: Documentation: Create stub pages for overviews of general topics - https://phabricator.wikimedia.org/T291880 (10AndyRussG) [19:44:15] 10Fundraising-Backlog, 10fr-donorservices, 10Wikipedia-Android-App-Backlog (Android Release FY2021-22): "No App" error from monthly convert on mobile - https://phabricator.wikimedia.org/T291199 (10ABorbaWMF) Appears to be fixed on 2.7.50376-r-2021-09-27 I was able to reproduce the issue on a previous build... [19:51:17] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into HEAD [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724167 [19:51:32] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into HEAD [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724167 (owner: 10Eileen) [19:55:54] civicrm revision changed from 18228490ae to ef5367bffc, config revision is 77cb7ec866 [19:55:57] !log civicrm revision changed from 18228490ae to ef5367bffc, config revision is 77cb7ec866 [19:56:02] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:56:21] dwisehaupt: I just deployed - shall I run the trigger drop? [19:56:52] trigger drop, oh trigger drop... trigger gonna drop on you-oo-oo [19:57:00] trigger drop or index drop? [19:57:25] index [19:57:35] index drop, oh index drop... [19:57:39] lol [19:57:41] ok. yeah. index drop is good. [19:58:04] let me know when you are set and i'll drop and recreate that unique_entity_id index. [19:59:38] oh - so I'll run it & let you know? [19:59:51] sounds good. [20:00:15] runnin [20:00:21] done [20:00:52] so next one is https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/719163 [20:01:01] which I think ejegg needs to look [20:01:20] ok. i'll delete and recreate that index. [20:01:40] yep, just trying to run ucf locally tells me there are too many indexes. Trying to make sure the drop index patch actually got applied [20:03:13] index dropped and recreated: Query OK, 0 rows affected (50.692 sec) [20:04:25] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fr-donorservices: Civi Sidebar "recent items" --> no longer showing the top records - https://phabricator.wikimedia.org/T290767 (10Eileenmcnaughton) @SHust it should be back.... [20:10:21] 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 10Epic: [Epic] Architecture, separation of concerns and refactors in DonationInterface (2021) - https://phabricator.wikimedia.org/T291697 (10Ejegg) We now have 2 (or 3) of 5 integrations using mostly SmashPig calls (the iffy one is Amazon) rath... [20:17:22] fr-tech, after inviting the entire mediawiki-l list to that meeting.. it's only gone and been cancelled! [20:18:02] awwww [20:18:53] maybe there were concerns about pandemic stuff with that many possible attendees [20:20:16] LOL [20:20:28] that's probably it AndyRussG :) [20:21:06] last time I invited all of wikimedia-l to my house not too many folks showed up, sadly [20:21:45] I guess maybe the fact that I asked everyone to bring a server might have been a deterrent...... :/ [20:21:51] lol [20:22:29] you could probably boil water on the wifi router if even half the list turned up to the same event [20:22:45] hehehe [20:24:39] right all the wifi might interfere with the correct functioning of my microwave [20:26:17] lol [20:28:42] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fr-donorservices: Civi Sidebar "recent items" --> no longer showing the top records - https://phabricator.wikimedia.org/T290767 (10SHust) It is, thank you so much! @Eileenmcnaughton [20:32:23] 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 10Epic: [Epic] Architecture, separation of concerns and refactors in DonationInterface (2021) - https://phabricator.wikimedia.org/T291697 (10AndyRussG) >>! In T291697#7381793, @Ejegg wrote: > We now have 2 (or 3) of 5 integrations using mostly... [20:39:27] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10FR-Adyen: Set up new fr-dev staging server for Apple Pay - https://phabricator.wikimedia.org/T291577 (10jgleeson) @XenoRyet @Damilare and @Eileenmcnaughton are now added as admins on the project. [20:49:22] 10Fundraising-Backlog, 10fr-donorservices, 10Wikipedia-Android-App-Backlog (Android Release FY2021-22): "No App" error from monthly convert on mobile - https://phabricator.wikimedia.org/T291199 (10MBeat33) Thank you @ABorbaWMF for the info. If we see any more of this error, should we recommend that the don... [20:51:23] 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 10Epic: [Epic] Architecture, separation of concerns and refactors in DonationInterface (2021) - https://phabricator.wikimedia.org/T291697 (10Ejegg) We have to reintegrate PayPal next year. Not sure when D*Local will push us off our current (dep... [20:58:24] ejegg: are you still looking at that field add one locally? [20:59:06] eileen: right, sorry, got called away to help with baby and when I came back I got distracted with the staging forwarding thing [20:59:21] ok, so I just need to delete some indexes locally so that it lets me make those fields [20:59:32] shoot, have to help with baby again [21:00:46] 10Fundraising-Backlog, 10fr-donorservices, 10Wikipedia-Android-App-Backlog (Android Release FY2021-22): "No App" error from monthly convert on mobile - https://phabricator.wikimedia.org/T291199 (10Dbrant) >>! In T291199#7382000, @MBeat33 wrote: > Thank you @ABorbaWMF for the info. > > If we see any more of... [21:02:12] 10Fundraising-Backlog, 10fr-donorservices, 10Wikipedia-Android-App-Backlog (Android Release FY2021-22): "No App" error from monthly convert on mobile - https://phabricator.wikimedia.org/T291199 (10MBeat33) Awesome, thank you. [21:19:33] ok, so, let me try that patch in my docker-civi that shouldn't have any extra fields or indexes [21:23:47] cool - as long as the index drop runs first. [21:24:24] right, i thought I should have gotten that from the cvapi extension.upgrade [21:32:58] yep [21:34:42] grr, now trying to load pages at my local docker is giving me 'Secure Connection Failed' [21:35:04] not even the normal self-signed cert thing, doesn't let me put in any exception [21:35:12] whelp, guess I'll try rebuilding [21:41:19] ug [21:43:51] ok, it's currently mid-rebuilding with the already-merged-to-master patches [21:44:02] so I can pull down yours to see them applied to an existing db [21:46:19] cool [21:54:24] well damn, rebuilt and still the same error [21:54:41] that's in FF [21:54:49] and in Chromium I get ERR_EMPTY_RESPONSE [21:55:09] going to try again with my non-docker civi instance [22:00:16] grr, let's try just forcing the extension schema version in SQL [22:11:48] ohhh dang, it created the custom_field records but crashed on the table alter last time [22:12:05] yeah - you have the latest of the civicrm submodule? [22:12:15] yes i do [22:13:18] just going to delete the custom_field records and try ucf again [22:16:10] derp, "delect from" [22:16:23] * ejegg needs sleep [22:17:26] argh, again too many keys [22:17:51] so what the heck columns do I have locally that aren't on prod [22:21:20] grr, exactly the same columns [22:25:12] and all the same indexes! [22:27:33] so I see 24 'ADD INDEX' lines in the failing SQL [22:30:07] and 41 existing indexes locally (including the primary key and unique key) [22:30:16] eileen: how many do you have locally? [22:30:22] and how many do you expect to add? [22:32:45] I wind up with 64 I think https://etherpad.wikimedia.org/p/wmf-d [22:33:35] ejegg: note I ran it on staging so that should be the same [22:41:21] ejegg: this is weird - locally I have a drop index patch [22:41:41] eileen: 64 including the primary key? [22:41:53] yeah I think so [22:41:56] here, let me put up a paste with mine (and prod) [22:42:36] (03PS1) 10Eileen: Drop indexes on wmf_donor fields identified as not required to be searchable. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) [22:43:16] https://phabricator.wikimedia.org/P17332 [22:43:17] I think local must be a duplicate patch [22:43:30] eileen: which of ^^^ is extra? [22:44:24] oh right, I can look at dev_civicrm; [22:45:42] hmm, I only see 61 lines on dev_civicrm [22:46:00] yeah - I'm trying to figure out whether the patch that was merged was the right one [22:47:03] there are 2 more fields in it [22:48:31] ejegg: I suspect the one that got merged wasn't the 'right' one [22:48:38] ah damn [22:48:53] oh, so we need to drop more indexes? [22:49:20] (03PS2) 10Eileen: Drop indexes on wmf_donor fields identified as not required to be searchable. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) [22:49:41] yeah - I think I actually decided to ditch 2014 based on it being needed [22:50:11] oh wait, but that makes my 'too many indexes' problem worse [22:50:11] oh hang on [22:50:18] that diff looks like the RIGHT one merged [22:51:27] (03PS3) 10Eileen: Drop indexes on wmf_donor fields identified as not required to be searchable. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) [22:54:25] (03CR) 10Ejegg: [C: 04-1] "looks like one typo" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [22:54:29] ejegg: so the drop routine here might drop more https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/724201/3/drupal/sites/default/civicrm/extensions/wmf-civicrm/CRM/Wmf/Upgrader.php - since we've run the other we may as well put it in 4008 & run it [22:56:18] hmm - I think master looks ok tho - this might all be a red herring [22:56:48] wait, but my local has exactly the same indexes as master [22:57:06] (03CR) 10jerkins-bot: [V: 04-1] Drop indexes on wmf_donor fields identified as not required to be searchable. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [22:57:07] none of those listed to drop are left on local or on master [22:57:16] but that leaves 41 indexes [22:57:20] and 24 to add [22:57:37] =65, too many [22:58:06] so how the heck does dev just have 61 ? [22:58:20] including 2 2014 ones that aren't on prod? [22:58:43] ok, lemme see if I can find which that it tries to add locally aren't on dev [22:58:50] that is, aren't on staging [23:00:27] oho, all_funds_change_XXXX_YYYY is added as an index when I try running it [23:00:57] but isn't on staging [23:01:04] let's see what the spreadsheet says [23:02:24] eileen: that's the discrepancy [23:02:44] spreadsheet says all_funds_change_xxxx_yyyy need indexes, and the patch tries to add them [23:02:53] but they weren't added on staging [23:03:05] we need to take at least 2 away [23:03:48] ah ok [23:04:06] ejegg: & yet the patch passes on CI [23:04:18] :P [23:06:11] oh hey, my docker db is missing the entity_total index [23:06:28] is that actually needed? [23:06:51] seems to be the only compound index [23:06:58] on entity_id and lifetime_usd_total [23:07:04] oh, I guess for searches [23:07:12] major gift groups [23:07:25] anyway, we should add that to the dev / CI setup [23:19:03] how does the compound indev get there? [23:19:16] I don't know if it makes sense [23:19:52] i don't know how it got on to prod + my old local db [23:20:44] i guess we could drop it and see what breaks [23:24:21] ok, i don't see anything in civicrm_group with a where_clause including lifetime_usd_total [23:26:23] yeah I'm just not sure it makes sense in that I don't think it's how the index would work [23:27:23] oh, so it wouldn't even be useful to use greater/less than on the lifetime_usd_total? [23:27:27] or ? [23:27:32] I don't think so [23:27:50] just for searching for exact values? [23:28:16] I'd have to test -but generally the keys are used in order - so if you didn't specify entity_id AND lifetime_usd_total in your where clause it wouldn't be used [23:28:41] ok. i need to head out to get kid #1, back in 45 or so unless i have to get kid #2 also in which case it'll be at least 90 mins. [23:29:07] oh right, you'd want to just use the lifetime_usd_total index generally [23:29:10] i'll check in on the status when i return. p-c jobs are still currently disabled on civi and frdev. [23:29:46] eileen: ok, want to put that in the drop list then? [23:29:57] ejegg: I'm confused - this is prod [23:29:58] https://docs.google.com/spreadsheets/d/10i_vyKMqxYtjitOrxd8nS7jiQXU3Uyi_Ipy7pqn7P58/edit#gid=798986747 [23:30:01] I don't see it [23:30:15] on hang on KEY `entity_total` (`entity_id`,`lifetime_usd_total`), [23:30:27] yep yep [23:30:59] so we drop that and it brings the existing total down to 41 [23:31:13] meaning we can only add 23, not the 24 the script is trying to add [23:31:15] ejegg: hmm - see wmf_civicrm_update_8110 [23:31:48] oh drat, so it's been shown to be useful [23:31:54] seems so [23:32:09] lol you just added that in February [23:32:29] i swear there's something about this job that kills our long-term memories [23:33:04] :-) [23:33:07] like with that CentralNotice JS error - neither me nor AndyRussG remembered anything about that file but gerrit says he fixed it 2 years ago and I reviewed it [23:33:22] so the spreadsheet says we have 64 fields [23:33:30] + that one = 65 [23:33:46] what else is missing from the spreadsheet? [23:34:20] oh, that FK doesn't count as an index, right? [23:35:00] hmm - good question [23:35:11] if we drop 1 more index does it work for you? [23:35:15] trying [23:35:45] ahhh, 8110 isn't in the _install list [23:37:34] oh [23:38:07] oh weird, now it fails with Table 'log_wmf_donor' already exists [23:38:11] I guess we can move it to the upgrader (number that live doesn't run) [23:38:29] that log_wmf_donor issue was happening before we patched civicrm submodule [23:38:44] but I guess that means it got past the alter [23:38:52] oh hmm, thought I had latest master [23:39:02] maybe i only did that on my docker src checkout [23:39:47] ok, there were 3 commits missing [23:40:13] but it DID work to create the new columns and indexes [23:40:27] so... which 1 index do we drop from prod? [23:41:35] I'm gonna say we just give them 1 less year [23:41:43] that will remove 2 indexes but I think that's fine [23:41:56] to be honest I already unilaterally removed one year [23:42:07] but I think we can go one more [23:49:19] (03CR) 10Jforrester: [C: 03+2] build: Disable phan option null_casts_as_any_type and make pass [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/723684 (owner: 10Umherirrender) [23:49:28] (03PS7) 10Eileen: Add new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/719163 (https://phabricator.wikimedia.org/T288721) [23:51:50] ejegg: heheh could happen with any job [23:52:02] or also with lack of job [23:52:34] (03Merged) 10jenkins-bot: build: Disable phan option null_casts_as_any_type and make pass [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/723684 (owner: 10Umherirrender) [23:55:37] (03CR) 10jerkins-bot: [V: 04-1] Add new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/719163 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen)