[00:00:31] eileen: any clues why CI would say "sh: 1: cv: not found" [00:00:33] ? [00:00:41] buildkit updates? [00:01:17] (03PS1) 10Eileen: Drop 2 more indexes - make way make way [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724210 (https://phabricator.wikimedia.org/T288721) [00:01:51] it shouldn't be - not changes today - let's see what that one does ^^ [00:01:58] k [00:04:08] it probably gets cv via composer or something that buildkit runs [00:04:21] let's hope it's just momentary thing [00:04:38] eileen: oh hey, should we set is_searchable to 0 for those fields whose indexes we're manually removing here? [00:05:01] would that take them out of some dropdowns so folks are less likely to get in trouble? [00:05:02] ejegg: yeah I thought the same - but I think that change goes in the 'add' patch [00:05:08] ok [00:05:15] because I think that's where it is for the others [00:05:21] right [00:06:04] hmm, it looks like ucf doesn't update definitions for existing fields though [00:07:01] ejegg: no - so we drop the indexes through the upgrader [00:07:13] but we have to update the definition for dev sites [00:07:54] and do we want to set is_searchable=0 on prod somehow? [00:08:03] (03PS8) 10Eileen: Add new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/719163 (https://phabricator.wikimedia.org/T288721) [00:08:05] for the fields whose indexes we're dropping in the upgrader? [00:09:08] yeah so the impact of is_searchable is that [00:09:08] - for reports - the fields become unavailable (at all - view or filter) [00:09:08] - for search kit - no change [00:09:08] - for advanced search they become unavailable for filters [00:10:13] ah, so we can't set it on prod because ppl will compain that they can't add 2014 giving to a report output [00:10:46] and we accept the risk of ppl trying to filter on it in advanced search and causing some deadlocks [00:11:41] right - that's the trade off [00:12:00] thanks for the explanation! [00:12:17] my guess is they care less than they think about those older year fields.... [00:12:33] I guess I could go either way so maybe err on doing nothing [00:12:45] right right [00:14:17] (03Abandoned) 10Eileen: Drop indexes on wmf_donor fields identified as not required to be searchable. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724201 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [00:16:25] (03CR) 10jerkins-bot: [V: 04-1] Drop 2 more indexes - make way make way [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724210 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [00:17:01] (03CR) 10Ejegg: [C: 03+2] "Looks good, worked locally. I like the refactor." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724210 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [00:17:28] oh grr, still missing cv [00:18:47] hmph, it works fine up a few lines at cv en --ignore-missing search_kit wmf-civicrm [00:19:41] oh, the 'cv not found' error comes soon after resetting the PATH [00:19:52] so why do we reset it there, and what are we missing? [00:20:07] + PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/src/civicrm/civicrm-buildkit/bin [00:20:18] (03CR) 10jerkins-bot: [V: 04-1] Add new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/719163 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [00:21:05] hmm, not explicitly set elsewhere in that jenkins log [00:22:12] (03CR) 10jerkins-bot: [V: 04-1] Drop 2 more indexes - make way make way [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724210 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [00:23:17] (03PS1) 10Eileen: Sync with upstream [wikimedia/fundraising/crm/civicrm-buildkit] - 10https://gerrit.wikimedia.org/r/724212 [00:23:54] hmm, cividiscount cache url? [00:23:59] ejegg: I just checked upstream buildkit & nothing seems relevant. I'll sync anyway [00:24:06] yeah I don't think that's it [00:24:15] but I do try to sync regularly [00:24:16] (03CR) 10Ejegg: [C: 03+2] Sync with upstream [wikimedia/fundraising/crm/civicrm-buildkit] - 10https://gerrit.wikimedia.org/r/724212 (owner: 10Eileen) [00:24:46] let's see if CI config has changed [00:24:56] (sync = make our code the same as upstream & accept history is a mess & our repo adds no value) [00:25:47] looks like a minor bump in the civi docker image [00:26:27] https://gerrit.wikimedia.org/r/c/integration/config/+/724156/2/jjb/wm-fundraising.yaml [00:26:27] yeah - our last buildkit update was 2 weeks ago & upstream not much more recent [00:28:04] ah, that patch says it's updating stretch-based images - we should get our CI onto a newer OS [00:30:38] hmm, the dockerfile update is pretty generic: [00:30:39] + Refresh for update in parent image ci-stretch: [00:30:39] + Rebuild for Let's Encrypt update - T291425 [00:30:40] T291425: Rebuild CI images affected by OpenSSL compat issue with new Let's Encrypt issuance chain - https://phabricator.wikimedia.org/T291425 [00:32:13] gawd [00:32:25] timing for Ci to get tricky on us! [00:32:40] so cv exists but it has a path issue? [00:32:46] hmph, so cv should be in /src/civicrm/civicrm-buildkit/bin [00:33:02] is it because it's being exec'ed under php? [00:33:11] > in cv() (line 33 of /src/wikimedia/fundraising/crm/drupal/sites/default/civicrm/extensions/org.wikimedia.omnimail/tests/phpunit/bootstrap.php). [00:33:15] (03CR) 10Eileen: [V: 03+2] Sync with upstream [wikimedia/fundraising/crm/civicrm-buildkit] - 10https://gerrit.wikimedia.org/r/724212 (owner: 10Eileen) [00:33:46] eileen: yeah, it's used a ways before in the setup directly from bash with no problems [00:33:50] to enable the extensions [00:33:54] hmm [00:35:10] oh wait, that path is wrong [00:35:24] /src/wikimedia/fundraising/crm/civicrm-buildkit is the actual src [00:35:32] ok, so let's fix that export PATH [00:35:47] (03PS1) 10Eileen: see if order matters [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724214 [00:36:26] oh ok - I am surprised it appears to be the LAST suite - but maybe it runs from the end [00:37:44] gotta help with baby, back in a bit! [00:38:00] found the fix btw, i think [00:38:08] but it'll need a docker image rebuild [00:38:45] https://gerrit.wikimedia.org/r/c/integration/config/+/724215 [00:41:57] (03CR) 10jerkins-bot: [V: 04-1] see if order matters [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724214 (owner: 10Eileen) [00:42:45] why did that start to matter? AndyRussG do we have to do something for the rebuild? ^^ [00:44:17] eileen: unit test order? [00:44:50] AndyRussG: nothing changed at the civi end between passing & failing - no I mean elliott's comment above that [00:45:03] 1.38 (nzdt :-) [00:48:40] eileen: I have no idea [00:49:06] really don't know how that CI stuff works [00:49:50] oh wait, that path was last changed in May: https://gerrit.wikimedia.org/r/c/integration/config/+/688350/4/dockerfiles/civicrm/run.sh [00:50:32] ahh, I see, so that patch was put up assuming the next patch would be merged, which it wasn't [00:50:37] https://gerrit.wikimedia.org/r/c/integration/config/+/689947/1/jjb/wm-fundraising.yaml [00:56:02] ohhhh [01:05:04] fr-tech I'm gonna put the CentralNotice updates on the train [01:05:17] 10-4 [01:06:19] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 14 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civ... - https://phabricator.wikimedia.org/T277500 [01:09:41] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 13 others: CiviCRM CI tests failing due to half-resolution of T277500 - https://phabricator.wikimedia.org/T291897 (10Ejegg) [01:10:25] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 14 others: CiviCRM CI tests failing due to half-resolution of T277500 - https://phabricator.wikimedia.org/T291897 (10Ejegg) [01:10:45] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 15 others: CiviCRM CI tests failing due to half-resolution of T277500 - https://phabricator.wikimedia.org/T291897 (10Ejegg) [01:10:57] ok, i got spammy with it [01:11:04] ejegg: are we stuck on needing help from releng? [01:11:09] hopefully that pushes it forward [01:11:11] yep eileen [01:11:16] if so I think we should v+2 on these 2 [01:11:20] yep yep [01:11:26] since they passed before... [01:13:52] (03CR) 10Ejegg: [V: 03+2 C: 03+2] "CI is currently broken, manual V+2" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724210 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [01:14:51] (03CR) 10Ejegg: [V: 03+2 C: 03+2] "Code and local results look good! Adding a manual V+2 because CI is currently broken." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/719163 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [01:16:41] k, manually merged them [01:17:03] so now trigger updates and 'fill new ...' [01:18:43] howdy. [01:19:13] that was an adventure. but i'm here if we need to do something. [01:19:28] keep in mind, we still have more days in the maint window, so no need to rush anything out. [01:19:44] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724224 [01:20:00] dwisehaupt: I'm just mergint to deployment now - I guess I have to v+1? [01:20:20] no it won't run tests so should be ok [01:26:04] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724224 (owner: 10Eileen) [01:26:40] !log civicrm revision changed from ef5367bffc to b8f756b60e, config revision is 77cb7ec866 [01:26:45] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:28:20] dwisehaupt: ready to run the script to add the custom fields - what do we need to do? disable triggers, jobs , ui? [01:28:34] jobs are stopped. [01:29:10] there is no way to temporarily disable the triggers, we need to remove them from that table and then reapply them. [01:29:44] yep - sounds good - it's specifically the ones that would update contact.modified_date [01:30:08] as long as no data entry happens while the triggers are off [01:30:30] yeah. i can put civi into maint mode. [01:30:34] let me start with that. [01:30:37] ok - are there users on? [01:33:32] not that i can see. but we are clear to do the work. [01:33:52] cool - so triggers off - I should kick it off? [01:35:29] i haven't turned of the triggers yet. just moving civi to maint mode. [01:36:14] (03PS4) 10Eileen: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) [01:36:35] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [01:36:43] we should create a script to disable triggers on a specific table. [01:36:49] (03PS5) 10Eileen: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) [01:37:10] dwisehaupt: you could also just drop them all [01:37:20] if nothing else is happening [01:37:50] yeah. i just like things scripted to keep me from messing up. :) [01:38:53] :-) [01:40:19] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [01:41:28] (03PS1) 10AndyRussG: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/724226 [01:42:10] (03CR) 10AndyRussG: [C: 03+2] Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/724226 (owner: 10AndyRussG) [01:43:55] ok. triggers should be dropped. [01:44:01] verifying. [01:44:57] ok. we are all set to run the script. [01:46:51] (03Merged) 10jenkins-bot: Merge branch 'master' into wmf_deploy [extensions/CentralNotice] (wmf_deploy) - 10https://gerrit.wikimedia.org/r/724226 (owner: 10AndyRussG) [01:47:45] if it rolls like it did on frdb2002 in my testing, it should take ~40 minutes [01:48:23] ok kicking it [01:48:46] crossing fingers. [01:49:55] dwisehaupt: ok it's running now - there is a further update but it took about 3.5 hours on staging - are you thinking tomorrow? [01:50:26] could do tomorrow for sure. [01:51:22] the big thing we have for tomorrow is the clone of frdb1003 to frdb2003. that will have superset unavailable but no reason we couldn't do additional work while that rolls along. when the dbs are done they will just catch up on replication. [01:51:49] i'm pretty sure jeff was going to kick it off early so it may be done before you are online. :) [01:53:06] (03PS6) 10Eileen: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) [01:53:33] i'm going to step away for a few to check something out on the house. back in a few. [01:58:23] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [01:58:51] cstone: you didn't take a look at that upgrade script did you? [01:59:11] back [01:59:13] elliott had started looking at them so I didn't [01:59:18] i can look now though [02:03:21] oh yeah - it's just the last one [02:03:23] the update one [02:03:42] okie [02:04:35] dwisehaupt: it's still running - - do you prefer to kick it off tonight if we can or wait to tomorrow - it will take a while although I think it was slower on staging as triggers were stil on [02:05:32] when you mean kick it off tonight, do you mean the second update that would be looking at 3+ hours? [02:10:16] dwisehaupt: yeah - I guess you can't just set & forget? [02:10:58] not with the triggers, queues, and ui off, unless we want to leave them that way overnight. [02:11:28] yeah - better to leave for tomorrow I guess - so cstone no need to rush it now [02:11:42] yeah. tomorrow will have me fresher mentally also. :) [02:11:49] okie ill just finish looking it over and +1 if good [02:11:55] cool [02:38:36] (03CR) 10Cstone: [C: 03+1] "looks good" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [02:45:30] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1312 [02:45:36] dwisehaupt: it's finished [02:45:55] ha. yes. i see the replication alerts. :) [02:47:03] i see the new indexes and columns. i'll readd the triggers. [02:47:30] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1429 [02:47:36] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1434 [02:50:13] triggers triggers look good. [02:50:36] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1612 [02:50:52] ok. anything else you want to check before i reenable the queues and civiui? [02:52:30] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1730 [02:52:36] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1734 [02:55:36] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1912 [02:57:30] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2029 [02:57:36] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2034 [02:57:55] ACKNOWLEDGEMENT - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2034 Dwisehaupt known - replication lag after alters. [02:57:55] ACKNOWLEDGEMENT - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2029 Dwisehaupt known - replication lag after alters. [02:57:55] ACKNOWLEDGEMENT - check_mysql on frdb2002 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) Dwisehaupt known - replication lag after alters. [03:00:36] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2212 [03:01:55] ACKNOWLEDGEMENT - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2212 Dwisehaupt known - replication lag after alters. [03:02:30] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 2710706 Threads: 11 Questions: 163141764 Slow queries: 1274 Opens: 2579 Flush tables: 1 Open tables: 1075 Queries per second avg: 60.184 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [03:03:12] ok. civi ui and process control are back in service. puppet run complete on civi1001 so it should be in action now. [03:06:20] thanks dwisehaupt [03:06:50] just sending out an email and then i'm gonna get some dinner. [03:10:30] RECOVERY - check_mysql on frdev1001 is OK: Uptime: 47592 Threads: 14 Questions: 359457 Slow queries: 38844 Opens: 2595041 Flush tables: 1 Open tables: 200 Queries per second avg: 7.552 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 290 [03:12:36] RECOVERY - check_mysql on frdb1002 is OK: Uptime: 2437754 Threads: 11 Questions: 155141207 Slow queries: 1227 Opens: 627611711 Flush tables: 1 Open tables: 210 Queries per second avg: 63.641 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 307 [03:41:30] RECOVERY - check_mysql on frdb2002 is OK: Uptime: 2779409 Threads: 12 Questions: 164229571 Slow queries: 1329 Opens: 2378 Flush tables: 1 Open tables: 997 Queries per second avg: 59.087 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [04:00:55] (03PS1) 10Eileen: [WIP] Add recurring failed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724231 [04:05:32] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add recurring failed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724231 (owner: 10Eileen) [04:12:45] (03PS1) 10Cstone: WIP Switch getHostedPaymentStatus to return a PaymentDetailResponse [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/724232 (https://phabricator.wikimedia.org/T291223) [04:12:50] (03PS1) 10Cstone: WIP DI part of switching getHostedPaymentStatus to return a PaymentDetailResponse [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/724233 (https://phabricator.wikimedia.org/T291223) [04:13:12] (03CR) 10jerkins-bot: [V: 04-1] WIP Switch getHostedPaymentStatus to return a PaymentDetailResponse [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/724232 (https://phabricator.wikimedia.org/T291223) (owner: 10Cstone) [04:13:50] (03PS2) 10Cstone: WIP Switch getHostedPaymentStatus to return a PaymentDetailResponse [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/724232 (https://phabricator.wikimedia.org/T291223) [04:14:16] (03CR) 10jerkins-bot: [V: 04-1] WIP Switch getHostedPaymentStatus to return a PaymentDetailResponse [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/724232 (https://phabricator.wikimedia.org/T291223) (owner: 10Cstone) [04:15:27] (03CR) 10jerkins-bot: [V: 04-1] WIP DI part of switching getHostedPaymentStatus to return a PaymentDetailResponse [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/724233 (https://phabricator.wikimedia.org/T291223) (owner: 10Cstone) [04:27:00] 10Fundraising-Backlog: Clean up new adyen ideal recurrings - https://phabricator.wikimedia.org/T291712 (10Cstone) [08:28:17] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 14 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civ... - https://phabricator.wikimedia.org/T277500 [10:11:00] (03PS1) 10Eileen: Clone buildkit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724358 [10:11:23] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 14 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civ... - https://phabricator.wikimedia.org/T277500 [10:16:00] (03CR) 10jerkins-bot: [V: 04-1] Clone buildkit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724358 (owner: 10Eileen) [13:38:47] 10Fundraising-Backlog, 10fr-donorservices: Clean up new adyen ideal recurrings - https://phabricator.wikimedia.org/T291712 (10MBeat33) [13:46:08] (03PS1) 10Umherirrender: Use IDatabase::ALL_ROWS in Campaign::getAllCampaignNames [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/724420 [14:59:38] (03CR) 10Abijeet Patro: [C: 03+1] Avoid writing empty prioritylangs rows to translate_metadata [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/720743 (https://phabricator.wikimedia.org/T204026) (owner: 10Nikerabbit) [17:47:56] 10fundraising-tech-ops: lightweight high-availability for payments-listener and frdata services - https://phabricator.wikimedia.org/T170318 (10Jgreen) [17:50:14] 10fundraising-tech-ops: lightweight high-availability for payments-listener and frdata services - https://phabricator.wikimedia.org/T170318 (10Jgreen) BIRD and anycast-healthchecker have been forked from production puppet to the frack puppet repo. It seems to be at a point where it generally installs and appears... [18:02:26] (03CR) 10Ejegg: [V: 03+2] Add PrivateBin to services [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/720092 (https://phabricator.wikimedia.org/T285881) (owner: 10Ejegg) [18:15:48] fr-tech I think we should all have maintainer access to the docker repo here - want to tell me if you are able to merge this? https://gitlab.wikimedia.org/releng/dev-images/-/merge_requests/4 [18:18:45] nope [18:19:02] Ask someone with write access to this repository to merge this request [18:19:06] yeah nope [18:24:58] 10Fundraising-Backlog, 10Analytics-Radar, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, 10User-Johan: Understand impact of Apple's Relay Service - https://phabricator.wikimedia.org/T289795 (10DStrine) [18:26:28] 10Fundraising-Backlog: Spanish Monthly Convert for the US / Does es-419 fall back to es? - https://phabricator.wikimedia.org/T291993 (10HNordeenWMF) [18:37:35] 10Fundraising-Backlog, 10Analytics-Radar, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, 10User-Johan: Understand impact of Apple's Relay Service - https://phabricator.wikimedia.org/T289795 (10JMinor) [18:44:30] wow [18:44:44] Given that IPs are a primary signal and only way to prevent repeated account creation and sock-puppetry, this means many Wikis will likely need to or will at least consider blocking all edits from Apple's IPs, as we do with VPN. [18:52:57] jgleeson and cstone you were logged in at the gitlab page? [18:53:13] yep [18:53:42] hmm, looks like we should all be maintainers: https://gitlab.wikimedia.org/releng/dev-images/-/project_members?tab=groups [18:54:25] oh wait, that group only includes me and Brennen [18:54:32] ok, lemme add the rest of us [18:54:46] my username is ceec on there too to cause more confusion [18:55:56] heh, yep, found it [18:56:23] damilare, do you want to set up access at https://gitlab.wikimedia.org/ ? [18:57:08] eileen doesn't seem to have an account there either [18:57:27] mergable now ejegg [18:57:34] woohoo [19:00:05] same i can too [19:01:41] be there in 1 minute fr-tech [19:04:38] ejegg: yes please, how do I go about it [19:06:10] damilare: try signing in with the link at top right, using your wikitech credentials [19:07:29] yh I got this error: "Service access denied due to missing privileges." [19:07:50] oh odd [19:09:01] 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10FR-Adyen, 10FR-Smashpig, and 2 others: Update SmashPig + DonationInterface under CRM - https://phabricator.wikimedia.org/T291219 (10XenoRyet) 05Open→03Resolved [19:10:47] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fr-donorservices: Civi Sidebar "recent items" --> no longer showing the top records - https://phabricator.wikimedia.org/T290767 (10XenoRyet) 05Open→03Resolved [19:11:10] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10MW-1.37-notes (1.37.0-wmf.23; 2021-09-13): Adyen Checkout display bug on Chrome Android - https://phabricator.wikimedia.org/T290528 (10XenoRyet) 05Open→03Resolved [19:12:04] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10FR-Adyen, 10MW-1.37-notes (1.37.0-wmf.23; 2021-09-13): Adyen Checkout: Further CSS tweaks - https://phabricator.wikimedia.org/T290105 (10XenoRyet) 05Open→03Resolved [19:12:35] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10MediaWiki-extensions-CentralNotice, 10JavaScript, and 2 others: Central notice projects and language choices not loading - https://phabricator.wikimedia.org/T291410 (10XenoRyet) 05Open→03Resolved [19:12:47] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster: Disable post-payment (Ingenico) monthly convert during September 21 en6C pre-test - https://phabricator.wikimedia.org/T291429 (10XenoRyet) 05Open→03Resolved [19:13:19] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster: Stripe import issues 9/16/21 - https://phabricator.wikimedia.org/T291220 (10XenoRyet) 05Open→03Resolved [19:16:04] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10Patch-For-Review: Update CiviCRM extensions - https://phabricator.wikimedia.org/T290050 (10XenoRyet) 05Open→03Resolved [19:16:11] (03Abandoned) 10Eileen: Geocoder update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/723021 (https://phabricator.wikimedia.org/T290050) (owner: 10Eileen) [19:21:32] 10Fundraising-Backlog: Help unblock Language team - https://phabricator.wikimedia.org/T292008 (10XenoRyet) [19:42:38] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Help unblock Language team - https://phabricator.wikimedia.org/T292008 (10DStrine) [19:43:44] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Help unblock Language team - https://phabricator.wikimedia.org/T292008 (10XenoRyet) [19:51:28] 10Fundraising-Backlog, 10fundraising sprint S 2021, 10MediaWiki-extensions-CentralNotice: Help unblock Language team - https://phabricator.wikimedia.org/T292008 (10XenoRyet) [19:51:29] 10Fundraising-Backlog, 10fundraising sprint S 2021: Spanish Monthly Convert for the US / Does es-419 fall back to es? - https://phabricator.wikimedia.org/T291993 (10XenoRyet) [19:51:33] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 16 others: CiviCRM CI tests failing due to half-resolution of T277500 - https://phabricator.wikimedia.org/T291897 (10XenoRyet) [19:51:37] 10Fundraising-Backlog, 10fundraising sprint S 2021, 10fr-donorservices: Clean up new adyen ideal recurrings - https://phabricator.wikimedia.org/T291712 (10XenoRyet) [19:51:41] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10FR-Adyen: Set up new fr-dev staging server for Apple Pay - https://phabricator.wikimedia.org/T291577 (10XenoRyet) [19:51:47] 10Fundraising-Backlog, 10fundraising sprint S 2021, 10MediaWiki-extensions-DonationInterface, 10Patch-For-Review: Make it possible to turn off post-payments monthly convert by URL parameter - https://phabricator.wikimedia.org/T291507 (10XenoRyet) [19:51:53] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10MediaWiki-extensions-CentralNotice: Update CentralNotice admin UI to fix or replace broken JQ multiselect widget - https://phabricator.wikimedia.org/T291431 (10XenoRyet) [19:52:01] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021: Apple Pay: Prevent flash of normal blue donate button before branded button appears - https://phabricator.wikimedia.org/T291143 (10XenoRyet) [19:52:05] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021: sprint goal: deploy apple pay and begin the 600/800 task - https://phabricator.wikimedia.org/T290107 (10XenoRyet) [19:52:09] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, and 2 others: Switch over to using WMF-generated certificates with Adyen Apple Pay - https://phabricator.wikimedia.org/T290108 (10XenoRyet) [19:52:13] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 14 others: Fix civicrm repo to be non-symlinked - https://phabricator.wikimedia.org/T289100 (10XenoRyet) [19:52:17] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, and 2 others: Adyen Checkout: Remove flash of unused card type selection buttons - https://phabricator.wikimedia.org/T290103 (10XenoRyet) [19:52:21] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, and 2 others: Recurring for apple pay through adyen - https://phabricator.wikimedia.org/T290106 (10XenoRyet) [19:52:25] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Parallel processing roller coaster, 10fundraising sprint Quantum Leapfrog, and 3 others: Send Receipts not working from civi - https://phabricator.wikimedia.org/T288149 (10XenoRyet) [19:52:29] 10Fundraising-Backlog, 10fundraising sprint Parallel processing roller coaster, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, and 4 others: Adyen Checkout: Pull donor contact from Apple Pay sheet - https://phabricator.wikimedia.org/T285722 (10XenoRyet) [19:52:33] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10FR-Smashpig: Add RiskScorer for Ingenico to SmashPig - https://phabricator.wikimedia.org/T291230 (10XenoRyet) [19:52:37] 10Fundraising Sprint Mandatory corn dogs, 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint Parallel processing roller coaster, and 6 others: Adyen Checkout: normalize API error codes - https://phabricator.wikimedia.org/T283307 (10XenoRyet) [19:52:41] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10fr-donorservices: Epic: Automatically send status 800 on prior donors stopped at 600 - https://phabricator.wikimedia.org/T276183 (10XenoRyet) [19:52:47] 10Fundraising Sprint Mandatory corn dogs, 10Fundraising-Backlog, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint Legø Trampoline, and 4 others: Adyen Checkout: create new recurring iDEAL donations - https://phabricator.wikimedia.org/T284606 (10XenoRyet) [19:52:51] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021: Update PaymentsInitQueueConsumer to leave FAILED/REJECTED pending rows for Ingenico - https://phabricator.wikimedia.org/T291228 (10XenoRyet) [19:52:55] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021: Add existing donor matching to PendingTransaction::resolve - https://phabricator.wikimedia.org/T291226 (10XenoRyet) [19:52:59] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10Patch-For-Review: Add PendingTable::Consume API call that implements pending resolver outer loop - https://phabricator.wikimedia.org/T291225 (10XenoRyet) [19:53:03] 10Fundraising-Backlog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10Recurring-Donations, 10fr-donorservices: Civi: Auto Recurring Fail email, batch of copy corrections - https://phabricator.wikimedia.org/T289617 (10XenoRyet) [19:53:07] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10Patch-For-Review: Add PendingTransaction::resolve API call that replicates Ingenico orphan rectifier - https://phabricator.wikimedia.org/T291223 (10XenoRyet) [19:53:11] 10Fundraising-Backlog, 10fundraising sprint Parallel processing roller coaster, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021: Catch Adyen Checkout API errors on the DI side - https://phabricator.wikimedia.org/T288830 (10XenoRyet) [19:53:15] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Parallel processing roller coaster, 10fundraising sprint Quantum Leapfrog, and 3 others: Review upstream PRs related to contracted token work - https://phabricator.wikimedia.org/T288698 (10XenoRyet) [19:53:19] 10Fundraising Sprint Ketchup Flume Ride, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Legø Trampoline, and 2 others: Would it be possible to have an 'Owns the Donor Advised Fund' field that would link that Individual to their DAF? - https://phabricator.wikimedia.org/T286520 (... [19:53:23] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint Parallel processing roller coaster, and 5 others: Create new WMF donor fields - https://phabricator.wikimedia.org/T288721 (10XenoRyet) [19:53:27] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising Sprint NULL calorie food cart, 10fundraising sprint Parallel processing roller coaster, and 4 others: update the docs on the civi-acoustic import and export scripts - https://phabricator.wikimedia.org/T286934 (10XenoRyet) [19:53:31] 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, and 2 others: simplify download UI for the donor data delivery tool - https://phabricator.wikimedia.org/T285881 (10XenoRyet) [19:53:35] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Parallel processing roller coaster, 10fundraising sprint Quantum Leapfrog, and 3 others: Search kit links not loading when sharing out to Engage - https://phabricator.wikimedia.org/T279686 (10XenoRyet) [19:53:39] 10Fundraising Sprint Nitpicking, 10Fundraising Sprint Octopus Untangling, 10Fundraising Sprint Pretending This Isn't Happening, 10Fundraising-Backlog, and 5 others: Documentation: UML activity and sequence diagrams for all processors - https://phabricator.wikimedia.org/T141617 (10XenoRyet) [19:53:43] 10Fundraising Sprint Esperantoland, 10Fundraising Sprint File Systems Stage Show, 10Fundraising Sprint Git Rebase Jump, 10Fundraising Sprint Humongous bacteria petting zoo, and 15 others: Update Fundraising tech CI image to use upstream buildkit, no symlink for civ... - https://phabricator.wikimedia.org/T277500 [19:53:47] 10Fundraising Sprint Princess Mongodb, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, and 3 others: Email preference center: create new mail type "Email Preference Country" - https://phabricator.wikimedia.org/T280674 (10XenoRyet) [19:53:51] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10fr-email-preference-center: Snooze for 90 days setting on email preference center - https://phabricator.wikimedia.org/T278859 (10XenoRyet) [19:53:55] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Roller Toaster, 10fundraising sprint S 2021, 10fr-email-preference-center: Add 3 email preference fields to civi - https://phabricator.wikimedia.org/T278858 (10XenoRyet) [19:53:59] 10Fundraising Sprint Airline Passenger Experience, 10Fundraising Sprint Bee Wheel, 10Fundraising Sprint Corrugated super slide, 10Fundraising Sprint Downed power line jump rope, and 27 others: Fr-tech chores list - https://phabricator.wikimedia.org/T258527 (10XenoRyet) [20:16:01] 10Fundraising-Backlog, 10fundraising sprint Screaming Goats Yelling Zoo: Spanish Monthly Convert for the US / Does es-419 fall back to es? - https://phabricator.wikimedia.org/T291993 (10Ejegg) Fallback should work on payments-wiki. We've got a little bit of code there specifically to make es-419 fall back to e... [20:27:11] ejegg: fr-tech, I spent a bit of time earlier trying to set up the equivalent of what we have for the jackgleeson.co.uk domain test tunnels on a single wmf cloud instance within our project. I got as far as getting the tunnel running and receiving traffic from the VM but could not get the WMF proxy thingy on the frontend to send traffic to the VM. I think you mentioned something similar ejegg. [20:27:13] The 502 gateway errors :/ [20:27:31] jgleeson: yeah, i'm having the same problem [20:27:43] I can't seem to make it allow traffic into the other ports [20:29:36] if y'all don't figure it out on tech-talk, might be one for the wmf cloud services irc chan maybe? [20:29:49] yep [20:30:18] it would be sweet if we could manage all of that complexity in a handful of individual web proxy URLs [20:31:32] srsly, not having to manage ssl certs would be great [20:33:50] ohhh, it might be this: https://serverfault.com/questions/910526/ssh-local-port-forwarding-working-from-localhost-only [20:33:59] I think you had to fix that on your servers too [20:34:20] oh wait, not that [20:34:29] you had to fix something in sshd, right? [20:34:33] GATEWAY_PORTS? [20:34:44] yeah I added the gateway ports config [20:34:50] on the wmf cloud instance [20:34:56] oh cool [20:35:05] but I think that config fix is different [20:35:10] i'm messing around on cn-staging-1 and had totally forgotten that one [20:35:15] lemme quickly try that, it might work [20:35:53] 10Fundraising-Backlog, 10Analytics-Radar, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, and 2 others: Understand impact of Apple's Relay Service - https://phabricator.wikimedia.org/T289795 (10Krinkle) [20:36:53] oh that's interesting [20:37:02] the GatewayPorts line is gone [20:37:08] from /etc/sshd_config [20:37:14] did I not save it? [20:37:22] or maybe puppet overwrote it? [20:37:54] woot, i just got something on my local netcat listener! [20:37:55] ok I'll reapply that, try and again and if still not success try your link fix [20:38:04] ah nice! [20:38:22] yeah, after applying GatewayPorts I didn't need to do the stuff at that link [20:40:12] oh so when I reload sshd after applying that setting, it dissapears [20:40:26] disappears* [20:41:54] hmmm not if I add it right at the bottom of the file [20:46:01] still timing out for me on the proxy side although I can hit my local machine from the cloud VM [20:52:45] 10Fundraising Sprint Ketchup Flume Ride, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Legø Trampoline, and 2 others: Would it be possible to have an 'Owns the Donor Advised Fund' field that would link that Individual to their DAF? - https://phabricator.wikimedia.org/T286520 (... [21:03:29] best of luck figuring it all out! [21:06:33] cstone: ejegg can we get this merged now? https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/722486 [21:07:23] looking eileen [21:08:59] (03PS1) 10Eileen: Trigger update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724509 [21:09:03] seems about right, but those few wmf_donor. prefixes look a bit odd [21:13:34] (03PS7) 10Eileen: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) [21:13:41] ejegg: ^^ [21:14:17] (03CR) 10jerkins-bot: [V: 04-1] Trigger update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724509 (owner: 10Eileen) [21:14:41] I guess CI is still broken [21:19:23] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [21:24:53] ejegg: cstone I'm going out to get my second jab soon -so I just want to pressure a little bit on ^^ so I can get the trigger / update sql to dwisehaupt before I go [21:27:05] (03PS1) 10Eileen: Fix our message template install to do both reserved & editable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724513 [21:27:47] eileen i'm looking at the trigger one and I don't see the new all_funds fields being calculated there [21:28:16] ejegg: yeah - I generated that off live now & then realised the other patch had to be deployed before I could generate the right sql [21:28:27] oh ok, got it [21:30:16] urg, getting 'unknown column coupon_code' locally [21:30:37] that was dropped a while back, right? [21:30:54] ejegg: yeah - you had that before one time too? [21:31:05] yep, but I deleted it locally [21:31:17] i guess it's sometimes helpful to have the smae warts as prod [21:32:06] :-) [21:32:33] ejegg: is that when trying to run the sql or the code to generate the sql? [21:32:41] (03CR) 10jerkins-bot: [V: 04-1] Fix our message template install to do both reserved & editable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724513 (owner: 10Eileen) [21:32:55] do we still have a field on prod that we shouldn't? [21:34:27] (03PS8) 10Eileen: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) [21:34:29] (03PS2) 10Eileen: [WIP] Add recurring failed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724231 [21:37:48] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add recurring failed [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724231 (owner: 10Eileen) [21:39:31] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [21:42:44] eileen: sorry, that was when trying to run the sql locally [21:43:00] so yeah, if that sql was generated from prod it looks like we have that obsolete field there [21:43:17] ejegg: yeah - I think prod maybe has the extra field - we must have figured that out last time - we should drop on prod so we don't get confused next time [21:43:18] but the changed one looks good [21:44:03] (03CR) 10Ejegg: [C: 03+2] "This looks good! As per Eileen in IRC, another will be coming later to add the all_funds_ fields" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724509 (owner: 10Eileen) [21:44:11] (03CR) 10Ejegg: [V: 03+2 C: 03+2] Trigger update [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724509 (owner: 10Eileen) [21:44:51] (03PS9) 10Ejegg: Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [21:46:23] (03CR) 10Ejegg: [V: 03+2 C: 03+2] "This looks good and works to fill those fields for me locally" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [21:46:32] thanks ejegg [21:46:45] yw! [21:52:42] (03CR) 10jerkins-bot: [V: 04-1] Fill new wmf_donor fields. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/722486 (https://phabricator.wikimedia.org/T288721) (owner: 10Eileen) [22:14:38] 10fundraising-tech-ops: frdb2003: configure RAID, install OS, and add to fr-analytics db replication - https://phabricator.wikimedia.org/T290484 (10Dwisehaupt) [22:15:07] 10fundraising-tech-ops: frdb2003: configure RAID, install OS, and add to fr-analytics db replication - https://phabricator.wikimedia.org/T290484 (10Dwisehaupt) Database cloned and replicating off of frdb1003. [22:17:41] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724517 [22:17:59] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/724517 (owner: 10Eileen) [22:25:31] !log civicrm revision changed from b8f756b60e to a1929b3dfd, config revision is 77cb7ec866 [22:25:39] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [22:48:53] (03PS1) 10Eileen: Remove events in cart triggers - so we can remove the coupon_code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724522 [22:49:34] ejegg: dang that cal fields wasn't quite enough - but @dwisehaupt we can start that db update when suites - I just still have to nail down the trigger script to reenale [22:50:12] ok. let me put things in maint mode. [22:53:44] (03CR) 10jerkins-bot: [V: 04-1] Remove events in cart triggers - so we can remove the coupon_code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724522 (owner: 10Eileen) [22:54:10] ok. civi in maint mode and p-c jobs stopped. triggers disabled. [22:54:20] you are clear to do the update [22:55:01] as i mentioned, i have to head off for 60+ minutes. i'll be reachable by phone but the queries should take longer than that. [22:55:26] dwisehaupt: kicking it off now [22:57:11] ok - it's running [23:13:59] ejegg: I do need to do another trigger PR - working on it now [23:14:07] also - I'm gonna drop that field... [23:18:09] (03PS1) 10Eileen: Remove legacy event cart field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724531 [23:18:40] ejegg: ^^ - will run that before I re-enable/ generate triggers [23:21:39] (03CR) 10Ejegg: [V: 03+2 C: 03+2] "Hooray for wart removal!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/724531 (owner: 10Eileen) [23:45:33] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2943 [23:46:33] PROBLEM - check_mysql on frdb2002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3002 [23:47:31] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3060 [23:47:35] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3065 [23:50:33] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3243 [23:51:31] RECOVERY - check_mysql on frdb2002 is OK: Uptime: 2852009 Threads: 12 Questions: 165179624 Slow queries: 1371 Opens: 3001 Flush tables: 1 Open tables: 974 Queries per second avg: 57.916 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 [23:52:31] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3360 [23:52:35] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3365 [23:55:35] PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3543 [23:57:29] PROBLEM - check_mysql on frdb2001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3660 [23:57:35] PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3665