[00:13:11] (03PS1) 10Lars SG: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308240 [00:13:44] (03CR) 10CI reject: [V:04-1] Add tab for DonorHistory [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308231 (https://phabricator.wikimedia.org/T430865) (owner: 10Eileen) [00:15:00] (03PS2) 10Eileen: Port menu cache clear fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 [00:15:11] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 (owner: 10Eileen) [00:24:17] 06Fundraising-Backlog: Contributions against anonymous contact (72) being sent thank yous - https://phabricator.wikimedia.org/T431516 (10Eileenmcnaughton) 03NEW [00:24:46] (03CR) 10Wfan: [C:03+2] Set mysql_fts to false [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308228 (owner: 10Eileen) [00:32:30] (03PS1) 10Lars SG: Test fix for donor segment / status history [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308241 (https://phabricator.wikimedia.org/T430865) [00:33:21] (03CR) 10Lars SG: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308240 (owner: 10Lars SG) [00:35:05] (03PS1) 10Wfan: Cast null to '' to overpass sql return, since string as response expected [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308242 (https://phabricator.wikimedia.org/T431468) [00:40:28] wfan: are you also doing an upstream PR for ^^ [00:41:17] (03CR) 10Eileen: [C:03+2] "This isn't a full fix but the errors might be better & it should be done regardless IMHO" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308242 (https://phabricator.wikimedia.org/T431468) (owner: 10Wfan) [00:42:35] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Omnirecipient.load null return - https://phabricator.wikimedia.org/T431468#12097690 (10Lars) Possibly we are running into problems when we try to pass ~2m contact ids in an IN to the API in _civicrm_api3_omnirecipient_load_batch_get_... [01:03:02] (03CR) 10CI reject: [V:04-1] Set mysql_fts to false [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308228 (owner: 10Eileen) [01:03:58] (03CR) 10Eileen: [V:03+2] "Forcing this cos unrelatedd" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308228 (owner: 10Eileen) [01:03:59] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308240 (owner: 10Lars SG) [01:07:01] (03PS1) 10Lars SG: Only map the contacts we'll load on this run, no the whole file which can be large [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308243 (https://phabricator.wikimedia.org/T431468) [01:07:11] (03CR) 10CI reject: [V:04-1] Port menu cache clear fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 (owner: 10Eileen) [01:09:59] (03PS2) 10Lars SG: Only map the contacts we'll load on this run, not the whole file which can be large [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308243 (https://phabricator.wikimedia.org/T431468) [01:12:00] !log civicrm upgraded from 941505c3 to 0057f9a1 [01:12:01] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [01:23:47] (03CR) 10CI reject: [V:04-1] Cast null to '' to overpass sql return, since string as response expected [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308242 (https://phabricator.wikimedia.org/T431468) (owner: 10Wfan) [01:24:50] (03CR) 10Eileen: [C:03+2] Test fix for donor segment / status history [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308241 (https://phabricator.wikimedia.org/T430865) (owner: 10Lars SG) [01:26:23] (03CR) 10Eileen: [C:03+2] "Let's give it a whirl!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308243 (https://phabricator.wikimedia.org/T431468) (owner: 10Lars SG) [01:32:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Omnirecipient.load null return - https://phabricator.wikimedia.org/T431468#12097752 (10AnnWF) https://github.com/civicrm/civicrm-core/pull/36182 upstream of the case convert for typeError [01:34:45] (03CR) 10Lars SG: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308242 (https://phabricator.wikimedia.org/T431468) (owner: 10Wfan) [01:35:23] (03CR) 10Lars SG: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 (owner: 10Eileen) [01:35:31] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Omnirecipient.load null return - https://phabricator.wikimedia.org/T431468#12097753 (10AnnWF) >>! In T431468#12097690, @Lars wrote: > Possibly we are running into problems when we try to pass ~2m contact ids in an IN to the API in _c... [01:38:37] (03PS1) 10Wfan: add array_chunk to use batch for contacts avoid large contacts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308248 (https://phabricator.wikimedia.org/T431468) [01:40:01] (03PS3) 10Eileen: Port menu cache clear fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 [01:40:22] (03PS2) 10Wfan: Cast null to '' to overpass sql return, since string as response expected [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308242 (https://phabricator.wikimedia.org/T431468) [01:42:27] (03PS2) 10Eileen: Document a few more properties [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307008 [01:45:58] (03CR) 10Lars SG: [C:03+2] Document a few more properties [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307008 (owner: 10Eileen) [01:46:43] (03Merged) 10jenkins-bot: Document a few more properties [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307008 (owner: 10Eileen) [01:55:39] FIRING: HighPaymentMethodFailures: Average card payment method failures are high [6.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentMethodFailures [01:55:39] FIRING: HighPaymentSchemeFailures: Average payment method scheme failures are high [6.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentSchemeFailures [01:55:39] FIRING: HighPaymentGatewayFailures: Average gravy payment gateway failures are high [6.2] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentGatewayFailures [01:57:39] FIRING: SystemdService: Systemd service coworker.service stopped on civi1002.frack.eqiad.wmnet:9100 - https://alerts.wikimedia.org/?q=alertname%3DSystemdService [02:01:16] ^^^ acked the ones there that are related to the maintenance. [02:13:41] FIRING: [3x] EndpointResultCritical: Endpoint check for civiproxy on civi1002 is CRITICAL - https://alerts.wikimedia.org/?q=alertname%3DEndpointResultCritical [02:16:31] (03PS1) 10Eileen: Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) [02:16:34] (03CR) 10Lars SG: [C:03+2] "Let's give it a try" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 (owner: 10Eileen) [02:18:27] (03PS2) 10Eileen: Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) [02:18:45] (03CR) 10CI reject: [V:04-1] Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [02:20:16] (03PS3) 10Eileen: Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) [02:20:33] (03CR) 10Lars SG: [C:03+2] "Looks good to me, works well, thanks!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1307279 (https://phabricator.wikimedia.org/T430011) (owner: 10Dmiranda) [02:28:43] FIRING: RedisQueueSize: Redis Queue contribution_tracking is Critical: [2152] - https://frmon.wikimedia.org/d/R5m3iU1Wk/queue?orgId=1&from=now-24h&to=now&timezone=utc - https://alerts.wikimedia.org/?q=alertname%3DRedisQueueSize [02:39:57] FIRING: [2x] MariadbReplicationRunning: MariaDB replica thread not running on : frdb1008.frack.eqiad.wmnet:9004 for >30m - https://alerts.wikimedia.org/?q=alertname%3DMariadbReplicationRunning [02:40:29] (03Merged) 10jenkins-bot: Port menu cache clear fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308233 (owner: 10Eileen) [02:40:31] (03Merged) 10jenkins-bot: Updated to use an array instead of string + concatenation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1307279 (https://phabricator.wikimedia.org/T430011) (owner: 10Dmiranda) [02:47:53] (03PS1) 10Lars SG: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308253 [02:51:33] (03PS1) 10Lars SG: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308254 [02:52:03] (03CR) 10Lars SG: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308253 (owner: 10Lars SG) [02:53:08] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308253 (owner: 10Lars SG) [02:55:09] (03PS1) 10Eileen: chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) [02:55:39] (03CR) 10CI reject: [V:04-1] chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [02:57:35] (03PS2) 10Eileen: chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) [02:58:06] (03CR) 10CI reject: [V:04-1] chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [02:59:30] (03CR) 10Cstone: "It's late but I think CheckoutDotCom is just slightly less worse naming wise, will look" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307682 (https://phabricator.wikimedia.org/T431240) (owner: 10Eileen) [03:01:45] (03CR) 10Lars SG: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308254 (owner: 10Lars SG) [03:03:03] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308254 (owner: 10Lars SG) [03:07:20] (03PS3) 10Eileen: chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) [03:08:43] FIRING: [2x] RedisQueueSize: Redis Queue contribution_tracking is Critical: [3187] - https://frmon.wikimedia.org/d/R5m3iU1Wk/queue?orgId=1&from=now-24h&to=now&timezone=utc - https://alerts.wikimedia.org/?q=alertname%3DRedisQueueSize [03:19:06] (03PS1) 10Lars SG: Update triggers for new audience framework [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308257 (https://phabricator.wikimedia.org/T426818) [03:23:27] (03CR) 10Eileen: [C:03+2] "Seems as we expect `civicrm_value_test_51_after_insert` is a bit odd but we can look at that" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308257 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [03:26:30] (03PS12) 10Eileen: CheckoutCom process audit file [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307682 (https://phabricator.wikimedia.org/T431240) [03:31:17] (03CR) 10Eileen: [V:03+2 C:03+2] Update triggers for new audience framework [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308257 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [03:33:43] FIRING: RedisQueueSize: Redis Queue payments_init is Critical: [2106] - https://frmon.wikimedia.org/d/R5m3iU1Wk/queue?orgId=1&from=now-24h&to=now&timezone=utc - https://alerts.wikimedia.org/?q=alertname%3DRedisQueueSize [03:36:14] (03PS1) 10Lars SG: Additional trigger update to remove test table [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308258 (https://phabricator.wikimedia.org/T426818) [03:41:19] (03CR) 10Cstone: [C:03+2] Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [03:41:21] (03PS1) 10Eileen: chariot: split out total, net_fee for matching_gift and individual_gift amounts [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308259 (https://phabricator.wikimedia.org/T415825) [03:41:55] (03CR) 10Eileen: [C:03+2] Additional trigger update to remove test table [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308258 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [03:41:58] (03CR) 10Eileen: [V:03+2 C:03+2] Additional trigger update to remove test table [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308258 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [03:41:59] (03Merged) 10jenkins-bot: Chariot: record another metadata field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308252 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [03:42:45] (03CR) 10Cstone: [C:03+2] chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [03:42:56] (03PS1) 10Eileen: Chariot: document another field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308260 [03:43:19] (03Merged) 10jenkins-bot: chariot: enhance donation amount helpers [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308256 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [03:45:10] (03CR) 10Cstone: [C:03+2] "Naming is hard, gravy sends it to us as checkoutcom so matching that..." [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307682 (https://phabricator.wikimedia.org/T431240) (owner: 10Eileen) [03:45:48] (03Merged) 10jenkins-bot: CheckoutCom process audit file [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307682 (https://phabricator.wikimedia.org/T431240) (owner: 10Eileen) [03:46:43] (03PS1) 10Lars SG: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308261 [03:46:54] (03CR) 10Lars SG: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308261 (owner: 10Lars SG) [03:46:58] (03CR) 10Lars SG: [V:03+2 C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308261 (owner: 10Lars SG) [03:48:53] !log civicrm upgraded from 0057f9a1 to ead61459 [03:48:54] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [03:49:38] (03PS1) 10Eileen: Chariot - rename individual & matching_gift amount to total_amount [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308262 (https://phabricator.wikimedia.org/T415825) [04:00:16] FIRING: ContextSwitchingSpike: Host context switching high (instance frpm1002.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [04:02:33] (03PS1) 10Eileen: Chariot audit: add new fields to test csv files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308263 (https://phabricator.wikimedia.org/T415825) [04:07:27] (03CR) 10CI reject: [V:04-1] Chariot - rename individual & matching_gift amount to total_amount [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308262 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [04:07:27] RESOLVED: MariadbReplicationRunning: MariaDB replica thread not running on : frdb2004.frack.codfw.wmnet:9004 for >30m - https://frmon.wikimedia.org/d/000000273/mariadb-system?orgId=1&var-server=frdb2004.frack.codfw.wmnet - https://alerts.wikimedia.org/?q=alertname%3DMariadbReplicationRunning [04:08:00] (03PS1) 10Eileen: Update accounting batch view to hold defaults [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308264 [04:08:16] (03PS2) 10Eileen: Update accounting batch view to hold defaults [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308264 [04:13:09] RESOLVED: SystemdService: Systemd service coworker.service stopped on civi1002.frack.eqiad.wmnet:9100 - https://alerts.wikimedia.org/?q=alertname%3DSystemdService [04:13:36] FIRING: HostTotalProcs: Host high process count (instance frdb1008.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DHostTotalProcs [04:15:16] FIRING: ContextSwitchingSpike: Host context switching high (instance frdb1008.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [04:18:13] RESOLVED: RedisQueueSize: Redis Queue pending is Critical: [3247] - https://frmon.wikimedia.org/d/R5m3iU1Wk/queue?orgId=1&from=now-24h&to=now&timezone=utc - https://alerts.wikimedia.org/?q=alertname%3DRedisQueueSize [04:18:36] RESOLVED: HostTotalProcs: Host high process count (instance frdb1008.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DHostTotalProcs [04:20:16] RESOLVED: ContextSwitchingSpike: Host context switching high (instance frdb1008.frack.eqiad.wmnet:9100) - https://alerts.wikimedia.org/?q=alertname%3DContextSwitchingSpike [04:22:15] (03PS1) 10Eileen: Upgrading wikimedia/smash-pig (v1.2.4.18 => v1.2.4.19) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308265 [04:24:48] (03CR) 10CI reject: [V:04-1] Chariot audit: add new fields to test csv files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308263 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [04:26:44] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Omnirecipient.load null return - https://phabricator.wikimedia.org/T431468#12097874 (10Lars) a:03Lars [04:28:37] FIRING: FrTechFailMail: CiviCRM failmail activity exceeded 3 msg/min for the past 5 minutes. - https://alerts.wikimedia.org/?q=alertname%3DFrTechFailMail [04:30:20] (03CR) 10Cstone: [C:03+2] chariot: split out total, net_fee for matching_gift and individual_gift amounts [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308259 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [04:30:27] (03CR) 10Cstone: [C:03+2] Chariot: document another field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308260 (owner: 10Eileen) [04:30:55] (03Merged) 10jenkins-bot: chariot: split out total, net_fee for matching_gift and individual_gift amounts [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308259 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [04:31:02] (03Merged) 10jenkins-bot: Chariot: document another field [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308260 (owner: 10Eileen) [04:33:37] RESOLVED: FrTechFailMail: CiviCRM failmail activity exceeded 3 msg/min for the past 5 minutes. - https://alerts.wikimedia.org/?q=alertname%3DFrTechFailMail [04:42:54] (03PS1) 10Eileen: chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) [04:43:23] (03CR) 10CI reject: [V:04-1] chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [04:44:34] (03PS3) 10Eileen: Add handling for incomplete deposits [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1306997 (https://phabricator.wikimedia.org/T415825) [04:45:43] (03PS2) 10Eileen: Use property rather than passing api parameter around [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307005 (https://phabricator.wikimedia.org/T415825) [04:46:09] (03PS2) 10Eileen: Chariot: add maintenance script to list & set properties [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307006 (https://phabricator.wikimedia.org/T415825) [04:46:27] (03CR) 10Eileen: [C:03+2] Upgrading wikimedia/smash-pig (v1.2.4.18 => v1.2.4.19) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308265 (owner: 10Eileen) [04:47:11] RESOLVED: [3x] EndpointResultCritical: Endpoint check for civiproxy on civi1002 is CRITICAL - https://alerts.wikimedia.org/?q=alertname%3DEndpointResultCritical [04:52:25] (03PS2) 10Eileen: chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) [04:53:30] (03PS4) 10Eileen: Initial checkout.com audit processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1307867 (https://phabricator.wikimedia.org/T431240) [04:56:02] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Add Refunded donation find FormBuilder with name of user who did the refund - https://phabricator.wikimedia.org/T431463#12097895 (10Lars) @MBeat33 We added the index during today's downtime, it's speedy now. [04:56:58] (03PS5) 10Eileen: Initial checkout.com audit processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1307867 (https://phabricator.wikimedia.org/T431240) [05:03:13] 06Fundraising-Backlog, 07payments-orchestration: Set up the Braintree -> Gravy settlement report - https://phabricator.wikimedia.org/T396147#12097899 (10Eileenmcnaughton) [05:03:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-WMF-Audit: Bug with move of venmo to gravy - the gr4vy ID rather than the braintree ID is being stored as the braintree ID - https://phabricator.wikimedia.org/T431230#12097903 (10Eileenmcnaughton) We seem to have another 128... eg https://civicrm.w... [05:17:26] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Adding check number field to DAFpay payment method - https://phabricator.wikimedia.org/T429835#12097911 (10Eileenmcnaughton) [05:18:13] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Adding check number field to DAFpay payment method - https://phabricator.wikimedia.org/T429835#12097912 (10Eileenmcnaughton) @AKanji-WMF yes - I've discussed with Melanie - not too big a deal - onc... [05:53:11] FIRING: NodeDown: Node fransw1003 is down. - https://frmon.wikimedia.org/d/000000377/host-overview?orgId=1&var-host=fransw1003 - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [05:54:28] FIRING: OOM: OOM killer active on fransw1003.frack.eqiad.wmnet:9100 - https://alerts.wikimedia.org/?q=alertname%3DOOM [05:58:11] RESOLVED: NodeDown: Node fransw1003 is down. - https://frmon.wikimedia.org/d/000000377/host-overview?orgId=1&var-host=fransw1003 - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [05:59:28] RESOLVED: OOM: OOM killer active on fransw1003.frack.eqiad.wmnet:9100 - https://alerts.wikimedia.org/?q=alertname%3DOOM [06:18:05] (03PS2) 10Eileen: Chariot - rename individual & matching_gift amount to total_amount [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308262 (https://phabricator.wikimedia.org/T415825) [06:18:05] (03PS2) 10Eileen: Chariot audit: add new fields to test csv files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308263 (https://phabricator.wikimedia.org/T415825) [06:18:32] (03PS1) 10Eileen: Notice fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308344 [06:35:54] (03CR) 10CI reject: [V:04-1] Chariot - rename individual & matching_gift amount to total_amount [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308262 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [06:36:38] (03CR) 10CI reject: [V:04-1] Notice fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308344 (owner: 10Eileen) [06:40:38] (03CR) 10CI reject: [V:04-1] Chariot audit: add new fields to test csv files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308263 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [07:38:38] 06Fundraising-Backlog, 10fundraising-tech-ops: Monitor the success percentage for each payment methods - https://phabricator.wikimedia.org/T407927#12098110 (10RKumar_WMF) In Gr4vy the Insights shows the data with the time lag of 15 mins or so. We can drill down to information like which country, BIN, issuer et... [07:40:58] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Add demo message template to CiviCRM - https://phabricator.wikimedia.org/T427975#12098120 (10Priyankar22) a:03Priyankar22 Hi, i'll pick this up and get it fixed ASAP, will submit a patch for review shortly. [07:41:35] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Add demo message template to CiviCRM - https://phabricator.wikimedia.org/T427975#12098124 (10Priyankar22) [07:47:15] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, and 2 others: Add documentation of all transactional emails that we send to Wikitech - https://phabricator.wikimedia.org/T428937#12098159 (10Priyankar22) a:03Priyankar22 Hi, i'd like to work on t... [07:52:45] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Add conditional text to failed recurring emails for messaging when failure is on initial donation - https://phabricator.wikimedia.org/T422092#12098185 (10Priyankar22) a:03Priyankar22 Hi, i'd like to work on this task and will get an update / fix out ver... [07:56:01] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Add conditional text to failed recurring emails for messaging when failure is on initial donation - https://phabricator.wikimedia.org/T422092#12098202 (10Priyankar22) [07:59:07] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Remove advertised source_contact_id:name fields in SK which don't work - https://phabricator.wikimedia.org/T428005#12098216 (10Priyankar22) a:03Priyankar22 Hi, i'd like to work on this task and will get an update / fix... [07:59:08] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1308403 (owner: 10L10n-bot) [07:59:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Remove advertised source_contact_id:name fields in SK which don't work - https://phabricator.wikimedia.org/T428005#12098218 (10Priyankar22) [08:02:24] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email, 10FR-Donor-portal, and 2 others: Update instructions to cancel or modify recurring in EOY TY email - https://phabricator.wikimedia.org/T429808#12098233 (10Priyankar22) a:03Priyankar22 Hi, i'd like to work on this task and will get... [08:02:46] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email, 10FR-Donor-portal, and 3 others: Update instructions to cancel or modify recurring in EOY TY email - https://phabricator.wikimedia.org/T429808#12098235 (10Priyankar22) [08:10:10] (03PS3) 10Eileen: chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) [08:10:10] (03PS1) 10Eileen: chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) [08:10:44] (03CR) 10CI reject: [V:04-1] chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [08:16:03] (03PS2) 10Eileen: chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) [08:16:33] (03CR) 10CI reject: [V:04-1] chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [08:18:51] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations: Add setting to allow auto-adding % to the end of Is Like searches in FormBuilder - https://phabricator.wikimedia.org/T430685#12098303 (10Priyankar22) a:03Priyankar22 Hi, i'd like to work on this task and will get an update / fix o... [08:19:14] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 06RoadToWiki: Add setting to allow auto-adding % to the end of Is Like searches in FormBuilder - https://phabricator.wikimedia.org/T430685#12098306 (10Priyankar22) [08:20:14] (03PS3) 10Eileen: chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) [08:21:32] (03CR) 10Nikerabbit: [C:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1308403 (owner: 10L10n-bot) [08:40:16] (03PS1) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [08:40:58] (03CR) 10CI reject: [V:04-1] chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [08:42:02] (03PS2) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [08:59:09] (03PS3) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [08:59:50] (03CR) 10CI reject: [V:04-1] chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [09:02:19] (03PS4) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [09:23:37] (03PS5) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [09:29:11] (03PS1) 10Eileen: chariot: fix comparison for string zero values [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308579 (https://phabricator.wikimedia.org/T415825) [09:52:32] (03CR) 10CI reject: [V:04-1] chariot: fix comparison for string zero values [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308579 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [10:17:56] (03PS1) 10Eileen: chariot: fix benevity fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308590 (https://phabricator.wikimedia.org/T415825) [10:48:50] (03CR) 10CI reject: [V:04-1] chariot: fix benevity fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308590 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [12:37:51] (03PS2) 10Jgleeson: WIP: Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) [12:37:58] (03PS3) 10Jgleeson: Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) [12:38:44] (03PS4) 10Jgleeson: Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) [12:39:07] (03PS5) 10Jgleeson: WIP: Add Gravy card form to ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307491 (https://phabricator.wikimedia.org/T428019) [12:39:12] (03PS6) 10Jgleeson: Add Gravy card form to ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307491 (https://phabricator.wikimedia.org/T428019) [12:42:39] FIRING: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [24.6] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [12:44:45] (03CR) 10CI reject: [V:04-1] Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) (owner: 10Jgleeson) [12:45:10] (03CR) 10CI reject: [V:04-1] Add Gravy card form to ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307491 (https://phabricator.wikimedia.org/T428019) (owner: 10Jgleeson) [13:39:40] 06Fundraising-Backlog, 10fundraising-tech-ops: Monitor the success percentage for each payment methods - https://phabricator.wikimedia.org/T407927#12100099 (10Jgreen) >>! In T407927#12098110, @RKumar_WMF wrote: > In Gr4vy the Insights shows the data with the time lag of 15 mins or so. We can drill down to info... [13:46:27] !log payments-wiki upgraded from f34901d7 to 196c7bc1 [13:46:29] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [14:32:39] RESOLVED: HighPaymentFraudMessages: More than 10 payment fraud messages in the past hour [14.3] - https://alerts.wikimedia.org/?q=alertname%3DHighPaymentFraudMessages [14:45:20] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Backfill donor status & segment history - https://phabricator.wikimedia.org/T431586 (10Lars) 03NEW [14:54:16] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 06RoadToWiki: Add setting to allow auto-adding % to the end of Is Like searches in FormBuilder - https://phabricator.wikimedia.org/T430685#12100662 (10Lars) @Priyankar22 Just so we're on the same page, note this is an upstream cha... [14:56:12] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email, 10FR-Donor-portal, and 3 others: Update instructions to cancel or modify recurring in EOY TY email - https://phabricator.wikimedia.org/T429808#12100667 (10Lars) @Priyankar22 Appreciate the enthusiasm, but this one is something our e... [14:57:46] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Remove advertised source_contact_id:name fields in SK which don't work - https://phabricator.wikimedia.org/T428005#12100673 (10Lars) @Priyankar22 This one is also an upstream change in CiviCRM. It is also... [14:58:15] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Add conditional text to failed recurring emails for messaging when failure is on initial donation - https://phabricator.wikimedia.org/T422092#12100674 (10Lars) @Priyankar22 This is also one for our email team, not a code change. [14:58:58] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, and 2 others: Add documentation of all transactional emails that we send to Wikitech - https://phabricator.wikimedia.org/T428937#12100680 (10Lars) @Priyankar22 This one would require knowledge of o... [15:39:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Why do a few donors have a last name and no first name? - https://phabricator.wikimedia.org/T412534#12100870 (10AnnWF) Contact ID Sort Name Contact Contributions: Total Amount Contact Contributions: Financial Typ... [15:42:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Omnirecipient.load null return - https://phabricator.wikimedia.org/T431468#12100879 (10AnnWF) Validate the job run without issue after the upgrade [16:50:34] (03PS5) 10Jgleeson: Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) [16:50:50] (03PS7) 10Jgleeson: Add Gravy card form to ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307491 (https://phabricator.wikimedia.org/T428019) [16:52:37] (03CR) 10CI reject: [V:04-1] Handle Gravy card donations for ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307490 (https://phabricator.wikimedia.org/T430751) (owner: 10Jgleeson) [16:54:37] (03CR) 10CI reject: [V:04-1] Add Gravy card form to ComboWiki [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1307491 (https://phabricator.wikimedia.org/T428019) (owner: 10Jgleeson) [17:16:51] 03Fundraising Sprint: Loud carpet, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-payments-general, 10FR-Smashpig: Failing Trustly recurring charges with No Institution ID configured - https://phabricator.wikimedia.org/T430789#12101292 (10Cstone) [17:19:31] 03Fundraising Sprint: Loud carpet, 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-payments-general, 10FR-Smashpig: Failing Trustly recurring charges with No Institution ID configured - https://phabricator.wikimedia.org/T430789#12101298 (10Cstone) Gravy updated that trustly said that these w... [17:24:11] 03Fundraising Sprint: Loud carpet, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Add source_record_id to merge activities - https://phabricator.wikimedia.org/T429532#12101325 (10Lars) Since this is just string parsing, we might as well backfill all of them. [17:26:50] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, and 2 others: Add documentation of all transactional emails that we send to Wikitech - https://phabricator.wikimedia.org/T428937#12101331 (10Priyankar22) a:05Priyankar22→03None Thanks for letti... [17:28:13] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Add conditional text to failed recurring emails for messaging when failure is on initial donation - https://phabricator.wikimedia.org/T422092#12101338 (10Priyankar22) a:05Priyankar22→03None Got it, thanks @Lars! I'll unclaim this one [17:29:16] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Remove advertised source_contact_id:name fields in SK which don't work - https://phabricator.wikimedia.org/T428005#12101348 (10Priyankar22) a:05Priyankar22→03None Understood, thanks @Lars! I'll unclai... [17:30:42] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-AutoTY-Email, 10FR-Donor-portal, and 3 others: Update instructions to cancel or modify recurring in EOY TY email - https://phabricator.wikimedia.org/T429808#12101354 (10Priyankar22) a:05Priyankar22→03None Understood, thanks for the guidance @... [17:33:56] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 06RoadToWiki: Add setting to allow auto-adding % to the end of Is Like searches in FormBuilder - https://phabricator.wikimedia.org/T430685#12101377 (10Priyankar22) a:05Priyankar22→03None [17:37:45] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Update getMergedTo to use source_record_id if not found in first pass - https://phabricator.wikimedia.org/T431604 (10Lars) 03NEW [17:38:45] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing: Develop process to prevent direct mail being sent to long-deleted cids - https://phabricator.wikimedia.org/T406771#12101399 (10Lars) Per discussion with @CHudson-WMF, we will provide a csv with mer... [17:44:44] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 06RoadToWiki: Add demo message template to CiviCRM - https://phabricator.wikimedia.org/T427975#12101421 (10Priyankar22) a:05Priyankar22→03None [18:31:44] 06Fundraising-Backlog, 10Fundraising-CiviCRM-Starting-Tasks, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, and 2 others: Show double opt-in on contact summary - https://phabricator.wikimedia.org/T407018#12101633 (10Lars) [18:40:21] (03PS1) 10Lars SG: Add last_recurring_amount_change & date to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1308708 (https://phabricator.wikimedia.org/T430877) [19:15:27] (03PS1) 10Cstone: Add in explaining comment [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1308739 [19:21:19] (03CR) 10Jgleeson: [C:03+2] Add in explaining comment [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1308739 (owner: 10Cstone) [19:22:29] 10Fundraising analytics stack, 10fundraising-tech-ops, 05WMF-NDA: connect metabase and slack - https://phabricator.wikimedia.org/T431496#12101824 (10AStein-WMF) permissions are largely in place. the only update needed was adding files.slack.com to the squid ACL. i also need to update the metabase systemd Ex... [19:22:51] 10Fundraising analytics stack, 10fundraising-tech-ops, 05WMF-NDA: connect metabase and slack - https://phabricator.wikimedia.org/T431496#12101826 (10AStein-WMF) a:03AStein-WMF [19:24:35] (03Merged) 10jenkins-bot: Add in explaining comment [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1308739 (owner: 10Cstone) [19:34:44] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308745 [19:35:21] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308745 (owner: 10Jgleeson) [19:36:52] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1308745 (owner: 10Jgleeson) [19:45:52] 03Fundraising Sprint: Loud carpet, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 13Patch-For-Review: Add last recurring amount change and last otg amount change fields to Civi & Acoustic - https://phabricator.wikimedia.org/T430877#12101924 (10Lars) Todo: backfill last_recurring_amount_change from... [19:46:49] (03PS1) 10Lars SG: Bulk update all donor segments & statuses and new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308758 (https://phabricator.wikimedia.org/T426818) [19:47:11] FIRING: [2x] NodeDown: Node civi2002 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [19:50:11] !log payments-wiki upgraded from 196c7bc1 to 5d2bb8e8 [19:50:12] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [19:52:11] RESOLVED: [2x] NodeDown: Node civi2002 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [19:56:32] (03Abandoned) 10Wfan: add array_chunk to use batch for contacts avoid large contacts [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308248 (https://phabricator.wikimedia.org/T431468) (owner: 10Wfan) [20:00:57] (03CR) 10Eileen: [C:03+2] Bulk update all donor segments & statuses and new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308758 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [20:02:11] FIRING: [3x] NodeDown: Node civi2002 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [20:03:01] ^^^ we'll see some of these warnings as we roll through these kernel updates. [20:03:47] FIRING: HAProxyExporterDown: HAProxy on fransc2001.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [20:07:11] RESOLVED: [6x] NodeDown: Node civi2002 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [20:08:47] RESOLVED: [2x] HAProxyExporterDown: HAProxy on fransc2001.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [20:19:18] (03Merged) 10jenkins-bot: Bulk update all donor segments & statuses and new wmf_donor fields [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308758 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [20:48:11] FIRING: NodeDown: Node fransw2002 is down. - https://frmon.wikimedia.org/d/000000377/host-overview?orgId=1&var-host=fransw2002 - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [20:49:47] FIRING: HAProxyExporterDown: HAProxy on fransw2002.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [20:53:11] RESOLVED: [2x] NodeDown: Node franio2003 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [20:54:47] RESOLVED: HAProxyExporterDown: HAProxy on fransw2002.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [21:09:18] (03PS1) 10Lars SG: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308774 [21:09:58] (03CR) 10CI reject: [V:04-1] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308774 (owner: 10Lars SG) [21:15:14] (03PS1) 10Lars SG: Update SmashPig [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1308776 [21:19:39] (03PS2) 10Lars SG: Upgrading wikimedia/smash-pig (v1.2.4.18 => v1.2.4.19) [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1308776 [21:19:49] (03CR) 10Lars SG: [C:03+2] Upgrading wikimedia/smash-pig (v1.2.4.18 => v1.2.4.19) [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1308776 (owner: 10Lars SG) [21:20:28] 06Fundraising-Backlog: Get our ID in the checkout.com console - https://phabricator.wikimedia.org/T431628 (10Cstone) 03NEW [21:20:41] 06Fundraising-Backlog: Get our ID in the checkout.com console - https://phabricator.wikimedia.org/T431628#12102250 (10Cstone) [21:20:43] 03Fundraising Sprint: Loud carpet, 06Fundraising-Backlog, 07Epic: Epic: Enable checkout.com through gravy - https://phabricator.wikimedia.org/T430569#12102251 (10Cstone) [21:24:29] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 06FR-donorrelations, 03Civi RoadMap FY 26-27: Provide API for key CiviCRM data by email address to load in Zendesk - https://phabricator.wikimedia.org/T421245#12102266 (10AKanji-WMF) @jgleeson can we check in on this with... [21:31:23] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap, 10Wikimedia-Fundraising-CiviCRM, 10FR-Donation-Processing, 03Civi RoadMap FY 26-27: Chariot import mapping template - 2 soft credits - https://phabricator.wikimedia.org/T411555#12102274 (10Lars) 05Open→03Invalid a:05Lars→03None Eileen has don... [21:31:39] (03CR) 10Cstone: [C:03+2] chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:32:16] (03Merged) 10jenkins-bot: chariot: more bubbling up of new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308268 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:34:01] (03PS2) 10Lars SG: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308774 [21:34:20] (03CR) 10Cstone: [C:03+2] chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:34:58] (03Merged) 10jenkins-bot: chariout: Add early validation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308554 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:35:58] (03CR) 10Cstone: chariot: also validate early on the DonationAudit processor (032 comments) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:37:16] (03Merged) 10jenkins-bot: Upgrading wikimedia/smash-pig (v1.2.4.18 => v1.2.4.19) [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1308776 (owner: 10Lars SG) [21:37:42] (03CR) 10Lars SG: [C:03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308774 (owner: 10Lars SG) [21:38:39] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1308774 (owner: 10Lars SG) [21:40:39] (03CR) 10Eileen: chariot: also validate early on the DonationAudit processor (031 comment) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:42:29] (03CR) 10Cstone: [C:03+2] "I am curious how many of these will get and if webhooks can make this more straightforward in the future" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1306997 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:43:33] (03Merged) 10jenkins-bot: Add handling for incomplete deposits [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1306997 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:44:15] (03CR) 10Cstone: [C:03+2] Use property rather than passing api parameter around [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307005 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:44:25] (03CR) 10CI reject: [V:04-1] Use property rather than passing api parameter around [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307005 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:44:44] (03PS3) 10Eileen: Use property rather than passing api parameter around [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307005 (https://phabricator.wikimedia.org/T415825) [21:47:20] (03PS6) 10Eileen: chariot: also validate early on the DonationAudit processor [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) [21:48:23] (03CR) 10Eileen: chariot: also validate early on the DonationAudit processor (031 comment) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1308559 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [21:58:39] (03PS3) 10Eileen: Chariot: add maintenance script to list & set properties [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307006 (https://phabricator.wikimedia.org/T415825) [22:13:43] !log donorwiki upgraded from c2cdbf24 to d59684ed [22:13:44] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [22:15:54] !log civicrm upgraded from ead61459 to f4ff79c8 [22:15:55] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [22:21:01] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1307005 (https://phabricator.wikimedia.org/T415825) (owner: 10Eileen) [22:30:57] (03CR) 10Eileen: Add statuses and segments from new audience framework to export (031 comment) [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [22:33:38] (03CR) 10Eileen: Add statuses and segments from new audience framework to export (031 comment) [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [22:36:30] (03CR) 10Eileen: Add statuses and segments from new audience framework to export (031 comment) [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [22:58:09] (03PS3) 10Lars SG: Add statuses and segments from new audience framework to export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) [22:58:09] (03PS3) 10Lars SG: Add last_donation_date fields from new audience framework to export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306437 (https://phabricator.wikimedia.org/T426818) [22:58:09] (03PS2) 10Lars SG: Add first_donation_usd to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306744 (https://phabricator.wikimedia.org/T418885) [22:58:10] (03PS2) 10Lars SG: Add last_recurring_amount_change & date to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1308708 (https://phabricator.wikimedia.org/T430877) [23:16:11] FIRING: NodeDown: Node franio2004 is down. - https://frmon.wikimedia.org/d/000000377/host-overview?orgId=1&var-host=franio2004 - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:21:11] FIRING: [2x] NodeDown: Node franio2004 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:21:14] (03CR) 10Lars SG: "All updated and added donor_status_recur_overall" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [23:21:47] FIRING: HAProxyExporterDown: HAProxy on fransw2003.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [23:22:56] (03CR) 10Eileen: "Looks good overall - one comment suggestion (could be made in more places) & question about the higher values" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [23:26:11] RESOLVED: [2x] NodeDown: Node franio2004 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:26:47] RESOLVED: HAProxyExporterDown: HAProxy on fransw2003.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [23:28:56] (03CR) 10Eileen: Add statuses and segments from new audience framework to export (031 comment) [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [23:31:11] FIRING: [4x] NodeDown: Node fran2001 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:34:28] (03CR) 10Eileen: [C:03+2] Add last_donation_date fields from new audience framework to export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306437 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [23:36:11] FIRING: [4x] NodeDown: Node fran2001 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:37:44] (03CR) 10Eileen: [C:03+2] "It might be ugly but it's very clever" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306744 (https://phabricator.wikimedia.org/T418885) (owner: 10Lars SG) [23:39:16] (03CR) 10Eileen: [C:03+2] Add last_recurring_amount_change & date to Acoustic export [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1308708 (https://phabricator.wikimedia.org/T430877) (owner: 10Lars SG) [23:40:00] (03CR) 10Eileen: "I've given everything above this +2 so if you come back to this & I'm not here feel free to up it to +2 once you have considered my code c" [wikimedia/fundraising/tools] - 10https://gerrit.wikimedia.org/r/1306436 (https://phabricator.wikimedia.org/T426818) (owner: 10Lars SG) [23:41:11] FIRING: [5x] NodeDown: Node fran2001 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:43:21] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/SmashPig into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1308791 [23:43:56] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/SmashPig into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1308791 (owner: 10Eileen) [23:44:26] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/SmashPig into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1308791 (owner: 10Eileen) [23:46:11] FIRING: [6x] NodeDown: Node fran2001 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:51:11] FIRING: [8x] NodeDown: Node fran2001 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:52:16] !log SmashPig upgraded from d8393666 to 2042f7f7 [23:52:17] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [23:54:47] FIRING: HAProxyExporterDown: HAProxy on pay-lb2001.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown [23:55:17] (03PS3) 10Eileen: Chariot - rename individual & matching_gift amount to total_amount [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308262 (https://phabricator.wikimedia.org/T415825) [23:55:17] (03PS3) 10Eileen: Chariot audit: add new fields to test csv files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308263 (https://phabricator.wikimedia.org/T415825) [23:55:18] (03PS2) 10Eileen: Notice fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308344 [23:55:18] (03PS2) 10Eileen: chariot: fix comparison for string zero values [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308579 (https://phabricator.wikimedia.org/T415825) [23:55:18] (03PS2) 10Eileen: chariot: fix benevity fees [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1308590 (https://phabricator.wikimedia.org/T415825) [23:56:11] FIRING: [8x] NodeDown: Node frbast2002 is down. - https://alerts.wikimedia.org/?q=alertname%3DNodeDown [23:59:47] RESOLVED: HAProxyExporterDown: HAProxy on pay-lb2001.frack.codfw.wmnet is down - https://alerts.wikimedia.org/?q=alertname%3DHAProxyExporterDown