[00:00:28] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07Epic: Fix batch date per feedback from Julie - https://phabricator.wikimedia.org/T420290 (10Eileenmcnaughton) 03NEW [00:01:21] (03PS1) 10Eileen: Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 (https://phabricator.wikimedia.org/T420290) [00:08:48] (03PS27) 10Wfan: Assign location type for 3rd party donation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1237617 (https://phabricator.wikimedia.org/T416052) [00:09:34] (03CR) 10Wfan: "Thanks Lars, updated again~" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1237617 (https://phabricator.wikimedia.org/T416052) (owner: 10Wfan) [00:09:48] (03PS2) 10Eileen: Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 [00:12:45] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 13Patch-For-Review: Dlocal - use batch settlement date for individual transactions - https://phabricator.wikimedia.org/T420157#11716547 (10Eileenmcnaughton) [00:21:25] (03CR) 10Wfan: [C:03+2] Dlocal - fix row-level settled_date [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253086 (https://phabricator.wikimedia.org/T420157) (owner: 10Eileen) [00:21:59] (03Merged) 10jenkins-bot: Dlocal - fix row-level settled_date [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253086 (https://phabricator.wikimedia.org/T420157) (owner: 10Eileen) [00:22:31] (03PS2) 10Eileen: Add utm_source index to contribution tracking. Bug: T420289 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253674 (https://phabricator.wikimedia.org/T420289) [00:22:32] (03PS1) 10Eileen: Add new tracking columns to wmf_contribution_extra. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253684 (https://phabricator.wikimedia.org/T419649) [00:27:02] (03PS1) 10Eileen: Add new columns to smashpig.pending [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253685 (https://phabricator.wikimedia.org/T420057) [00:29:46] (03CR) 10CI reject: [V:04-1] Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 (owner: 10Eileen) [00:31:26] (03PS2) 10Eileen: Add new columns to smashpig.pending [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253685 (https://phabricator.wikimedia.org/T420057) [00:31:47] thanks wfan [00:32:03] np~ [00:55:54] (03PS1) 10Eileen: Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) [00:56:25] (03CR) 10CI reject: [V:04-1] Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [00:57:06] (03PS3) 10Eileen: Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 (https://phabricator.wikimedia.org/T420290) [00:59:33] (03PS2) 10Eileen: Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) [01:00:02] (03CR) 10CI reject: [V:04-1] Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [01:03:32] (03PS3) 10Eileen: Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) [01:06:19] (03CR) 10Lars SG: [C:03+2] Add utm_source index to contribution tracking. Bug: T420289 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253674 (https://phabricator.wikimedia.org/T420289) (owner: 10Eileen) [01:08:40] (03CR) 10Lars SG: [C:03+2] Add new tracking columns to wmf_contribution_extra. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253684 (https://phabricator.wikimedia.org/T419649) (owner: 10Eileen) [01:20:10] PROBLEM - check_load on fran2001 is CRITICAL: LOAD CRITICAL - total load average: 22.05, 27.32, 20.18 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran2001&service=check_load [01:21:52] (03PS1) 10Wfan: WIP [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) [01:22:23] (03CR) 10CI reject: [V:04-1] WIP [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) (owner: 10Wfan) [01:24:36] (03CR) 10Lars SG: [C:03+2] Add GL_Code field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253070 (https://phabricator.wikimedia.org/T420156) (owner: 10Eileen) [01:29:19] (03CR) 10Lars SG: [C:03+2] Add new columns to smashpig.pending [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253685 (https://phabricator.wikimedia.org/T420057) (owner: 10Eileen) [01:32:16] (03PS1) 10Eileen: Remove interim account code hack [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253712 (https://phabricator.wikimedia.org/T415626) [01:44:50] (03PS1) 10Eileen: Put back memo suffix for endowment [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253713 (https://phabricator.wikimedia.org/T415626) [01:45:10] RECOVERY - check_load on fran2001 is OK: LOAD OK - total load average: 0.83, 1.12, 4.95 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran2001&service=check_load [01:48:21] (03Merged) 10jenkins-bot: Add GL_Code field [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253070 (https://phabricator.wikimedia.org/T420156) (owner: 10Eileen) [01:48:23] (03Merged) 10jenkins-bot: Add utm_source index to contribution tracking. Bug: T420289 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253674 (https://phabricator.wikimedia.org/T420289) (owner: 10Eileen) [01:48:26] (03Merged) 10jenkins-bot: Add new tracking columns to wmf_contribution_extra. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253684 (https://phabricator.wikimedia.org/T419649) (owner: 10Eileen) [01:48:29] (03Merged) 10jenkins-bot: Add new columns to smashpig.pending [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253685 (https://phabricator.wikimedia.org/T420057) (owner: 10Eileen) [01:51:15] (03PS1) 10Eileen: Add fields to the TransactionLog view Bug: T420057 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253714 (https://phabricator.wikimedia.org/T420057) [01:51:25] (03CR) 10CI reject: [V:04-1] Remove interim account code hack [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253712 (https://phabricator.wikimedia.org/T415626) (owner: 10Eileen) [01:51:30] (03CR) 10CI reject: [V:04-1] Add fields to the TransactionLog view Bug: T420057 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253714 (https://phabricator.wikimedia.org/T420057) (owner: 10Eileen) [01:52:02] (03PS2) 10Eileen: Add fields to the TransactionLog view Bug: T420057 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253714 (https://phabricator.wikimedia.org/T420057) [01:53:58] (03PS2) 10Eileen: Remove interim account code hack [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253712 (https://phabricator.wikimedia.org/T415626) [01:54:37] (03PS2) 10Eileen: Put back memo suffix for endowment [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253713 (https://phabricator.wikimedia.org/T415626) [02:08:09] (03PS2) 10Wfan: WIP [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) [02:08:34] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Delete disabled fields from wmf_contribution_extra - https://phabricator.wikimedia.org/T420301 (10Eileenmcnaughton) 03NEW [02:08:37] (03CR) 10CI reject: [V:04-1] WIP [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) (owner: 10Wfan) [02:10:46] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253716 (https://phabricator.wikimedia.org/T420289) [02:10:56] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253716 (https://phabricator.wikimedia.org/T420289) (owner: 10Eileen) [02:11:41] 06Fundraising-Backlog, 10fundraising-tech-ops: FR-Tech FY2526Q4 maintenance window - https://phabricator.wikimedia.org/T364186#11716803 (10Dwisehaupt) [02:11:49] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253716 (https://phabricator.wikimedia.org/T420289) (owner: 10Eileen) [02:13:47] !log civicrm upgraded from 3283e3ca to e73c6b50 [02:13:48] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [02:16:09] (03CR) 10CI reject: [V:04-1] Add fields to the TransactionLog view Bug: T420057 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253714 (https://phabricator.wikimedia.org/T420057) (owner: 10Eileen) [02:26:10] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1557 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [02:26:14] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1562 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [02:28:00] ACKNOWLEDGEMENT - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1557 Dwisehaupt known db alters https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [02:28:00] ACKNOWLEDGEMENT - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1562 Dwisehaupt known db alters https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [02:30:42] (03PS1) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 [02:31:41] (03CR) 10CI reject: [V:04-1] Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 (owner: 10Eileen) [02:38:25] (03PS2) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 [02:41:54] larssandergreen: not sure if you want to eyeball them ^^ [02:42:06] - that excludes the wmf-contribution-extra but... [02:55:18] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` - https://phabricator.wikimedia.org/T420057#11716865 (10Dwisehaupt) ` MariaDB [civicrm]> ALTER TABLE smashpig.pending -> ADD COLUMN backend_p... [02:55:29] (03CR) 10CI reject: [V:04-1] Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 (owner: 10Eileen) [02:56:46] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add fields to Civi to better reflect reporting - https://phabricator.wikimedia.org/T420156#11716867 (10Dwisehaupt) ` MariaDB [civicrm]> ALTER table civicrm_value_1_gift_data_7 ADD COLUMN gl_code VARCHAR(32) NULL, ADD INDEX index_gl_code(... [02:58:14] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit, 07Epic: New WMFContributionExtra fields backend_processor_reversal_id & payment_orchestrator_reversal_id & auth_id & capture_id - https://phabricator.wikimedia.org/T419649#11716869 (10Dwisehaupt) ` MariaDB [civicrm]> ALTER TABLE wmf_co... [02:58:52] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add index to utm_source on contribution tracking - https://phabricator.wikimedia.org/T420289#11716871 (10Dwisehaupt) ` MariaDB [civicrm]> CREATE INDEX utm_source ON civicrm_contribution_tracking (utm_source); Query OK, 0 rows affected (... [03:03:45] (03PS1) 10Eileen: Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 [03:08:13] (03PS1) 10Eileen: Opps - it's trying to alter the text length [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253739 [03:10:08] (03CR) 10Dwisehaupt: [V:03+2 C:03+2] Opps - it's trying to alter the text length [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253739 (owner: 10Eileen) [03:19:04] this one-line fix is one I'm v keen to get out https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1253676 [03:19:20] (03CR) 10Lars SG: [C:03+2] Put back memo suffix for endowment [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253713 (https://phabricator.wikimedia.org/T415626) (owner: 10Eileen) [03:20:02] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253740 [03:20:04] (03CR) 10Lars SG: [C:03+2] Remove interim account code hack [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253712 (https://phabricator.wikimedia.org/T415626) (owner: 10Eileen) [03:20:36] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253740 (owner: 10Eileen) [03:21:11] (03CR) 10Dwisehaupt: [C:03+2] Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 (owner: 10Eileen) [03:21:17] (03CR) 10Dwisehaupt: [V:03+2 C:03+2] Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 (owner: 10Eileen) [03:21:33] (03CR) 10CI reject: [V:04-1] Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 (owner: 10Eileen) [03:23:52] (03PS3) 10Eileen: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 [03:24:00] (03PS2) 10Eileen: Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 [03:25:00] FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [03:28:54] (03PS1) 10Eileen: Add early return for already run upgrade [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253742 [03:30:11] (03CR) 10Eileen: [V:03+2 C:03+2] Add early return for already run upgrade [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253742 (owner: 10Eileen) [03:30:59] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253743 [03:31:07] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253743 (owner: 10Eileen) [03:31:51] (03CR) 10Eileen: [V:03+2 C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253743 (owner: 10Eileen) [03:43:16] (03CR) 10Dwisehaupt: [V:03+2 C:03+2] Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253725 (owner: 10Eileen) [03:45:48] (03CR) 10Eileen: [V:03+2] Update triggers - contribution_extra [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253738 (owner: 10Eileen) [03:47:00] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253744 [03:47:20] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253744 (owner: 10Eileen) [03:48:39] !log civicrm upgraded from 5360f9ad to e622a222 [03:48:40] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [03:50:40] (03Merged) 10jenkins-bot: Remove interim account code hack [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253712 (https://phabricator.wikimedia.org/T415626) (owner: 10Eileen) [03:50:42] (03Merged) 10jenkins-bot: Put back memo suffix for endowment [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253713 (https://phabricator.wikimedia.org/T415626) (owner: 10Eileen) [04:39:20] (03CR) 10Lars SG: [C:03+2] Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [04:41:08] RECOVERY - check_mysql on frdb2004 is OK: Uptime: 284588 Threads: 4 Questions: 22798763 Slow queries: 171 Opens: 117317 Open tables: 896 Queries per second avg: 80.111 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [04:41:14] RECOVERY - check_mysql on frdb2005 is OK: Uptime: 283471 Threads: 4 Questions: 22719613 Slow queries: 170 Opens: 116676 Open tables: 869 Queries per second avg: 80.147 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [04:53:36] (03CR) 10Lars SG: [C:03+2] "Looking good, let's put this through. I do have a few small things, but I'll do them in a follow up since we're up to PS 27 here. Thanks f" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1237617 (https://phabricator.wikimedia.org/T416052) (owner: 10Wfan) [05:00:53] (03PS1) 10Eileen: Fix failure to skip if .gz exists [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253759 [05:02:22] (03Merged) 10jenkins-bot: Fix batch date payout row [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253676 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [05:03:40] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253761 [05:03:54] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253761 (owner: 10Eileen) [05:04:59] 06Fundraising-Backlog: Contribution tracking clean up from small failstorm - https://phabricator.wikimedia.org/T420307 (10Cstone) 03NEW [05:05:16] (03CR) 10Cstone: [C:03+2] Fix failure to skip if .gz exists [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253759 (owner: 10Eileen) [05:05:52] (03Merged) 10jenkins-bot: Fix failure to skip if .gz exists [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253759 (owner: 10Eileen) [05:06:22] !log civicrm upgraded from e622a222 to 7fe14629 [05:06:23] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [05:11:40] (03Merged) 10jenkins-bot: Assign location type for 3rd party donation [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1237617 (https://phabricator.wikimedia.org/T416052) (owner: 10Wfan) [05:13:30] 06Fundraising-Backlog: Contribution tracking clean up from small failstorm - https://phabricator.wikimedia.org/T420307#11716994 (10Dwisehaupt) [05:34:12] (03PS1) 10Eileen: Fix dlocal passing date not timestamp in aggregate row [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253914 [05:37:53] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1253828 (owner: 10L10n-bot) [05:39:05] (03CR) 10Eileen: [C:03+2] Fix dlocal passing date not timestamp in aggregate row [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253914 (owner: 10Eileen) [05:49:30] (03PS1) 10Eileen: Upgrading wikimedia/smash-pig (v1.2.2.17 => v1.2.2.18) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253962 [05:51:54] (03PS1) 10Eileen: Upgrading wikimedia/smash-pig (v1.2.2.17 => v1.2.2.18) [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1253964 [05:54:52] (03CR) 10Eileen: [C:03+2] Upgrading wikimedia/smash-pig (v1.2.2.17 => v1.2.2.18) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1253962 (owner: 10Eileen) [05:55:07] (03CR) 10Eileen: [C:03+2] Upgrading wikimedia/smash-pig (v1.2.2.17 => v1.2.2.18) [wikimedia/fundraising/crm/vendor] - 10https://gerrit.wikimedia.org/r/1253964 (owner: 10Eileen) [05:58:45] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_43) - 10https://gerrit.wikimedia.org/r/1253828 (owner: 10L10n-bot) [06:16:33] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253966 [06:16:45] (03PS2) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253966 [06:17:12] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1253966 (owner: 10Eileen) [06:20:23] !log civicrm upgraded from 7fe14629 to adc36173 [06:20:24] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [06:34:16] (03PS1) 10Eileen: Braintree also passing date rather than timestamp [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253973 [07:25:00] FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [08:03:37] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1254058 (owner: 10L10n-bot) [08:23:05] (03PS3) 10Wfan: Move redirect specific mapper from general mapper to redirect request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) [08:23:39] (03CR) 10CI reject: [V:04-1] Move redirect specific mapper from general mapper to redirect request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) (owner: 10Wfan) [08:30:11] (03PS1) 10Wfan: Enable pix recurring [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1254102 (https://phabricator.wikimedia.org/T389865) [08:57:30] (03CR) 10Raimond Spekking: [C:03+2] "false positive" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1254058 (owner: 10L10n-bot) [09:24:42] (03PS4) 10Wfan: Move redirect specific mapper from general mapper to redirect request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) [09:27:18] (03PS7) 10Wfan: Re-enable active paypal-ec recurring donor to update their EmailPreference Primary Email [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1237626 (https://phabricator.wikimedia.org/T416056) [09:27:18] (03PS2) 10Wfan: Enable pix recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248051 (https://phabricator.wikimedia.org/T389865) [09:57:57] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 07payments-orchestration: Enable iDeal via Gr4vy in Netherlands - https://phabricator.wikimedia.org/T392068#11717563 (10AnnWF) a:03AnnWF [11:07:24] 06Fundraising-Backlog: Japan Donations created by Seilo on 17-Mar-2026 do not show up in Civi - https://phabricator.wikimedia.org/T420331 (10RKumar_WMF) 03NEW [11:25:00] FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [13:36:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Japan Donations created by Seilo on 17-Mar-2026 do not show up in Civi - https://phabricator.wikimedia.org/T420331#11718541 (10AKanji-WMF) [13:40:53] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Japan Donations created by Seilo on 17-Mar-2026 do not show up in Civi - https://phabricator.wikimedia.org/T420331#11718569 (10jgleeson) I think this might be related to the Civi scheduled downtime [14:15:58] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 07fr-current-sprint, 05FY25-26 WE3.5 Donor Identification and recognition: Evaluate donor badge experiment - https://phabricator.wikimedia.org/T415918#11718792 (10AKanji-WMF) 05Open→03Resolved Responded! To confirm, the donor bad... [14:26:39] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 10FR-Donation-Processing: Large file for March DM Activity import - https://phabricator.wikimedia.org/T419301#11718832 (10AKanji-WMF) [14:26:56] 03Fundraising Sprint - Entire room is a couch, 14Fundraising Sprint: Vixen, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 3 others: Implement recurring donation monthly-to-annual conversion - https://phabricator.wikimedia.org/T388753#11718833 (10AKanji-WMF) [14:53:40] RESOLVED: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [15:23:44] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-email-preferences: Scoping: Email signup form into Civi - https://phabricator.wikimedia.org/T417326#11719205 (10AKanji-WMF) [15:42:51] FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [15:46:51] FIRING: [2x] SwitchCoreInterfaceDown: Switch core interface down - fasw1-f5a-codfw:et-0/0/47 (Core: pfw1-codfw:et-0/1/0 {#122505}) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Switch_interface_down - https://alerts.wikimedia.org/?q=alertname%3DSwitchCoreInterfaceDown [15:52:51] RESOLVED: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown [15:56:51] RESOLVED: [2x] SwitchCoreInterfaceDown: Switch core interface down - fasw1-f5a-codfw:et-0/0/47 (Core: pfw1-codfw:et-0/1/0 {#122505}) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Switch_interface_down - https://alerts.wikimedia.org/?q=alertname%3DSwitchCoreInterfaceDown [16:03:43] (03PS9) 10Wfan: Match secondary email with location type for dedupe [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1248961 (https://phabricator.wikimedia.org/T418660) [16:23:44] (03CR) 10Lars SG: "Few new comments, see also older comments" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1248961 (https://phabricator.wikimedia.org/T418660) (owner: 10Wfan) [16:43:06] (03CR) 10Lars SG: "We already have 430k billing type emails, so maybe we should make a different email type here instead. Would be clearer as well if it expl" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1251209 (https://phabricator.wikimedia.org/T418790) (owner: 10Wfan) [16:46:38] (03PS1) 10Wfan: add gravy ideal test link [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1254250 [17:00:12] (03PS1) 10Wfan: Map ideal to gravy payment method for capture [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1254252 (https://phabricator.wikimedia.org/T392068) [17:01:15] (03PS1) 10Wfan: Enable ideal with recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1254253 (https://phabricator.wikimedia.org/T392068) [17:16:47] 10fundraising-tech-ops: Upgrade frtech hosts to kernel 6.1.164-1 - https://phabricator.wikimedia.org/T420018#11720164 (10Dwisehaupt) [17:17:32] (03PS5) 10Wfan: Move redirect specific mapper from general mapper to redirect request mapper [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253711 (https://phabricator.wikimedia.org/T389865) [17:18:01] Thanks Lars, checking~ [17:18:51] (03PS3) 10Wfan: Enable pix recurring [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248051 (https://phabricator.wikimedia.org/T389865) [17:19:22] (03PS1) 10Wfan: update smashpig to 1.2.2.18 [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1254261 [17:24:49] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 10FR-Donation-Processing: Large file for March DM Activity import - https://phabricator.wikimedia.org/T419301#11720198 (10Lars) a:03Eileenmcnaughton [17:36:43] (03CR) 10Wfan: [V:03+2 C:03+2] add gravy ideal test link [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1254250 (owner: 10Wfan) [17:37:17] (03CR) 10Jgleeson: "Thanks for all the work on this, Dami! I spotted a few small things which I'll push up fixes for as you're OOO, but on the whole, it looks" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248518 (owner: 10Damilare Adedoyin) [17:38:14] (03PS11) 10Jgleeson: Add manage donations view [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248518 (owner: 10Damilare Adedoyin) [17:39:46] anilk: once that passes on CI I'll merge it and push it out ^ [17:43:41] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Epic, 07fr-current-sprint: Set double opt in activity to cancelled when double opted in contact opts out - https://phabricator.wikimedia.org/T417695#11720323 (10Lars) After looking at this, I think we... [17:45:16] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 07fr-current-sprint, 05FY25-26 WE3.5 Donor Identification and recognition: Answer some data size/churn questions for the Donor ID export - https://phabricator.wikimedia.org/T418164#11720345 (10Lars) a:03Lars [17:46:07] (03CR) 10Jgleeson: [C:03+2] Add manage donations view [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248518 (owner: 10Damilare Adedoyin) [17:46:13] 06Fundraising-Backlog: Native app payments don't fill in backend processor field - https://phabricator.wikimedia.org/T420401 (10Cstone) 03NEW [17:47:24] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07fr-current-sprint: Send double opt-in confirmation message whenever a contact's email is changed - https://phabricator.wikimedia.org/T407261#11720363 (10Lars) Also waiting for better pre-hook [18:03:00] (03Merged) 10jenkins-bot: Add manage donations view [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1248518 (owner: 10Damilare Adedoyin) [18:03:33] thanks again jgleeson appreciate [18:14:30] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1254277 [18:14:40] (03CR) 10Jgleeson: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1254277 (owner: 10Jgleeson) [18:17:43] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1254277 (owner: 10Jgleeson) [18:40:43] !log donorwiki upgraded from 4c09db39 to 7d1666f9 [18:40:44] Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL [18:41:42] anilk: that is live [18:41:50] I just tested with my own donor portal account [18:42:13] and I see "manage my donation" on my 2 active recurrings [18:48:54] I just pinged Kevin on the DP channel https://wikimedia.slack.com/archives/C084EG935MG/p1773773242460859 [19:02:15] I really appreciate this jgleeson - and our entire team's culture of taking ownership of things ❤️ [19:09:01] (03PS1) 10Jgleeson: Merge branch 'master' into deployment [wikimedia/fundraising/SmashPig] (deployment) - 10https://gerrit.wikimedia.org/r/1254294 [19:26:21] (03CR) 10Lars SG: [C:03+2] Braintree also passing date rather than timestamp [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253973 (owner: 10Eileen) [19:27:45] (03Merged) 10jenkins-bot: Braintree also passing date rather than timestamp [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253973 (owner: 10Eileen) [19:45:06] (03PS1) 10Cstone: Add in new fields [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1254306 (https://phabricator.wikimedia.org/T420057) [19:46:58] np anilk [20:18:50] (03PS1) 10Umherirrender: Pass GatewayAdapter to Form::setGateway to match class property [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1254312 [20:22:42] (03PS5) 10Umherirrender: build: Updating dependencies [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1246240 (owner: 10Libraryupgrader) [20:22:55] (03PS6) 10Umherirrender: build: Updating dependencies [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1246240 (owner: 10Libraryupgrader) [20:54:29] (03PS12) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628) [20:57:32] (03PS1) 10Eileen: is it running [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1254327 [21:00:35] (03CR) 10CI reject: [V:04-1] is it running [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1254327 (owner: 10Eileen) [22:37:50] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 07fr-current-sprint, 05FY25-26 WE3.5 Donor Identification and recognition: Answer some data size/churn questions for the Donor ID export - https://phabricator.wikimedia.org/T418164#11721201 (10Jdlrobson-WMF) @lars great, yes these are gr... [23:06:17] 03Fundraising Sprint - Entire room is a couch, 06Fundraising-Backlog, 07fr-current-sprint, 05FY25-26 WE3.5 Donor Identification and recognition: Answer some data size/churn questions for the Donor ID export - https://phabricator.wikimedia.org/T418164#11721248 (10Lars) @Jdlrobson-WMF In theory, yes, we coul... [23:16:25] 10fundraising-tech-ops: Upgrade frtech hosts to kernel 6.1.164-1 - https://phabricator.wikimedia.org/T420018#11721267 (10Dwisehaupt) [23:44:32] (03PS1) 10Wfan: seperate ach location type to define one from bank one from form [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1254366 (https://phabricator.wikimedia.org/T418790) [23:50:11] (03CR) 10Cstone: "Lots of comments, I know we want to iterate but do we want to move a bunch of that logic out of the Maintenance script and into the Parser" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628) (owner: 10Eileen) [23:51:50] (03CR) 10Cstone: [C:03+2] Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [23:52:24] (03Merged) 10jenkins-bot: Fix batch settlement date & ref to run off batch not first transaction [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1253696 (https://phabricator.wikimedia.org/T420290) (owner: 10Eileen) [23:59:50] (03CR) 10Lars SG: [C:03+2] seperate ach location type to define one from bank one from form [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1254366 (https://phabricator.wikimedia.org/T418790) (owner: 10Wfan)