[05:01:51] (03CR) 10Cstone: [C:03+2] Add ability to output raw files to Braintree maintenance, add statusHistory [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220404 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [05:01:56] (03CR) 10Cstone: [C:03+2] Add ability to pass in alternate date-types [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220407 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [05:02:23] (03Merged) 10jenkins-bot: Add ability to output raw files to Braintree maintenance, add statusHistory [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220404 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [05:02:26] (03Merged) 10jenkins-bot: Add ability to pass in alternate date-types [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220407 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [06:13:57] (03CR) 10Cstone: [C:03+2] Set journals to reflect batches as agreed on call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220049 (owner: 10Eileen) [06:16:38] (03CR) 10CI reject: [V:04-1] Set journals to reflect batches as agreed on call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220049 (owner: 10Eileen) [07:58:03] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1220580 (owner: 10L10n-bot) [08:02:54] (03CR) 10Raimond Spekking: [C:03+2] "false positive" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1220580 (owner: 10L10n-bot) [15:11:14] PROBLEM - check_memory on fransw2003 is CRITICAL: CRIT Memory 96% used. Largest process: trino-server-co (3848879) = 94.4% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_memory [15:12:12] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 96% used. Largest process: trino-server-co (3806159) = 94.9% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [15:16:14] RECOVERY - check_memory on fransw2003 is OK: OK Memory 73% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2003&service=check_memory [15:17:14] RECOVERY - check_memory on fransw2002 is OK: OK Memory 68% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [15:39:18] (03PS1) 10Damilare Adedoyin: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1220638 [15:42:27] (03CR) 10Damilare Adedoyin: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1220638 (owner: 10Damilare Adedoyin) [15:43:54] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1220638 (owner: 10Damilare Adedoyin) [15:46:33] !log payments-wiki upgraded from 5c9a955f to 81340350 [15:46:35] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:56:55] (03CR) 10Jgleeson: [C:03+2] Add 'Cancel Support' to the cancel reasons as it is specified on DonorPortal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1219915 (owner: 10Damilare Adedoyin) [15:57:05] (03PS2) 10Damilare Adedoyin: Add 'Cancel Support' to the cancel reasons as it is specified on DonorPortal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1219915 [15:57:08] (03CR) 10CI reject: [V:04-1] Add 'Cancel Support' to the cancel reasons as it is specified on DonorPortal [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1219915 (owner: 10Damilare Adedoyin) [17:42:08] PROBLEM - check_memory on fransw2001 is CRITICAL: CRIT Memory 95% used. Largest process: trino-server-co (3808047) = 93.8% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_memory [17:47:10] PROBLEM - check_memory on fransw2001 is CRITICAL: CRIT Memory 97% used. Largest process: trino-server-co (3808047) = 96.0% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_memory [17:52:10] PROBLEM - check_memory on fransw2001 is CRITICAL: CRIT Memory 97% used. Largest process: trino-server-co (3808047) = 96.0% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_memory [17:57:08] RECOVERY - check_memory on fransw2001 is OK: OK Memory 55% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2001&service=check_memory [19:51:31] (03PS1) 10Cstone: Add in bad contact id test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220648 [20:00:36] 06Fundraising-Backlog: contact_id in email link having %%CONTACTIY%% instead of contact_id=1234 is breaking the queues - https://phabricator.wikimedia.org/T413452 (10Cstone) 03NEW [20:02:28] 06Fundraising-Backlog: contact_id in email link having contact_id=%%CONTACTID%% instead of contact_id=1234 is breaking the queues - https://phabricator.wikimedia.org/T413452#11483756 (10Cstone) [20:04:46] (03CR) 10Cstone: [C:03+2] "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220049 (owner: 10Eileen) [20:06:06] (03PS1) 10Eileen: Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) [20:06:31] (03CR) 10CI reject: [V:04-1] Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [20:06:54] (03PS2) 10Eileen: Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) [20:07:33] (03CR) 10CI reject: [V:04-1] Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [20:08:29] (03CR) 10CI reject: [V:04-1] Add in bad contact id test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220648 (owner: 10Cstone) [20:10:19] (03PS1) 10Eileen: Fix output path for raw braintree [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220650 (https://phabricator.wikimedia.org/T412174) [20:10:51] (03CR) 10CI reject: [V:04-1] Fix output path for raw braintree [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220650 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [20:13:06] (03PS3) 10Eileen: Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) [20:13:06] (03PS2) 10Eileen: Fix output path for raw braintree [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220650 (https://phabricator.wikimedia.org/T412174) [20:13:32] (03CR) 10CI reject: [V:04-1] Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [20:13:36] (03CR) 10CI reject: [V:04-1] Fix output path for raw braintree [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220650 (https://phabricator.wikimedia.org/T412174) (owner: 10Eileen) [20:15:37] (03PS4) 10Eileen: Allow start & end date instead of hours as options [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220649 (https://phabricator.wikimedia.org/T412174) [20:15:37] (03PS3) 10Eileen: Fix output path for raw braintree [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1220650 (https://phabricator.wikimedia.org/T412174) [20:21:27] (03Merged) 10jenkins-bot: Set journals to reflect batches as agreed on call [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220049 (owner: 10Eileen) [20:33:57] 06Fundraising-Backlog, 10Wikimedia-Portals, 13Patch-For-Review: Fundraising banner copy appearing in google search snippets for portals - https://phabricator.wikimedia.org/T413454#11483813 (10DLynch) [21:02:48] (03PS3) 10Eileen: Add validation to api push job [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1220312 [22:45:58] 06Fundraising-Backlog, 10fundraising-tech-ops, 06Fundraising-Tech-Roadmap, 06FR-donorrelations, and 2 others: Routing issue from payments hosts to 62.146.255.205 - https://phabricator.wikimedia.org/T413382#11483888 (10Dwisehaupt) More details from adyen, this particular IP is behind cloudflare: ` This IP a... [23:12:12] PROBLEM - check_load on fransw2002 is CRITICAL: LOAD CRITICAL - total load average: 64.95, 70.91, 54.04 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [23:14:48] ^^ aware of that. just some trino queries cooking along. [23:22:12] RECOVERY - check_load on fransw2002 is OK: LOAD OK - total load average: 19.90, 42.12, 49.13 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_load [23:25:41] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1220656 [23:28:52] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1220656 (owner: 10Eileen) [23:29:21] I'm pushing out the journals fix - doesn't touch code that we rely on at this time of year... [23:29:37] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1220656 (owner: 10Eileen) [23:30:53] !log civicrm upgraded from 9cba6b6d to 03ff6ee3 [23:30:55] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log