[00:01:40] (03Merged) 10jenkins-bot: Improve adyen audit logging [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188474 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [00:18:06] PROBLEM - Host pay-lb1002 is DOWN: PING CRITICAL - Packet loss = 100% [00:21:17] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Cannot continue existing import of type = Import data from Search Kit - https://phabricator.wikimedia.org/T404660#11183322 (10Lars) [[ https://github.com/civicrm/civicrm-core/pull/33599 | Upstream PR ]] [00:22:12] RECOVERY - Host pay-lb1002 is UP: PING OK - Packet loss = 0%, RTA = 2.81 ms [00:34:23] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188480 [00:34:50] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188480 (owner: 10Eileen) [00:35:49] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188480 (owner: 10Eileen) [00:45:06] !log civicrm upgraded from 223550b5 to ebb37b83 [00:45:09] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:30:46] (03PS1) 10Eileen: Testing testing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188488 [01:32:55] (03CR) 10Eileen: [C:03+1] "This seems right but when I ran it locally I didn't get on with your handling of the casing on `opt_in` - check the follow on patch" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184810 (https://phabricator.wikimedia.org/T401353) (owner: 10Ejegg) [01:54:48] (03CR) 10CI reject: [V:04-1] Testing testing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188488 (owner: 10Eileen) [02:13:58] (03PS1) 10Eileen: Remove empty files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188490 (https://phabricator.wikimedia.org/T402876) [02:15:52] cstone: another easy one I think ^^ [02:16:38] ah yeah those empty files are new with gravy [02:16:52] (03CR) 10Cstone: [C:03+2] Remove empty files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188490 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [02:22:00] (03PS7) 10Eileen: Action to backfill opt-in [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184810 (https://phabricator.wikimedia.org/T401353) (owner: 10Ejegg) [02:22:00] (03PS2) 10Eileen: Testing testing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188488 [02:33:30] (03Merged) 10jenkins-bot: Remove empty files [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188490 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [02:48:24] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188494 [02:48:39] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188494 (owner: 10Eileen) [02:49:28] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188494 (owner: 10Eileen) [02:50:05] (03CR) 10Eileen: [C:03+1] Queue up opt-ins to backfill (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1185228 (https://phabricator.wikimedia.org/T401353) (owner: 10Ejegg) [02:53:20] !log civicrm upgraded from 223550b5 to ebb37b83 [02:53:22] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [02:56:23] dang the file survived - looking [03:06:57] !log config revision changed from aca52498 to 76dfebab fundraise-up jobs gone [03:06:58] cstone wfan after the fact +2 if you can - conceptual only is fine [03:06:58] deleting civi1002/fundraise-up_audit_parse.yaml [03:06:58] deleting civi1002/fundraise-up_audit_download.yaml [03:06:58] deleting civi1002/fundraise-up_audit.yaml [03:07:00] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:07:34] hah for deleting eileen ? [03:07:42] yep [03:50:00] (03PS1) 10Eileen: Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) [03:56:48] cstone: another one to try to sort out that adyen audit noise ^^ [03:58:05] (03CR) 10Wfan: [C:03+2] Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [03:58:18] thanks wfan [03:58:40] hah speed! [03:58:49] :-) [04:00:45] Haha~ good night cstone [04:00:50] haha night! [04:02:21] (03CR) 10CI reject: [V:04-1] Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [04:03:11] (03PS2) 10Eileen: Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) [04:03:55] too speedy - the csv wasn't committed [04:04:31] hah [04:04:33] holy cow [04:05:38] (03CR) 10Cstone: [C:03+2] Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [04:05:55] lol - the holy cow may be temporary.... [04:18:37] cstone: if you are still there.. [04:19:04] haha yeah ill be here [04:19:10] im praticing for the salon on wednesday LOL [04:19:48] Well I'm looking at the failed adyen transaction - it is a gravy adyen refund [04:20:23] I know we decided that for chargebacks we just treat them as adyen transactions as gravy doesn't do them yet - what about refunds? [04:20:34] gravy should be doing refunds is info missing? [04:20:49] i know osme have slipped through that were accidentally refunded at adyen [04:21:07] ah sorry re read what you said [04:21:25] it should be ignored if its a gravy adyen refund in the adyen audit [04:21:28] and should be in the gravy audit [04:22:59] cstone: so this is the transaction in the adyen file [04:22:59] https://docs.google.com/document/d/1pl3LQ5odMFBbuyaHZNgguVJ4InOVgwwzCqYkns3qcJg/edit?usp=sharing [04:22:59] But I don't think we have processed this refund anywhere [04:23:24] (03Merged) 10jenkins-bot: Prefer backoffice credentials for gravy lookup [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188498 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [04:24:38] hmmm [04:25:04] cant find it in gravys console either [04:27:05] oh hmm i think its in civi [04:27:26] cid=68211923 [04:31:09] ok - so I could have found it from the payment orchestrator ID - maybe I should try that first [04:31:36] i did cheat and go into adyen to find the email [04:37:02] (03PS1) 10Eileen: Try payment_orchestrator_reconcilliation_id first [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188503 (https://phabricator.wikimedia.org/T402876) [04:37:08] lol [04:37:15] cstone: I think ^^ will do it then [04:38:25] im going through old pictures i apparently took only selfies for the nyc civi sprint [04:39:31] (03CR) 10Cstone: [C:03+2] Try payment_orchestrator_reconcilliation_id first [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188503 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [04:40:05] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188504 [04:42:50] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188504 (owner: 10Eileen) [04:44:11] !log civicrm upgraded from eff6c786 to 3c895373 [04:44:13] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [04:49:47] sigh & now I can't find another one [04:50:18] - in the same doc [04:52:19] cstone: can you find the one now in the doc? [04:52:32] lemme see [04:53:22] hmmm [04:53:43] (03CR) 10Eileen: "I'm Ok with this being upgraded to +2 once you have processed my feedback" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1184810 (https://phabricator.wikimedia.org/T401353) (owner: 10Ejegg) [04:54:22] yeah its in gravy but not in civi at all [04:54:36] hmm ok - I wonder wy now [04:55:30] yeah entirely possible gravy bug ill make a phab for it incase others were stuck in the same way [04:55:47] weve had a bunch of instances where they didnt make it out of gravy [04:58:05] I found the transaction in pending db [04:58:44] they could have never sent us the capture ipn [04:59:34] :-( [05:00:08] i wonder why the gravy audit didnt find it though [05:00:41] In terms of what smashpig is serving up - we are putting up the base62 version of it - I think I should change the adyen audit parser to covert to the long-form since that is what is in gateway_txn_id in the pending db [05:01:04] (03Merged) 10jenkins-bot: Try payment_orchestrator_reconcilliation_id first [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188503 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [05:01:19] that base62 thing has always been annoying [05:01:59] (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188505 [05:02:18] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188505 (owner: 10Eileen) [05:02:50] yeah it is [05:03:04] (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1188505 (owner: 10Eileen) [05:09:35] alright pictures scanned [05:09:54] im gona leave my desk eileen but ill be up for another hour if you need anything else [05:18:55] cool cool [06:07:12] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 96% used. Largest process: trino-server-co (1560) = 95.4% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:12:14] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 96% used. Largest process: trino-server-co (1560) = 95.4% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:17:12] PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 96% used. Largest process: trino-server-co (1560) = 95.4% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:22:12] RECOVERY - check_memory on fransw2002 is OK: OK Memory 89% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory [06:32:36] (03PS1) 10Eileen: Render valid gravy gateway [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1188513 (https://phabricator.wikimedia.org/T402876) [06:40:29] 06Fundraising-Backlog, 06Fundraising-Tech-Roadmap: [Action required] Bulgarian Lev (BGN) currency changeover to Euro (EUR) - https://phabricator.wikimedia.org/T402929#11183583 (10RKumar_WMF) Thank you! [06:52:24] (03PS1) 10Eileen: Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) [06:52:51] (03CR) 10CI reject: [V:04-1] Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [06:56:13] (03PS2) 10Eileen: Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) [06:56:43] (03CR) 10CI reject: [V:04-1] Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [07:02:43] (03PS3) 10Eileen: Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) [07:03:09] (03CR) 10CI reject: [V:04-1] Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876) (owner: 10Eileen) [07:05:03] (03PS4) 10Eileen: Decode gateway_txn_id for gravy [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1188654 (https://phabricator.wikimedia.org/T402876)