[00:14:22] (03Merged) 10jenkins-bot: Log where we got the weird value from [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054683 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [00:16:40] (03CR) 10CI reject: [V:04-1] Test removing fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [01:47:45] cstone: ok so an opps - - we need Dallas to make the change? [02:08:37] (03PS2) 10Eileen: Finish removing log_search_past_days variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054676 (https://phabricator.wikimedia.org/T270675) [02:14:25] (03PS3) 10Eileen: Finish removing log_search_past_days variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054676 (https://phabricator.wikimedia.org/T270675) [02:15:25] (03PS4) 10Eileen: Finish removing log_search_past_days variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054676 (https://phabricator.wikimedia.org/T270675) [02:43:54] (03PS2) 10Eileen: Remove fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) [02:49:10] (03PS3) 10Eileen: Remove last caller on wmf_common_locale_civi_to_mediawiki [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1051480 (https://phabricator.wikimedia.org/T365415) [02:50:47] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1054690 [02:53:09] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1054690 (owner: 10Eileen) [02:56:46] !log civicrm upgraded from 4f919c1e to 1ac3e7be [02:56:48] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:22:49] (03PS3) 10Eileen: Remove fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) [04:11:14] (03PS2) 10Eileen: Use setting for wmf_audit_directory_payments_log [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054678 (https://phabricator.wikimedia.org/T270675) [04:27:57] (03PS1) 10Eileen: Remove audit settings form [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054693 (https://phabricator.wikimedia.org/T270675) [04:27:59] (03PS1) 10Eileen: move function to audit processor class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054694 (https://phabricator.wikimedia.org/T270675) [04:33:28] (03PS1) 10Eileen: Call wmf_echo via the class function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054695 (https://phabricator.wikimedia.org/T270675) [04:38:40] (03PS1) 10Eileen: Move tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) [04:50:24] (03CR) 10CI reject: [V:04-1] Call wmf_echo via the class function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054695 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [04:51:28] (03CR) 10CI reject: [V:04-1] Remove audit settings form [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054693 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [04:54:19] (03CR) 10CI reject: [V:04-1] Move tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [06:18:21] (03CR) 10CI reject: [V:04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_41) - 10https://gerrit.wikimedia.org/r/1054723 (owner: 10L10n-bot) [09:10:45] (03PS2) 10Eileen: Move tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) [09:34:47] (03CR) 10CI reject: [V:04-1] Move tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [11:57:54] 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, 07payments-orchestration: Setup monthly convert flow in Gravy - https://phabricator.wikimedia.org/T367579#9989797 (10Damilare) a:03Damilare [12:01:24] (03PS1) 10Damilare Adedoyin: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1054873 [12:08:32] (03CR) 10Damilare Adedoyin: [C:03+2] Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1054873 (owner: 10Damilare Adedoyin) [12:11:37] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [extensions/DonationInterface] (deployment) - 10https://gerrit.wikimedia.org/r/1054873 (owner: 10Damilare Adedoyin) [12:37:27] (03PS7) 10Damilare Adedoyin: Add recurring payment token to data sent to queue [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1047493 (https://phabricator.wikimedia.org/T367578) [13:43:18] 03Fundraising Sprint: meow(), 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, 07Epic, 07payments-orchestration: Gravy - Import donations to CIviCRM - https://phabricator.wikimedia.org/T368039#9990083 (10jgleeson) One other approach I ruled out was having processor-specific Gravy gateway adapters... [13:44:54] damilare: FYI https://phabricator.wikimedia.org/T368039#9986336 [13:45:12] looking [13:53:29] my immediate assumption was that donor services and folks over in civiland and analytics will need to know who the backend processor is. i still think that might be true but I wanna confirm that before deciding how to solve that problem mentioned [13:54:47] jgleeson: I think one of the reasons we considered highlighting the processor in the transaction was because of the chargeback msg that gr4vy said they don't support [13:55:53] if I recall correctly payments were not interested in the processor specifics in the trxns, I think they were fine in just knowing the trxn is in the gravy bucket [13:56:14] we can confirm in the call today [13:56:15] does gravy not show the chargeback in the gravy console? [13:56:43] (allowing manual routing to civi) [13:57:23] I'm gonna mention the processor visibility issue today and see what Evelyn says. Might be one for after the call [14:06:36] 06Fundraising-Backlog, 10fundraising-tech-ops, 10SRE Observability (FY2024/2025-Q1): Shift frack alerting to use alertmanager instead of icinga - https://phabricator.wikimedia.org/T367370#9990415 (10lmata) [14:55:03] jgleeson: out of the loop but trying to catch up do we still get the adyen id for what you were talking about ^ or is it no id or backend processor name? [14:55:25] yep we do cstone [14:55:32] processor-specific ids [14:55:39] are available [15:02:21] ok cool [15:03:56] im awake during this meeting now so coming to lurk hah [15:04:05] (03CR) 10Abijeet Patro: [V:03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_41) - 10https://gerrit.wikimedia.org/r/1054723 (owner: 10L10n-bot) [15:30:45] 03Fundraising Sprint: meow(), 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, 07Epic, 07payments-orchestration: Gravy - Import donations to CIviCRM - https://phabricator.wikimedia.org/T368039#9990724 (10jgleeson) Gravy recommended using the `payment_service_definition_id` e.g. payment_service_de... [16:13:10] 06Fundraising-Backlog: Rename fundraise-up audit directory to fundraiseup - https://phabricator.wikimedia.org/T370307 (10Cstone) 03NEW [16:30:51] 06Fundraising-Backlog, 10fundraising-tech-ops: Rename fundraise-up audit directory to fundraiseup - https://phabricator.wikimedia.org/T370307#9991033 (10Dwisehaupt) [16:53:34] 06Fundraising-Backlog, 10LPL Technical Support, 10MediaWiki-extensions-DonationInterface, 07affects-translatewiki.net, and 3 others: Remove " 06Fundraising-Backlog, 10LPL Technical Support, 10MediaWiki-extensions-DonationInterface, 07affects-translatewiki.net, and 3 others: Remove " 06Fundraising-Backlog, 10fundraising-tech-ops, 07Epic, 07payments-orchestration: Network configuration for Gravy integration - https://phabricator.wikimedia.org/T370319 (10Damilare) 03NEW [17:15:44] 06Fundraising-Backlog, 10fundraising-tech-ops: Copy Gravy certificate to dedicated Cert directory - https://phabricator.wikimedia.org/T370159#9991382 (10Damilare) @Dwisehaupt I've created this phab T370319 for the network configuration and I've reached out to the Gr4vy team for their hostnames and IP ranges. A... [17:16:10] 06Fundraising-Backlog, 10fundraising-tech-ops: Rename fundraise-up audit directory to fundraiseup - https://phabricator.wikimedia.org/T370307#9991390 (10Dwisehaupt) a:03Dwisehaupt New directory set up. All puppet instances of fundraise-up changed to fundraiseup. Files on disk on civi1002 copied to new location. [17:59:34] 06Fundraising-Backlog, 10FR-donorservices: Donation history in a single email feature request - https://phabricator.wikimedia.org/T368668#9991826 (10AKanji-WMF) Ideally this functionality would be in place for Jan TY sends. [18:01:29] 06Fundraising-Backlog, 10FR-donorservices: Snooze expired date request - https://phabricator.wikimedia.org/T368664#9991834 (10AKanji-WMF) Desired for Q2 - greatly streamlines DR workflow - CC @Eileenmcnaughton [18:01:59] 06Fundraising-Backlog, 10FR-donorservices: PDF Civi receipts - feature request - https://phabricator.wikimedia.org/T368671#9991837 (10AKanji-WMF) Ideally in place for Jan TY emails [18:04:56] 06Fundraising-Backlog, 10FR-donorservices: Civi Tag sync with Acoustic - feature request - https://phabricator.wikimedia.org/T366212#9991846 (10AKanji-WMF) @AKanji-WMF to review @SHust's [[ https://docs.google.com/document/d/1-yVpM3-XeUBmwP8OVOCr5hK8yQiRF-T442nVuVClCn4/edit?hl=en | notes ]] with comms preferen... [18:06:22] (03CR) 10Ejegg: [C:03+2] Remove last caller on wmf_common_locale_civi_to_mediawiki [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1051480 (https://phabricator.wikimedia.org/T365415) (owner: 10Eileen) [18:06:35] 06Fundraising-Backlog, 10FR-donorservices: Donation history in a single email feature request - https://phabricator.wikimedia.org/T368668#9991862 (10AKanji-WMF) [18:06:36] 06Fundraising-Backlog, 10FR-donorservices: Civi "thank you letter summary" feature request - https://phabricator.wikimedia.org/T299096#9991861 (10AKanji-WMF) [18:09:40] 06Fundraising-Backlog, 10Wikimedia-Fundraising-Banners, 10donate.wikimedia.org, 10FR-donorservices, 10FR-Japan: Investigate Japanese unintended recurring donations - https://phabricator.wikimedia.org/T262361#9991879 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF Tx @Pcoombe - according to @SHu... [18:10:18] 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10Fr-CiviCRM-dedupe-FY2017/18: Civi: search for duplicates when updating email address not working - https://phabricator.wikimedia.org/T201652#9991883 (10AKanji-WMF) [18:10:19] 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10Fr-CiviCRM-dedupe-FY2017/18: Civi: make results page number sticky for Find & Merge Duplicates - https://phabricator.wikimedia.org/T192161#9991885 (10AKanji-WMF) [18:10:20] 14Fundraising Sprint Junebugs prefer July, 14Fundraising Sprint Karma chameleons hide amongst us, 14Fundraising Sprint Lactose is unusually tolerant, 06Fundraising-Backlog, and 2 others: Historical MG data lost in same-moment merges - https://phabricator.wikimedia.org/T198131#9991884 (10AKanji-WMF) [18:10:21] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe, 10Fr-CiviCRM-dedupe-FY2017/18: Add functionality to be able to do mesh dedupes on selected search results - https://phabricator.wikimedia.org/T176702#9991887 (10AKanji-WMF) [18:12:09] 14Fundraising Sprint: didAnyoneTryThis(), 03Fundraising Sprint: meow(), 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, and 2 others: Sandra can't dedupe Name + address matching contacts (no email match) - https://phabricator.wikimedia.org/T353971#9991897 (10AKanji-WMF) p:05Triage→03High [18:13:02] 14Fundraising Sprint: didAnyoneTryThis(), 03Fundraising Sprint: meow(), 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, and 2 others: Sandra can't dedupe Name + address matching contacts (no email match) - https://phabricator.wikimedia.org/T353971#9991909 (10AKanji-WMF) Flagging this as the priori... [18:15:40] 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10Fr-CiviCRM-dedupe-FY2017/18: Civi: search for duplicates when updating email address not working - https://phabricator.wikimedia.org/T201652#9991936 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust recommends closing. [18:16:44] 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10Fr-CiviCRM-dedupe-FY2017/18: Civi: make results page number sticky for Find & Merge Duplicates - https://phabricator.wikimedia.org/T192161#9991952 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust recommends closing. [18:17:26] 14Fundraising Sprint Junebugs prefer July, 14Fundraising Sprint Karma chameleons hide amongst us, 14Fundraising Sprint Lactose is unusually tolerant, 06Fundraising-Backlog, and 2 others: Historical MG data lost in same-moment merges - https://phabricator.wikimedia.org/T198131#9991959 (10AKanji-WMF) 05... [18:20:29] 06Fundraising-Backlog, 10FR-Civi-Dedupe: Dedupe -note issues - https://phabricator.wikimedia.org/T363960#9992019 (10AKanji-WMF) [18:20:31] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe - latin American names rule - https://phabricator.wikimedia.org/T278899#9992020 (10AKanji-WMF) [18:21:00] 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10FR-donorservices: Deduper Issue - https://phabricator.wikimedia.org/T363958#9992047 (10AKanji-WMF) p:05Triage→03High [18:21:06] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 13Patch-For-Review: CentralNotice should catch banner errors and log them to its own channel - https://phabricator.wikimedia.org/T361680#9992048 (10Jdlrobson) [18:21:30] 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, 10FR-Civi-Dedupe: Create button to merge "Unknown: Shares contact information" - https://phabricator.wikimedia.org/T367234#9992052 (10AKanji-WMF) p:05Triage→03High [18:22:05] 14Fundraising Sprint: didAnyoneTryThis(), 03Fundraising Sprint: meow(), 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, and 2 others: Sandra can't dedupe Name + address matching contacts (no email match) - https://phabricator.wikimedia.org/T353971#9992059 (10AKanji-WMF) p:05High→03Unbreak! [18:22:26] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Revisit deduper interface with a view to making it possible to do a search in search kit to dedupe - https://phabricator.wikimedia.org/T327728#9992069 (10AKanji-WMF) p:05Triage→03Medium [18:23:05] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Return of duplicate active employer relationships crashing silverpop export - https://phabricator.wikimedia.org/T306822#9992084 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust flags as done [18:25:03] 06Fundraising-Backlog, 10FR-Civi-Dedupe: Alternate name field for donors with different alphabet characters - https://phabricator.wikimedia.org/T275678#9992100 (10AKanji-WMF) [18:25:42] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: How to deal with clearly bad email addresses in Civi - https://phabricator.wikimedia.org/T363946#9992112 (10AKanji-WMF) p:05Triage→03Medium [18:26:44] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe - latin American names rule - https://phabricator.wikimedia.org/T278899#9992116 (10AKanji-WMF) p:05Triage→03Low [18:27:56] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe exception search field keeps adding the text "test" to searches - https://phabricator.wikimedia.org/T225670#9992120 (10AKanji-WMF) p:05Triage→03Low [18:28:51] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe exception search field keeps adding the text "test" to searches - https://phabricator.wikimedia.org/T225670#9992132 (10AKanji-WMF) team is currently working around this issue [18:29:34] 14Fundraising Sprint Git Rebase Jump, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe - document for DS the various bits of handling that are in the dedupe code - https://phabricator.wikimedia.org/T276392#9992143 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF con... [18:30:46] (03Merged) 10jenkins-bot: Remove last caller on wmf_common_locale_civi_to_mediawiki [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1051480 (https://phabricator.wikimedia.org/T365415) (owner: 10Eileen) [18:32:02] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10FR-donorservices: Deduper causing deadlocks - https://phabricator.wikimedia.org/T355677#9992157 (10AKanji-WMF) @Eileenmcnaughton @SHust says this is closeable [18:32:21] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Civi-Dedupe, 10FR-donorservices: Recent donations not saved after merging of two CIDs - https://phabricator.wikimedia.org/T326583#9992161 (10AKanji-WMF) 05In progress→03Resolved [18:34:11] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Deduper resolvable conflict- handle suffixes & prefix in names - https://phabricator.wikimedia.org/T242305#9992165 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust flags this as done. [18:34:53] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe resolvable conflict - use most recent if names are reversed - https://phabricator.wikimedia.org/T242304#9992181 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust flags as done [18:35:56] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-Civi-Dedupe: Dedupe resolvable conflict resolve misspellings where email disambiguates - https://phabricator.wikimedia.org/T242303#9992185 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF @SHust flags this has been resolved with email ame... [19:02:38] 06Fundraising-Backlog, 10fundraising-tech-ops, 07Epic, 07payments-orchestration: Network configuration for Gravy integration - https://phabricator.wikimedia.org/T370319#9992293 (10Dwisehaupt) Ok. Looks like they are using google cloud computing for those endpoints. I'll have to check our config on the pfw,... [19:03:39] 06Fundraising-Backlog, 10fundraising-tech-ops: Copy Gravy certificate to dedicated Cert directory - https://phabricator.wikimedia.org/T370159#9992299 (10Dwisehaupt) The private key has been configured and is in place on the civicrm, frdev, payments, and payments-listener roles. The file is available at: * /etc... [19:14:55] (03PS1) 10Ejegg: Only log TY ages for donations from front-end [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054929 (https://phabricator.wikimedia.org/T342812) [19:16:35] (03PS2) 10Eileen: Stop stopping logging on civicrm_mailing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1053418 (https://phabricator.wikimedia.org/T369782) [19:18:26] (03PS3) 10Eileen: Stop stopping logging on civicrm_mailing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1053418 (https://phabricator.wikimedia.org/T369782) [19:18:28] (03CR) 10Ejegg: [C:03+2] Stop stopping logging on civicrm_mailing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1053418 (https://phabricator.wikimedia.org/T369782) (owner: 10Eileen) [19:22:43] 06Fundraising-Backlog, 10FR-donorservices: Pop ups blocking our Civi view - https://phabricator.wikimedia.org/T370067#9992374 (10Cstone) @SHust We cleared the alert that was making this popup, it shouldn't come back [19:23:23] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Pop ups blocking our Civi view - https://phabricator.wikimedia.org/T370067#9992375 (10Cstone) [19:32:37] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Turns out wmffraud tests not running - https://phabricator.wikimedia.org/T365414#9992408 (10Cstone) [19:32:49] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Donation charged after cancellation and missed by the autocancelation - https://phabricator.wikimedia.org/T369748#9992411 (10Cstone) [19:33:56] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 13Patch-For-Review: Native (inapp) donations donor_country is the full country name - https://phabricator.wikimedia.org/T369775#9992412 (10Cstone) [19:34:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Figure out why we have civicrm_mailing_job records with a status of running - https://phabricator.wikimedia.org/T369783#9992413 (10Cstone) [19:35:56] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Drupal admin civi contact getting activites for all contributions - https://phabricator.wikimedia.org/T367612#9992435 (10Cstone) [19:36:19] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: refund process question - India - https://phabricator.wikimedia.org/T369094#9992437 (10Cstone) [19:36:33] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-Adyen, 10FR-Smashpig: We're recording the wrong amount from Adyen Capture IPNs in JPY - https://phabricator.wikimedia.org/T364392#9992438 (10Cstone) [19:36:55] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Fail mail - auto rescue with no predecessor - https://phabricator.wikimedia.org/T364438#9992439 (10Cstone) [19:37:34] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Japan Email Campaign Records - Incorrect utm_campaign Causing Duplicate Records - https://phabricator.wikimedia.org/T364515#9992440 (10Cstone) [19:37:36] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog: Address too long causing failmail - https://phabricator.wikimedia.org/T361981#9992441 (10Cstone) [19:45:24] (03Merged) 10jenkins-bot: Stop stopping logging on civicrm_mailing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1053418 (https://phabricator.wikimedia.org/T369782) (owner: 10Eileen) [19:45:26] !log config revision changed from 85336766 to 4ea1c745 [19:45:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [19:50:09] anilk: we dug into the civi-mails - the job wasn't sending them all each time you ran it - it needed to be run multiple times to send to all of them. That meant there were still some unsent - cstone confirmed it was OK to still send to the ones that didn't have it yet so we ran the ones that hadn't gone (500ish on the India one) and they have now gone. The job is now scheduled so you should not have to do that step [19:50:11] in future [19:52:57] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Figure out why we have civicrm_mailing_job records with a status of running - https://phabricator.wikimedia.org/T369783#9992482 (10Cstone) Since the jobs weren't scheduled, it wasn't actually sending all of the mailings... [19:53:39] (03CR) 10Cstone: [C:03+2] "rip" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054676 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [19:57:39] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10FR-donorservices: Pop ups blocking our Civi view - https://phabricator.wikimedia.org/T370067#9992495 (10SHust) Thank you, @Cstone [20:18:21] (03Merged) 10jenkins-bot: Finish removing log_search_past_days variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054676 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [20:50:58] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054695 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [20:57:48] (03PS3) 10Eileen: Move main audit classes & tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) [21:03:13] (03PS1) 10Eileen: Increase time limit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054938 [21:05:43] can I get a merge on this one - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1054684/3 - cstone I think the fact tests passed is enough? I'm gonna try to re-order the ones on top to see if the test time outs change at all if I do [21:06:03] sure eileen when gerrit decides to load for me [21:06:08] looking too [21:06:30] my theory is this one might be in some way to blame for the time outs https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1054678/2 [21:06:54] but we might just have reached that point [21:09:28] im having internet issues ejegg if you are ready to +2 go ahead [21:11:34] (03CR) 10CI reject: [V:04-1] Move main audit classes & tests to extension [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054696 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [21:14:29] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Figure out why we have civicrm_mailing_job records with a status of running - https://phabricator.wikimedia.org/T369783#9992803 (10Eileenmcnaughton) @AKanji-WMF we dug into the civi-mails - the job wasn't sending them a... [21:18:59] PROBLEM - Host frdb2003 is DOWN: PING CRITICAL - Packet loss = 100% [21:19:01] PROBLEM - Host frdb2002 is DOWN: PING CRITICAL - Packet loss = 100% [21:21:09] RECOVERY - Host frdb2002 is UP: PING OK - Packet loss = 0%, RTA = 30.41 ms [21:22:09] RECOVERY - Host frdb2003 is UP: PING OK - Packet loss = 0%, RTA = 30.49 ms [21:23:39] looks like my downtimes expired. those were expected. [21:29:46] may expect some puppet alerts as i reboot frpm2001. just fyi [21:30:47] (03PS4) 10Eileen: Remove fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) [21:31:32] (03CR) 10Ejegg: [C:03+2] "Looks good! Less painful than I was expecting. And thanks for adding the labels on those test data arrays." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [21:34:27] 06Fundraising-Backlog: New design layout variant for payments wiki - https://phabricator.wikimedia.org/T370369 (10ppenloglou) 03NEW [21:36:15] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 34 failures. Last run 4 minutes ago with 34 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [21:37:11] PROBLEM - check_puppetrun on frdb2003 is CRITICAL: CRITICAL: Puppet has 39 failures. Last run 5 minutes ago with 39 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [21:37:11] PROBLEM - check_puppetrun on frbackup2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 5 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [21:37:15] PROBLEM - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 5 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [21:37:17] PROBLEM - check_puppetrun on frban2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 5 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [21:37:17] PROBLEM - check_puppetrun on civi2001 is CRITICAL: CRITICAL: Puppet has 40 failures. Last run 5 minutes ago with 40 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [21:37:19] PROBLEM - check_puppetrun on frdata2001 is CRITICAL: CRITICAL: Puppet has 36 failures. Last run 5 minutes ago with 36 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [21:38:11] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 6 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [21:38:19] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 41 failures. Last run 6 minutes ago with 41 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [21:38:19] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 41 failures. Last run 6 minutes ago with 41 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [21:40:11] PROBLEM - check_puppetrun on frlog2001 is CRITICAL: CRITICAL: Puppet has 32 failures. Last run 8 minutes ago with 32 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [21:41:13] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 39 failures. Last run 9 minutes ago with 39 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [21:41:13] PROBLEM - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 34 failures. Last run 9 minutes ago with 34 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [21:41:15] PROBLEM - check_puppetrun on frauth2002 is CRITICAL: CRITICAL: Puppet has 35 failures. Last run 9 minutes ago with 35 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2002&service=check_puppetrun [21:41:15] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 34 failures. Last run 9 minutes ago with 34 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [21:41:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 41 failures. Last run 9 minutes ago with 41 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [21:41:21] PROBLEM - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 36 failures. Last run 9 minutes ago with 36 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [21:42:01] and there they are. [21:42:11] PROBLEM - check_puppetrun on frdb2003 is CRITICAL: CRITICAL: Puppet has 39 failures. Last run 10 minutes ago with 39 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [21:42:13] PROBLEM - check_puppetrun on frbackup2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [21:42:13] should clear on the next run [21:42:15] PROBLEM - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [21:42:17] PROBLEM - check_puppetrun on frban2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [21:42:17] PROBLEM - check_puppetrun on civi2001 is CRITICAL: CRITICAL: Puppet has 40 failures. Last run 10 minutes ago with 40 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [21:42:19] PROBLEM - check_puppetrun on frdata2001 is CRITICAL: CRITICAL: Puppet has 36 failures. Last run 10 minutes ago with 36 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [21:43:11] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 11 minutes ago with 31 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [21:43:19] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 41 failures. Last run 11 minutes ago with 41 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [21:43:19] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 41 failures. Last run 11 minutes ago with 41 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [21:45:09] PROBLEM - check_puppetrun on frlog2001 is CRITICAL: CRITICAL: Puppet has 32 failures. Last run 13 minutes ago with 32 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [21:45:36] ACKNOWLEDGEMENT - check_puppetrun on civi2001 is CRITICAL: CRITICAL: Puppet has 40 failures. Last run 10 minutes ago with 40 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frauth2002 is CRITICAL: CRITICAL: Puppet has 35 failures. Last run 9 minutes ago with 35 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2002&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frbackup2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frban2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frdata2001 is CRITICAL: CRITICAL: Puppet has 36 failures. Last run 10 minutes ago with 36 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 39 failures. Last run 9 minutes ago with 39 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [21:45:37] ACKNOWLEDGEMENT - check_puppetrun on frdb2003 is CRITICAL: CRITICAL: Puppet has 39 failures. Last run 10 minutes ago with 39 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [21:45:38] ACKNOWLEDGEMENT - check_puppetrun on frlog2001 is CRITICAL: CRITICAL: Puppet has 32 failures. Last run 13 minutes ago with 32 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [21:45:39] ACKNOWLEDGEMENT - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 36 failures. Last run 9 minutes ago with 36 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [21:45:39] ACKNOWLEDGEMENT - check_puppetrun on frpm2001 is CRITICAL: CRITICAL: Puppet has 58 failures. Last run 10 minutes ago with 58 failures. Failed resources (up to 3 shown): File[/usr/local/bin/package_update_check],File[/etc/vim/vimrc.local],File[/usr/local/bin/yubikey_otp_filter],File[/bin/nopeshell] Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpm2001&service=check_puppetru [21:45:39] ACKNOWLEDGEMENT - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 11 minutes ago with 31 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [21:45:40] ACKNOWLEDGEMENT - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 31 failures. Last run 10 minutes ago with 31 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [21:45:40] ACKNOWLEDGEMENT - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 34 failures. Last run 9 minutes ago with 34 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [21:45:41] ACKNOWLEDGEMENT - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 34 failures. Last run 9 minutes ago with 34 failures. Failed resources (up to 3 shown): Dwisehaupt fallout for puppetserver reboot https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [21:46:11] RECOVERY - check_puppetrun on pay-lvs2001 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [21:46:13] RECOVERY - check_puppetrun on frdb2002 is OK: OK: Puppet is currently enabled, last run 17 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [21:46:13] RECOVERY - check_puppetrun on pay-lvs2002 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [21:46:15] RECOVERY - check_puppetrun on frauth2002 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2002&service=check_puppetrun [21:46:15] RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [21:46:21] RECOVERY - check_puppetrun on frpig2001 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [21:47:02] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1054941 [21:47:11] RECOVERY - check_puppetrun on frdb2003 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [21:47:13] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1054941 (owner: 10Eileen) [21:47:13] RECOVERY - check_puppetrun on frbackup2002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [21:47:15] RECOVERY - check_puppetrun on frqueue2002 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [21:47:15] RECOVERY - check_puppetrun on civi2001 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [21:47:15] RECOVERY - check_puppetrun on frban2001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [21:47:17] RECOVERY - check_puppetrun on frdata2001 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [21:48:11] RECOVERY - check_puppetrun on frqueue2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [21:48:17] RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [21:48:17] RECOVERY - check_puppetrun on payments2003 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [21:50:11] RECOVERY - check_puppetrun on frlog2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [21:51:45] !log civicrm upgraded from 1ac3e7be to 384fe444 [21:51:47] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [21:54:22] (03CR) 10CI reject: [V:04-1] Remove fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [21:54:43] (03Merged) 10jenkins-bot: Remove fakedb [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054684 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [21:59:18] 03Fundraising Sprint: neveruse(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Investigate test issue with tests running out of time - https://phabricator.wikimedia.org/T370371 (10Eileenmcnaughton) 03NEW [22:00:00] (03PS2) 10Eileen: Increase time limit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054938 (https://phabricator.wikimedia.org/T370371) [22:00:17] ejegg: it ran out of time again - I'm gonna dig but as a quick-fix https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1054938 [22:05:51] (03PS3) 10Eileen: Increase time limit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054938 (https://phabricator.wikimedia.org/T370371) [22:05:52] (03CR) 10Ejegg: [C:03+2] Increase time limit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054938 (https://phabricator.wikimedia.org/T370371) (owner: 10Eileen) [22:30:05] (03Merged) 10jenkins-bot: Increase time limit [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054938 (https://phabricator.wikimedia.org/T370371) (owner: 10Eileen) [22:30:58] (03PS3) 10Eileen: Use setting for wmf_audit_directory_payments_log [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054678 (https://phabricator.wikimedia.org/T270675) [22:33:17] (03PS1) 10Eileen: Disable debug in unit tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054945 (https://phabricator.wikimedia.org/T370371) [22:33:43] (03PS2) 10Eileen: Call wmf_echo via the class function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054695 (https://phabricator.wikimedia.org/T270675) [22:50:12] (03CR) 10CI reject: [V:04-1] Call wmf_echo via the class function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054695 (https://phabricator.wikimedia.org/T270675) (owner: 10Eileen) [22:55:07] (03CR) 10CI reject: [V:04-1] Disable debug in unit tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054945 (https://phabricator.wikimedia.org/T370371) (owner: 10Eileen) [23:05:46] (03PS2) 10Eileen: Disable debug in unit tests [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1054945 (https://phabricator.wikimedia.org/T370371)