[00:07:21] 10Fundraising Sprint Fish HEAD^, 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10FR-dlocal: DLocal Audit - Test with new integration - https://phabricator.wikimedia.org/T324347 (10Cstone) Ran the audit successfully with the dlocal log f... [00:16:35] (03CR) 10Cstone: [C: 03+2] Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917414 (owner: 10Eileen) [00:28:38] (03Merged) 10jenkins-bot: Update triggers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917414 (owner: 10Eileen) [00:30:21] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=2005 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [00:35:23] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=2013 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [00:38:29] looking into this ^^ [00:40:19] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=2763 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [00:40:33] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917418 [00:40:43] (03CR) 10Eileen: [C: 03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917418 (owner: 10Eileen) [00:41:52] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917418 (owner: 10Eileen) [00:43:21] !log civicrm upgraded from d5229d22 to 301e24e4 [00:43:23] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:45:19] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=2870 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [00:50:19] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=2782 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [00:55:19] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=3585 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:00:19] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=3858 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:05:21] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=3302 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:10:23] PROBLEM - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=3302 [critical =2000] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:11:58] ^^ this is known now. acking in icinga. [01:13:45] ACKNOWLEDGEMENT - check_missing_thank_yous on frdb1005 is CRITICAL: CRITICAL missing_thank_yous=3302 [critical =2000] Dwisehaupt Known issue with duplicate donations. Being cleared up. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:30:13] (03PS1) 10Cstone: Add fix for stopping ty emails for duplicated audit donations. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917422 (https://phabricator.wikimedia.org/T324347) [01:33:33] (03CR) 10Eileen: [C: 03+2] Add fix for stopping ty emails for duplicated audit donations. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917422 (https://phabricator.wikimedia.org/T324347) (owner: 10Cstone) [01:46:47] (03Merged) 10jenkins-bot: Add fix for stopping ty emails for duplicated audit donations. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917422 (https://phabricator.wikimedia.org/T324347) (owner: 10Cstone) [01:49:17] (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917425 [01:51:49] (03CR) 10Cstone: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917425 (owner: 10Cstone) [01:52:46] (03Merged) 10jenkins-bot: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/917425 (owner: 10Cstone) [01:53:43] !log civicrm upgraded from 301e24e4 to d8a1a562 [01:53:45] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:55:19] RECOVERY - check_missing_thank_yous on frdb1005 is OK: OK missing_thank_yous=1 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_missing_thank_yous [01:57:26] (03PS3) 10Eileen: Fix tests not running [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) [02:01:04] (03PS4) 10Eileen: Fix tests not running [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) [02:10:46] eileen: did you mean to say 'NOW fully populated' instead of '...database is not fully populated' [02:10:49] ? [02:10:56] erm yes [02:12:13] PROBLEM - check_mysql on frdb2001 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_mysql [02:13:19] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:13:57] (03CR) 10CI reject: [V: 04-1] Fix tests not running [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) (owner: 10Eileen) [02:14:38] ^^ those db ones are known while we have things stopped for the trigger update. [02:17:09] PROBLEM - check_mysql on frdb2001 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_mysql [02:18:17] PROBLEM - check_mysql on frdb1004 is CRITICAL: Slave IO: No Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:20:41] (03CR) 10Eileen: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) (owner: 10Eileen) [02:22:09] RECOVERY - check_mysql on frdb2001 is OK: Uptime: 532516 Threads: 4 Questions: 13643142 Slow queries: 270 Opens: 3455 Open tables: 1178 Queries per second avg: 25.620 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_mysql [02:23:13] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 443428 Threads: 4 Questions: 10345017 Slow queries: 226 Opens: 3529 Open tables: 1285 Queries per second avg: 23.329 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [02:23:26] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Patch-For-Review: Update triggers to not save details for thank you letters - https://phabricator.wikimedia.org/T332074 (10Dwisehaupt) The trigger updates are complete. [02:38:52] woohoo - I got this to pass https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/915032/ [03:02:18] (03PS1) 10Eileen: Copy live template back to our local [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917429 (https://phabricator.wikimedia.org/T309349) [04:05:58] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Patch-For-Review: Update triggers to not save details for thank you letters - https://phabricator.wikimedia.org/T332074 (10Eileenmcnaughton) @AnnWF - Dallas did the trig... [06:22:00] (03CR) 10CI reject: [V: 04-1] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/917497 (owner: 10L10n-bot) [08:07:30] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_35) - 10https://gerrit.wikimedia.org/r/916970 (owner: 10L10n-bot) [08:56:51] (03CR) 10Nikerabbit: [V: 03+2] Localisation updates from https://translatewiki.net. [extensions/DonationInterface] (REL1_39) - 10https://gerrit.wikimedia.org/r/917497 (owner: 10L10n-bot) [09:11:11] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests: As an FR-Tech SRE, we want to be able to designate a host for decommissioning - https://phabricator.wikimedia.org/T334154 (10jbond) >>! In T334154#8833707, @jbond wrote: > i have created a CR wh... [09:18:45] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations: Migrate jgreen out of the ops group and into the fr-tech-admins group - https://phabricator.wikimedia.org/T336231 (10jbond) [09:18:58] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations: Migrate jgreen out of the ops group and into the fr-tech-admins group - https://phabricator.wikimedia.org/T336231 (10jbond) p:05Triage→03Medium [09:49:38] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10Tracking-Neverending: Banner editor should be more consistent with other editing (tracking) - https://phabricator.wikimedia.org/T116140 (10Tacsipacsi) [09:49:43] 10Fundraising Sprint Snoop (Dogg|Lion), 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, and 3 others: Central Notice message groups are slow to index - https://phabricator.wikimedia.org/T111189 (10Tacsipacsi) [09:49:46] 10Fundraising Sprint Snoop (Dogg|Lion), 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate: New banner messages not added to message group - https://phabricator.wikimedia.org/T106856 (10Tacsipacsi) [09:49:49] 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate: BUG: CentralNotice banner translations sometimes go missing from Translate interface - https://phabricator.wikimedia.org/T90863 (10Tacsipacsi) [09:49:52] 10Fundraising Sprint N*E*R*D, 10Fundraising Sprint ODB, 10Fundraising Sprint Snoop (Dogg|Lion), 10Fundraising-Backlog, and 3 others: Publishing translations for central notice banners fails - https://phabricator.wikimedia.org/T104774 (10Tacsipacsi) [09:49:55] 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate, 10I18n: Can't use Translate extension's variables feature in CentralNotice - https://phabricator.wikimedia.org/T87448 (10Tacsipacsi) [09:49:58] 10Fundraising-Backlog, 10Fundraising-Backlog-Old, 10MediaWiki-extensions-CentralNotice, 10TranslationNotifications, 10I18n: Can't send translation notifications for translatable banners - https://phabricator.wikimedia.org/T58187 (10Tacsipacsi) [09:50:04] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10Wikimedia-production-error: Localizable messages with spaces in the name cause fatal exception - should fail gracefully - https://phabricator.wikimedia.org/T58105 (10Tacsipacsi) [09:50:10] 10Fundraising Sprint Rocky Horror Presentation Layer, 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, and 2 others: CentralNotice reloads old translatable messages on form submit - https://phabricator.wikimedia.org/T72939 (10Tacsipacsi) [09:50:20] 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, 10I18n: Allow CentralNotice banner variables to be marked as non translated - https://phabricator.wikimedia.org/T53470 (10Tacsipacsi) [09:50:42] 10Fundraising-Backlog, 10Fr-CentralNotice-translations, 10MediaWiki-extensions-CentralNotice: CentralNotice banners should exist in their own namespace (stop using the MediaWiki namespace) - https://phabricator.wikimedia.org/T33595 (10Tacsipacsi) [09:50:48] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10I18n: CentralNotice should respect language fallbacks - https://phabricator.wikimedia.org/T34482 (10Tacsipacsi) [09:50:58] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10I18n: All messages in CentralNotice are defaulted to English regardless of content language - https://phabricator.wikimedia.org/T18321 (10Tacsipacsi) [09:51:08] 10Fundraising-Backlog, 10Fr-CentralNotice-Translation-Bugs, 10MediaWiki-extensions-CentralNotice, 10Epic, 10I18n: [Epic] CentralNotice translation should move closer to MediaWiki i18n standards and the code cleaned up - https://phabricator.wikimedia.org/T116235 (10Tacsipacsi) 05Declined→03Open A task... [09:58:22] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations: Migrate jgreen out of the ops group and into the fr-tech-admins group - https://phabricator.wikimedia.org/T336231 (10MoritzMuehlenhoff) AFAICS are two remaining use cases for the private repo: - There's a kafka-fundraising-clie... [10:33:34] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations: Migrate jgreen out of the ops group and into the fr-tech-admins group - https://phabricator.wikimedia.org/T336231 (10MoritzMuehlenhoff) One other actionable: We'd need a new LDAP group (e.g. cn=fr-tech-admins or cn=fr-tech-sres)... [12:28:49] (03CR) 10Damilare Adedoyin: [C: 03+2] Move test code to test class [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/915731 (owner: 10Ejegg) [12:30:45] (03Merged) 10jenkins-bot: Move test code to test class [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/915731 (owner: 10Ejegg) [12:49:56] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations: Migrate jgreen out of the ops group and into the fr-tech-admins group - https://phabricator.wikimedia.org/T336231 (10Jgreen) >>! In T336231#8836198, @MoritzMuehlenhoff wrote: > AFAICS are two remaining use cases for the private... [13:15:18] (03CR) 10Jgleeson: [C: 03+2] "LGTM. Thanks!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/909316 (owner: 10Ejegg) [13:27:10] (03Merged) 10jenkins-bot: Fix end-of-run message for pending resolver [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/909316 (owner: 10Ejegg) [14:36:02] 10Fundraising-Backlog, 10fundraising-tech-ops, 10fr-donorservices: Expose email-prefs-wiki to public - https://phabricator.wikimedia.org/T335793 (10MSuijkerbuijk_WMF) Hi @Ejegg and @AKanji-WMF We need to discuss this first, as the Email preference center and the Recurring self-service/upgrade have very diff... [14:56:34] thanks for the code review damilare and jgleeson ! [14:56:47] np [14:57:58] np [14:58:10] thanks for all the work! [15:10:01] 10Fundraising-Backlog, 10fundraising-tech-ops, 10fr-donorservices: Expose email-prefs-wiki to public - https://phabricator.wikimedia.org/T335793 (10Ejegg) Hi @MSuijkerbuijk_WMF! I asked @Jgreen and @Dwisehaupt about the domain names yesterday, and it turns out they don't have a problem with multiple names po... [15:14:09] 10Fundraising-Backlog, 10fundraising-tech-ops, 10fr-donorservices: Expose email-prefs-wiki to public - https://phabricator.wikimedia.org/T335793 (10Ejegg) [15:24:06] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Export currency and amount of active recurring donation to Acoustic - https://phabricator.wikimedia.org/T335926 (10Ejegg) Hi @MSuijkerbuijk_WMF So we probably don't want to solicit upgrades from ALL donors where AF_has_active_rec... [15:24:32] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Indicate in Acoustic export if a donor is eligible for monthly upgrade - https://phabricator.wikimedia.org/T335824 (10Ejegg) [15:24:43] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Indicate in Acoustic export if a donor is eligible for monthly upgrade - https://phabricator.wikimedia.org/T335824 (10Ejegg) [15:25:31] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Export currency and amount of active recurring donation to Acoustic - https://phabricator.wikimedia.org/T335926 (10Ejegg) [15:47:28] hi fr-tech, heads up that Brazil & Mexico banner pre-test is starting in 15 minutes [15:49:21] ty haley_ [15:54:27] (03CR) 10Ejegg: "This change is ready for review." [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [15:57:24] (03PS12) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [15:58:56] (03CR) 10CI reject: [V: 04-1] Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:00:06] (03PS13) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [16:00:16] man, that minfraud test is really annoying [16:00:32] we're catching an exception around where I need to make the assertion [16:00:47] so it's suppressing the output of the expected vs actual in CI [16:01:05] and I need to push up dumb patch sets like that last one to log the stuff [16:01:46] (03CR) 10CI reject: [V: 04-1] Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:02:56] (03PS14) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [16:04:47] (03CR) 10CI reject: [V: 04-1] Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:08:05] (03PS15) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [16:08:37] oh hey, let's do some log watching [16:09:36] (03CR) 10CI reject: [V: 04-1] Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:16:16] (03PS16) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [16:18:09] (03CR) 10CI reject: [V: 04-1] Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:21:03] (03PS17) 10Ejegg: Delete GlobalCollect [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/913977 (https://phabricator.wikimedia.org/T260725) [16:21:11] hmm, that's HUGE [16:21:31] Let's see if I can split that up without getting in a huge git mess [16:27:34] passing tests now, at least! [16:35:13] (03PS1) 10Ejegg: Make tests use Ingenico as default adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/917921 (https://phabricator.wikimedia.org/T260725) [16:35:22] let's see if it still passes broken up [16:36:47] (03CR) 10CI reject: [V: 04-1] Make tests use Ingenico as default adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/917921 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:38:44] nahhhh, that looks like a whole nother horrible annoyance [16:38:54] I'll leave it as the big ball of wax [16:39:28] (03Abandoned) 10Ejegg: Make tests use Ingenico as default adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/917921 (https://phabricator.wikimedia.org/T260725) (owner: 10Ejegg) [16:47:27] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Export currency and amount of active recurring donation to Acoustic - https://phabricator.wikimedia.org/T335926 (10MSuijkerbuijk_WMF) Thanks @Ejegg this makes sense. We will exclude PayPal donors from upgrades (we can talk if thi... [16:51:04] (03PS3) 10Ejegg: Consider FallbackCurrency when choosing gateway [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/804485 (https://phabricator.wikimedia.org/T309755) [16:52:33] (03CR) 10CI reject: [V: 04-1] Consider FallbackCurrency when choosing gateway [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/804485 (https://phabricator.wikimedia.org/T309755) (owner: 10Ejegg) [16:53:17] whew, invalid CPF error failstorm! [16:55:58] 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Export currency and amount of active recurring donation to Acoustic - https://phabricator.wikimedia.org/T335926 (10Ejegg) @MSuijkerbuijk_WMF those rows are correct as far as I know. Beyond the ones mentioned there, I *think* we c... [17:02:15] PROBLEM - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 1 minute ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_puppetrun [17:05:13] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 4 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [17:05:19] PROBLEM - check_puppetrun on payments1008 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1008&service=check_puppetrun [17:06:15] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [17:06:15] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [17:06:15] PROBLEM - check_puppetrun on fran1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_puppetrun [17:06:17] PROBLEM - check_puppetrun on frmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon1001&service=check_puppetrun [17:06:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [17:06:18] PROBLEM - check_puppetrun on pay-lvs1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1004&service=check_puppetrun [17:06:19] PROBLEM - check_puppetrun on frauth1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth1002&service=check_puppetrun [17:06:25] PROBLEM - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [17:07:15] PROBLEM - check_puppetrun on frqueue1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_puppetrun [17:07:15] PROBLEM - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_puppetrun [17:07:15] PROBLEM - check_puppetrun on pay-lvs1003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1003&service=check_puppetrun [17:07:16] PROBLEM - check_puppetrun on frdb1003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_puppetrun [17:07:16] RECOVERY - check_puppetrun on frdb2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_puppetrun [17:07:16] PROBLEM - check_puppetrun on frdev1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [17:07:16] PROBLEM - check_puppetrun on frmx1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx1001&service=check_puppetrun [17:07:17] PROBLEM - check_puppetrun on frdb2003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [17:07:17] PROBLEM - check_puppetrun on frpm2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 5 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpm2001&service=check_puppetrun [17:07:17] PROBLEM - check_puppetrun on frbackup2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [17:07:18] PROBLEM - check_puppetrun on civi2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [17:07:18] PROBLEM - check_puppetrun on frban1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1001&service=check_puppetrun [17:07:19] PROBLEM - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [17:07:19] PROBLEM - check_puppetrun on frdata2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [17:08:12] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [17:08:14] PROBLEM - check_puppetrun on frbast2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 6 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbast2001&service=check_puppetrun [17:08:14] PROBLEM - check_puppetrun on frnetmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frnetmon1001&service=check_puppetrun [17:08:16] PROBLEM - check_puppetrun on frqueue1003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_puppetrun [17:08:18] PROBLEM - check_puppetrun on frdb1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_puppetrun [17:08:18] PROBLEM - check_puppetrun on frlog1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog1002&service=check_puppetrun [17:08:18] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [17:08:20] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 7 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [17:10:14] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [17:10:16] PROBLEM - check_puppetrun on frdev1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_puppetrun [17:10:16] PROBLEM - check_puppetrun on frdata1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata1002&service=check_puppetrun [17:10:18] PROBLEM - check_puppetrun on frmx2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [17:10:18] PROBLEM - check_puppetrun on payments1007 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 8 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1007&service=check_puppetrun [17:10:20] PROBLEM - check_puppetrun on civi1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 8 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1001&service=check_puppetrun [17:10:20] PROBLEM - check_puppetrun on payments1008 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1008&service=check_puppetrun [17:10:22] PROBLEM - check_puppetrun on frdb1005 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 9 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_puppetrun [17:11:14] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [17:11:14] PROBLEM - check_puppetrun on fran1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_puppetrun [17:11:14] PROBLEM - check_puppetrun on frmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon1001&service=check_puppetrun [17:11:15] RECOVERY - check_puppetrun on pay-lvs2002 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [17:11:15] PROBLEM - check_puppetrun on pay-lvs1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1004&service=check_puppetrun [17:11:16] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [17:11:16] PROBLEM - check_puppetrun on frauth1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth1002&service=check_puppetrun [17:11:24] RECOVERY - check_puppetrun on frpig2001 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [17:12:14] RECOVERY - check_puppetrun on frdb1003 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_puppetrun [17:12:14] PROBLEM - check_puppetrun on frqueue1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_puppetrun [17:12:14] PROBLEM - check_puppetrun on frbackup2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [17:12:15] PROBLEM - check_puppetrun on frav1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_puppetrun [17:12:15] PROBLEM - check_puppetrun on frdb2003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [17:12:15] PROBLEM - check_puppetrun on pay-lvs1003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1003&service=check_puppetrun [17:12:15] PROBLEM - check_puppetrun on frpm2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 10 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpm2001&service=check_puppetrun [17:12:16] PROBLEM - check_puppetrun on civi2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [17:12:16] PROBLEM - check_puppetrun on frmx1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx1001&service=check_puppetrun [17:12:16] PROBLEM - check_puppetrun on frdev1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [17:12:17] PROBLEM - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [17:12:17] PROBLEM - check_puppetrun on frban1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1001&service=check_puppetrun [17:12:18] RECOVERY - check_puppetrun on frdata2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata2001&service=check_puppetrun [17:13:15] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 12 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [17:13:15] PROBLEM - check_puppetrun on frbast2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 11 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbast2001&service=check_puppetrun [17:13:15] PROBLEM - check_puppetrun on frnetmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 12 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frnetmon1001&service=check_puppetrun [17:13:17] PROBLEM - check_puppetrun on frqueue1003 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 12 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_puppetrun [17:13:17] RECOVERY - check_puppetrun on frdb1004 is OK: OK: Puppet is currently enabled, last run 18 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_puppetrun [17:13:18] RECOVERY - check_puppetrun on frlog1002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog1002&service=check_puppetrun [17:13:19] RECOVERY - check_puppetrun on payments2003 is OK: OK: Puppet is currently enabled, last run 59 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [17:13:19] RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 40 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [17:15:15] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 14 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [17:15:17] PROBLEM - check_puppetrun on frdev1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 14 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_puppetrun [17:15:17] RECOVERY - check_puppetrun on frdata1002 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdata1002&service=check_puppetrun [17:15:19] RECOVERY - check_puppetrun on frmx2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [17:15:19] PROBLEM - check_puppetrun on payments1007 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 13 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1007&service=check_puppetrun [17:15:19] RECOVERY - check_puppetrun on civi1001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1001&service=check_puppetrun [17:15:19] PROBLEM - check_puppetrun on payments1008 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 14 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1008&service=check_puppetrun [17:15:25] RECOVERY - check_puppetrun on frdb1005 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1005&service=check_puppetrun [17:16:15] RECOVERY - check_puppetrun on frdb2002 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [17:16:15] PROBLEM - check_puppetrun on fran1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 15 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_puppetrun [17:16:15] PROBLEM - check_puppetrun on frmon1001 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 15 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon1001&service=check_puppetrun [17:16:17] RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 12 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [17:16:17] PROBLEM - check_puppetrun on pay-lvs1004 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 15 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1004&service=check_puppetrun [17:16:19] PROBLEM - check_puppetrun on frauth1002 is CRITICAL: CRITICAL: Puppet has 3 failures. Last run 15 minutes ago with 3 failures. Failed resources (up to 3 shown): Exec[systemctl-daemon-reload],File[/etc/systemd/system/prometheus-node-exporter.service],Service[nsca] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth1002&service=check_puppetrun [17:17:15] RECOVERY - check_puppetrun on frdb2003 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_puppetrun [17:17:15] RECOVERY - check_puppetrun on pay-lvs1003 is OK: OK: Puppet is currently enabled, last run 41 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1003&service=check_puppetrun [17:17:15] RECOVERY - check_puppetrun on frav1002 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_puppetrun [17:17:16] RECOVERY - check_puppetrun on frqueue1004 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1004&service=check_puppetrun [17:17:16] RECOVERY - check_puppetrun on frbackup2002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbackup2002&service=check_puppetrun [17:17:16] RECOVERY - check_puppetrun on frpm2001 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpm2001&service=check_puppetrun [17:17:16] RECOVERY - check_puppetrun on frmx1001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx1001&service=check_puppetrun [17:17:16] RECOVERY - check_puppetrun on frdev1002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1002&service=check_puppetrun [17:17:17] RECOVERY - check_puppetrun on civi2001 is OK: OK: Puppet is currently enabled, last run 50 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi2001&service=check_puppetrun [17:17:17] RECOVERY - check_puppetrun on frqueue2002 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [17:17:18] RECOVERY - check_puppetrun on frban1001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1001&service=check_puppetrun [17:18:15] RECOVERY - check_puppetrun on frqueue2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [17:18:15] RECOVERY - check_puppetrun on frbast2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frbast2001&service=check_puppetrun [17:18:15] RECOVERY - check_puppetrun on frnetmon1001 is OK: OK: Puppet is currently enabled, last run 55 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frnetmon1001&service=check_puppetrun [17:18:17] RECOVERY - check_puppetrun on frqueue1003 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue1003&service=check_puppetrun [17:20:07] hi icinga-wm [17:20:15] RECOVERY - check_puppetrun on frauth2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [17:20:17] RECOVERY - check_puppetrun on frdev1001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_puppetrun [17:20:19] RECOVERY - check_puppetrun on payments1007 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1007&service=check_puppetrun [17:20:19] RECOVERY - check_puppetrun on payments1008 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments1008&service=check_puppetrun [17:21:15] RECOVERY - check_puppetrun on frmon1001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon1001&service=check_puppetrun [17:21:15] RECOVERY - check_puppetrun on fran1001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_puppetrun [17:21:17] RECOVERY - check_puppetrun on pay-lvs1004 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs1004&service=check_puppetrun [17:21:19] RECOVERY - check_puppetrun on frauth1002 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth1002&service=check_puppetrun [17:24:17] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests: As an FR-Tech SRE, we want to be able to designate a host for decommissioning - https://phabricator.wikimedia.org/T334154 (10Dzahn) @Dwisehaupt @Jgreen It looks like this should be resolved now.... [17:25:34] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests: As an FR-Tech SRE, we want to be able to designate a host for decommissioning - https://phabricator.wikimedia.org/T334154 (10Dwisehaupt) @Dzahn Thanks. I'll try and test it later today or tomorrow. [17:30:45] 10Fundraising-Backlog: Update the information on donation form for Netherlands: update the word from "uw" to "je" - https://phabricator.wikimedia.org/T336287 (10RKumar_WMF) [17:33:42] oh my [17:33:47] (icinga spam) [17:38:27] I tried giving him some attention [18:15:09] so https://phabricator.wikimedia.org/T336209 would quiet things down for the next Brazil test [18:15:22] let's see how that would go [18:28:25] ejegg: so these failmails are not a concern? asking cause i was wondering if i needed to worry. [18:29:33] dwisehaupt: they're annoying, but they're not actually serious [18:29:47] we're sending failmail when we should be showing a validation error [18:29:56] ok cool. thanks! [18:30:13] so I'm just trying to build a test case to replicate it, then will fix the code to show the error [18:41:49] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: CT-to-Civi-Phase 1.5 cleanup & update contribution_tracking documentation - https://phabricator.wikimedia.org/T334538 (10AnnWF) https://wikitech.wikimedia.org/wiki/Fundraising_Tracking is the main part of what is contribution tracking... [18:43:07] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Patch-For-Review: Update triggers to not save details for thank you letters - https://phabricator.wikimedia.org/T332074 (10AnnWF) Thanks @Dwisehaupt and @Eileenmcnaughton [18:43:17] (03Abandoned) 10Wfan: Update Upgrader [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915833 (https://phabricator.wikimedia.org/T332074) (owner: 10Wfan) [18:53:08] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests: As an FR-Tech SRE, we want to be able to designate a host for decommissioning - https://phabricator.wikimedia.org/T334154 (10Dzahn) a:03Dwisehaupt Cool! Feel free to reach out to me if you hav... [19:05:19] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10FR-Paypal, 10FR-Smashpig: PayPal unsupported message type: recurring_payment_outstanding_payment - https://phabricator.wikimedia.org/T219833 (10XenoRyet) 05Open→03Resolved [19:05:44] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Geocoding data does not appear to be updated correctly on merge - https://phabricator.wikimedia.org/T334151 (10XenoRyet) [19:05:46] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Geocoding data needs to be updated as some have not been done correctly on merge - https://phabricator.wikimedia.org/T334152 (10XenoRyet) 05Open→03Resolved [19:06:10] 10Fundraising-Backlog, 10FR-dlocal, 10Epic: Upgrade Dlocal to leverage Smartfields/no redirect - https://phabricator.wikimedia.org/T274300 (10XenoRyet) [19:06:21] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10FR-dlocal, and 3 others: Enable Monthly Convert for DLocal in all markets that support recurring cards - https://phabricator.wikimedia.org/T324624 (10XenoRyet) 05Open→03Resolved [19:06:35] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10FR-Smashpig, 10Recurring-Donations: Stop hardcoding payment_method=cc in SmashPig recurring donation messages - https://phabricator.wikimedia.org/T334991 (10XenoRyet) 05Open→03Resolved [19:06:58] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) [19:07:09] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: CT-to-Civi-Phase 1 Create a view to replace the drupal table - https://phabricator.wikimedia.org/T334535 (10XenoRyet) 05Open→03Resolved [19:07:20] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) [19:07:23] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: CT-to-Civi-Phase 1 - expose in the UI - https://phabricator.wikimedia.org/T334540 (10XenoRyet) 05Open→03Resolved [19:07:45] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: CT-to-Civi-Phase 1.5 cleanup & update contribution_tracking documentation - https://phabricator.wikimedia.org/T334538 (10XenoRyet) 05Open→03Resolved [19:07:47] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) [19:08:20] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: Update config for Payments Staging - https://phabricator.wikimedia.org/T327855 (10XenoRyet) 05Open→03Resolved [19:08:56] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10FR-Adyen, 10MediaWiki-extensions-DonationInterface: Upgrade to newer version of Adyen JS SDK - https://phabricator.wikimedia.org/T334999 (10XenoRyet) 05Open→03Resolved [19:09:02] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Patch-For-Review: Add Online Banking for Czech Republic - https://phabricator.wikimedia.org/T328717 (10XenoRyet) [19:10:02] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) 05Open→03Resolved [19:10:40] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Make contribution tracking data visible in CiviCRM - https://phabricator.wikimedia.org/T158140 (10XenoRyet) [19:10:54] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Make contribution tracking data visible in CiviCRM - https://phabricator.wikimedia.org/T158140 (10XenoRyet) 05Open→03Resolved [19:10:56] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) [19:11:13] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Patch-For-Review: Add Online Banking for Czech Republic - https://phabricator.wikimedia.org/T328717 (10XenoRyet) 05Open→03Resolved [19:11:19] 10Fundraising Sprint Everything I Merge I Merge For You, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Try creating a new activity_type_id for Thank yous & see if the UI is still fine - if so switch - https://phabricator.wikimedia.org/T327963 (10XenoRyet) [19:11:28] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising-tech-ops, 10Patch-For-Review: Update triggers to not save details for thank you letters - https://phabricator.wikimedia.org/T332074 (10XenoRyet) 05Open→03Resolved [19:11:38] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: civicrm notification top blocked if toolbar visiable - https://phabricator.wikimedia.org/T335992 (10XenoRyet) 05Open→03Resolved [19:11:54] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: ContributionTracking form - split into 2 - https://phabricator.wikimedia.org/T335717 (10XenoRyet) 05Open→03Resolved [19:12:56] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Epic: Scope contribution tracking - https://phabricator.wikimedia.org/T182030 (10XenoRyet) [19:13:10] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog, 10Patch-For-Review: CT-to-Civi-Phase 1 - populate the civi fields - https://phabricator.wikimedia.org/T334534 (10XenoRyet) 05Open→03Resolved [19:14:16] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: Monthly Convert Popup with Other amount error UI not working - https://phabricator.wikimedia.org/T334308 (10XenoRyet) 05Open→03Resolved [19:14:57] 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog: mc-debit not found for ingenico - https://phabricator.wikimedia.org/T334769 (10XenoRyet) 05Open→03Resolved [20:01:25] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising-Backlog: Investigate feasiblily of the queries for the contact coding - https://phabricator.wikimedia.org/T336305 (10Eileenmcnaughton) [20:01:33] 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Investigate feasiblily of the queries for the contact coding - https://phabricator.wikimedia.org/T336305 (10Eileenmcnaughton) [20:11:21] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Investigate feasiblily of the queries for the contact coding - https://phabricator.wikimedia.org/T336305 (10XenoRyet) [20:11:27] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, and 3 others: Some Thank-you related tests won't run.... - https://phabricator.wikimedia.org/T335933 (10XenoRyet) [20:11:32] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Export currency and amount of active recurring donation to Acoustic - https://phabricator.wikimedia.org/T335926 (10XenoRyet) [20:11:37] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Recurring-Donations, 10fr-donorservices, 10FR-Email: Indicate in Acoustic export if a donor is eligible for monthly upgrade - https://phabricator.wikimedia.org/T335824 (10XenoRyet) [20:11:39] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10FR-dlocal: Test India Recurring - https://phabricator.wikimedia.org/T335818 (10XenoRyet) [20:11:41] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 10Recurring-Donations, 10fr-donorservices: Create recurring upgrade form - https://phabricator.wikimedia.org/T335795 (10XenoRyet) [20:11:44] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Recurring-Donations, 10fr-donorservices: Update CiviProxy config to allow new API action for recurring upgrade - https://phabricator.wikimedia.org/T335794 (10XenoRyet) [20:11:47] 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10Recurring-Donations: Create API action to get active recurrings based on contact ID and checksum - https://phabricator.wikimedia.org/T335788 (10XenoRyet) [20:11:50] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: Remove code references to old contribution tracking from wmf_audit_get_contribution_tracking_data - https://phabricator.wikimedia.org/T335711 (10XenoRyet) [20:11:52] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: Investigation: Scope out and write tasks for Recurring donor upgrade project - https://phabricator.wikimedia.org/T335375 (10XenoRyet) [20:11:55] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising Tech - Chaos Crew, 10Fundraising-Backlog, and 2 others: "Order id is duplicated" error from dlocal - https://phabricator.wikimedia.org/T334905 (10XenoRyet) [20:12:00] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: Update fundraising-dev SmashPig image Apache config - https://phabricator.wikimedia.org/T334894 (10XenoRyet) [20:12:02] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: CT-to-Civi-Phase 1.5 Go through phab tasks & clean up relating to contribution tracking - https://phabricator.wikimedia.org/T334539 (10XenoRyet) [20:12:04] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: CT-to-Civi-Phase 1.5 - Update code references to contribution_tracking - https://phabricator.wikimedia.org/T334537 (10XenoRyet) [20:12:07] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10fundraising-tech-ops: Ingenico Audit - They are moving their SFTP server to MFT - https://phabricator.wikimedia.org/T334239 (10XenoRyet) [20:12:11] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, and 3 others: Handle Refund/Chargeback IPNS for Dlocal - https://phabricator.wikimedia.org/T330723 (10XenoRyet) [20:12:15] 10Fundraising Sprint Fish HEAD^, 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, and 2 others: DLocal Audit - Test with new integration - https://phabricator.wikimedia.org/T324347 (10XenoRyet) [20:12:21] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, and 4 others: Update DLocal createPayment for UPI / DIRECT method - https://phabricator.wikimedia.org/T324294 (10XenoRyet) [20:12:33] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, and 2 others: Delete Globalcollect adapter, fold still-used code into Ingenico adapter - https://phabricator.wikimedia.org/T260725 (10XenoRyet) [20:12:36] 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising-Backlog: Investigation: Scope Segmentation in Civi & Acoustic - https://phabricator.wikimedia.org/T335376 (10XenoRyet) [20:12:45] 10Fundraising Sprint Go Your Own Pay, 10Fundraising Sprint Istanbool (not Constantidouble), 10Fundraising Sprint J 2023, 10Fundraising Sprint Juggalology 2022, and 8 others: Currency code/currency symbol in Thank You emails received by Donors issues - https://phabricator.wikimedia.org/T309349 (10XenoRyet) [20:27:11] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:32:11] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:33:06] ^^ looking into that. [20:34:19] hey eileen sorry about that. chrome froze for me but I could still hear you [20:34:24] I just couldn't see anything or unmute [20:34:53] lol - I just wasn't sure if we had catch ups still planned & then we seemed to be the only 2 in the hangout [20:35:12] I added a note to our call later just saying it's getting difficult making that slot and if we could catch up async [20:35:31] I was curious if you're attending any of the upcoming civi events? [20:35:38] (03PS1) 10Ejegg: Validation error instead of failmail for bad CPF [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/917946 (https://phabricator.wikimedia.org/T336209) [20:36:13] ok I just saw your email eileen ha [20:36:26] (03CR) 10CI reject: [V: 04-1] Validation error instead of failmail for bad CPF [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/917946 (https://phabricator.wikimedia.org/T336209) (owner: 10Ejegg) [20:36:30] fr-tech i /think/ that's the right way to get those failmails quieted, i just need to make a test for it... ^^^ damilare or jgleeson I think you were working on that code last, right? [20:36:54] yeah - I think heading the the UK in December would be a lot - but those residentials are really good [20:37:08] (03PS1) 10Jgleeson: WIP/BROKE: Add method to resolve the rare OrderId / ct_id mismatches [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/917947 (https://phabricator.wikimedia.org/T334905) [20:37:13] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:38:45] (03CR) 10CI reject: [V: 04-1] WIP/BROKE: Add method to resolve the rare OrderId / ct_id mismatches [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/917947 (https://phabricator.wikimedia.org/T334905) (owner: 10Jgleeson) [20:39:06] ejegg: I added the catch block for the ApiExceptions on the createPayment. This seems different, unfortunately I haven't been able to reproduce this locally [20:39:46] brb in about 10 mins and then will also take a look ejegg [20:42:13] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:45:40] ejegg: looks like this param was just added or a different issue. Here's the ticket I worked on https://phabricator.wikimedia.org/T333725 initially [20:46:41] so yh a validation error will make sense in this case [20:47:13] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:52:09] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [20:57:13] PROBLEM - check_mysql on frdb2003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [21:00:05] 10Fundraising Sprint Jealousy Driven Development, 10Fundraising-Backlog, 10Recurring-Donations: Create API action to get active recurrings based on contact ID and checksum - https://phabricator.wikimedia.org/T335788 (10AnnWF) a:03AnnWF [21:02:13] RECOVERY - check_mysql on frdb2003 is OK: Uptime: 600554 Threads: 4 Questions: 137611963 Slow queries: 242 Opens: 5072 Open tables: 1776 Queries per second avg: 229.141 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [21:37:16] (03CR) 10Jgleeson: Validation error instead of failmail for bad CPF (031 comment) [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/917946 (https://phabricator.wikimedia.org/T336209) (owner: 10Ejegg) [22:01:01] thanks damilare and jgleeson|away [22:01:30] np [22:24:06] (03CR) 10Wfan: [C: 03+2] Add dlocal refund and chargeback ipn handling [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/910610 (https://phabricator.wikimedia.org/T330723) (owner: 10Cstone) [22:24:46] (03PS1) 10Eileen: Add render test [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917960 [22:24:48] (03PS1) 10Eileen: Add test for Render code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917961 (https://phabricator.wikimedia.org/T309349) [22:28:31] (03PS1) 10Eileen: Update es_MX text to match prod [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917962 (https://phabricator.wikimedia.org/T309349) [22:38:17] (03CR) 10CI reject: [V: 04-1] Add test for Render code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917961 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [22:41:00] (03CR) 10CI reject: [V: 04-1] Update es_MX text to match prod [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917962 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [22:54:57] eileen: for the ct view where is it supposed to be shown? in the view pop up window it doesnt show the data [22:55:14] cstone: hmm I have definitely seen it show the data! [22:55:23] its there just like hidden? [22:55:25] by css [22:55:41] if I go to https://civicrm.wikimedia.org/civicrm/contact/view/contribution?action=view&reset=1&id=88907464&cid=53601124&context=home [22:56:11] and scroll down I see it & then If I click on the link to the contribution on the search display it pops up with it also there [22:56:37] eileen: want to join tech talk real quick :P [22:56:41] yep [23:08:27] 10Fundraising-Backlog: Update the information on donation form for Netherlands: update the word from "uw" to "je" - https://phabricator.wikimedia.org/T336287 (10Cstone) The adyen field name for the dropdown placeholder is `idealIssuer.selectField.placeholder` [23:14:50] 10Fundraising-Backlog: contribution info not load from pop up window - https://phabricator.wikimedia.org/T336318 (10AnnWF) [23:24:46] (03PS2) 10Eileen: Update es_MX text to match prod [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917962 (https://phabricator.wikimedia.org/T309349) [23:25:08] (03CR) 10Cstone: [C: 03+2] Copy live template back to our local [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917429 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [23:25:10] (03CR) 10Wfan: [C: 03+2] Copy live template back to our local [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917429 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [23:29:24] (03CR) 10Wfan: [C: 03+2] Fix tests not running [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) (owner: 10Eileen) [23:31:26] (03CR) 10Cstone: [C: 03+2] "Mind blown!!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) (owner: 10Eileen) [23:32:32] (03CR) 10Cstone: [C: 03+2] Update es_MX text to match prod [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917962 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [23:37:41] (03CR) 10Cstone: [C: 03+2] "thanks!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/917356 (owner: 10Ejegg) [23:42:24] (03Merged) 10jenkins-bot: Fix tests not running [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/915032 (https://phabricator.wikimedia.org/T335933) (owner: 10Eileen) [23:42:26] (03Merged) 10jenkins-bot: Copy live template back to our local [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917429 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [23:45:17] (03Merged) 10jenkins-bot: Update es_MX text to match prod [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/917962 (https://phabricator.wikimedia.org/T309349) (owner: 10Eileen) [23:45:19] (03Merged) 10jenkins-bot: Clarify logging around RecurringContract IPNs [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/917356 (owner: 10Ejegg) [23:47:18] 10fundraising-tech-ops, 10Infrastructure Security, 10Infrastructure-Foundations, 10SRE, 10SRE-Access-Requests: As an FR-Tech SRE, we want to be able to designate a host for decommissioning - https://phabricator.wikimedia.org/T334154 (10Dzahn) 05Open→03In progress