[00:02:49] 10Wikimedia-Fundraising-Banners, 10Wikipedia-iOS-App-Backlog, 10Patch-For-Review, 10iOS Release FY2022-23: Japan fundraising messages in iOS app - https://phabricator.wikimedia.org/T332229 (10Tsevener) {F36922275} {F36922277} [03:10:39] (03PS1) 10Eileen: [Not for merge] - upstream PR [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901732 (https://phabricator.wikimedia.org/T303986) [03:10:41] (03PS1) 10Eileen: [do not merge] Upstream PR [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901733 (https://phabricator.wikimedia.org/T303986) [03:10:43] (03PS1) 10Eileen: Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) [03:25:42] (03CR) 10CI reject: [V: 04-1] Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) (owner: 10Eileen) [03:36:08] (03PS2) 10Eileen: Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) [03:36:10] (03PS1) 10Eileen: [not for merge] [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901735 (https://phabricator.wikimedia.org/T303986) [03:51:03] (03CR) 10CI reject: [V: 04-1] Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) (owner: 10Eileen) [04:45:23] (03PS1) 10Eileen: [Not for merge [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901738 [04:53:44] (03PS3) 10Eileen: Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) [04:53:46] (03PS2) 10Eileen: [Not for merge] - port label fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901738 [04:53:48] (03PS1) 10Eileen: Enable Civi-import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901739 [04:57:29] (03CR) 10CI reject: [V: 04-1] Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901734 (https://phabricator.wikimedia.org/T303986) (owner: 10Eileen) [04:57:46] (03CR) 10CI reject: [V: 04-1] Enable Civi-import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901739 (owner: 10Eileen) [04:57:58] (03CR) 10CI reject: [V: 04-1] [Not for merge] - port label fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901738 (owner: 10Eileen) [04:59:29] (03CR) 10CI reject: [V: 04-1] [Not for merge] - port label fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901738 (owner: 10Eileen) [05:46:52] (03PS1) 10Eileen: [not for merge] - user template listing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901743 [06:00:55] (03CR) 10CI reject: [V: 04-1] [not for merge] - user template listing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901743 (owner: 10Eileen) [06:25:18] 10Fundraising Sprint Amazing grep, 10Fundraising Sprint Vwl Cnsrvtn, 10Fundraising Sprint Wibbly Wobbly Timey Wimey, 10Fundraising-Backlog, and 2 others: Figure out storage implications of varchar vs text in possible logging solution - https://phabricator.wikimedia.org/T325789 (10Eileenmcnaughton) I am jus... [06:41:32] (03PS2) 10Eileen: [not for merge] - user template listing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901743 [06:56:24] (03CR) 10CI reject: [V: 04-1] [not for merge] - user template listing [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901743 (owner: 10Eileen) [08:16:45] (03PS1) 10Eileen: [not for merge] empty contact_id fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901997 [08:32:00] (03CR) 10CI reject: [V: 04-1] [not for merge] empty contact_id fix [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/901997 (owner: 10Eileen) [10:44:30] (03PS1) 10Eileen: Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902025 (https://phabricator.wikimedia.org/T303986) [10:59:51] (03CR) 10CI reject: [V: 04-1] Add hooks to import to do organization lookup from nick-name [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902025 (https://phabricator.wikimedia.org/T303986) (owner: 10Eileen) [11:24:20] (03PS1) 10Eileen: Fix permissions for Benevity in test suite?: [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902059 [11:38:35] (03CR) 10CI reject: [V: 04-1] Fix permissions for Benevity in test suite?: [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902059 (owner: 10Eileen) [14:10:56] (03CR) 10Zoranzoki21: "Failure of CI isn't related to this patch, so this patch should be force-merged, like done previously for patches made by Libup. 😊" [extensions/FundraisingEmailUnsubscribe] - 10https://gerrit.wikimedia.org/r/900711 (https://phabricator.wikimedia.org/T220036) (owner: 10Zoranzoki21) [15:06:13] PROBLEM - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 5 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [15:06:13] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [15:06:15] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 5 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [15:06:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 27 failures. Last run 5 minutes ago with 27 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [15:06:23] PROBLEM - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 5 minutes ago with 23 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [15:07:13] PROBLEM - check_puppetrun on frdb2001 is CRITICAL: CRITICAL: Puppet has 24 failures. Last run 6 minutes ago with 24 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_puppetrun [15:07:17] PROBLEM - check_puppetrun on frban2001 is CRITICAL: CRITICAL: Puppet has 18 failures. Last run 6 minutes ago with 18 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [15:07:17] PROBLEM - check_puppetrun on frqueue2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 14 seconds ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [15:07:18] PROBLEM - check_puppetrun on frmon2001 is CRITICAL: CRITICAL: Puppet has 18 failures. Last run 6 minutes ago with 18 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon2001&service=check_puppetrun [15:08:13] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 18 failures. Last run 7 minutes ago with 18 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [15:08:17] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 7 minutes ago with 10 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [15:08:18] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 7 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [15:10:13] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 22 failures. Last run 9 minutes ago with 22 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [15:10:13] PROBLEM - check_puppetrun on frlog2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 2 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [15:10:17] PROBLEM - check_puppetrun on frmx2001 is CRITICAL: CRITICAL: Puppet has 14 failures. Last run 9 minutes ago with 14 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [15:11:13] PROBLEM - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 10 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [15:11:13] PROBLEM - check_puppetrun on frdb2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 8 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [15:11:15] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 10 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [15:11:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 27 failures. Last run 10 minutes ago with 27 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [15:11:27] PROBLEM - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 23 failures. Last run 10 minutes ago with 23 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [15:13:11] hmmm fr-tech ^ [15:13:17] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 10 failures. Last run 12 minutes ago with 10 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [15:13:18] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 12 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [15:15:13] RECOVERY - check_puppetrun on frauth2001 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [15:15:17] RECOVERY - check_puppetrun on frmx2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [15:16:13] RECOVERY - check_puppetrun on pay-lvs2001 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [15:16:15] RECOVERY - check_puppetrun on pay-lvs2002 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [15:16:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 second ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [15:17:17] RECOVERY - check_puppetrun on frqueue2002 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2002&service=check_puppetrun [15:17:18] RECOVERY - check_puppetrun on frmon2001 is OK: OK: Puppet is currently enabled, last run 16 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon2001&service=check_puppetrun [15:18:17] RECOVERY - check_puppetrun on payments2003 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [15:18:18] RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [15:20:13] RECOVERY - check_puppetrun on frlog2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frlog2001&service=check_puppetrun [15:21:17] RECOVERY - check_puppetrun on frdb2002 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2002&service=check_puppetrun [15:21:17] RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [15:21:23] RECOVERY - check_puppetrun on frpig2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [15:22:17] RECOVERY - check_puppetrun on frban2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban2001&service=check_puppetrun [15:27:13] RECOVERY - check_puppetrun on frdb2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2001&service=check_puppetrun [15:28:13] RECOVERY - check_puppetrun on frqueue2001 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [15:39:25] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate: Can’t mark CentralNotice translations as reviewed - https://phabricator.wikimedia.org/T308614 (10AndyRussG) [15:44:26] woah [15:57:16] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate: Can’t mark CentralNotice translations as reviewed - https://phabricator.wikimedia.org/T308614 (10Ciell) Perfect, thanks Andy! [16:02:34] 10Fundraising Sprint Anti-matter doesn't matter, 10Fundraising Sprint Cheese String Theory, 10Fundraising Sprint Discworld reformatted as ntfs, 10Fundraising Sprint Fibonachos, and 17 others: Fr-tech chores list - https://phabricator.wikimedia.org/T258527 (10AndyRussG) Notes for current sprint: https://eth... [16:10:16] huh, what was that puppet flap? [16:21:44] 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 10MediaWiki-extensions-Translate: Can’t mark CentralNotice translations as reviewed - https://phabricator.wikimedia.org/T308614 (10XenoRyet) Just to reiterate, moving this off the Chaos Crew board was just internal board organization, and not a sig... [16:35:52] greg-g: it was a typo when shifting puppet traffic off of frpm2001 in prep for it's rebuild. [16:48:53] cool :) [17:11:13] PROBLEM - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 8 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [17:11:14] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 8 minutes ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [17:16:13] PROBLEM - check_puppetrun on pay-lvs2001 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 54 seconds ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [17:16:14] PROBLEM - check_puppetrun on pay-lvs2002 is CRITICAL: CRITICAL: Puppet has 20 failures. Last run 57 seconds ago with 20 failures. Failed resources (up to 3 shown): https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [17:21:13] RECOVERY - check_puppetrun on pay-lvs2001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2001&service=check_puppetrun [17:21:13] RECOVERY - check_puppetrun on pay-lvs2002 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=pay-lvs2002&service=check_puppetrun [17:46:52] fr-tech so yeah sounds like we do need to switch to monthly vs on-demand... should we make a task for that also= [17:46:53] ? [17:47:24] also apologies for asking stuff about scheduling that I guess I should have known or at least known where to look to find out about [17:53:45] Have we confirmed that the monthly flow doesn't have that 5000 statement in it? [17:58:06] I'm glad you asked AndyRussG [17:58:36] anilk: ah thx :) [18:00:01] XenoRyet: that's a good question. I imagine it doesn't but I think indeed we're not 100% sure? [18:00:41] We should check before we commit to switching. [18:07:19] PROBLEM - check_puppetrun on frmon2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 5 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon2001&service=check_puppetrun [18:08:11] PROBLEM - check_puppetrun on frqueue2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [18:08:21] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [18:08:21] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [18:09:04] ^^^ these are somewhat expected as we shift frpm2001 back into services [18:09:32] 10fundraising-tech-ops: Fundraising environment support for Debian 11 (Bullseye) - https://phabricator.wikimedia.org/T277184 (10Jgreen) [18:09:34] 10fundraising-tech-ops: reimage fundraising build (frpm) role servers to Bullseye - https://phabricator.wikimedia.org/T305972 (10Jgreen) 05Open→03Resolved p:05Triage→03Medium [18:10:11] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 8 minutes ago with 2 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [18:10:17] PROBLEM - check_puppetrun on frmx2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 8 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [18:11:13] XenoRyet: yeah.. I imagine we can figure out a specific action item to verify that in tech-talk in less than 1 hr [18:11:17] PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 9 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [18:11:27] PROBLEM - check_puppetrun on frpig2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 9 minutes ago with 2 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [18:12:17] PROBLEM - check_puppetrun on frmon2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 10 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon2001&service=check_puppetrun [18:13:11] RECOVERY - check_puppetrun on frqueue2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frqueue2001&service=check_puppetrun [18:13:21] PROBLEM - check_puppetrun on payments2003 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 11 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [18:13:21] PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 11 minutes ago with 1 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [18:15:17] PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 13 minutes ago with 2 failures. Failed resources (up to 3 shown): Service[puppet] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [18:15:17] RECOVERY - check_puppetrun on frmx2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmx2001&service=check_puppetrun [18:16:04] XenoRyet: I think we can make it configurable on our side [18:16:17] RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun [18:16:21] (the monthly vs ondemand) [18:16:27] RECOVERY - check_puppetrun on frpig2001 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frpig2001&service=check_puppetrun [18:17:19] RECOVERY - check_puppetrun on frmon2001 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frmon2001&service=check_puppetrun [18:18:19] RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun [18:18:19] RECOVERY - check_puppetrun on payments2003 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2003&service=check_puppetrun [18:20:17] RECOVERY - check_puppetrun on frauth2001 is OK: OK: Puppet is currently enabled, last run 3 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun [18:33:40] 10fundraising-tech-ops: reimage fundraising queue role servers to Bullseye - https://phabricator.wikimedia.org/T327453 (10Dwisehaupt) [18:36:37] 10fundraising-tech-ops: reimage fundraising auth role servers to Bullseye - https://phabricator.wikimedia.org/T332808 (10Dwisehaupt) [18:53:38] 10Fundraising-Backlog: Error after Civi import - https://phabricator.wikimedia.org/T332810 (10MDemosWMF) [18:58:29] gonna be a few mins late to the meeting fr-tech. just finishing up some homework with Oscar [19:05:07] 10Fundraising-Backlog: Error after Civi import - https://phabricator.wikimedia.org/T332810 (10greg) Confirmed by Eileen in the Civi Fortnightly that this is a thing she knows how to fix and it's not negatively impacting the imports (just annoying/noisy). [19:05:33] (03PS1) 10Eileen: Fix for Civi not found [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902171 [19:06:36] (03PS2) 10Greg Grossmeier: Fix for Civi not found [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902171 (https://phabricator.wikimedia.org/T332810) (owner: 10Eileen) [19:09:24] (03CR) 10Ejegg: [C: 03+2] Fix for Civi not found [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902171 (https://phabricator.wikimedia.org/T332810) (owner: 10Eileen) [19:11:22] (03PS1) 10Ejegg: Respect result.redirect in dlocal.js [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/902177 (https://phabricator.wikimedia.org/T324302) [19:24:34] (03Merged) 10jenkins-bot: Fix for Civi not found [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/902171 (https://phabricator.wikimedia.org/T332810) (owner: 10Eileen) [19:24:53] ty for the review damilare ! [19:25:05] np [20:04:42] 10Fundraising-Backlog: dLocal: add config and code to allow switching between UPI monthly and on-demand - https://phabricator.wikimedia.org/T332816 (10AndyRussG) [20:07:06] XenoRyet: greg-g ^ so quick update on monthly vs on-demand: we agreed to make it a config switch so we can easily try out the monthly option and see what it does [20:07:29] that'll let us keep trying out both and compare flows [20:11:08] Sounds good. Did we find out yet if monthly lets us lose the problematic language? [20:25:42] (03PS1) 10Ejegg: Add processor name to log lines [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/902185 [20:26:00] There's your SmashPig-Dlocal cstone ^^^ [20:26:27] XenoRyet: not yet, once we deploy the configurable version we can find out [20:26:47] thanks ejegg ! [20:26:57] Cool. I'm just really curious is all, don't mean to rush it. [21:04:25] 10Fundraising-Backlog: Issue testing Gpay and PayTM through in India - confusing indicators on recurring flow - https://phabricator.wikimedia.org/T332828 (10EMartin) [21:06:58] 10Fundraising-Backlog: Testing: Issues with One time donation through UPI on new Dlocal API - https://phabricator.wikimedia.org/T332830 (10EMartin) [21:11:41] 10Fundraising-Backlog: Testing: Card test in India is paused at pending for 3DS authorization - https://phabricator.wikimedia.org/T332831 (10EMartin) [21:15:54] 10Fundraising-Backlog: Testing: card test in Brazil remains in pending status awaiting 3DS - https://phabricator.wikimedia.org/T332832 (10EMartin) [21:22:49] 10Fundraising-Backlog: Issue testing Gpay and PayTM through in India - confusing indicators on recurring flow - https://phabricator.wikimedia.org/T332828 (10EMartin) PayTM recurring then stays in pending status as attached: {F36923118} [21:25:41] 10Fundraising-Backlog: Issue testing Gpay and PayTM through in India - confusing indicators on recurring flow and trx remaining in pending status - https://phabricator.wikimedia.org/T332828 (10EMartin) [21:31:12] (03PS1) 10Ejegg: Make subscription frequency configurable [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/902200 (https://phabricator.wikimedia.org/T332828) [21:32:03] (03CR) 10Jgleeson: [C: 03+2] "thanks. this makes our logging lives easier!" [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/902185 (owner: 10Ejegg) [21:32:08] :) [21:32:38] (03Merged) 10jenkins-bot: Add processor name to log lines [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/902185 (owner: 10Ejegg) [21:38:55] 10Fundraising Sprint Fish HEAD^, 10MW-1.41-notes (1.41.0-wmf.1; 2023-03-20): Please turn on Monthly Convert in Czech (csCZ) - https://phabricator.wikimedia.org/T331665 (10HNordeenWMF) Thanks @Ejegg ! @TSkaff my fellow MC test warrior - I just completed a one-time donation + MC add on recurring and everything... [21:43:26] 10Fundraising-Backlog: Testing: card test in Brazil remains in pending status awaiting 3DS - https://phabricator.wikimedia.org/T332832 (10Aklapper) [21:47:36] 10Fundraising-Backlog, 10Patch-For-Review: Issue testing Gpay and PayTM through in India - confusing donation amounts and trx remaining in pending status - https://phabricator.wikimedia.org/T332828 (10EMartin) [21:49:22] 10Fundraising-Backlog: Testing: Paytm recurring remains in pending status - https://phabricator.wikimedia.org/T332835 (10EMartin) [21:50:51] ejegg: I want to start moving some of our logic from munge-message to a pre hook - specifically populating `source` on new contact records so it can be handled by imports without us having to go through the wmf-specific contribution create. But, going forwards it would be more things like calculating the exchange rates at that point [22:11:56] ok, interesting [22:12:34] letting us use standard core stuff is always good [22:14:10] 10Fundraising-Backlog: Testing: Paytm recurring remains in pending status - https://phabricator.wikimedia.org/T332835 (10Ejegg) In tech talk we were looking for payment confirmation notificationss and never got them either. We should ask dlocal if we need to make another API call to capture or something [22:16:40] (03PS1) 10Ejegg: Dlocal: set use_3d_secure when configured [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/902204 (https://phabricator.wikimedia.org/T332831) [22:17:11] ok, i /think/ that ^^^ should help testing its parent patch [22:17:36] ejegg: cool - I just wanted to check if you had any re-action before starting. [22:17:36] Also - where did we get to on this - https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/893095 [22:18:04] ah yeah, that. [22:18:43] definitely good code direction, i just wanted to double check we wouldn't lose info in logs [22:47:07] ok, i need to stop typing for today [22:47:21] dang wrist [22:50:33] (03PS3) 10Ejegg: Log rawResponse on recurring charge failure [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/885902 (https://phabricator.wikimedia.org/T239770) [22:51:05] Handle initial scheme transaction ID [22:51:07] [22:51:07] Handle initial scheme transaction ID [22:51:10] ack [22:51:59] 10Fundraising-Backlog: Testing: card test in Brazil remains in pending status awaiting 3DS - https://phabricator.wikimedia.org/T332832 (10Ejegg) [22:52:03] 10Fundraising-Backlog, 10Patch-For-Review: Testing: Card test in India is paused at pending for 3DS authorization - https://phabricator.wikimedia.org/T332831 (10Ejegg) [22:52:54] 10Fundraising Sprint Fish HEAD^, 10Fundraising-Backlog, 10FR-dlocal, 10Patch-For-Review: Testing: Card tests in India and in Brazil are paused at pending for 3DS authorization - https://phabricator.wikimedia.org/T332831 (10Ejegg) [22:53:43] (03PS2) 10Ejegg: Respect result.redirect in dlocal.js [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/902177 (https://phabricator.wikimedia.org/T332831)