[06:23:22] PROBLEM - check_puppetrun on frav1003 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 6 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[initramfs-tools] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1003&service=check_puppetrun [06:28:18] PROBLEM - check_puppetrun on frav1003 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 11 minutes ago with 1 failures. Failed resources (up to 3 shown): Package[initramfs-tools] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1003&service=check_puppetrun [06:33:22] RECOVERY - check_puppetrun on frav1003 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1003&service=check_puppetrun [07:43:10] (03CR) 10Raimond Spekking: [C: 03+2] "false positive" [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/947237 (owner: 10L10n-bot) [12:30:55] PROBLEM - Host payments1005 is DOWN: PING CRITICAL - Packet loss = 100% [12:32:13] RECOVERY - Host payments1005 is UP: PING OK - Packet loss = 0%, RTA = 0.43 ms [14:04:41] 10Wikimedia-Fundraising-Banners, 10Wikipedia-iOS-App-Backlog, 10iOS Release FY2023-24: India fundraising messages in iOS app - https://phabricator.wikimedia.org/T335919 (10Tsevener) a:03Tsevener [14:45:30] 10fundraising-tech-ops, 10SRE, 10decommission-hardware, 10ops-codfw: decommission frmon2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T342694 (10Jhancock.wm) 05Open→03Resolved [14:46:17] 10fundraising-tech-ops, 10SRE, 10decommission-hardware, 10ops-codfw: decommission frbast2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T340156 (10Jhancock.wm) 05Open→03Resolved [15:36:01] 10Fundraising-Backlog: Acoustic Click-tracking - SSL cert for links.email.wikimedia.org - https://phabricator.wikimedia.org/T318188 (10AKanji-WMF) 05Open→03Resolved a:03AKanji-WMF Thank you @KHaggard ! Resolving. [15:39:28] 10fundraising-tech-ops, 10SRE, 10decommission-hardware, 10ops-codfw: decommission frbast2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T340156 (10Papaul) interfaces are disable ` papaul@fasw-c-codfw# run show interfaces ge-[0-1]/0/15 descriptions Interface Admin Link Description ge-0/0/... [15:45:00] 10fundraising-tech-ops, 10SRE, 10decommission-hardware, 10ops-codfw: decommission frmon2001.frack.codfw.wmnet - https://phabricator.wikimedia.org/T342694 (10Papaul) interfaces disable ` papaul@fasw-c-codfw# run show interfaces descriptions ge-[0-1]/0/16 Interface Admin Link Description ge-0/0/16... [15:49:25] 10Wikimedia-Fundraising-Banners, 10Wikipedia-iOS-App-Backlog, 10Patch-For-Review, 10iOS Release FY2023-24: India fundraising messages in iOS app - https://phabricator.wikimedia.org/T335919 (10Tsevener) a:05Tsevener→03None [15:49:34] 10Wikimedia-Fundraising-Banners, 10Wikipedia-iOS-App-Backlog, 10Patch-For-Review, 10iOS Release FY2023-24: India fundraising messages in iOS app - https://phabricator.wikimedia.org/T335919 (10Tsevener) Screenshot from development testing: {F37396234} [16:05:23] (03CR) 10Wfan: [C: 03+2] Upgrade CiviCRM Image to PHP 7.4 [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/944320 (https://phabricator.wikimedia.org/T342195) (owner: 10Jgleeson) [16:06:14] (03CR) 10Wfan: [V: 03+2 C: 03+2] "LGTM" [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/944320 (https://phabricator.wikimedia.org/T342195) (owner: 10Jgleeson) [16:14:19] 10Wikimedia-Fundraising-Banners, 10Wikipedia-Android-App-Backlog (Android Release - FY2023-24): India fundraising messages in Android app - https://phabricator.wikimedia.org/T335918 (10Tsevener) Gerrit patch: https://gerrit.wikimedia.org/r/c/mediawiki/services/wikifeeds/+/948156 [17:33:04] (03CR) 10Damilare Adedoyin: [C: 03+2] "LGT" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/944357 (https://phabricator.wikimedia.org/T342722) (owner: 10Eileen) [17:46:29] 10Fundraising Sprint Owner of a Lonely Product, 10Fundraising-Backlog, 10MW-1.41-notes (1.41.0-wmf.22; 2023-08-15): Pass additional params from Payments to thank you page URL - https://phabricator.wikimedia.org/T340058 (10HNordeenWMF) a million thanks @Cstone for all your work on this! It's super exciting. [17:47:37] (03Merged) 10jenkins-bot: E-notice fix on import [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/944357 (https://phabricator.wikimedia.org/T342722) (owner: 10Eileen) [17:48:37] (03PS1) 10Damilare Adedoyin: This patch would go upstream, pushing up here to discuss if this is a potential solution for this unique problem. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/948174 (https://phabricator.wikimedia.org/T342722) [17:54:00] 10Fundraising Sprint Owner of a Lonely Product, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Import template editing issue - https://phabricator.wikimedia.org/T342722 (10Damilare) Hi @Eileenmcnaughton I dug into this today and saw a potential cause of the problem. I added som... [20:07:11] PROBLEM - check_ipsec on frban1001 is CRITICAL: Strongswan CRITICAL - ipsec statusall: connecting to unix:///var/run/charon.ctl failed: Connection refused https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1001&service=check_ipsec [20:12:17] RECOVERY - check_ipsec on frban1001 is OK: Strongswan OK - 1 ESP OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frban1001&service=check_ipsec [20:51:51] 10Fundraising-Backlog: Civi TY Receipt Unsubscribe form passes email address through the URL - https://phabricator.wikimedia.org/T317929 (10KHaggard) 05Open→03Invalid Hi @AnnWF - I think you meant to ping me in your comment, correct? Unfortunately I'm not sure what the plan is for the Civi-side unsubscribe... [21:04:39] (03PS2) 10Damilare Adedoyin: This patch would go upstream, pushing up here to discuss if this is a potential solution for this unique problem. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/948174 (https://phabricator.wikimedia.org/T342722) [23:13:30] 10Fundraising Sprint Owner of a Lonely Product, 10Fundraising-Backlog, 10MediaWiki-extensions-DonationInterface, 10fr-email-preference-center, 10FR-Email: Make unsubscribe work in EmailPreferences, delete Unsub extension - https://phabricator.wikimedia.org/T223330 (10AKanji-WMF)