[00:02:36] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 3 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:07:36] <icinga-wm>	 RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:15:55] <wikibugs>	 10Fundraising-Backlog, 10fundraising-tech-ops: Fundraising access request for Emma Hughes - https://phabricator.wikimedia.org/T296940 (10Dwisehaupt)
[00:17:30] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 4 [=1],Paypal_endpoint_warning 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:22:30] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 8 [=1],Paypal_endpoint_warning 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:27:36] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 8 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:32:40] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 9 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:33:35] <wikibugs>	 10Fundraising-Backlog: Enable South Africa through Dlocal - https://phabricator.wikimedia.org/T293508 (10EMartin) Here is the link to the documentation, which you may already have. API information: https://docs.dlocal.com/api-documentation/payins-api-reference/country-reference#documents.
[00:37:36] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 9 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:42:40] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 8 [=1],Paypal_endpoint_warning 1 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:47:36] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 11 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:52:40] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 9 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:57:38] <dwisehaupt>	 i'm going to ack this alert but keep an eye on it.
[00:57:40] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 9 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[00:58:24] <icinga-wm>	 ACKNOWLEDGEMENT - check_log_messages on frav1002 is CRITICAL: CRITICAL: Paypal_endpoint_critical 9 [=1] Dwisehaupt known - keeping an eye on it. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[01:17:36] <icinga-wm>	 RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[03:23:49] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fr-donorservices: Amazon  and applepay donations TY email doesn’t have the donor's name - https://phabricator.wikimedia.org/T296881 (10Ejegg) I just emailed our Adyen contact this question regarding Apple Pay:  > Do you know if other merchants p...
[03:25:35] <cstone>	 ejegg: on that forgetme failmail there is one that is still there but it looks like I don't have the powder to redo the GDPR request
[03:58:29] <ejegg>	 oh fun, I can try
[03:58:35] <ejegg>	 cstone: ^^
[03:59:01] <cstone>	 Thanks ejegg 
[03:59:09] <cstone>	 Power not powder haha
[03:59:22] <ejegg>	 heh, sounded like a piratical way of saying it
[04:00:18] <ejegg>	 cstone so you deleted the rest of the jobs that you checked already?
[04:01:24] <ejegg>	 oh man, that's a good address too
[04:04:29] <ejegg>	 hmm cstone I only see one job in the api4 explorer, against the Endowment db
[04:05:02] <ejegg>	 and I don't find the email associated with that job in either the Endowment db or the All db
[04:06:08] <cstone>	 The one I saw left was still in the suppression list 
[04:06:15] <ejegg>	 ahh ok
[04:06:58] <ejegg>	 yep, i see it now
[04:08:13] <ejegg>	 ok, submitted
[04:08:20] <cstone>	 Thanks !
[04:08:25] <ejegg>	 have a good weekend!
[04:08:29] <cstone>	 You too !
[07:40:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1239 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[07:42:12] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1286 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[07:45:18] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1463 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[07:47:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1497 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[07:50:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1684 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[07:52:16] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1711 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[07:52:35] <RhinosF1>	 AndyRussG: this is 3 days in a row ^
[07:55:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1900 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[07:57:12] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1853 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:00:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2125 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:02:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1752 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:05:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1950 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:07:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1970 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:10:18] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2174 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:12:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2011 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:15:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2399 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:17:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2143 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:20:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2361 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:22:18] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2007 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:25:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2585 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:27:12] <icinga-wm>	 PROBLEM - check_mysql on frdb1002 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2212 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:30:18] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2451 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:32:18] <icinga-wm>	 RECOVERY - check_mysql on frdb1002 is OK: Uptime: 1506538 Threads: 18 Questions: 180201252 Slow queries: 1021 Opens: 786428207 Flush tables: 1 Open tables: 287 Queries per second avg: 119.612 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2418 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1002&service=check_mysql
[08:35:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2666 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:40:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2878 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:45:12] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3095 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[08:55:14] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3534 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:00:16] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3594 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:05:16] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3764 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:10:16] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3175 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:15:16] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2550 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:20:16] <icinga-wm>	 PROBLEM - check_mysql on frdev1001 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1798 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[09:30:13] <icinga-wm>	 RECOVERY - check_mysql on frdev1001 is OK: Uptime: 1509593 Threads: 15 Questions: 154033813 Slow queries: 1234653 Opens: 833267932 Flush tables: 1 Open tables: 301 Queries per second avg: 102.036 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 344 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdev1001&service=check_mysql
[12:59:14] <jgleeson>	 damilare: my car is back in the garage so I won't be able to meet today before the school run as I'm walking today so have to set off earlier 
[12:59:27] <jgleeson>	 did you catch my message last night? 
[13:10:02] <damilare>	 yes I did jgleeson
[13:11:10] <damilare>	 That's ok, we can catch up in the evening. I should be done by then 
[13:16:41] <jgleeson>	 sounds good!
[13:33:11] <jgleeson>	 hmm damilare did that just reinvite you to all the meetings you'd already declined over the christmas period?
[13:33:32] <jgleeson>	 I just removed the recurring tuesday slot thinking it would only update those
[13:33:39] <jgleeson>	 to go down to 3 days a week as discussed
[13:50:14] <damilare>	 ok no problem, I didn't get any notifications anyway
[13:51:19] <jgleeson>	 ah ok. I got six emails relating to meeting declines shortly after updating the invite
[13:51:26] <jgleeson>	 maybe google wanted to remind me :)
[13:58:25] <damilare>	 that's possible
[14:13:18] <icinga-wm>	 PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 6 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun
[14:15:14] <icinga-wm>	 PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 6 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun
[14:18:18] <icinga-wm>	 RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun
[14:20:14] <icinga-wm>	 RECOVERY - check_puppetrun on frauth2001 is OK: OK: Puppet is currently enabled, last run 4 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun
[14:26:24] <icinga-wm>	 PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 9 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun
[14:31:18] <icinga-wm>	 RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun
[14:42:14] <wikibugs>	 10Fundraising-Backlog: Post-payment Monthly Convert *Yes* button text overflows container on some devices - https://phabricator.wikimedia.org/T297012 (10spatton)
[14:42:20] <wikibugs>	 10Fundraising-Backlog: Post-payment Monthly Convert *Yes* button text overflows container on some devices - https://phabricator.wikimedia.org/T297012 (10spatton) p:05Triage→03High
[15:21:18] <icinga-wm>	 PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 5 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun
[15:23:18] <icinga-wm>	 PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 6 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun
[15:25:14] <icinga-wm>	 PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 8 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun
[15:26:18] <icinga-wm>	 PROBLEM - check_puppetrun on payments2002 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 10 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun
[15:28:18] <icinga-wm>	 PROBLEM - check_puppetrun on payments2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 11 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun
[15:30:14] <icinga-wm>	 PROBLEM - check_puppetrun on frauth2001 is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 13 minutes ago with 2 failures. Failed resources (up to 3 shown): Exec[fetch_geoip_database] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun
[15:31:18] <icinga-wm>	 RECOVERY - check_puppetrun on payments2002 is OK: OK: Puppet is currently enabled, last run 31 seconds ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2002&service=check_puppetrun
[15:33:18] <icinga-wm>	 RECOVERY - check_puppetrun on payments2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=payments2001&service=check_puppetrun
[15:35:14] <icinga-wm>	 RECOVERY - check_puppetrun on frauth2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frauth2001&service=check_puppetrun
[16:01:46] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Post-payment Monthly Convert *Yes* button text overflows container on some devices - https://phabricator.wikimedia.org/T297012 (10DStrine)
[16:42:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 20 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[16:47:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 46 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[16:52:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 78 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[16:57:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 78 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:02:35] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 78 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:07:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 78 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:12:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 68 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:16:37] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Documentation: Create a place for text source for UML diagrams, and put existing diagrams there - https://phabricator.wikimedia.org/T296344 (10DStrine) p:05Triage→03Medium
[17:17:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 47 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:18:40] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fr-donorservices: Add a second layer of confirmation to the batch merge deduper button - https://phabricator.wikimedia.org/T296042 (10DStrine) p:05Triage→03Medium
[17:19:00] <wikibugs>	 10Fundraising Sprint Visual C Saw, 10Fundraising-Backlog, 10fundraising sprint Universal Cereal Bus, 10fundraising sprint Wireless Zipline, 10Patch-For-Review: Payment session lost causing error page throwing exception - https://phabricator.wikimedia.org/T295149 (10DStrine) p:05Triage→03Medium
[17:19:10] <wikibugs>	 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Wireless Zipline: Automatically populate start date on relationships - https://phabricator.wikimedia.org/T294487 (10DStrine) p:05Triage→03Medium
[17:19:16] <wikibugs>	 10Fundraising Sprint Visual C Saw, 10Fundraising-Backlog, 10fundraising sprint Universal Cereal Bus, 10fundraising sprint Wireless Zipline: Tech talk on EOY receipt email - https://phabricator.wikimedia.org/T291572 (10DStrine) p:05Triage→03Medium
[17:19:30] <wikibugs>	 10Fundraising Sprint Technical debt house of horrors, 10Fundraising Sprint Visual C Saw, 10Fundraising-Backlog, 10fundraising sprint Quantum Leapfrog, and 6 others: Enhance Pending Transaction Resolver to use all available fraud filter scores - https://phabricator.wikimedia.org/T296555 (10DStrine) p:05Tri...
[17:19:39] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fr-donorservices: Civi is no longer displaying the recurring amount in the same format - correctly. - https://phabricator.wikimedia.org/T296663 (10DStrine) p:05Triage→03High
[17:19:48] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fundraising-tech-ops: Document Ingenico integration - https://phabricator.wikimedia.org/T291874 (10DStrine) p:05Triage→03Medium
[17:19:56] <wikibugs>	 10Fundraising Sprint Visual C Saw, 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Document Adyen Checkout - https://phabricator.wikimedia.org/T288403 (10DStrine) p:05Triage→03Medium
[17:20:11] <wikibugs>	 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Wireless Zipline, 10fr-donorservices: Recurring "Cancel" button reposition request - https://phabricator.wikimedia.org/T291133 (10DStrine) p:05Triage→03Medium
[17:20:21] <wikibugs>	 10Fundraising Sprint Visual C Saw, 10Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10fundraising sprint Universal Cereal Bus, 10fundraising sprint Wireless Zipline: Question about blocked contacts in Civi and how they sync to Acoustic - https://phabricator.wikimedia.org/T293587 (10DStrine) p:05T...
[17:20:50] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10Patch-For-Review: Set up Civilint in composer scripts - https://phabricator.wikimedia.org/T296636 (10DStrine) p:05Triage→03Medium
[17:21:27] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Orphan Slayer deep dive documentation - https://phabricator.wikimedia.org/T295926 (10DStrine) p:05Triage→03Medium
[17:21:37] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10FR-Smashpig, 10Patch-For-Review: Adyen payments-init messages getting NULL final_status - https://phabricator.wikimedia.org/T296266 (10DStrine) p:05Triage→03Medium
[17:22:15] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fr-donorservices: In need of access to see the actual "email content" under mailing events tab in civi. - https://phabricator.wikimedia.org/T296039 (10DStrine) p:05Triage→03Medium
[17:22:21] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10MW-1.38-notes (1.38.0-wmf.12; 2021-12-06): Matching Gifts - Employer IDs showing in Field - https://phabricator.wikimedia.org/T292582 (10DStrine) p:05Triage→03Medium
[17:22:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 29 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:27:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 30 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:32:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 30 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:37:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 30 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:42:41] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 30 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:43:38] <wikibugs>	 (03PS1) 10Jgleeson: WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881)
[17:43:49] <jgleeson>	 cstone: ^^
[17:46:20] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881) (owner: 10Jgleeson)
[17:47:04] <jgleeson>	 cstone: gonna grab dinner but will dig in more when back
[17:47:12] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Transactions at status 600 - https://phabricator.wikimedia.org/T296956 (10DStrine) p:05Triage→03High
[17:47:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 25 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:52:31] <icinga-wm>	 PROBLEM - check_log_messages on frav1002 is CRITICAL: CRITICAL: ipset_error 5 [=1] https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[17:57:31] <icinga-wm>	 RECOVERY - check_log_messages on frav1002 is OK: OK https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frav1002&service=check_log_messages
[18:01:02] <wikibugs>	 (03PS1) 10Damilare Adedoyin: Pending resover: incorporate scores from fredge [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743467 (https://phabricator.wikimedia.org/T296555)
[18:14:25] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Pending resover: incorporate scores from fredge [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743467 (https://phabricator.wikimedia.org/T296555) (owner: 10Damilare Adedoyin)
[18:43:09] <wikibugs>	 (03PS1) 10Cstone: Add last_name check before displaying first_name in TY subject [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743481 (https://phabricator.wikimedia.org/T296881)
[18:46:13] <jgleeson>	 hmm my macbook isn't turning on
[18:47:02] <jgleeson>	 possibly hypothermia. it was 0c here midweek 
[18:47:26] <cstone>	 aww hah
[18:47:28] <jgleeson>	 ah it's come back
[18:48:04] <cstone>	 most unsettling thing about my 2015 macbook is when the trackpad wont click when it has no power
[18:48:16] <jgleeson>	 cstone: so to test this we've gotta do the whole ssh tunnel with our cloud vps dev box haven't we
[18:48:27] <cstone>	 yeo
[18:48:28] <cstone>	 p
[18:48:28] <jgleeson>	 been a good few weeks since I run that
[18:49:20] <cstone>	 if you have issues setting it up I think mine is behaving we could test on a call
[18:49:48] <jgleeson>	 ok that might be quicker
[18:50:02] <jgleeson>	 we also need to login as our sandbox users right?
[18:50:09] <cstone>	 mine is already like that
[18:50:18] <cstone>	 or I set it up wrong from the start haha
[18:50:21] <jgleeson>	 ha!
[18:50:27] <jgleeson>	 today's standup link?
[18:50:31] <cstone>	 sure
[18:56:30] <wikibugs>	 (03PS2) 10Jgleeson: WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881)
[18:57:15] <wikibugs>	 10Fundraising-Backlog, 10Desktop Improvements, 10MediaWiki-extensions-CentralNotice, 10Product-Design-Strategy, and 7 others: [Discussion] Prevent SEO decline due to Google "page experience": mid-June 2021 - https://phabricator.wikimedia.org/T280476 (10AndyRussG) Thanks so much @Peter and and @awight!  >>!...
[18:58:54] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881) (owner: 10Jgleeson)
[19:02:05] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Transactions at status 600 - https://phabricator.wikimedia.org/T296956 (10EMartin) To give a magnitude around this issue, there have been 598 unscored stopped transactions on 11/30  & 12/1.  Here is a sample of transactions unscored but remain at 60...
[19:11:46] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline: Post-payment Monthly Convert *Yes* button text overflows container on some devices - https://phabricator.wikimedia.org/T297012 (10AndyRussG) a:03AndyRussG
[19:38:44] <wikibugs>	 (03PS3) 10Jgleeson: WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881)
[19:40:50] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] WIP: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881) (owner: 10Jgleeson)
[19:41:59] <wikibugs>	 10Fundraising Sprint Vagranty McVagrantface, 10Fundraising Sprint We all meet again, 10Fundraising-Backlog, 10SRE, and 5 others: Deal with donatewiki Thank You page launching in apps - https://phabricator.wikimedia.org/T259312 (10DStrine) Hey all,  Bumping this  @Ppena  just saw a TY page in app and an in-...
[19:54:46] <wikibugs>	 (03PS4) 10Jgleeson: Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881)
[19:58:03] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Add method to pick out best apple pay contact name [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/743464 (https://phabricator.wikimedia.org/T296881) (owner: 10Jgleeson)
[19:59:01] <wikibugs>	 (03CR) 10Jgleeson: [C: 03+2] "Looks good to me!" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743481 (https://phabricator.wikimedia.org/T296881) (owner: 10Cstone)
[20:00:40] <jgleeson>	 dami I'm hanging around for another 15 minutes or so if you fancied jumping on today's standup to catch up on the pending trxn stuff? If not we can do Monday
[20:01:03] <jgleeson>	 damilare: *
[20:10:04] <wikibugs>	 (03Merged) 10jenkins-bot: Add last_name check before displaying first_name in TY subject [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743481 (https://phabricator.wikimedia.org/T296881) (owner: 10Cstone)
[20:13:45] <wikibugs>	 (03PS1) 10Cstone: Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/743495
[20:15:27] <wikibugs>	 (03CR) 10Cstone: [C: 03+2] Merge branch 'master' into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/743495 (owner: 10Cstone)
[20:18:34] <damilare>	 Hi jgleeson
[20:19:07] <damilare>	 Sorry for keeping you waiting
[20:19:18] <damilare>	 joining the call now
[20:21:28] <dstrine>	 hey fr-tech using search kit: If I want to search for contacts created after a certain date, is there a field for that? If so I'm hoping it's available in search kit
[20:21:53] <wikibugs>	 (03CR) 10Cstone: [C: 03+2] "recheck" [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/743495 (owner: 10Cstone)
[20:22:17] <jgleeson>	 sorry dstrine I'm a complete search kit noob
[20:22:21] <jgleeson>	 AndyRussG: cstone damilare ?
[20:22:23] <dstrine>	 lol ok
[20:22:48] <cstone>	 dstrine: theres a created_date on the contact table
[20:24:29] <dstrine>	 ok I tried searching on created date and it's not automatically giving me date range UI controls maybe there's a bug
[20:26:02] <wikibugs>	 (03CR) 10Damilare Adedoyin: "recheck" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/743467 (https://phabricator.wikimedia.org/T296555) (owner: 10Damilare Adedoyin)
[20:27:02] <dstrine>	 ok never-mind I figured out the date format it was looking for
[20:27:25] <cstone>	 fr-tech I just deployed that ty email subject line change, you can test it in the testmeister UI by removing the last name
[20:29:03] <wikibugs>	 10Fundraising-Backlog, 10fundraising sprint Wireless Zipline, 10fr-donorservices, 10Patch-For-Review: Amazon  and applepay donations TY email doesn’t have the donor's name - https://phabricator.wikimedia.org/T296881 (10Cstone) I just deployed the change to the TY email subject to only display the first_nam...
[20:29:19] <cstone>	 !log revision changed from 2c2e22cd to b82183b9
[20:29:22] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log
[20:48:17] <wikibugs>	 10Fundraising-Backlog, 10fundraising-tech-ops: Fundraising access request for Emma Hughes - https://phabricator.wikimedia.org/T296940 (10Dwisehaupt)
[20:50:08] <wikibugs>	 10Fundraising-Backlog, 10fundraising-tech-ops: Fundraising access request for Emma Hughes - https://phabricator.wikimedia.org/T296940 (10Dwisehaupt) SSL client certificate created and sent via email. Key password send via sms. civicrm account created. superset account created and password sent via SMS. Instruc...