[00:06:40] lol - I just thought it was checking in on all of us [00:09:46] ha! even better. [00:44:01] (03PS4) 10Eileen: Simplify timestamp handling in contribution tracking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111699 [01:00:25] I'm just pushing a process control change to keep eoy receipts going for 1 more day - they just stopped due to UTC rolling over to the 16th [01:00:26] the change is trivial & no-one is around so am just doing it [01:01:31] !log config revision changed from 0f227a32 to d7fbb972 [01:01:33] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [01:07:39] (03CR) 10CI reject: [V:04-1] Simplify timestamp handling in contribution tracking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111699 (owner: 10Eileen) [01:13:53] (03PS5) 10Eileen: Simplify timestamp handling in contribution tracking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111699 [02:20:52] (03PS4) 10Eileen: Fix another place not to use legacy date function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111700 [02:20:52] (03PS4) 10Eileen: Remove last reference to one of our legacy date functions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111701 [02:20:52] (03PS3) 10Eileen: Use Message->getDate() for all retrievals of message date [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111714 (https://phabricator.wikimedia.org/T270677) [02:38:29] (03PS6) 10Eileen: Switch new address create to use apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1071707 [03:14:22] (03PS1) 10Eileen: Remove some never-used exception-type pseudoconstants [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111729 (https://phabricator.wikimedia.org/T270677) [03:14:24] (03PS1) 10Eileen: Remove WMFException type that is partially-used in outgoing code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111730 (https://phabricator.wikimedia.org/T270677) [03:14:26] (03PS1) 10Eileen: Remove never used Fredge, get_contact, get_banner etc exception types [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111731 (https://phabricator.wikimedia.org/T270677) [03:17:48] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1111732 [03:17:56] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1111732 (owner: 10Eileen) [03:18:44] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1111732 (owner: 10Eileen) [03:21:26] !log civicrm upgraded from 4206a253 to c671fd23 [03:21:29] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [03:25:13] 03Fundraising Sprint: A, 06Fundraising-Backlog, 10FR-donorservices: Activity field failing - https://phabricator.wikimedia.org/T383174#10465192 (10Eileenmcnaughton) @shust - https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=43432239&selectedChild=activity should be working for everyone now? [03:37:09] (03CR) 10CI reject: [V:04-1] Remove never used Fredge, get_contact, get_banner etc exception types [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111731 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [03:47:34] (03PS6) 10Eileen: Switch thankyou.send from apiv3 to v4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108536 (https://phabricator.wikimedia.org/T299096) [03:47:34] (03PS4) 10Eileen: Simplify thank you code in RecurringQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108870 (https://phabricator.wikimedia.org/T299096) [03:47:34] (03PS4) 10Eileen: Fold thank_you_send back into only caller [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108871 (https://phabricator.wikimedia.org/T299096) [03:47:35] (03PS5) 10Eileen: Update function signature to reflect callers [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108872 (https://phabricator.wikimedia.org/T299096) [03:47:36] (03PS4) 10Eileen: Consolidate handling of contributionID [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108873 (https://phabricator.wikimedia.org/T299096) [03:47:37] (03PS1) 10Eileen: Code comments - just some of my thoughts about exceptiosn [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111734 [03:51:50] (03Abandoned) 10Eileen: Remove never used Fredge, get_contact, get_banner etc exception types [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111731 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [03:56:40] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: PHP-queue predis backend is losing data when exit() is call in popAtomic callback - https://phabricator.wikimedia.org/T178104#10465285 (10Eileenmcnaughton) 05Open→03Resolved a:03Eileenmcnaughton a lot has changed since this [03:57:10] 14Fundraising Sprint YAMLton, the Musical, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07Unplanned-Sprint-Work: Ensure we are coping with deadlock Exceptions properly & our q doesn't corrupt at high volume, reduce unecessary potentially blocki... - https://phabricator.wikimedia.org/T182148#10465288 [04:33:16] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1301 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [04:38:16] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1601 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [04:43:16] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1902 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [04:45:35] ACKNOWLEDGEMENT - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1902 Dwisehaupt known - query chugging away writing to the log table https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [05:08:16] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 237 Threads: 4 Questions: 241283 Slow queries: 0 Opens: 952 Open tables: 946 Queries per second avg: 1018.071 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [11:34:43] (03CR) 10Jgleeson: [C:03+2] Move function to last remaining caller [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1110955 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [11:38:19] (03CR) 10Jgleeson: [C:03+2] Move last usage of function back to calling class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1110956 (owner: 10Eileen) [11:39:29] (03CR) 10Jgleeson: [C:03+2] Remove call to legacy WMF date function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111707 (owner: 10Eileen) [11:41:39] (03CR) 10Jgleeson: [C:03+2] "Nice." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111708 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [11:48:25] (03CR) 10Jgleeson: [C:03+2] Move getContributionStatus to the message class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111711 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [11:54:14] (03CR) 10Jgleeson: "b3d1b26b817bad1dfc14b1a8d936deb6040c83d2 will need rebasing over this :)" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111712 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:00:00] (03CR) 10Jgleeson: [C:03+2] Switch markRefund from using legacy function to message function (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111712 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:00:44] (03Merged) 10jenkins-bot: Move function to last remaining caller [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1110955 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:03:03] (03Merged) 10jenkins-bot: Move last usage of function back to calling class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1110956 (owner: 10Eileen) [12:03:03] (03CR) 10Jgleeson: [C:03+2] Remove unused variable [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111713 (owner: 10Eileen) [12:03:52] (03Merged) 10jenkins-bot: Remove call to legacy WMF date function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111707 (owner: 10Eileen) [12:07:32] (03Merged) 10jenkins-bot: Minor signature fixes [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111708 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:07:56] (03CR) 10Jgleeson: [C:03+2] Switch markRefund from using legacy function to message function (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111712 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:08:44] (03CR) 10Jgleeson: [C:03+2] Remove one of our legacy timestamp functions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111696 (owner: 10Eileen) [12:11:35] (03CR) 10Jgleeson: [C:03+2] Remove legacy constant [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111697 (owner: 10Eileen) [12:14:10] (03Merged) 10jenkins-bot: Move getContributionStatus to the message class [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111711 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:15:19] (03CR) 10Jgleeson: [C:03+2] Fold date back into only caller [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111698 (owner: 10Eileen) [12:21:07] (03CR) 10Jgleeson: [C:03+2] Simplify timestamp handling in contribution tracking [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111699 (owner: 10Eileen) [12:23:19] (03CR) 10Jgleeson: [C:03+2] Fix another place not to use legacy date function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111700 (owner: 10Eileen) [12:24:57] (03Merged) 10jenkins-bot: Switch markRefund from using legacy function to message function [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111712 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:29:11] (03CR) 10Jgleeson: [C:03+2] Remove last reference to one of our legacy date functions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111701 (owner: 10Eileen) [12:42:40] (03CR) 10Jgleeson: [C:03+2] Use Message->getDate() for all retrievals of message date (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111714 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:44:47] (03CR) 10Jgleeson: [C:03+2] "This one took me back to a kids show Oscar used to like. Stomp Stomp Stomp https://www.youtube.com/watch?v=-6w9CqSqx9E" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111729 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [12:46:47] (03CR) 10Jgleeson: [C:03+2] Remove WMFException type that is partially-used in outgoing code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111730 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [13:00:57] (03CR) 10Jgleeson: [C:03+2] Switch new address create to use apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1071707 (owner: 10Eileen) [13:09:30] (03Merged) 10jenkins-bot: Remove some never-used exception-type pseudoconstants [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111729 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [13:10:35] (03Merged) 10jenkins-bot: Remove WMFException type that is partially-used in outgoing code [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111730 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [13:26:53] (03Merged) 10jenkins-bot: Switch new address create to use apiv4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1071707 (owner: 10Eileen) [13:44:14] (03PS3) 10Jgleeson: Extract exchange conversion out and check it worked ok. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111704 [13:44:36] (03PS4) 10Jgleeson: Extract exchange conversion out. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111704 [13:45:52] (03PS1) 10Jgleeson: Remove unused exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112006 [14:02:06] (03PS1) 10Jgleeson: FAO eileen; Remove unused $refund_date. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112021 [14:08:28] (03PS1) 10Jgleeson: Remove unused paypal method. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112024 [14:20:21] (03PS1) 10Jgleeson: Add extra type hints to RefundQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112032 [15:18:06] (03CR) 10Damilare Adedoyin: [C:03+2] Switch thankyou.send from apiv3 to v4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108536 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [15:18:27] (03CR) 10Damilare Adedoyin: [C:03+2] "Thanks for all the work on this and for adding the test." [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108536 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [15:43:36] (03Merged) 10jenkins-bot: Switch thankyou.send from apiv3 to v4 [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108536 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [16:02:24] (03CR) 10Damilare Adedoyin: "Thank you for pushing this out, the test "testRecurringPaymentMonthlyConvertNotificationEmailSend" provides coverage for this part of the " [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108870 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [16:02:28] (03CR) 10Damilare Adedoyin: [C:03+2] Simplify thank you code in RecurringQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108870 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [16:27:49] (03Merged) 10jenkins-bot: Simplify thank you code in RecurringQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108870 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [16:41:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1489 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:43:50] (03PS2) 10Jgleeson: Run fraud filters for Adyen ApplePay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111333 (https://phabricator.wikimedia.org/T383352) [16:44:19] (03PS2) 10Jgleeson: Constantize app payment method strings in Adyen Adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111334 (https://phabricator.wikimedia.org/T383352) [16:44:23] (03PS2) 10Jgleeson: Constantize other payment method strings in Adyen Adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111335 (https://phabricator.wikimedia.org/T383352) [16:44:27] (03PS2) 10Jgleeson: To Run, RunDo or DoRun. That is the question. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111339 [16:46:10] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1789 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:51:10] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2090 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [16:56:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2389 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:01:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2689 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:06:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2990 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:11:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3289 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:16:08] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3589 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:21:10] PROBLEM - check_mysql on frdb2004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:26:09] RECOVERY - check_mysql on frdb2004 is OK: Uptime: 141 Threads: 4 Questions: 74302 Slow queries: 0 Opens: 984 Open tables: 976 Queries per second avg: 526.964 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [17:26:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16004 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:28:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16126 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [17:31:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16304 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:33:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16426 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [17:36:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16604 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:38:17] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16726 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [17:41:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 16904 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:43:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17026 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [17:46:13] PROBLEM - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17204 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [17:48:03] ACKNOWLEDGEMENT - check_mysql on frdb1004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17026 Dwisehaupt known - tracking down issues with replication - acking to keep irc noise down. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [17:48:04] ACKNOWLEDGEMENT - check_mysql on frdb2005 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 17204 Dwisehaupt known - tracking down issues with replication - acking to keep irc noise down. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql [18:33:15] (03CR) 10Wfan: [C:03+2] Refactor: add type hints to adyen expatriated message & action files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109489 (owner: 10Jgleeson) [18:33:41] (03CR) 10Wfan: [C:03+2] Refactor: add type hints to adyen jobs files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109492 (owner: 10Jgleeson) [18:33:47] (03Merged) 10jenkins-bot: Refactor: add type hints to adyen expatriated message & action files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109489 (owner: 10Jgleeson) [18:34:08] (03Merged) 10jenkins-bot: Refactor: add type hints to adyen jobs files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109492 (owner: 10Jgleeson) [18:34:15] ty wfan !!! [18:34:36] (03CR) 10Wfan: [C:03+2] Refactor: add type hints to adyen maintenance files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109493 (owner: 10Jgleeson) [18:34:56] (03CR) 10Wfan: [C:03+2] Refactor: add type hints to adyen audit and rest listener files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109494 (owner: 10Jgleeson) [18:35:03] (03Merged) 10jenkins-bot: Refactor: add type hints to adyen maintenance files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109493 (owner: 10Jgleeson) [18:35:22] (03CR) 10Wfan: [C:03+2] Remove unused exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112006 (owner: 10Jgleeson) [18:35:24] (03Merged) 10jenkins-bot: Refactor: add type hints to adyen audit and rest listener files [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1109494 (owner: 10Jgleeson) [18:36:28] (03CR) 10Wfan: [C:03+2] Add extra type hints to RefundQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112032 (owner: 10Jgleeson) [18:37:06] (03CR) 10Wfan: [C:03+2] Extract exchange conversion out. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111704 (owner: 10Jgleeson) [18:45:49] 03Fundraising Sprint: A, 06Fundraising-Backlog, 10FR-donorservices: Activity field failing - https://phabricator.wikimedia.org/T383174#10467840 (10SHust) I'm happy to confirm that we can see the activities for the CIDs shared here. Thanks a million, @Eileenmcnaughton, for the speedy fix! [18:59:34] (03Merged) 10jenkins-bot: Extract exchange conversion out. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111704 (owner: 10Jgleeson) [18:59:48] (03Merged) 10jenkins-bot: Remove unused exceptions [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112006 (owner: 10Jgleeson) [19:01:01] (03CR) 10Wfan: [C:03+2] FAO eileen; Remove unused $refund_date. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112021 (owner: 10Jgleeson) [19:01:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1461 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:01:15] (03CR) 10Wfan: [C:03+2] Remove unused paypal method. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112024 (owner: 10Jgleeson) [19:02:16] np~ thanks for those fixes! [19:03:48] (03CR) 10Wfan: [C:03+2] Ensure payment reconciliation id isn't wiped off by approve payment response [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111220 (owner: 10Damilare Adedoyin) [19:06:02] (03Merged) 10jenkins-bot: Ensure payment reconciliation id isn't wiped off by approve payment response [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111220 (owner: 10Damilare Adedoyin) [19:06:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 1761 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:07:27] (03CR) 10Wfan: [C:03+1] "LGTM" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108871 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [19:10:13] (03CR) 10Wfan: "I think it's ok to keep them, for us to manual testing a different email template?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108872 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [19:11:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2061 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:12:20] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 06Product-Analytics, 06Web-Team, 07SEO: CentralNotice severely impacts CLS score - https://phabricator.wikimedia.org/T318046#10467950 (10Jdlrobson) 05Open→03Resolved a:03Jdlrobson The web team discussed this in our grooming meeting. S... [19:16:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2361 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:21:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2661 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:26:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 2961 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:27:18] (03Merged) 10jenkins-bot: FAO eileen; Remove unused $refund_date. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112021 (owner: 10Jgleeson) [19:27:20] (03Merged) 10jenkins-bot: Remove unused paypal method. [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112024 (owner: 10Jgleeson) [19:27:22] (03Merged) 10jenkins-bot: Add extra type hints to RefundQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112032 (owner: 10Jgleeson) [19:31:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3261 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:36:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3561 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:36:13] thanks for all the review wfan ! [19:37:46] (03PS3) 10Jgleeson: Run fraud filters for Adyen ApplePay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111333 (https://phabricator.wikimedia.org/T383352) [19:37:46] (03PS3) 10Jgleeson: Constantize app payment method strings in Adyen Adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111334 (https://phabricator.wikimedia.org/T383352) [19:37:46] (03PS3) 10Jgleeson: Constantize other payment method strings in Adyen Adapter [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111335 (https://phabricator.wikimedia.org/T383352) [19:37:47] (03PS3) 10Jgleeson: To Run, RunDo or DoRun. That is the question. [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1111339 [19:41:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 3861 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:46:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4161 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:47:42] Np Jack, they all looks good! Thanks for the works u did and may u have a good night :) [19:51:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4461 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:53:33] PROBLEM - Host frdb1004 is DOWN: PING CRITICAL - Packet loss = 100% [19:56:11] PROBLEM - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4762 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:58:15] RECOVERY - check_mysql on frdb1004 is OK: Uptime: 153 Threads: 4 Questions: 4321 Slow queries: 0 Opens: 952 Open tables: 946 Queries per second avg: 28.241 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [19:58:15] RECOVERY - Host frdb1004 is UP: PING OK - Packet loss = 0%, RTA = 0.41 ms [19:58:56] ACKNOWLEDGEMENT - check_mysql on frdb2004 is CRITICAL: SLOW_SLAVE CRITICAL: Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 4762 Dwisehaupt known - working on it. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [19:59:28] (03PS1) 10Jgleeson: Refactor: Add type hints to QueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112085 [20:12:14] (03PS1) 10Jgleeson: Refactor: Add type hints to DonationQueueConsumer [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1112087 [21:00:59] 06Fundraising-Backlog: Queue consume failed with code 137 - (ERROR Killed) on the donations queue consumer and recurring queue consumer - https://phabricator.wikimedia.org/T383937 (10Cstone) 03NEW [21:09:31] 06Fundraising-Backlog: Adyen recurring failmails being labelled as Paypal - https://phabricator.wikimedia.org/T383940 (10Cstone) 03NEW [21:09:43] 06Fundraising-Backlog: Adyen recurring failmails being labelled as Paypal - https://phabricator.wikimedia.org/T383940#10468500 (10Cstone) [21:41:44] (03CR) 10Eileen: "wfa" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1108871 (https://phabricator.wikimedia.org/T299096) (owner: 10Eileen) [21:43:17] (03CR) 10Eileen: Use Message->getDate() for all retrievals of message date (031 comment) [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1111714 (https://phabricator.wikimedia.org/T270677) (owner: 10Eileen) [22:36:09] RECOVERY - check_mysql on frdb2004 is OK: Uptime: 249 Threads: 3 Questions: 1589 Slow queries: 0 Opens: 940 Open tables: 934 Queries per second avg: 6.381 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [22:46:13] RECOVERY - check_mysql on frdb2005 is OK: Uptime: 57 Threads: 3 Questions: 316 Slow queries: 0 Opens: 923 Open tables: 917 Queries per second avg: 5.543 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2005&service=check_mysql