[00:00:29] (03PS19) 10Jdlrobson: Introduce the Donate skin [extensions/FundraiserLandingPage] - 10https://gerrit.wikimedia.org/r/1020408 (https://phabricator.wikimedia.org/T257862) [00:07:15] 06Fundraising-Backlog: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10315094 (10Eileenmcnaughton) Just noting that this field was added to Acoustic the maintenance window before last on the expectation we needed to do the field add when we had the window - but ho... [00:19:05] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07FR-Imports: Engage import_Matching - Ind contact import *updated - https://phabricator.wikimedia.org/T377605#10315104 (10Eileenmcnaughton) [00:19:47] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 07FR-Imports: Engage import_Matching - Ind contact import *updated - https://phabricator.wikimedia.org/T377605#10315105 (10Eileenmcnaughton) 05Open→03Resolved a:03Eileenmcnaughton [00:23:56] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Make cancellation reason available to Acoustic - https://phabricator.wikimedia.org/T379206#10315109 (10Eileenmcnaughton) @MSuijkerbuijk_WMF - I'm a bit confused about the meeting cos it was in my calendar - but not for yest... [00:25:03] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-donorservices: Civi merge is not saving the state when a mailing address is present. - https://phabricator.wikimedia.org/T379684#10315111 (10Eileenmcnaughton) [00:29:07] (03PS1) 10Ejegg: Update symfony/http-foundation [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1090574 [00:35:20] !log payments-wiki upgraded from 7d24a942 to 459f259b [00:35:21] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [00:53:02] ejegg|food: if you come back I'd like to pick your brain on something [02:13:07] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702 (10Eileenmcnaughton) 03NEW [02:17:19] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315220 (10Eileenmcnaughton) [02:35:18] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315247 (10Eileenmcnaughton) [02:47:18] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315255 (10Eileenmcnaughton) [03:07:15] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315261 (10Eileenmcnaughton) [03:08:54] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315273 (10Eileenmcnaughton) [04:48:18] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315294 (10Eileenmcnaughton) [04:50:17] 03Fundraising Sprint: void(), 06Fundraising-Backlog: Develop process for tracking & handling consent records - https://phabricator.wikimedia.org/T379702#10315295 (10Eileenmcnaughton) Copying in notes from elsewhere about flows - not sure how useful they are just yet To think of a work flow 1) A person fill... [04:58:10] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 10FR-Tech-Analytics, 10FR-tech-data-integrity: Non Campaign Emails in Database - https://phabricator.wikimedia.org/T361621#10315296 (10Eileenmcnaughton) Just updating this to note I asked Brian about this and here is what is going on - normal email sen... [05:08:07] (03CR) 10Eileen: [C:03+2] "yep - ran it on staging & looks good" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1088625 (https://phabricator.wikimedia.org/T379013) (owner: 10Ejegg) [05:09:46] (03PS1) 10Eileen: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1090591 [05:10:39] (03CR) 10Eileen: [C:03+2] Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1090591 (owner: 10Eileen) [05:11:27] (03Merged) 10jenkins-bot: Merge branch 'master' of ssh://gerrit.wikimedia.org:29418/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1090591 (owner: 10Eileen) [05:17:37] !log civicrm upgraded from ad008134 to 23e08fc2 [05:17:39] Logged the message at https://wikitech.wikimedia.org/wiki/Server_Admin_Log [07:08:47] 06Fundraising-Backlog, 10fundraising-tech-ops, 10FR-Tech-Analytics: swap analytics db origin from frdb1003 to frdb2003 - https://phabricator.wikimedia.org/T378859#10315405 (10Dwisehaupt) [07:12:09] RECOVERY - check_mysql on frdb2003 is OK: Uptime: 2098 Threads: 6 Questions: 1042464 Slow queries: 0 Opens: 1456 Open tables: 1370 Queries per second avg: 496.884 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2003&service=check_mysql [07:16:01] 03Fundraising Sprint: void(), 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Make cancellation reason available to Acoustic - https://phabricator.wikimedia.org/T379206#10315416 (10MSuijkerbuijk_WMF) @Eileenmcnaughton Thursday at that time I have a doctor appointment, I could meet Friday or Monday. T... [08:06:09] PROBLEM - check_mysql on frdb2004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb2004&service=check_mysql [08:08:02] just acked that. it's a known clone. [08:08:37] well, put in a downtime, not ack;d [13:18:30] 14fundraising Sprint Grep works IRL, 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 07Epic: EPIC: Find alternative to Special:HideBanners cookies to mitigate the loss of 3rd-party cookie support - https://phabricator.wikimedia.org/T244699#10316490 (10Tgr) [13:35:20] 06Fundraising-Backlog: Suppress Annual recurring donors from January receipt. Set up a receipt right after the yearly donation is processed - https://phabricator.wikimedia.org/T379751 (10MSuijkerbuijk_WMF) 03NEW [14:49:58] (03PS4) 10Damilare Adedoyin: WIP: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) [14:50:27] (03CR) 10CI reject: [V:04-1] WIP: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [14:52:23] (03PS5) 10Damilare Adedoyin: WIP: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) [14:52:56] 06Fundraising-Backlog: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10317029 (10JMando) Hey there, so I would maybe do something like this (contribution tracking fields): ` CASE WHEN utm_medium IN ('email', 'endowment','MGEventEmail', 'MGAppeal','MGAnnual','... [15:13:42] (03PS6) 10Damilare Adedoyin: WIP: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) [15:33:58] (03PS5) 10Damilare Adedoyin: Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) [15:35:50] (03CR) 10CI reject: [V:04-1] Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [15:36:05] (03PS7) 10Damilare Adedoyin: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) [15:49:41] 06Fundraising-Backlog, 07fr-direct-mail: Direct mail returns not linked to Acoustic - https://phabricator.wikimedia.org/T378717#10317447 (10MSuijkerbuijk_WMF) Hi @Eileenmcnaughton - @MDemosWMF and I were talking and thought we'd share the updated [[ https://docs.google.com/spreadsheets/d/12E36Pprlr6aZpX45m1Ene... [16:33:44] (03PS1) 10Wfan: create two new options in the communications area - No Direct Mail and No Paper TY [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1090895 (https://phabricator.wikimedia.org/T374720) [16:40:12] (03CR) 10Wfan: [C:03+2] "LGTM" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1089971 (owner: 10Eileen) [16:44:02] (03CR) 10Wfan: "I think we use 'custom_' . \CRM_Core_BAO_CustomField::getCustomFieldID($field_name, $group_name); at the other patch, wanna tie here?" [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1089970 (owner: 10Eileen) [16:45:07] (03PS6) 10Damilare Adedoyin: Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) [16:50:46] (03PS8) 10Damilare Adedoyin: Setup Gravy GooglePay payment provider class [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1084829 (https://phabricator.wikimedia.org/T367806) [16:50:52] (03CR) 10CI reject: [V:04-1] Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [16:59:26] (03PS1) 10Damilare Adedoyin: Add gravy google pay test links [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1090909 [17:00:57] (03PS2) 10Damilare Adedoyin: Add gravy google pay test links [wikimedia/fundraising/dev] - 10https://gerrit.wikimedia.org/r/1090909 [17:47:45] 10fundraising-tech-ops: Fundraising access request for skivlehan - https://phabricator.wikimedia.org/T378541#10318171 (10SKivlehan-WMF) Yubikey public key: ` cccccbrkvikn ` Key-pair: ` ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOu9fUk0dyjON264Xkv8mCi5j988CICGuUtT0KKNHebD skivlehan@wikimedia.org ` [18:16:11] PROBLEM - check_memory on fran1001 is CRITICAL: CRIT Memory 95% used. Largest process: python3 (1342406) = 58.3% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_memory [18:26:47] 10fundraising-tech-ops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 2 others: Frack eqiad network upgrade: design, installation and configuration - https://phabricator.wikimedia.org/T377381#10318425 (10cmooney) 05Open→03Resolved @Jclark-ctr I've erased the config on all the old devices no... [18:28:59] 10fundraising-tech-ops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 2 others: Q1:eqiad:frack network upgrade tracking task - https://phabricator.wikimedia.org/T371435#10318437 (10cmooney) @robh the migration work is now done, all that remains is to remove the old devices and any cables connecting... [18:35:59] (03CR) 10Eileen: [C:03+2] create two new options in the communications area - No Direct Mail and No Paper TY [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1090895 (https://phabricator.wikimedia.org/T374720) (owner: 10Wfan) [18:36:13] PROBLEM - check_memory on fran1001 is CRITICAL: CRIT Memory 95% used. Largest process: python3 (1342406) = 59.2% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_memory [18:36:38] 10fundraising-tech-ops, 06DC-Ops, 06Infrastructure-Foundations, 10netops, and 2 others: Q1:eqiad:frack network upgrade tracking task - https://phabricator.wikimedia.org/T371435#10318507 (10RobH) a:03Jclark-ctr I'd hand this over to either John or Valerie as ops-eqiad for them to remove any devices and ca... [18:40:26] discussion on tests https://softwareengineering.stackexchange.com/questions/455588/would-a-middle-ground-between-unit-and-integration-tests-be-optimal?utm_source=iterable&utm_medium=email&utm_campaign=the-overflow-newsletter [18:41:13] PROBLEM - check_memory on fran1001 is CRITICAL: CRIT Memory 96% used. Largest process: python3 (1342406) = 60.2% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_memory [18:43:36] ACKNOWLEDGEMENT - check_memory on fran1001 is CRITICAL: CRIT Memory 96% used. Largest process: python3 (1342406) = 60.2% Dwisehaupt known - testing. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fran1001&service=check_memory [18:57:18] (03Merged) 10jenkins-bot: create two new options in the communications area - No Direct Mail and No Paper TY [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1090895 (https://phabricator.wikimedia.org/T374720) (owner: 10Wfan) [19:01:53] (03PS7) 10Damilare Adedoyin: Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) [19:03:54] (03CR) 10CI reject: [V:04-1] Setup Gravy Google Pay [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1084823 (https://phabricator.wikimedia.org/T367806) (owner: 10Damilare Adedoyin) [19:16:44] 06Fundraising-Backlog, 07fr-direct-mail: Direct mail returns not linked to Acoustic - https://phabricator.wikimedia.org/T378717#10318729 (10Eileenmcnaughton) Hi @MSuijkerbuijk_WMF in terms of how we would store the information in CiviCRM I think we would ideally create an activity against each contact who is... [19:23:44] 06Fundraising-Backlog: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10318785 (10Eileenmcnaughton) @JMando so that doesn't feel great as an approach. It looks to me like the field 'channel' that Nora was working on was intended for this purpose - but we do not se... [19:33:31] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10318866 (10Eileenmcnaughton) [19:33:41] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10318867 (10JMando) Totally fair. I was actually just looking at civicrm_value_1_gift_data_7.channel . If that was fully populated, analytics could rely on it... [19:35:31] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10318872 (10Eileenmcnaughton) @JMando what could possibly be not to love about that sql :-) Re "had more "channels." - have you got a full list? The list abo... [19:41:54] 06Fundraising-Backlog, 07fr-direct-mail: Direct mail returns not linked to Acoustic - https://phabricator.wikimedia.org/T378717#10318930 (10JMando) @Eileenmcnaughton I am exporting two files for Faircom each month. A transactional file (donations from the last month for US donors) as well as a contacts data ta... [19:51:52] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10318972 (10JMando) @Eileenmcnaughton I could probably generate a list that works for reporting, but I am wondering if we should loop in some more people and st... [19:53:06] 06Fundraising-Backlog, 07fr-direct-mail: Direct mail returns not linked to Acoustic - https://phabricator.wikimedia.org/T378717#10318989 (10Eileenmcnaughton) @JMando ok - so whoever deals with Faircom - (@MDemosWMF ? ) will need to get them to provide a file of the contact IDs who receive mailings if we want t... [19:57:49] 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Populating both_funds_latest_donation_source field - https://phabricator.wikimedia.org/T379700#10319028 (10Eileenmcnaughton) @JMando I agree we should figure out who owns this! Maybe @AKanji-WMF can help? But I think it's worth seeing what your list looks... [20:17:40] 10fundraising-tech-ops: enable SSL database connection for fundraising superset - https://phabricator.wikimedia.org/T279058#10319168 (10Jgreen) [20:18:09] 10fundraising-tech-ops: enable SSL database connection for fundraising superset - https://phabricator.wikimedia.org/T279058#10319170 (10Jgreen) [21:28:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:33:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:38:13] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:43:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:48:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:53:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [21:58:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [22:03:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [22:08:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [22:13:15] PROBLEM - check_mysql on frdb1004 is CRITICAL: Cant connect to local server through socket /var/run/mysqld/mysqld.sock (2) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1004&service=check_mysql [22:14:03] oh. looks like the ack expired. [22:14:09] sorry about the spew. [22:15:26] downtime added. [22:20:47] 06Fundraising-Backlog: Connection to Gr4vy's consolidated output file - https://phabricator.wikimedia.org/T379837 (10EMartin) 03NEW [22:21:00] 06Fundraising-Backlog: Connection to Gr4vy's consolidated output file into Civi - https://phabricator.wikimedia.org/T379837#10319556 (10EMartin) [22:57:13] PROBLEM - check_mysql on frdb1003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_mysql [23:02:10] PROBLEM - check_mysql on frdb1003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_mysql [23:02:58] ACKNOWLEDGEMENT - check_mysql on frdb1003 is CRITICAL: Slave IO: Yes Slave SQL: No Seconds Behind Master: (null) Dwisehaupt investigating. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_mysql [23:07:14] RECOVERY - check_mysql on frdb1003 is OK: Uptime: 59380 Threads: 17 Questions: 24469628 Slow queries: 81 Opens: 2262 Open tables: 1658 Queries per second avg: 412.085 Slave IO: Yes Slave SQL: Yes Seconds Behind Master: 0 https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=frdb1003&service=check_mysql [23:09:28] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Slow thank you email - https://phabricator.wikimedia.org/T377906#10319735 (10Eileenmcnaughton) 05Open→03Resolved upstream now merged [23:09:30] 06Fundraising Tech - Chaos Crew, 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM: Slow thank you email - https://phabricator.wikimedia.org/T377906#10319737 (10Eileenmcnaughton) [23:24:18] 06Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice, 06Product-Analytics, 10Web-Team, 07SEO: CentralNotice severely impacts CLS score - https://phabricator.wikimedia.org/T318046#10319909 (10Jdlrobson) [23:40:10] PROBLEM - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [23:40:56] PROBLEM - Host frpig1002 is DOWN: PING CRITICAL - Packet loss = 100% [23:41:12] RECOVERY - Host frpig1002 is UP: PING OK - Packet loss = 0%, RTA = 0.84 ms [23:45:10] PROBLEM - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [23:50:08] PROBLEM - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker [23:51:26] let me ack that coworker alert. we need a better way than a process check to notify on that. [23:51:42] ACKNOWLEDGEMENT - check_coworker on civi1002 is CRITICAL: PROCS CRITICAL: 1 process with args php /srv/coworker/bin/coworker Dwisehaupt known https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=civi1002&service=check_coworker