[01:26:06] <wikibugs>	 (03PS9) 10Umherirrender: build: Updating dependencies [extensions/CentralNotice] - 10https://gerrit.wikimedia.org/r/1246163 (owner: 10Libraryupgrader)
[02:13:00] <wikibugs>	 (03PS5) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[02:13:32] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628) (owner: 10Eileen)
[02:30:21] <wikibugs>	 (03PS6) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[02:56:35] <wikibugs>	 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` - https://phabricator.wikimedia.org/T420057#11710797 (10BucheonFC) 05Open→03Resolved a:03BucheonFC
[03:18:05] <wikibugs>	 06Fundraising-Backlog, 10Wikimedia-Fundraising-CiviCRM, 10FR-WMF-Audit: Add backend_processor_id & payment_reconciliation_id fields to `smashpig.pending` - https://phabricator.wikimedia.org/T420057#11710882 (10AlexisJazz) 05Resolved→03Open a:05BucheonFC→03None
[03:24:59] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[04:02:40] <wikibugs>	 (03PS7) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[04:03:14] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628) (owner: 10Eileen)
[04:14:21] <wikibugs>	 (03PS8) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[04:15:12] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628) (owner: 10Eileen)
[04:17:23] <wikibugs>	 (03PS9) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[04:49:49] <wikibugs>	 (03PS10) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[04:53:03] <wikibugs>	 (03PS11) 10Eileen: Initial Stripe SmashPig commit [wikimedia/fundraising/SmashPig] - 10https://gerrit.wikimedia.org/r/1251207 (https://phabricator.wikimedia.org/T419628)
[05:11:32] <wikibugs>	 (03PS1) 10Eileen: Initial Stripe Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1252306 (https://phabricator.wikimedia.org/T419628)
[05:35:38] <wikibugs>	 (03CR) 10CI reject: [V:04-1] Initial Stripe Audit Processor [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1252306 (https://phabricator.wikimedia.org/T419628) (owner: 10Eileen)
[06:17:13] <icinga-wm>	 PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 97% used. Largest process: trino-server-co (1563) = 95.9% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory
[06:22:13] <icinga-wm>	 PROBLEM - check_memory on fransw2002 is CRITICAL: CRIT Memory 97% used. Largest process: trino-server-co (1563) = 95.9% https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory
[06:32:13] <icinga-wm>	 RECOVERY - check_memory on fransw2002 is OK: OK Memory 79% used https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=fransw2002&service=check_memory
[07:24:59] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[11:02:53] <wikibugs>	 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for dmorgan-ctr - https://phabricator.wikimedia.org/T415403#11711319 (10WMFOfffce) 05Resolved→03In progress
[11:03:21] <wikibugs>	 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for rholt - https://phabricator.wikimedia.org/T415400#11711322 (10WMFOfffce) 05Resolved→03Open p:05Triage→03Medium
[11:04:07] <wikibugs>	 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for rholt - https://phabricator.wikimedia.org/T415400#11711324 (10Johannnes89) 05Open→03Resolved p:05Medium→03Triage
[11:05:23] <wikibugs>	 06Fundraising-Backlog, 10fundraising-tech-ops: Issue new SSL Client Certificate for dmorgan-ctr - https://phabricator.wikimedia.org/T415403#11711327 (10Nemoralis) 05In progress→03Resolved
[11:24:59] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[15:24:59] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[19:24:59] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[19:40:15] <wikibugs>	 (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1252762
[19:40:29] <wikibugs>	 (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1252762 (owner: 10Eileen)
[19:41:32] <wikibugs>	 (03Merged) 10jenkins-bot: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1252762 (owner: 10Eileen)
[19:42:47] <wikibugs>	 (03PS1) 10Umherirrender: Add @throws documentation and enable PhanThrowTypeAbsent [extensions/DonationInterface] - 10https://gerrit.wikimedia.org/r/1252767
[19:43:37] <eileen>	 !log civicrm upgraded from a1279ee4 to dce257f0
[19:43:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL
[23:25:00] <jinxer-wm>	 FIRING: CoreRouterInterfaceDown: Core router interface down - pfw1-codfw:reth2 (fasw1-f5 2x25G) - https://wikitech.wikimedia.org/wiki/Network_monitoring#Router_interface_down - https://grafana.wikimedia.org/d/fb403d62-5f03-434a-9dff-bd02b9fff504/network-device-overview?var-instance=pfw1-codfw:9804 - https://alerts.wikimedia.org/?q=alertname%3DCoreRouterInterfaceDown
[23:33:56] <wikibugs>	 (03PS1) 10Eileen: Opps - remove accidentally committed debug [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1252946
[23:34:07] <wikibugs>	 (03CR) 10Eileen: [C:03+2] Opps - remove accidentally committed debug [wikimedia/fundraising/crm] - 10https://gerrit.wikimedia.org/r/1252946 (owner: 10Eileen)
[23:53:58] <wikibugs>	 (03PS1) 10Eileen: Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1252952
[23:54:09] <wikibugs>	 (03CR) 10Eileen: [C:03+2] Merge branch 'master' of https://gerrit.wikimedia.org/r/wikimedia/fundraising/crm into deployment [wikimedia/fundraising/crm] (deployment) - 10https://gerrit.wikimedia.org/r/1252952 (owner: 10Eileen)
[23:56:37] <eileen>	 !log civicrm upgraded from dce257f0 to 3283e3ca
[23:56:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Fundraising/SAL