[00:08:51] FIRING: FermMSS: Unexpected MSS value on 10.2.1.27:80 @ ms-fe2015 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=codfw&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS [00:13:51] RESOLVED: FermMSS: Unexpected MSS value on 10.2.1.27:80 @ ms-fe2015 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=codfw&var-cluster=swift - https://alerts.wikimedia.org/?q=alertname%3DFermMSS [02:26:06] FIRING: [2x] FermMSS: Unexpected MSS value on 10.2.2.38:8082 @ druid1012 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=druid_public - https://alerts.wikimedia.org/?q=alertname%3DFermMSS [06:22:08] 06Traffic, 06Data-Engineering, 06SRE: Add pageview information to turnilo's webrequest_sampled_live (is_pageview is always "-") - https://phabricator.wikimedia.org/T402612#11175061 (10JAllemandou) >>! In T402612#11174218, @CDanis wrote: > I can find time to reimplement the 'true' logic (or maybe a very sligh... [06:26:07] FIRING: [2x] FermMSS: Unexpected MSS value on 10.2.2.38:8082 @ druid1012 - https://wikitech.wikimedia.org/wiki/LVS#LVSRealserverMSS_alert - https://grafana.wikimedia.org/d/Y9-MQxNSk/ipip-encapsulated-services?orgId=1&viewPanel=4&var-site=eqiad&var-cluster=druid_public - https://alerts.wikimedia.org/?q=alertname%3DFermMSS [06:34:08] Acked and silenced the 'FermMSS: Unexpected MSS value on 10.2.2.38:8082 @ druid1012' error above. The host is still in setup on T397441 [06:34:09] T397441: Bring druid101[2-3] online to replace druid100[7-8] - https://phabricator.wikimedia.org/T397441 [08:39:16] stevemunene: ack [08:39:51] XioNoX: https://grafana.wikimedia.org/goto/TBCTvzjNR?orgId=1 did we have some maintenance on the Arelion transport from eqsin to codfw? [08:47:48] vgutierrez: nothing planned or declared by Arelion at least [09:04:49] 06Traffic, 10HaproxyKafka, 10Data-Platform-SRE (2025.09.05 - 2025.09.26), 07Essential-Work, 13Patch-For-Review: Replicate current low-message alerting from VarnishKafka - https://phabricator.wikimedia.org/T391810#11175407 (10BTullis) Hi @Fabfur - Just as an interesting data point, I noticed that we r... [11:30:32] FIRING: SLOMetricAbsent: haproxy-combined - https://slo.wikimedia.org/?search=haproxy-combined - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent [11:35:32] RESOLVED: SLOMetricAbsent: haproxy-combined - https://slo.wikimedia.org/?search=haproxy-combined - https://alerts.wikimedia.org/?q=alertname%3DSLOMetricAbsent [12:29:47] 10netops, 06Infrastructure-Foundations, 06SRE, 13Patch-For-Review: Move management routers ssh port - https://phabricator.wikimedia.org/T277438#11175906 (10ayounsi) 05Open→03Declined Feel free to re-open if you disagree, but looks like we might not need to get to that heavy port (and tooling) chang... [14:00:26] 06Traffic, 06MediaWiki-Platform-Team, 06Reader Experience Team, 10MobileFrontend (Core PHP), 13Patch-For-Review: Toggling desktop view doesn't toggle user back into mobile mode - https://phabricator.wikimedia.org/T403866#11176278 (10Krinkle) The above patch is now live in the Beta Cluster for all its wik... [14:57:29] 06Traffic, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q4:rack/setup/install cp20[43-58] codfw - https://phabricator.wikimedia.org/T392851#11176523 (10Jhancock.wm) @elukey correct, i hadn't set those up yet. How many of the servers do you want ips setup on? I'm still trying to leave some untouched... [15:01:41] 06Traffic, 06DC-Ops, 10ops-codfw, 06SRE, 13Patch-For-Review: Q4:rack/setup/install cp20[43-58] codfw - https://phabricator.wikimedia.org/T392851#11176560 (10elukey) >>! In T392851#11176523, @Jhancock.wm wrote: > @elukey correct, i hadn't set those up yet. How many of the servers do you want ips setup on?... [16:09:45] 06Traffic, 06SRE: Setting up Wikimedia Trust and Safety Help Center with Zendesk product: Seeking Guidance on host mapping - https://phabricator.wikimedia.org/T400952#11176819 (10BCornwall) 05Open→03In progress p:05Triage→03Low a:03BCornwall [16:10:29] 06Traffic, 10DNS: Setting up Wikimedia Trust and Safety Help Center with Zendesk product: Seeking Guidance on host mapping - https://phabricator.wikimedia.org/T400952#11176826 (10BCornwall) [17:20:44] 10Acme-chief, 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible, 13Patch-For-Review: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11177103 (10BCornwall) The unit has run and cleanu... [18:57:53] 06Traffic: 403 errors with user-agent - https://phabricator.wikimedia.org/T404219#11177388 (10Lupascriptix) It seems T402959 is a bit quiet - haven't gotten any responses there. I suspect that SPARQL is the issue with the OpenRefine reconciliation since the non-SPARQL ones are working fine, but the python script... [20:28:06] 10Acme-chief, 10Beta-Cluster-Infrastructure, 07Beta-Cluster-reproducible: Warning about /etc/acmecerts/unified contents during puppet run on deployment-cache-text08 & deployment-cache-upload08 - https://phabricator.wikimedia.org/T399419#11177647 (10bd808) >>! In T399419#11177103, @BCornwall wrote: > The unit... [21:50:28] 06Traffic, 06MediaWiki-Platform-Team, 06Reader Experience Team, 10MobileFrontend (Core PHP), 13Patch-For-Review: Toggling desktop view doesn't toggle user back into mobile mode - https://phabricator.wikimedia.org/T403866#11177880 (10Etonkovidova) >>! In T403866#11176278, @Krinkle wrote: > The above patch... [23:20:56] 10netops, 10fundraising-tech-ops, 06Infrastructure-Foundations: Move pfw1b-codfw to rack F5 - https://phabricator.wikimedia.org/T401297#11178008 (10Papaul) We tested the last 2 cross cage links for the frack migration and all is working now. We are ready for the move on the 22nd.