[00:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:19:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [00:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:14:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [01:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:14:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [02:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:14:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [03:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:19:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [04:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:19:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [05:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:14:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:34:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:39:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:39:23] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=eqiad%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [06:44:12] FIRING: [3x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [06:44:18] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=eqiad%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [06:54:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [07:04:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [07:09:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [07:14:12] FIRING: [4x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [07:24:12] RESOLVED: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [07:49:52] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 10Event-Platform: EventStreams: duplicate events from double compute (wdqs/rdf) streams - https://phabricator.wikimedia.org/T396564#11059763 (10dcausse) The eventstream config looks correct for eqiad (accessing its configmap via kubec... [07:58:43] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 10Event-Platform: EventStreams: duplicate events from double compute (wdqs/rdf) streams - https://phabricator.wikimedia.org/T396564#11059771 (10dcausse) Given the name `stream_config_defaults` perhaps that was the intent? Should we ha... [08:18:15] 06Data-Engineering, 10Pageviews-Anomaly, 07SecTeam-Processed, 07Security: Views data integrity compromised by entity running up fake views - https://phabricator.wikimedia.org/T366554#11059827 (10Cixtpide) >>! In T366554#11056356, @Stevietheman wrote: > @Cixtpide Are you subscribed to any particular newslet... [08:34:12] FIRING: AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [08:44:12] FIRING: [2x] AlertLintProblem: Linting problems found for HaproxyKafkaDeliveryErrors - https://wikitech.wikimedia.org/wiki/Alertmanager#Alert_linting_found_problems - TODO - https://alerts.wikimedia.org/?q=alertname%3DAlertLintProblem [09:17:36] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Change RefererClassifier to capture malformed but internal referer strings as internal (wmf.pageview_actor) - https://phabricator.wikimedia.org/T400799#11060006 (10JAllemandou) I don't think we use the `referer` classification for anything else than repo... [09:18:12] 06Data-Engineering, 10Data-Platform-SRE (2025.07.26 - 2025.08.15): Ensure that blunderbuss uses the minimum HDFS file system permissions required - https://phabricator.wikimedia.org/T401103#11060008 (10BTullis) a:03BTullis [11:52:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Spike: Assess feasibility of integrating the new file export with the legacy Dumps1 UI - https://phabricator.wikimedia.org/T400507#11060471 (10BTullis) I have a question here, in response to this part: >>! In T400507#11051996, @xcollazo wrote: > A `Worke... [11:57:54] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Spike: Assess feasibility of integrating the new file export with the legacy Dumps1 UI - https://phabricator.wikimedia.org/T400507#11060490 (10BTullis) Here's another idea for a breaking change that might be a relatively easy option, both for us and for... [13:03:07] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 10Event-Platform: EventGate: Add Prometheus metric for hoisting errors - https://phabricator.wikimedia.org/T398922#11060648 (10dr0ptp4kt) And, as expected, it's working in prod fine for `HoistingError` and `MalformedHeaderError... [13:07:32] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 10Event-Platform: EventGate: Log unparseable X-Experiment-Enrollments headers to an error stream - https://phabricator.wikimedia.org/T396359#11060668 (10dr0ptp4kt) As noted in T398922#11060648 it's working prod. Thanks @Ottomat... [13:26:21] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform, 10MW-1.45-notes (1.45.0-wmf.12; 2025-07-29), 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#11060702 (10Ottomata) [ApiMain - don't set meta.dt on mediawiki.api-r... [13:26:31] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform, 10MW-1.45-notes (1.45.0-wmf.12; 2025-07-29), 13Patch-For-Review: Update event-producing tools to overwrite `meta.dt` - https://phabricator.wikimedia.org/T376026#11060704 (10Ottomata) [13:26:44] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11060705 (10JAllemandou) @CDanis nothing special is needed, we just need to update the ingestion removing the field. [13:27:50] 06Data-Engineering: AlertLintProblem - https://phabricator.wikimedia.org/T401156#11060716 (10Ottomata) →14Duplicate dup:03T395539 [13:31:34] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 10Event-Platform: EventStreams: duplicate events from double compute (wdqs/rdf) streams - https://phabricator.wikimedia.org/T396564#11060730 (10Ottomata) Hm, good catch, you are right. I don't think there is a case right now where... [13:41:00] 06Data-Engineering, 10Event-Platform: More strictly validate X-Experiment-Enrollments-Header - https://phabricator.wikimedia.org/T401198 (10dr0ptp4kt) 03NEW [13:59:45] 06Data-Engineering, 10Event-Platform: More strictly validate X-Experiment-Enrollments-Header - https://phabricator.wikimedia.org/T401198#11060882 (10Ottomata) Also for consideration: T396359#11047999 > the /v1/events API allows for POSTing an array of events, each of which could potentially be a for a differe... [14:06:45] (03CR) 10Mforns: [V:03+2 C:03+2] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1175556 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [14:20:23] (03Merged) 10jenkins-bot: Update changelog.md to reflect the latest version 0.2.68 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1175556 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [14:26:07] Starting build #40 for job analytics-refinery-maven-release [14:38:05] (03CR) 10Aleksandar Mastilovic: "Could you please point me to this code? I'm not sure what code you're referring to." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [14:45:48] (03CR) 10Joal: "LGTM for this part. You also need to update the related schema in hql/mediawiki/history/create_mediawiki_categorylinks_table.hql" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [14:47:41] Project analytics-refinery-maven-release build #40: 09SUCCESS in 21 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/40/ [14:51:55] (03CR) 10Joal: [C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175159 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [14:54:52] (03CR) 10Joal: [C:03+2] "LGTM! Thank you :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1175156 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [14:57:01] (03CR) 10Aleksandar Mastilovic: "Looking for column commentary for `cl_target_id` and `cl_collation_id` I just realized that `cl_collation_id` is a foreign key for table/c" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [15:01:36] 10Analytics-Canonical-Data, 06Research, 10Movement-Insights (FY25-26 H1): Provide ISO 639 language codes in canonical wiki dataset - https://phabricator.wikimedia.org/T346855#11061164 (10CMyrick-WMF) 05Open→03In progress [15:01:57] 10Analytics-Canonical-Data, 06Research, 10Movement-Insights (FY25-26 H1): Create a canonical dataset for incubating wikis - https://phabricator.wikimedia.org/T393075#11061168 (10CMyrick-WMF) 05Open→03In progress [15:02:30] 10Analytics-Canonical-Data, 06Research, 07Epic: Add language and incubating wikis data to canonical data - https://phabricator.wikimedia.org/T388201#11061174 (10CMyrick-WMF) 05Open→03In progress [15:11:22] (03CR) 10Aleksandar Mastilovic: "@joal@wikimedia.org @dandreescu@wikimedia.org could any of you please merge? I don't seem to have that option." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175159 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [15:20:01] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 06SRE Observability, 10Event-Platform, 13Patch-For-Review: Eventstreams 'assignments' logstash field type - https://phabricator.wikimedia.org/T390140#11061304 (10tchin) >>! In T390140#11043685, @colewhite wrote: >>>! In T390140#11... [15:36:13] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 06SRE Observability, 10Event-Platform, 13Patch-For-Review: Eventstreams 'assignments' logstash field type - https://phabricator.wikimedia.org/T390140#11061393 (10colewhite) >>! In T390140#11061304, @tchin wrote: > Was this a recen... [15:39:17] (03CR) 10Ottomata: "Hm, I think you should? You are in the gerrit Analytics group?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175159 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [15:44:08] (03CR) 10Aleksandar Mastilovic: [V:03+2] Stop using cuc_ip field in cu_changes, switch to cuc_ip_hex [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175159 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [16:06:01] (03PS2) 10Aleksandar Mastilovic: Update Sqoop's schema for the categorylinks table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) [16:08:37] (03PS3) 10Aleksandar Mastilovic: Update Sqoop's schema for the categorylinks table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) [16:13:25] (03PS1) 10Aleksandar Mastilovic: Bring back the deprecated cuc_ip field in cu_changes table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) [16:49:24] 06Data-Engineering-Radar, 10CheckUser, 06DBA, 06Trust and Safety Product Team, 07Schema-change-in-production: Add '*_actor_ip_hex_time' indexes to 'cu_changes', 'cu_log_event', and 'cu_private_event' on WMF wikis - https://phabricator.wikimedia.org/T399728#11061818 (10FCeratto-WMF) [17:14:12] (03CR) 10Joal: "In addition to the schema change, you want to manulaly set the value to NULL in sqoop" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [17:17:04] (03CR) 10Joal: "One nit" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [17:21:05] (03PS2) 10Aleksandar Mastilovic: Bring back the deprecated cuc_ip field in cu_changes table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) [17:21:20] (03CR) 10Aleksandar Mastilovic: "Acknowledged" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [17:27:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Analyze temporary hourly webrequest traffic loss - https://phabricator.wikimedia.org/T399312#11061961 (10Ottomata) Since July 17, this alert has fired at error (above 2.0% loss) or warning 36 times. This is more than usual. @JAllemandou and I looked a... [17:28:40] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Analyze hourly webrequest traffic loss - July 2025 - https://phabricator.wikimedia.org/T399312#11061962 (10Ottomata) [19:08:07] (03CR) 10Aleksandar Mastilovic: Update Sqoop's schema for the categorylinks table (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [19:10:55] (03PS4) 10Aleksandar Mastilovic: Update Sqoop's schema for the categorylinks table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) [20:53:45] 06Data-Engineering, 10Pageviews-Anomaly, 07SecTeam-Processed, 07Security: Views data integrity compromised by entity running up fake views - https://phabricator.wikimedia.org/T366554#11062686 (10Stevietheman) More human beings visiting an article was never the issue. Mounds of bots posing as human beings... [20:55:10] 06Data-Engineering, 10Pageviews-Anomaly, 07SecTeam-Processed, 07Security: Views data integrity compromised by entity running up fake views - https://phabricator.wikimedia.org/T366554#11062688 (10Stevietheman) >>! In T366554#11059827, @Cixtpide wrote: >>>! In T366554#11056356, @Stevietheman wrote: >> @Cixtp... [20:59:02] 06Data-Engineering, 10Pageviews-Anomaly, 07SecTeam-Processed, 07Security: Views data integrity compromised by entity running up fake views - https://phabricator.wikimedia.org/T366554#11062693 (10Stevietheman) >>! In T366554#11057697, @Ottomata wrote: > Well you are in luck! IIUC ^, is in WMF's annual plan... [22:01:15] 06Data-Engineering, 06Data-Persistence: Data Persistence Design Review: Year in Review (YiR) - https://phabricator.wikimedia.org/T401260 (10Eevans) 03NEW [22:01:31] 06Data-Engineering, 06Data-Persistence: Data Persistence Design Review: Year in Review (YiR) - https://phabricator.wikimedia.org/T401260#11062848 (10Eevans) p:05Triage→03Medium [22:02:51] 06Data-Engineering, 06Data-Persistence: Data Persistence Design Review: Year in Review (YiR) - https://phabricator.wikimedia.org/T401260#11062854 (10Eevans)