[00:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [00:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [01:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [01:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [02:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [02:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [02:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [02:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [03:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [04:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [04:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [05:30:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [05:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [06:00:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [06:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [06:08:15] (03CR) 10KCVelaga: "Ah right, I can imagine it can be confusing. We can rename, one alternative I can think of is, cloudflare_bot_definitions. @guilherme@wiki" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [06:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [06:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [07:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [07:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [07:35:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Analyze and optimize Airflow Postgres backend performance - https://phabricator.wikimedia.org/T411990#12181755 (10Antoine_Quhen) 05Open→03Resolved Root cause was demanding xcoms network loads for airflow PG and location of PG instance on a machin... [07:36:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Airflow main performance instance optimization - https://phabricator.wikimedia.org/T411988#12181761 (10Antoine_Quhen) 05Open→03Resolved Root cause was demanding xcoms network loads for airflow PG and location of PG instance on a machine with 1GB... [07:38:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refine_to_hive dag optimizations - https://phabricator.wikimedia.org/T392668#12181769 (10Antoine_Quhen) I think we can push more config setup at execution time. This would be a big win and not huge work. [08:15:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12181861 (10Antoine_Quhen) @Addshore Within the data-platform infrastructure, we have a large Kafka setup and a way to generate events through an API called EventGate. So... [08:30:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [08:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [08:30:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12181916 (10Addshore) I expect the hook approach would not work for GitHub, as the scope of the data span multiple orgs, repositories and users, and hooks are either at t... [08:39:43] 06Data-Engineering: Migrate Google Search Console analytics off GCP BigQuery to the Data Lake - https://phabricator.wikimedia.org/T433927 (10Antoine_Quhen) 03NEW [09:00:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [09:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [09:05:20] (03CR) 10Guilherme Gonçalves: "Hm, I think `cloudflare_bot_directory` is fine to make it explicit where the data comes from." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [09:12:02] (03CR) 10KCVelaga: "Right, sounds good." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [09:47:30] 06Data-Engineering: Analyze Superset virtual datasets to find candidates for materialization - https://phabricator.wikimedia.org/T429226#12182312 (10GGoncalves-WMF) p:05Triage→03Low Marking this low priority: it would be nice to get to this in Q1, and perhaps we can keep this as a time-boxed spike, but it's... [09:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [09:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [10:34:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [10:34:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [10:43:29] (03PS2) 10KCVelaga: HQLs to create and import Cloudflare bot directory. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) [10:44:12] (03PS2) 10KCVelaga: HQLs to create and import Cloudflare bot directory. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) [10:59:57] (03CR) 10KCVelaga: "I have tested both the HQLs, ready for final review." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [11:00:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform: Create `webrequest-pageview` K8s application. - https://phabricator.wikimedia.org/T433962 (10JMonton-WMF) 03NEW [11:00:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform: Create `webrequest-pageview` K8s application. - https://phabricator.wikimedia.org/T433962#12182715 (10JMonton-WMF) [11:02:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform: Create `webrequest-pageview` K8s application. - https://phabricator.wikimedia.org/T433962#12182722 (10JMonton-WMF) [11:04:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform, 13Patch-For-Review: Create `webrequest-pageview` K8s application. - https://phabricator.wikimedia.org/T433962#12182754 (10JMonton-WMF) [11:04:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Classify requests to developer.wikimedia.org as pageviews - https://phabricator.wikimedia.org/T428550#12182755 (10Ahoelzl) [11:04:24] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12182756 (10Ahoelzl) [11:13:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform, 13Patch-For-Review: Create `webrequest-pageview` K8s application. - https://phabricator.wikimedia.org/T433962#12182768 (10JMonton-WMF) [11:13:58] 06Data-Engineering, 10Data-Engineering-Wikistats: Metric is not available for all projects error: show more constructive solution - https://phabricator.wikimedia.org/T433965 (10Effeietsanders) 03NEW [11:17:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic, 05Metrics-Sprint-2026-2027: Epic: dbt improvements to support metrics pipelines - https://phabricator.wikimedia.org/T433612#12182798 (10Ahoelzl) [11:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [11:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [11:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [11:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [12:13:38] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12182897 (10GGoncalves-WMF) [12:14:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Classify requests to developer.wikimedia.org as pageviews - https://phabricator.wikimedia.org/T428550#12182899 (10GGoncalves-WMF) [12:14:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Classify requests to developer.wikimedia.org as pageviews - https://phabricator.wikimedia.org/T428550#12182902 (10GGoncalves-WMF) [12:14:31] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12182903 (10GGoncalves-WMF) [12:30:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [12:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [13:00:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [13:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [13:21:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Machine-Learning-Team, 10Event-Platform: `mediawiki.page_change.v1`: negative namespace_id schema validation errors - https://phabricator.wikimedia.org/T421237#12183117 (10isarantopoulos) [13:35:54] 06Data-Engineering: dbt: delay job by arbitrary amounts of time - https://phabricator.wikimedia.org/T433976 (10Milimetric) 03NEW [13:40:13] 06Data-Engineering: dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622#12183315 (10Milimetric) [13:43:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic, 05Metrics-Sprint-2026-2027: Epic: dbt improvements to support metrics pipelines - https://phabricator.wikimedia.org/T433612#12183331 (10Milimetric) 05Open→03Declined I merged these thoughts into the official dbt epics maintained by Gu... [13:48:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12183356 (10Ottomata) Even if 'batch' generated, I think an event model here would be valuable. Commits and other git interactions are events, e.g. commit A at timestamp... [13:51:30] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Refine: persist resolved stream configurations to make reruns deterministic - https://phabricator.wikimedia.org/T418151#12183390 (10Antoine_Quhen) a:05Antoine_Quhen→03None [13:54:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: EPIC: DYK Archive - https://phabricator.wikimedia.org/T430319#12183416 (10Snwachukwu) [13:55:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: EPIC: DYK Archive - https://phabricator.wikimedia.org/T430319#12183423 (10Snwachukwu) [14:00:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [14:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [14:25:27] FYI am deploying a mw config change and rolling restarting eventgate-main... [14:26:39] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12183612 (10Addshore) I'd argue that the commit itself is not the event within git, though we could likely get away with modeling it that way. I suspect that is for examp... [14:30:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [14:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [14:30:10] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12183629 (10Ottomata) > I'd argue that the commit itself is not the event within git ACK, TBD on event data model. > If we've already fired off a commit event to eventga... [14:38:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12183711 (10Addshore) > No, but we can fire a different event that would invalidate the previous one. I'm not even sure if invalidation matters (at least not for the spe... [14:42:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Research-engineering, 10Event-Platform, 13Patch-For-Review: Expose mediawiki.page_html_feature_counts_change.v1 in public EventStreams API - https://phabricator.wikimedia.org/T433507#12183723 (10Ottomata) [14:42:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Research-engineering, 10Event-Platform, 13Patch-For-Review: Expose mediawiki.page_html_feature_counts_change.v1 in public EventStreams API - https://phabricator.wikimedia.org/T433507#12183727 (10Ottomata) a:03Ottomata [14:43:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: [DYK Archive] Enrichment resolve & enrich linked pages - https://phabricator.wikimedia.org/T433984 (10Snwachukwu) 03NEW [14:50:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: data lake: git commit history - https://phabricator.wikimedia.org/T433696#12183770 (10Ottomata) Yes, agree. I think I meant 'invalidate' at a higher level, but really it is just a change event that happens at a specific time. It would invalid... [14:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [14:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [15:08:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12183833 (10Snwachukwu) There's a total of 6448776... [15:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [15:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [15:51:13] 06Data-Engineering, 06Product-Analytics (Kanban): Switch to user_agents_info and deprecate user_agent_compliance_classified_hourly - https://phabricator.wikimedia.org/T433992 (10KCVelaga_WMF) 03NEW [15:51:35] 06Data-Engineering, 06Product-Analytics (Kanban): Switch to user_agents_info and deprecate user_agent_compliance_classified_hourly - https://phabricator.wikimedia.org/T433992#12184047 (10KCVelaga_WMF) p:05Triage→03Medium [16:00:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:24:19] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 07Epic: Incremental MediaWiki History - v2 - https://phabricator.wikimedia.org/T432025#12184262 (10Ottomata) [16:30:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:44:21] 06Data-Engineering: Refine: persist resolved stream configurations to make reruns deterministic - https://phabricator.wikimedia.org/T418151#12184327 (10Aklapper) Please keep related project tags around - thanks. [16:46:21] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Add automated test run config option to dbt scheduler YAML - https://phabricator.wikimedia.org/T434000 (10amastilovic) 03NEW [16:47:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Add an option to define an alert email at the level of team YAML config dbt scheduler file - https://phabricator.wikimedia.org/T434001 (10amastilovic) 03NEW [17:00:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [17:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [17:10:13] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Dumps-Generation, 10Wikidata: No new wikibase dump downloads being published. - https://phabricator.wikimedia.org/T433240#12184462 (10Melderick) Same for this week dumps. Directory 20260803 should have been created by now her... [17:30:03] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [17:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [18:00:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [18:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [18:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [18:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [18:50:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06cloud-services-team, 10Cloud-VPS (Debian Bullseye Deprecation), 10Event-Platform: Upgrade (replace) beta|deployment-prep event platform VPS instances - https://phabricator.wikimedia.org/T429497#12184900 (10Ottomata) FWIW: https://wikitech.wikime... [18:57:28] (03PS4) 10Snwachukwu: Add wmf_content.mediawiki_html_content_current_v0 table and backfill [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319873 [18:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [18:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:01:59] (03CR) 10Snwachukwu: "Yes, I have added a hql script to read from the events table and update the html_content_current iceberg table." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319873 (owner: 10Snwachukwu) [19:03:34] (03PS5) 10Snwachukwu: Add wmf_content.mediawiki_html_content_current_v0 table and backfill [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319873 [19:23:15] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12185040 (10Isaac) Yikes, so many wiki pages out t... [19:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:52:41] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 05MW-1.47-notes (1.47.0-wmf.10; 2026-07-07): mediawiki.page_change.v1 event - Add revision revert details - https://phabricator.wikimedia.org/T423583#12185106 (10Ottomata) [19:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [20:30:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [20:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [20:36:31] (03PS3) 10TChin: Add Java implentation of haproxy's sdbm hashing algorithm in C [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1311901 (https://phabricator.wikimedia.org/T425663) [20:37:22] (03PS4) 10TChin: Add Java implentation of haproxy's sdbm hashing algorithm in C [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1311901 (https://phabricator.wikimedia.org/T425663) [20:44:53] (03PS5) 10TChin: Add Java implentation of haproxy's sdbm hashing algorithm in C [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1311901 (https://phabricator.wikimedia.org/T425663) [21:00:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [21:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [21:30:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [21:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [21:42:37] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 07Technical-Debt, 10Test Kitchen (Experiment Platform Sprint 27): Deprecate and remove mw.eventLog.submitInteraction() - https://phabricator.wikimedia.org/T415362#12185366 (10phuedx) [21:42:50] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 07Technical-Debt, 10Test Kitchen (Experiment Platform Sprint 27): Deprecate and remove mw.eventLog.submitInteraction() - https://phabricator.wikimedia.org/T415362#12185368 (10phuedx) a:03phuedx [21:44:19] 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 07Technical-Debt, 10Test Kitchen (Experiment Platform Sprint 27): Deprecate and remove mw.eventLog.submitInteraction() - https://phabricator.wikimedia.org/T415362#12185372 (10phuedx) @cjming effectively did this as part of {T415370}. As I mentioned i... [22:00:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [22:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [22:30:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [22:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [22:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [22:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [23:13:29] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12185565 (10ppelberg) [23:16:39] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12185578 (10ppelberg) p:05Triage→03High [23:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [23:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [23:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [23:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected