[00:29:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [00:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [00:45:39] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): [BUGFIX] DbtSkeinOperator doesn't handle list variables properly - https://phabricator.wikimedia.org/T433447 (10amastilovic) 03NEW [00:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [00:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [01:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [01:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [02:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [02:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [02:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [02:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [03:15:37] 06Data-Engineering, 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging context - https://phabricator.wikimedia.org/T433457 (10AKhatun_WMF) 03NEW [03:23:25] 06Data-Engineering, 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging context - https://phabricator.wikimedia.org/T433457#12165754 (10AKhatun_WMF) [03:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [03:30:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [03:34:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:34:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [03:57:10] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for ~115 wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458 (10AKhatun_WMF) 03NEW [03:59:18] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:59:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [04:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [04:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [04:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [04:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [05:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [05:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [05:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [05:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [06:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [06:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [07:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [07:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [07:42:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Create a new PostgreSQL cluster database with read-only from Superset and write access from Spark - https://phabricator.wikimedia.org/T432104#12166066 (10... [07:45:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Create a new PostgreSQL cluster database with read-only from Superset and write access from Spark - https://phabricator.wikimedia.org/T432104#12166069 (10... [07:46:21] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Create a new PostgreSQL cluster database with read-only from Superset and write access from Spark - https://phabricator.wikimedia.org/T432104#12166071... [07:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [07:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [08:15:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027: Move all FY2026-2027 chart data to PostgreSQL - https://phabricator.wikimedia.org/T432017#12166237 (10Ahoelzl) [08:17:37] (03PS1) 10A-pizzata: spark: Add logback config to quiet DataHub/OpenLineage lineage agent [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) [08:20:16] (03Abandoned) 10A-pizzata: Change log level for all the datahub instances in MediawikiHistoryRunner. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1318663 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [08:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [08:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [09:01:00] 06Data-Engineering, 06DBA, 10DiscussionTools, 07Schema-change-in-production: Drop DiscussionTools database tables from wikis which don't need them - https://phabricator.wikimedia.org/T426341#12166355 (10Marostegui) Tables dropped from s5 (just two wikis) And now looping over s3, which will take a bit. [09:14:13] (03CR) 10Joal: [C:03+1] "It'd be nice to have an example of how to make the file be used on jobs :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [09:15:09] (03CR) 10Joal: [C:03+1] "I didn't realize the usage was in the file comment - disregard the previous comment :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [09:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [09:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [10:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [10:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [10:53:28] 06Data-Engineering, 06DBA, 10DiscussionTools, 07Schema-change-in-production: Drop DiscussionTools database tables from wikis which don't need them - https://phabricator.wikimedia.org/T426341#12166656 (10Marostegui) 05Open→03Resolved Done [10:56:33] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066#12166676 (10Marostegui) s5 one dropped. Looping over the ones in s3, will take a bit. [10:59:03] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [10:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [11:12:33] (03PS2) 10A-pizzata: spark: Add logback config to quiet DataHub/OpenLineage lineage agent [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) [11:13:12] (03CR) 10A-pizzata: "@joal@wikimedia.org what do you think now?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [11:29:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [11:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [11:52:58] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop AbuseFilter tables from closed wikis - https://phabricator.wikimedia.org/T425066#12166833 (10Marostegui) 05Open→03Resolved s3 done [11:59:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [11:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [12:02:47] 06Data-Engineering, 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging context - https://phabricator.wikimedia.org/T433457#12166876 (10Tgr) These fields are bein... [12:29:03] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [12:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [12:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [12:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [13:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [13:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [13:34:31] 10Analytics-Canonical-Data, 06Movement-Insights, 07Epic: Add language and incubating wikis data to canonical data - https://phabricator.wikimedia.org/T388201#12167171 (10CMyrick-WMF) [13:37:58] 06Data-Engineering, 06Data-Engineering-Radar, 10WMDE Analytics, 10Wmfdata-Python: Print the link to the scheduler UI when a Spark session is started - https://phabricator.wikimedia.org/T402993#12167181 (10Ahoelzl) [13:39:52] 06Data-Engineering, 10Event-Platform: eventgate-analytics - allow refetching of stream config - https://phabricator.wikimedia.org/T430154#12167189 (10Ottomata) a:03Ahoelzl [13:40:24] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: eventgate-analytics - allow refetching of stream config - https://phabricator.wikimedia.org/T430154#12167193 (10Ottomata) [13:40:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167195 [13:40:59] 06Data-Engineering, 06Data-Engineering-Radar, 10Test Kitchen, 07Essential-Work, 10Event-Platform: [EventGate] Reject events from inactive instruments and experiments - https://phabricator.wikimedia.org/T430541#12167200 (10Ottomata) [13:41:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167201 [13:41:10] 06Data-Engineering, 06Data-Engineering-Radar, 07Essential-Work, 10Event-Platform, 10Test Kitchen (Experiment Platform Sprint 27): [EventGate] Add configurable UA denylist - https://phabricator.wikimedia.org/T429898#12167204 (10Ottomata) [13:41:19] 06Data-Engineering, 06Data-Engineering-Radar, 07Essential-Work, 10Event-Platform, 10Test Kitchen (Experiment Platform Sprint 27): [EventGate] Record metrics for instruments and experiments - https://phabricator.wikimedia.org/T430322#12167206 (10Ottomata) [13:41:49] 06Data-Engineering: HDFS under 20% of free space - https://phabricator.wikimedia.org/T431391#12167208 (10Ottomata) 05Open→03Resolved a:03Ottomata Resolving, feel free to reopen if incorrect. [13:42:12] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167212 [13:42:25] 06Data-Engineering: HDFS under 20% of free space - https://phabricator.wikimedia.org/T431391#12167213 (10Ottomata) 05Resolved→03Open [13:42:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): HDFS under 20% of free space - https://phabricator.wikimedia.org/T431391#12167215 (10Ottomata) [13:42:48] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): HDFS under 20% of free space - https://phabricator.wikimedia.org/T431391#12167218 (10Ottomata) 05Open→03Resolved [13:44:16] 06Data-Engineering, 10Data Pipelines: Consider edits from system accounts as bot edits in mediawiki_history - https://phabricator.wikimedia.org/T378104#12167225 (10Ottomata) [13:44:19] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 07Epic: Incremental MediaWiki History - v2 - https://phabricator.wikimedia.org/T432025#12167226 (10Ottomata) [13:45:13] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE (2026-07-03 - 2026-07-31): hive metastore was down - https://phabricator.wikimedia.org/T432507#12167230 (10Ahoelzl) [13:45:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for ~115 wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167231 (10Milimetric) a:03APizzata-WMF [13:47:24] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Dumps-Generation, 10Wikidata: No new dump downloads being published. - https://phabricator.wikimedia.org/T433240#12167237 (10Ahoelzl) [13:47:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Dumps-Generation, 10Wikidata: No new dump downloads being published. - https://phabricator.wikimedia.org/T433240#12167238 (10Ottomata) [13:48:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list July 2026 - https://phabricator.wikimedia.org/T433437#12167240 (10Milimetric) [13:48:46] 06Data-Engineering, 10Test Kitchen, 07OKR-Work (DE1 FY2026-27): [DE 1.1.9] Iceberg table of constructive edits - https://phabricator.wikimedia.org/T433379#12167245 (10Ottomata) a:03Ahoelzl [13:48:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12167247 (10Milimetric) [13:48:59] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Test Kitchen, 07OKR-Work (DE1 FY2026-27): [DE 1.1.9] Iceberg table of constructive edits - https://phabricator.wikimedia.org/T433379#12167249 (10Ottomata) [13:50:45] 06Data-Engineering, 10MediaWiki-extensions-PageAssessments, 06MediaWiki-Core-Platform-Team (Radar), 05MW-1.47-notes (1.47.0-wmf.13; 2026-07-28), 07Wikimedia-production-error: PHP Warning: MediaWiki\ResourceLoader\Context::encodeJson failed to encode "wgP... - https://phabricator.wikimedia.org/T432770#12167259 [13:50:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10MediaWiki-extensions-PageAssessments, 06MediaWiki-Core-Platform-Team (Radar), 05MW-1.47-notes (1.47.0-wmf.13; 2026-07-28), 07Wikimedia-production-error: PHP Warning: MediaWiki\ResourceLoader\Co... - https://phabricator.wikimedia.org/T432770#12167263 [13:53:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt: per-group DAG generation + a dedicated dbt Airflow instance - https://phabricator.wikimedia.org/T429439#12167269 (10Ahoelzl) [13:53:40] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Content-Platform-Team, 10MediaWiki-extensions-PageAssessments, 06MediaWiki-Core-Platform-Team (Radar), and 2 others: PHP Warning: MediaWiki\ResourceLoader\Context::encodeJson failed to encode "wg... - https://phabricator.wikimedia.org/T432770#12167270 [13:53:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): DBT: Standardize start and end date variable configuration and usage across dbt-jobs - https://phabricator.wikimedia.org/T429483#12167275 (10Ahoelzl) [13:54:29] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for ~115 wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167278 (10AKhatun_WMF) [13:54:35] 06Data-Engineering, 06DBA, 10DiscussionTools, 07Schema-change-in-production: Drop DiscussionTools database tables from wikis which don't need them - https://phabricator.wikimedia.org/T426341#12167279 (10AKhatun_WMF) [13:55:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for ~115 wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167282 (10AKhatun_WMF) [13:55:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Persistence, 10DiscussionTools: Adapt sqoop configs to account for discussiontools tables only present on some wiki databases - https://phabricator.wikimedia.org/T428916#12167285 (10AKhatun_WMF) →14Duplicate dup:03T433458 [13:55:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167287 (10AKhatun_WMF) [13:57:17] 06Data-Engineering, 10Test Kitchen, 07Essential-Work: Improve web instrument event data data lake management - https://phabricator.wikimedia.org/T429385#12167296 (10Ottomata) a:03Ahoelzl [13:57:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Test Kitchen, 07Essential-Work: Improve web instrument event data data lake management - https://phabricator.wikimedia.org/T429385#12167298 (10Ottomata) [13:57:45] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search (2026.06.01 - 2026.07.03), 10Event-Platform: Extract eventutilities-flink outside of wikimedia-eventutilities - https://phabricator.wikimedia.org/T429566#12167300 (10Ottomata) [13:57:50] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search (2026.06.01 - 2026.07.03), 10Event-Platform: Upgrade flink-utilities to flink 2.0.2 - https://phabricator.wikimedia.org/T429565#12167302 (10Ottomata) [13:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [13:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [13:59:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt data tests validate production data - https://phabricator.wikimedia.org/T430786#12167323 (10Ahoelzl) a:03amastilovic [14:00:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: Dbt backfill: signal_driven_handling_bad_faith_accounts - https://phabricator.wikimedia.org/T431279#12167325 (10Ahoelzl) [14:03:06] 06Data-Engineering, 10Event-Platform: [EventGate] Sanitize stream name in all metrics - https://phabricator.wikimedia.org/T429799#12167348 (10Ottomata) a:03tchin :) [14:03:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: [EventGate] Sanitize stream name in all metrics - https://phabricator.wikimedia.org/T429799#12167351 (10Ottomata) [14:08:43] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167382 (10AKhatun_WMF) Per {T420052} there are more tables that are being dropped. We can use the `cl... [14:09:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167383 (10Dreamy_Jazz) > Note a naive all.dblist − closed.dblist subtraction is not reliable — as it... [14:10:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167387 (10AKhatun_WMF) @Dreamy_Jazz, you're right. They are indeed closed. Some were in s5 and missed... [14:13:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167400 (10AKhatun_WMF) [14:29:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [14:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [14:32:48] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167490 [14:36:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167515 [14:39:58] (03CR) 10Joal: "that's perfect :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [14:47:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167573 [14:49:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, 07Schema-change: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging ... - https://phabricator.wikimedia.org/T433457#12167596 [14:52:46] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12167618 [14:54:36] (03CR) 10Joal: Switch pageview_actor consumers to use webrequest_v2 instead. (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) (owner: 10Mforns) [14:55:49] (03PS1) 10AKhatun: Provide sqoopable_dbnames for tables that were dropped in some wikis [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319107 (https://phabricator.wikimedia.org/T433458) [14:56:40] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: refinery-sqoop-mediawiki-production-daily fails for wikis that lack the discussiontools_subscription table - https://phabricator.wikimedia.org/T433458#12167641 (10AKhatun_WMF) Of all the tables dropped or to be dropped, `discussiont... [14:57:44] 06Data-Engineering, 06Research: Incremental HTML wiki content dataset to support "Who are moderators" - https://phabricator.wikimedia.org/T380874#12167644 (10Ottomata) 05Open→03Resolved a:03Ottomata We now have mediawiki.page_html_content_change.v1. [14:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [14:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [14:59:28] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Library for unified_diffs - https://phabricator.wikimedia.org/T419969#12167655 (10Ottomata) @JMonton-WMF what was the resolution here? [15:03:36] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: DE3.1 - Logged-out Wikipedia 21-day retention on mobile web - https://phabricator.wikimedia.org/T424706#12167683 (10Test-Kitchen-Experiments-Bot) Your experiment [[ https://growthbook.wikimedia.org/expe... [15:08:18] (03CR) 10Joal: [C:03+1] "This is great! thank you for fixing this @akhatun@wikimedia.org" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319107 (https://phabricator.wikimedia.org/T433458) (owner: 10AKhatun) [15:08:36] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Library for unified_diffs - https://phabricator.wikimedia.org/T419969#12167698 (10AKhatun_WMF) No existing problem with the diff library as of now. There are 2 hacks/things to remember (in the 2 MRs attached). This task was to possibly ge... [15:09:53] 06Data-Engineering, 10Research-engineering, 10Event-Platform: Expose mediawiki.page_html_feature_counts_change.v1 in public EventStreams API - https://phabricator.wikimedia.org/T433507 (10Ottomata) 03NEW [15:14:38] (03CR) 10Snwachukwu: [C:03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319107 (https://phabricator.wikimedia.org/T433458) (owner: 10AKhatun) [15:16:01] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Dumps-Generation, 10Wikidata: No new wikibase dump downloads being published. - https://phabricator.wikimedia.org/T433240#12167737 (10BTullis) [15:17:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE, 10Dumps-Generation, 10Wikidata: No new wikibase dump downloads being published. - https://phabricator.wikimedia.org/T433240#12167763 (10BTullis) I have modified the title to reflect that this is **wikibase** dumps, as opposed t... [15:23:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Test Kitchen, 07OKR-Work (DE1 FY2026-27): [DE 1.1.9] Iceberg table of constructive edits - https://phabricator.wikimedia.org/T433379#12167777 (10mpopov) By the way @cjming is planning on launching the instrument (that will produce the `edit_survive... [15:28:18] (03CR) 10Snwachukwu: [C:03+2] Provide sqoopable_dbnames for tables that were dropped in some wikis [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319107 (https://phabricator.wikimedia.org/T433458) (owner: 10AKhatun) [15:29:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [15:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [15:29:24] (03CR) 10Snwachukwu: [V:03+2 C:03+2] Provide sqoopable_dbnames for tables that were dropped in some wikis [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319107 (https://phabricator.wikimedia.org/T433458) (owner: 10AKhatun) [15:35:56] 06Data-Engineering: Feature request: Enable dbt jobs to be scheduled hourly - https://phabricator.wikimedia.org/T433509 (10KCVelaga_WMF) 03NEW [15:38:32] 06Data-Engineering, 10Research-engineering, 10Event-Platform: Expose mediawiki.page_html_feature_counts_change.v1 in public EventStreams API - https://phabricator.wikimedia.org/T433507#12167846 (10Ottomata) [15:39:35] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: [Mediawiki History] make array fields empty as opposed to null when relevant but no values present - https://phabricator.wikimedia.org/T433511 (10Isaac) 03NEW [15:49:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12167904 [15:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [15:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:01:51] (03CR) 10A-pizzata: [C:03+2] spark: Add logback config to quiet DataHub/OpenLineage lineage agent [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [16:02:00] (03CR) 10A-pizzata: [V:03+2 C:03+2] spark: Add logback config to quiet DataHub/OpenLineage lineage agent [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319028 (https://phabricator.wikimedia.org/T424358) (owner: 10A-pizzata) [16:19:31] !log Deploying Refinery at 56695674 as part of weekly train [16:19:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:29:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:30:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:30:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:31:43] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: [Mediawiki History] Default boolean fields to False instead of NULL where relevant - https://phabricator.wikimedia.org/T433517 (10Isaac) 03NEW [16:33:00] (03PS1) 10Btullis: Update the scap target for deployment [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1319127 (https://phabricator.wikimedia.org/T406746) [16:34:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:34:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [16:34:57] 06Data-Engineering: Feature request: Enable dbt jobs to be scheduled hourly - https://phabricator.wikimedia.org/T433509#12168234 (10KCVelaga_WMF) [16:36:01] (03CR) 10Btullis: [V:03+2 C:03+2] Update the scap target for deployment [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1319127 (https://phabricator.wikimedia.org/T406746) (owner: 10Btullis) [16:49:45] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Event-Platform: Library for unified_diffs - https://phabricator.wikimedia.org/T419969#12168422 (10Ottomata) 05Open→03Resolved Great, resolving then. [16:56:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168486 [16:59:18] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [16:59:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [17:29:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [17:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [17:41:14] !log Deployed refinery using scap, then deployed onto hdfs [17:41:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:48:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168709 (10Snwachukwu) **Approach update** The plan is to build a `wmf... [17:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [17:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [18:29:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [18:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [18:32:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168846 (10Ottomata) Wow! No objection! What's the schema like? W... [18:33:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168850 (10Ottomata) > wmf_content.mediawiki_html_content_current Sugg... [18:40:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168872 [18:48:00] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168883 (10Snwachukwu) I'm proposing: CREATE EXTERNAL TABLE wmf_conten... [18:49:10] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168888 [18:50:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168890 (10Snwachukwu) > unless you need more metadata about the revisi... [18:54:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168902 [18:55:00] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168903 [18:55:38] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt improvement: Simplify logging the run context of a dbt model - https://phabricator.wikimedia.org/T433532 (10amastilovic) 03NEW [18:55:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168916 [18:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [18:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:00:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:00:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:00:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168930 [19:04:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:04:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:07:31] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168939 (10Ottomata) > I'd like to get the revision_id and dt via /with... [19:09:16] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168957 [19:10:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168961 [19:12:45] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168984 [19:16:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12168993 (10Ottomata) I don't have objections, just preferences! One th... [19:19:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12168999 [19:20:18] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169001 [19:29:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:29:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [19:38:24] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12169054 (10Eevans) p:05Triage→03Medium [19:40:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169057 [19:40:45] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12169058 (10Eevans) [19:47:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169084 [19:58:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169121 [19:59:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [19:59:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [20:02:17] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169139 [20:03:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169141 [20:03:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12169142 (10Eevans) Hi @JMonton-WMF, First off, thanks for the excellent proposal, it is very co... [20:06:02] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169149 [20:07:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06MediaWiki-Core-Platform-Team, 10MediaWiki-extensions-OAuth, 10Event-Platform, and 2 others: `mediawiki.api-request` validation errors from Extension:OAuth `context.oauth_consumer_*` logging conte... - https://phabricator.wikimedia.org/T433457#12169152 [20:29:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [20:29:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [20:59:18] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [20:59:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [21:29:19] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [21:29:19] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [21:59:04] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [21:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [22:29:03] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [22:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [22:34:55] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12169592 (10Snwachukwu) > I suppose from mediawiki_content_current_v1?... [22:59:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [22:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [23:29:04] FIRING: [2x] GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [23:29:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [23:59:04] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [23:59:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=eqiad.pageview.trending.relative.v1&viewPanel=24 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected