[02:06:53] 06Data-Engineering: Spark queries of Google Search Console data fail due to Zstandard error - https://phabricator.wikimedia.org/T434625 (10nshahquinn-wmf) 03NEW [02:28:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#12206333 (10nshahquinn-wmf) For the record, this data is now being automatically loaded into `wmf_raw.google_search_console_{site|url}_impression` (but note T4... [02:44:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: mediawiki-event-enrichment - set deterministic meta.id - https://phabricator.wikimedia.org/T424987#12206341 (10Ahoelzl) [02:44:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform: Review "page_view" schema, definition and operations. - https://phabricator.wikimedia.org/T427901#12206342 (10Ahoelzl) [02:45:08] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): mw_content_history_reconcile_enrich api call returned 503 - https://phabricator.wikimedia.org/T415264#12206343 (10Ahoelzl) [02:46:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data Pipelines, 06Data-Platform-SRE (2026-08-07 - 2026-08-28), 07Essential-Work: Airflow dynamic task mapping logs mix up when, on rerun, an id is mapped to a different map_index_template - https://phabricator.wikimedia.org/T408802#12206344 (10Aho... [02:46:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Add source='snapshot+events' for incrementally-enriched snapshot rows - https://phabricator.wikimedia.org/T428969#12206345 (10Ahoelzl) a:03Ahoelzl [02:46:51] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-07 - 2026-08-28), 13Patch-For-Review: [Infra] Add a presto query logger - https://phabricator.wikimedia.org/T269832#12206346 (10Ahoelzl) a:03Ahoelzl [02:47:41] FIRING: MediawikiPageHtmlContentChangeEnrichHighKafkaConsumerLag: ... [02:47:41] High Kafka consumer lag for mw_page_html_content_change_enrich in eqiad - https://wikitech.wikimedia.org/wiki/MediaWiki_Event_Enrichment/HTML_Enrichment#Alerting - https://grafana.wikimedia.org/d/K9x0c4aVk/flink-app?orgId=1&var-datasource=eqiad%20prometheus/k8s-dse&var-namespace=mw-page-html-content-change-enrich&var-helm_release=production&var-operator_name=All&var-flink_job_name=mw_page_html_content_change_enrich - ... [02:47:41] https://alerts.wikimedia.org/?q=alertname%3DMediawikiPageHtmlContentChangeEnrichHighKafkaConsumerLag [02:48:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12206348 (10Ahoelzl) Following up on todays meeting, @Eevans do you sign off on table 1 implement... [02:49:09] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12206353 (10Ahoelzl) a:03Eevans [02:52:41] RESOLVED: MediawikiPageHtmlContentChangeEnrichHighKafkaConsumerLag: ... [02:52:41] High Kafka consumer lag for mw_page_html_content_change_enrich in eqiad - https://wikitech.wikimedia.org/wiki/MediaWiki_Event_Enrichment/HTML_Enrichment#Alerting - https://grafana.wikimedia.org/d/K9x0c4aVk/flink-app?orgId=1&var-datasource=eqiad%20prometheus/k8s-dse&var-namespace=mw-page-html-content-change-enrich&var-helm_release=production&var-operator_name=All&var-flink_job_name=mw_page_html_content_change_enrich - ... [02:52:41] https://alerts.wikimedia.org/?q=alertname%3DMediawikiPageHtmlContentChangeEnrichHighKafkaConsumerLag [08:45:17] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 3 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#12206811 (10ops-monitoring-bot) Section s3: Wikis testwiki redacted - fceratto@cumin1003 [11:00:53] 06Data-Engineering, 06Data-Engineering-Radar, 06Privacy Engineering, 06Security-Team, and 3 others: Privacy review of x1 tables in preparation of adding them to wikireplicas - https://phabricator.wikimedia.org/T415219#12207217 (10ops-monitoring-bot) Section s3: Wikis testwiki redacted - fceratto@cumin1003 [12:03:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt: Jobs with associated data tests do not run the main job - https://phabricator.wikimedia.org/T434516#12207364 (10KCVelaga_WMF) I have cleared the task instances for https://airflow.wikimedia.org/dags/dbt_daily/grid?search=dbt_daily&task_id=product-an... [12:04:36] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: dbt: Jobs with associated data tests do not run the main job - https://phabricator.wikimedia.org/T434516#12207368 (10KCVelaga_WMF) [15:22:55] (03CR) 10Mforns: [C:03+2] HQLs to create and import Cloudflare bot directory. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [15:23:02] (03CR) 10Mforns: [V:03+2 C:03+2] HQLs to create and import Cloudflare bot directory. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319913 (https://phabricator.wikimedia.org/T430673) (owner: 10KCVelaga) [15:27:31] !log Starting deployment of Refinery for weekly deployment train at 1319913: HQLs to create and import Cloudflare bot directory. [15:27:33] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:42:07] !log Finished deployment of Refinery for weekly deployment train at 1319913: HQLs to create and import Cloudflare bot directory. [15:42:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:42:56] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: [Mediawiki History] Split table into separate user/page/revision history tables - https://phabricator.wikimedia.org/T434683 (10Isaac) 03NEW [16:12:32] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12208619 (10ppelberg) [16:12:42] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 10EditCheck, 10VisualEditor, and 3 others: Implement sampling requirements for Headless VE technical pilot - https://phabricator.wikimedia.org/T433024#12208621 (10ppelberg) [16:38:49] 06Data-Engineering, 06MediaWiki-API-Platform-Team, 10Event-Platform: mediawiki.page_change.v1 - adapt page change kind model to MediaWiki's PageUpdateCauses. - https://phabricator.wikimedia.org/T430588#12208832 (10Ottomata) [16:38:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#12208833 (10Ottomata) [16:39:33] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 10Event-Platform: Incremental MWH - MediaWiki event data source improvements - https://phabricator.wikimedia.org/T423935#12208837 (10Ottomata) [16:39:34] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 07Epic: Incremental MediaWiki History - v2 - https://phabricator.wikimedia.org/T432025#12208838 (10Ottomata) [16:39:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt isolated model orchestration - https://phabricator.wikimedia.org/T430625#12208841 (10Ahoelzl) a:03amastilovic [16:40:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622#12208845 (10Ahoelzl) a:03amastilovic [16:40:20] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE: Create Kerberos identity for Randall Scout - https://phabricator.wikimedia.org/T430598#12208847 (10Ottomata) [16:44:04] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE, 10SRE-Access-Requests: Create Kerberos identity for Randall Scout - https://phabricator.wikimedia.org/T430598#12208906 (10Milimetric) @Rscout The process to follow here is: https://wikitech.wikimedia.org/wiki/Data_Platform/Data_access#Requ... [16:47:34] 06Data-Engineering, 06Data-Platform-SRE: Enable Superset feature flag for Alerts - https://phabricator.wikimedia.org/T430682#12208924 (10Ahoelzl) a:03Ahoelzl [16:48:12] 06Data-Engineering, 06Data-Platform-SRE: Enable Superset feature flag for Alerts - https://phabricator.wikimedia.org/T430682#12208931 (10Ahoelzl) We might be missing the Superset scheduler for this... ? [16:50:36] 06Data-Engineering, 06Product-Analytics: Add cache_status to pageview_actor table - https://phabricator.wikimedia.org/T431246#12208948 (10Ottomata) 05Open→03Declined This data should be in [[ https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf_traffic.webrequest_v2,PROD)/... [16:51:38] 06Data-Engineering: Productionize the updates to the wmf_traffic.browser_info table - https://phabricator.wikimedia.org/T431317#12208958 (10Ottomata) a:03mforns [16:51:40] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Spark queries of Google Search Console data fail due to Zstandard error - https://phabricator.wikimedia.org/T434625#12208959 (10Ahoelzl) p:05Triage→03High a:03Antoine_Quhen [16:51:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Productionize the updates to the wmf_traffic.browser_info table - https://phabricator.wikimedia.org/T431317#12208965 (10Ottomata) [16:52:45] 06Data-Engineering, 10DPE-Mediawiki-Content, 10Dumps-Generation: Recently edited pages are not included in MediaWiki Content File Exports - https://phabricator.wikimedia.org/T431339#12208967 (10Ottomata) →14Duplicate dup:03T431872 [16:52:46] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Dumps-Generation: mediawiki_content_current exports silently omit all pages edited between the snapshot cutoff and dump generation time (enwiki 2026-07-01: ~3,560 pages missing per partition file) - https://phabricator.wikimedia.org/T431872#12208969 (... [16:56:00] 06Data-Engineering, 06Data-Engineering-Radar, 10EventStreams, 10Instrument-ClientError, 10MediaWiki-extensions-Gadgets: Strategy for avoiding or excluding client side errors from gadgets and user scripts - https://phabricator.wikimedia.org/T262493#12208990 (10Ottomata) [17:00:23] 06Data-Engineering, 06Commons, 06Data-Persistence, 07OKR-Work: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12209014 (10Ottomata) a:03Ahoelzl [17:00:33] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Commons, 06Data-Persistence, 07OKR-Work: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12209017 (10Ottomata) [17:01:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Spark queries of Google Search Console data fail due to Zstandard error - https://phabricator.wikimedia.org/T434625#12209022 (10Antoine_Quhen) We now have spark 3.5 on the cluster. If you try with `spark35-sql`, it works. We picked a higher compression... [17:35:03] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12209132 (10Snwachukwu) The numbers look good to m... [17:41:12] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Backfill commonswiki and enwiki HTML for latest HTML when non-existent in event.mediawiki_page_html_content_change_v1 - https://phabricator.wikimedia.org/T426347#12209143 (10Isaac) > The numbers look good to me.... [20:10:14] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Create gb_block_email in the globalblocks table on WMF wikis - https://phabricator.wikimedia.org/T413198#12209653 (10bd808) [21:12:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [21:27:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [22:55:44] 06Data-Engineering, 06Java-Scala-Standardization, 07Essential-Work, 13Patch-For-Review: Decouple airflow-dags repository from Archiva - https://phabricator.wikimedia.org/T434160#12210188 (10amastilovic) [22:56:50] 06Data-Engineering, 06Java-Scala-Standardization, 07Essential-Work, 13Patch-For-Review: Decouple airflow-dags repository from Archiva - https://phabricator.wikimedia.org/T434160#12210189 (10amastilovic)