[06:26:00] 06Data-Engineering: Dbt backfill: signal_driven_handling_bad_faith_accounts - https://phabricator.wikimedia.org/T437095#12293548 (10Aklapper) [07:12:26] 14Analytics, 06Data-Engineering, 10Data-Engineering-Wikistats: Wikimedia Statistics page-views-by-country no-data - https://phabricator.wikimedia.org/T426935#12293582 (10Skif.res) Hello, so are you guys going to do anything? Thanks! [07:17:00] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Persistence-Design-Review, 10Event-Platform, 13Patch-For-Review: [Data Persistence Design Review] Relative trending pages - https://phabricator.wikimedia.org/T432352#12293584 (10JMonton-WMF) In case someone could review it, this is the patch... [08:23:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: migrate dbt_jobs pipelines to use MWH incremental - https://phabricator.wikimedia.org/T429770#12293707 (10APizzata-WMF) [09:24:59] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): 7za is OOMkilled when dumps are running on a Bookworm-based image - https://phabricator.wikimedia.org/T436913#12294090 (10brouberol) [10:14:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Create a new PostgreSQL cluster database with read-only from Superset and write access from Spark - https://phabricator.wikimedia.org/T432104#12294234 (10brouberol) I'm going to be bold and close th... [10:15:17] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 05Metrics-Sprint-2026-2027, 13Patch-For-Review: Create a new PostgreSQL cluster database with read-only from Superset and write access from Spark - https://phabricator.wikimedia.org/T432104#12294236... [12:14:33] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we import the MediaWiki databases into the S3 data lake - https://phabricator.wikimedia.org/T437079#12294551 (10JAllemandou) Thanks a lot @BTullis for having formalized all this :) Here are some comments here about what's written in the task descri... [12:23:21] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we ingest the Kafka streams into the S3 data lake - https://phabricator.wikimedia.org/T437080#12294568 (10JAllemandou) The only comment I have here is about the assumption that we want to go to from kafka to Iceberg directly. Even if we decide to h... [12:30:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Dumps-Generation: mediawiki_content_current exports silently omit all pages edited between the snapshot cutoff and dump generation time - https://phabricator.wikimedia.org/T431872#12294594 (10Dodecaplex) Hi @xcollazo Thanks for the fix, I confirm t... [12:51:46] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we import the MediaWiki databases into the S3 data lake - https://phabricator.wikimedia.org/T437079#12294624 (10BTullis) [12:52:08] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we import the MediaWiki databases into the S3 data lake - https://phabricator.wikimedia.org/T437079#12294627 (10BTullis) [13:02:31] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we import the MediaWiki databases into the S3 data lake - https://phabricator.wikimedia.org/T437079#12294644 (10BTullis) >>! In T437079#12294551, @JAllemandou wrote: > * Trying to precise the sentence "This is the closest match in shape to what Sqo... [13:13:15] 06Data-Engineering, 06Data-Platform-SRE: [Tech evaluation] How we ingest the Kafka streams into the S3 data lake - https://phabricator.wikimedia.org/T437080#12294673 (10JMonton-WMF) >>! In T437080#12294568, @JAllemandou wrote: > Even if we decide to have an Iceberg only metastore, I think the new data platform... [13:47:02] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): 7za is OOMkilled when dumps are running on a Bookworm-based image - https://phabricator.wikimedia.org/T436913#12294764 (10brouberol) Thanks to @BTullis 's change (https://gerrit.wikimedia.org/r/c/operations/dumps/+/1334846... [13:54:14] 06Data-Engineering, 10Dumps-Generation: .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220 (10brouberol) 03NEW p:05Triage→03Medium [13:54:36] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220#12294812 (10brouberol) [13:54:40] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220#12294813 (10brouberol) 05Open→03In progress [14:15:38] 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12294953 (10GGoncalves-WMF) I'd split this data across two independent dimensions: //support// (i.e. does DPE fully own/support a data product, or share ownership with a te... [14:48:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Growth-Team, 05FY 2026-27 DE1.3.2 Impact + Dashboard, and 2 others: Make the AQS endpoints more usable for the ImpactModule extension - https://phabricator.wikimedia.org/T433983#12295084 (10Urbanecm_WMF) [14:53:00] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Growth-Team, 05FY 2026-27 DE1.3.2 Impact + Dashboard, and 2 others: Make the AQS endpoints more usable for the ImpactModule extension - https://phabricator.wikimedia.org/T433983#12295123 (10Urbanecm_WMF) [15:10:27] (03CR) 10Mforns: [C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308104 (https://phabricator.wikimedia.org/T431256) (owner: 10Joal) [15:11:26] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220#12295226 (10brouberol) We have about 1TB of temporary files we need to cleanup on each host: ` brouberol@clouddumps1001:/s... [15:25:06] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220#12295263 (10brouberol) ` brouberol@clouddumps1001:/srv/dumps/xmldatadumps/public$ sudo find . -name "*.inprog" -delete bro... [15:25:12] 06Data-Engineering, 10Dumps-Generation, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): .inprog files are published as part of the public dumps - https://phabricator.wikimedia.org/T437220#12295265 (10brouberol) 05In progress→03Resolved [17:00:38] 06Data-Engineering, 10Data-Platform, 06Growth-Team, 10GrowthExperiments-ImpactModule, 10Event-Platform: EventBus throws TypeError: array_map(): Argument #2 ($array) must be of type array, int given when the event is too large - https://phabricator.wikimedia.org/T398506#12295605 (10Michael)