[00:24:35] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Define domain folder ownership and implement it in dbt-jobs - https://phabricator.wikimedia.org/T438273 (10amastilovic) 03NEW [00:28:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Set catchup=True as default in dbt scheduler YAML configuration - https://phabricator.wikimedia.org/T438275 (10amastilovic) 03NEW [00:30:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Define and document the earliest start_date for dbt DAGs - https://phabricator.wikimedia.org/T438276 (10amastilovic) 03NEW [07:16:19] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18): Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12332050 (10Gehel) [07:17:20] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12332052 (10Gehel) [08:14:15] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12332199 (10GGoncalves-WMF) > Considering this, should we even attempt to run 7z for wikidatawiki, or should we exclude it? I think it makes... [09:51:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Fix iceberg table location in hive metastore - https://phabricator.wikimedia.org/T408939#12332647 (10JAllemandou) I have tested updating two tables location for their fully qualified versions, and it worked: ` ALTER TABLE wmf_traffi... [10:02:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Fix iceberg table location in hive metastore - https://phabricator.wikimedia.org/T408939#12332686 (10JAllemandou) Something I forgot to mention: changing the location to the fully qualified path work but doesn't change the already e... [10:05:32] (03CR) 10Milimetric: [V:03+2 C:03+2] sqoop-mediawiki-tables - add a shard override [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1335154 (https://phabricator.wikimedia.org/T431476) (owner: 10Ottomata) [10:09:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Move webrequest sequence statistics to a dashboard - https://phabricator.wikimedia.org/T436700#12332702 (10Milimetric) This has been deployed for a while and nobody seems to be complaining. The false positive checker is apparently outdated (nobody ran i... [10:54:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Growth-Team, 07OKR-Work (DE1 FY2026-27): DE1.3.2: Implement "Sliding 30 Day Top-K Pages for Editor" - https://phabricator.wikimedia.org/T437710#12332809 (10Urbanecm_WMF) This has essentially moved to DE1.3.10 if I'm understanding... [11:31:36] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: dbt: Jobs with associated data tests do not run the main job - https://phabricator.wikimedia.org/T434516#12333039 (10GGoncalves-WMF) 05Open→03Resolved [11:35:13] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Engineering-Roadmap, 07Epic: Write technical documentation for KAPOW - https://phabricator.wikimedia.org/T437332#12333058 (10Novem_Linguae) I've created https://wikitech.wikimedia.org/wiki/KAPOW as a starting point. Feel free to blank and redi... [11:40:51] 10Data-Engineering-Roadmap, 07Epic: dbt automated initial backfill - https://phabricator.wikimedia.org/T438322 (10GGoncalves-WMF) 03NEW [11:41:12] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Set catchup=True as default in dbt scheduler YAML configuration - https://phabricator.wikimedia.org/T438275#12333088 (10GGoncalves-WMF) [11:41:14] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622#12333090 (10GGoncalves-WMF) [11:41:15] 10Data-Engineering-Roadmap, 07Epic: dbt automated initial backfill - https://phabricator.wikimedia.org/T438322#12333089 (10GGoncalves-WMF) [11:50:30] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Define and document the earliest start_date for dbt DAGs - https://phabricator.wikimedia.org/T438276#12333130 (10GGoncalves-WMF) Moving this under {https://phabricator.wikimedia.org/T438322} since I think we're specifically trying to define the earliest... [11:50:54] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Define and document the earliest start_date for dbt DAGs - https://phabricator.wikimedia.org/T438276#12333134 (10GGoncalves-WMF) [11:50:55] 10Data-Engineering-Roadmap, 07Epic: dbt automated initial backfill - https://phabricator.wikimedia.org/T438322#12333135 (10GGoncalves-WMF) [11:50:57] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): dbt standardized workflow and repository structure - https://phabricator.wikimedia.org/T430622#12333136 (10GGoncalves-WMF) [12:26:02] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333336 (10brouberol) a:03brouberol [12:26:06] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333338 (10brouberol) 05Open→03In progress [12:33:56] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333410 (10brouberol) All done: {F103435530} @SCampos-WMF can you confirm it's working as intended? [13:16:35] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users level 3 for derenrich - https://phabricator.wikimedia.org/T437668#12333591 (10tappof) [13:17:25] (03CR) 10Xcollazo: [C:03+1] "Joseph has proved that the change is safe. Closing this." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1341854 (https://phabricator.wikimedia.org/T408939) (owner: 10Joal) [13:18:35] (03CR) 10Xcollazo: [C:03+1] "Discussion in 1341854 settled. We can move forward with this patch." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1341858 (https://phabricator.wikimedia.org/T437883) (owner: 10Joal) [13:18:44] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to analytics-privatedata-users level 3 for derenrich - https://phabricator.wikimedia.org/T437668#12333611 (10tappof) ` $ sudo manage_principals.py create derenrich --email=derenrich@wikimedia... [13:29:10] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333687 (10SCampos-WMF) Amazing, thank you so much @brouberol. Since this affects how long the data is sto... [13:30:51] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333711 (10brouberol) Sure thing! Let's mark this as solved and reopen (or create a new task) if you n... [13:30:55] 06Data-Engineering, 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 07Essential-Work: Matomo: Enable “for kept data above, also keep segmented reports” option - https://phabricator.wikimedia.org/T438249#12333719 (10brouberol) 05In progress→03Resolved [14:05:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Fix iceberg table location in hive metastore - https://phabricator.wikimedia.org/T408939#12333936 (10JAllemandou) List of repos to review from codesearch (regex: "using iceberg") [x] analytics/refinery (30 files) -- https://gerrit.w... [14:27:46] (03PS2) 10Snwachukwu: Add sliding 30-day top pages per editor load HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) [15:45:21] (03CR) 10Mforns: [C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) (owner: 10Snwachukwu) [15:46:50] (03CR) 10Snwachukwu: [V:03+2] Add sliding 30-day top pages per editor load HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1342105 (https://phabricator.wikimedia.org/T437230) (owner: 10Snwachukwu) [15:49:23] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12334740 (10JAllemandou) Great analysis - thank you @xcollazo. [16:03:25] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Load Google Search Console data from BigQuery as soon as it's available and reload when data corrections occur - https://phabricator.wikimedia.org/T436193#12334798 (10Antoine_Quhen) 05Open→03Resolved [16:03:28] (03CR) 10Joal: "While this looks good, I wonder if we shouldn't try a different approach for IO efficiency: We could move the XML dumper to spark 3.5, and" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [16:03:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: Load Google Search Console data into the Data Lake - https://phabricator.wikimedia.org/T420996#12334802 (10Antoine_Quhen) 05Open→03Resolved [16:06:21] (03CR) 10Joal: [V:03+2 C:03+2] "Merging" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1341854 (https://phabricator.wikimedia.org/T408939) (owner: 10Joal) [16:07:33] (03CR) 10Btullis: [C:03+1] "Looks good to me, bearing in mind that I don't really know Scala very well." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [16:10:02] 06Data-Engineering, 06Data-Engineering-Radar, 10Research-engineering: mwparserfromhtml errors - https://phabricator.wikimedia.org/T437669#12334830 (10AKhatun_WMF) Thanks! Upgraded to new version, everything is working well! [16:13:42] (03CR) 10Xcollazo: "I don't think that would help the overall cost because BZip2 compression is way faster than 7zip, so we would hold on dump-sized cluster r" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [17:06:35] (03CR) 10Xcollazo: "(reopening thread; didn't mean to close it until we agree here)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [17:23:19] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Finalize and/or standardize metrics - https://phabricator.wikimedia.org/T438383 (10AKhatun_WMF) 03NEW [17:26:22] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement alerts and create runbooks - https://phabricator.wikimedia.org/T438384 (10AKhatun_WMF) 03NEW [17:27:50] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Finalize and/or standardize metrics - https://phabricator.wikimedia.org/T438383#12335142 (10AKhatun_WMF) [17:27:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): IR2.1.2d: Feature count data product [Attribution API] - https://phabricator.wikimedia.org/T432604#12335144 (10AKhatun_WMF) [17:27:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement alerts and create runbooks - https://phabricator.wikimedia.org/T438384#12335145 (10AKhatun_WMF) [17:27:58] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): IR2.1.2d: Feature count data product [Attribution API] - https://phabricator.wikimedia.org/T432604#12335147 (10AKhatun_WMF) [17:31:41] (03PS1) 10Mforns: Switch Clickstream to use webrequest_v2 instead of pageview_actor. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342767 (https://phabricator.wikimedia.org/T425668) [18:04:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics: dbt: Weekly scheduling dependency checks only check the first date of the week - https://phabricator.wikimedia.org/T436744#12335366 (10KCVelaga_WMF) Thank you @amastilovic! [18:57:07] (03CR) 10Joal: [C:03+1] "It's a time-cost vs an IO (read) cost I guess. I'm ok with the current approach, and maybe your next investigations will advise differentl" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [19:20:50] (03CR) 10Xcollazo: "Not just time-cost though, for the biggest jobs like `enwiki`, we'd be keeping 48GB executors pinned for way longer for a job that only re" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [19:23:48] (03CR) 10Xcollazo: [C:03+2] refinery-job-35: Add 7z recompressor for the v2 history dumps [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [19:38:09] (03CR) 10CI reject: [V:04-1] refinery-job-35: Add 7z recompressor for the v2 history dumps [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342321 (https://phabricator.wikimedia.org/T437454) (owner: 10Xcollazo) [19:49:39] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Re-establish rsync dumps access to PDApps mirror - https://phabricator.wikimedia.org/T437896#12335774 (10xcollazo) Re-added PDApps to public list of mirrors at https://meta.wikimedia.org/w/index.php?title=Mirroring_Wikimedia_project... [19:49:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Re-establish rsync dumps access to PDApps mirror - https://phabricator.wikimedia.org/T437896#12335775 (10xcollazo) 05Open→03In progress [20:10:15] FIRING: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [20:10:15] FIRING: HdfsRpcQueueLatency: RPC queue latency on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=56&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLatency [20:20:15] RESOLVED: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [20:20:15] RESOLVED: HdfsRpcQueueLatency: RPC queue latency on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=56&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLatency [20:50:16] (03PS1) 10Xcollazo: refinery-spark: Distribute HdfsFileFingerprintWriter across executors [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1342809 (https://phabricator.wikimedia.org/T438244) [21:00:15] FIRING: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [21:00:15] FIRING: HdfsRpcQueueLatency: RPC queue latency on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=56&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLatency [21:05:15] RESOLVED: HdfsRpcQueueLength: RPC queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [21:05:15] RESOLVED: HdfsRpcQueueLatency: RPC queue latency on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue/latency - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=56&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLatency [22:26:05] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Explore ways of improving the dbt-jobs GitLab CI/CD process - https://phabricator.wikimedia.org/T438424 (10amastilovic) 03NEW [22:30:21] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Explore implementation options for the "dbt build" command in dbt-jobs GitLab CI/CD - https://phabricator.wikimedia.org/T438426 (10amastilovic) 03NEW