[07:32:34] 06Data-Engineering, 06Infrastructure-Foundations, 06Data-Platform-SRE (2025.09.26 - 2025.10.17), 13Patch-For-Review: Also intake Network Error Logging events into the Analytics Data Lake - https://phabricator.wikimedia.org/T304373#11270367 (10brouberol) @CDanis Sorry, I was on PTO. I can see from SAL that... [08:20:11] 06Data-Engineering, 10Gerrit, 06Release-Engineering-Team: Reduce size of analytics/superset/deploy.git Gerrit repo - https://phabricator.wikimedia.org/T406856#11270575 (10brouberol) I think we can delete it without any issue. @BTullis do you concur? [08:24:10] 06Data-Engineering, 10observability: Fix Kafka replicas skew - https://phabricator.wikimedia.org/T407185 (10JMonton-WMF) 03NEW [08:25:32] 06Data-Engineering, 10observability: Fix Kafka replicas skew - https://phabricator.wikimedia.org/T407185#11270615 (10brouberol) This is something I've gone through over kafka-jumbo (see https://phabricator.wikimedia.org/T401599). Do reach out if you need some help! [09:55:57] 06Data-Engineering, 06Traffic: Request for a new request dataset for caching research - https://phabricator.wikimedia.org/T401331#11270999 (10GGoncalves-WMF) Unfortunately it looks like we won't be able to prioritize this for the next couple of months in Data Engineering, but it's still something we want to ge... [10:14:57] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): SDS 1.3.6 Import 2 new datasets of Spurs in a parquet table - https://phabricator.wikimedia.org/T407104#11271070 (10Antoine_Quhen) here is the new table: wmf_traffic.spur_feed by snapshot and each partition is split in 8 parquet files. ` snapshot... [10:16:08] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): SDS 1.3.6 Import 2 new datasets of Spurs in a parquet table - https://phabricator.wikimedia.org/T407104#11271088 (10Antoine_Quhen) 05Open→03In progress [11:21:29] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): SDS 1.3.6 First analysis review - https://phabricator.wikimedia.org/T407103#11271406 (10Antoine_Quhen) 05Open→03Resolved Awesome study. Thanks for adding the split by countries. As you asked me, I've read your Notebook and I haven't detected a... [12:27:42] (03PS1) 10Joal: Update mediawiki_history job for rev_sha1 DB removal [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1196049 (https://phabricator.wikimedia.org/T406000) [13:25:41] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 Conduct Analysis on Alerting for changes in automated traffic distribution - https://phabricator.wikimedia.org/T406882#11272155 (10Snwachukwu) [13:29:35] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Rewrite wmf_content.mediawiki_content_*_v1 tables with a new column for origin_rev_id - https://phabricator.wikimedia.org/T405944#11272220 (10xcollazo) >>! In T405944#11264818, @xcollazo wrote: > From [[... [13:32:42] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 EPIC : Automated alerting for changes in automated traffic behavior - https://phabricator.wikimedia.org/T407235 (10Snwachukwu) 03NEW [13:33:12] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 EPIC : Automated alerting for changes in automated traffic behavior - https://phabricator.wikimedia.org/T407235#11272282 (10Snwachukwu) [13:33:14] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 Conduct Analysis on Alerting for changes in automated traffic distribution - https://phabricator.wikimedia.org/T406882#11272283 (10Snwachukwu) [13:33:31] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 [EPIC] Automated alerting for changes in automated traffic behavior - https://phabricator.wikimedia.org/T407235#11272285 (10Snwachukwu) [13:37:37] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Rewrite wmf_content.mediawiki_content_*_v1 tables with a new column for origin_rev_id - https://phabricator.wikimedia.org/T405944#11272306 (10xcollazo) We need to do just one more thing: The MERGE INTO... [13:40:02] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content: Wait till november wmf_raw.mediawiki_slots sqoop table is available, and apply origin_rev_id fix to mw_content tables - https://phabricator.wikimedia.org/T407237 (10xcollazo) 03NEW [13:40:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content: Wait till november wmf_raw.mediawiki_slots sqoop table is available, and apply origin_rev_id fix to mw_content tables - https://phabricator.wikimedia.org/T407237#11272326 (10xcollazo) [13:40:14] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 07Epic: Dumps 2.0 Phase III: Production level dumps - https://phabricator.wikimedia.org/T366752#11272327 (10xcollazo) [13:41:09] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Rewrite wmf_content.mediawiki_content_*_v1 tables with a new column for origin_rev_id - https://phabricator.wikimedia.org/T405944#11272329 (10xcollazo) Opened {T407237}. Ok now we good. [13:59:03] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 07OKR-Work: SDS 1.3.2 Implementation - https://phabricator.wikimedia.org/T407239 (10Snwachukwu) 03NEW [14:06:48] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE, 10Event-Platform: Avoid accepting Kafka messages with whacky timestamps - https://phabricator.wikimedia.org/T282887#11272407 (10JMonton-WMF) [16:07:15] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 13Patch-For-Review: Adapt mediawiki_history to the removal of mediawiki revision.rev_sha1 - https://phabricator.wikimedia.org/T406000#11273095 (10JAllemandou) a:03JAllemandou [16:39:48] 06Data-Engineering, 10Gerrit, 06Release-Engineering-Team: Reduce size of analytics/superset/deploy.git Gerrit repo - https://phabricator.wikimedia.org/T406856#11273296 (10BTullis) Agreed. Please feel free to delete it from Gerrit. We no longer need it. Thanks @hashar for checking. The new version in GitLab... [17:23:58] 06Data-Engineering, 06Traffic: Request for a new request dataset for caching research - https://phabricator.wikimedia.org/T401331#11273659 (10ssingh) Traffic has a dependency on Data Engineering for this to be executed, as we will need their help to run the query and generate the data set. That being said, we... [17:59:50] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Growth-Team, 10MediaWiki-Page-derived-data, 06Wikipedia-Android-App-Backlog, and 3 others: Global Editor Metrics - HTTP API endpoints - https://phabricator.wikimedia.org/T405041#11273847 (10Ottomata) [Add edits/per-user timeseries endpoint (119... [18:26:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10Data-Platform, 06Movement-Insights (FY25-26 H1): NEWFEATURE REQUEST: Add new referral sources to pageview data - https://phabricator.wikimedia.org/T406531#11273909 (10Mayakp.wiki) [18:26:34] 06Data-Engineering, 06Data-Persistence, 06DBA, 10MediaWiki-Core-Revision-backend, and 6 others: Rethink rev_sha1 field - https://phabricator.wikimedia.org/T389026#11273910 (10Zabe) [19:11:50] 06Data-Engineering, 06Data-Engineering-Radar, 06Traffic: improved x-analytics data on Edge Uniques status - https://phabricator.wikimedia.org/T405783#11274071 (10CDanis) [19:15:03] 06Data-Engineering, 06DBA, 10MediaWiki-Recent-changes, 06Moderator-Tools-Team, 07Schema-change: Duplicate rc_name_type_patrolled_timestamp index on rc_source - https://phabricator.wikimedia.org/T401912#11274088 (10Zabe) [20:09:58] (03PS4) 10Aleksandar Mastilovic: Add user_central_id to the mediawiki_history dataset(s) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1194951 (https://phabricator.wikimedia.org/T406263) [20:15:10] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11274339 (10achou) > I want to understand wh... [21:03:29] 06Data-Engineering, 10Data-Engineering-Wikistats, 06RoadToWiki, 13Patch-For-Review: "Designed by Aislinn Grigas" at the bottom of stats.wikimedia.org leads to an inactive Squarespace account - https://phabricator.wikimedia.org/T406502#11274467 (10BCornwall) I do not have merge permissions for that repo...