[00:50:42] 06Data-Engineering, 06Data-Engineering-Radar, 10Wmfdata-Python, 06Movement-Insights (FY25-26 H1): Support creating a Spark session with a GitLab-built Conda environment - https://phabricator.wikimedia.org/T378253#11431342 (10nshahquinn-wmf) >>! In T378253#10381138, @nshahquinn-wmf wrote: > @fkaelin has pre... [05:53:35] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11431555 (10Marostegui) [05:53:43] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11431556 (10Marostegui) [09:31:43] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598#11431834 (10JMonton-WMF) We had a discussion on Slack, we'll start with 3 partitions, it should help balance the storage across the brokers, speed... [10:29:52] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Create new Druid datasource based on the `mediawiki_revision_history_v1` table - https://phabricator.wikimedia.org/T411484#11431959 (10JAllemandou) First investigation: I've looked fo... [11:56:05] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Create new Druid datasource based on the `mediawiki_revision_history_v1` table - https://phabricator.wikimedia.org/T411484#11432242 (10JAllemandou) First thing: new snapshot (`2025-11... [12:12:19] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Create new Druid datasource based on the `mediawiki_revision_history_v1` table - https://phabricator.wikimedia.org/T411484#11432321 (10JAllemandou) One another thing I found is that s... [16:37:04] 06Data-Engineering, 10ChangeProp, 10EventStreams, 10iPoid-Service, and 17 others: Migrate node-based services in production to node22 - https://phabricator.wikimedia.org/T393434#11433400 (10JVanderhoop-WMF) [16:37:35] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598#11433405 (10xcollazo) > because this process runs only once a month. Daily reconcile does run every day, but it generates ~10k events. Monthly r... [16:53:41] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Fix reconcile bug where user_id is not being populated correctly. - https://phabricator.wikimedia.org/T411803 (10xcollazo) 03NEW [16:54:14] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Create new Druid datasource based on the `mediawiki_revision_history_v1` table - https://phabricator.wikimedia.org/T411484#11433508 (10xcollazo) Opened {T411803} to fix the reconcile... [17:31:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598#11433635 (10JMonton-WMF) That's interesting. In general, I wouldn't reduce the number of partitions of a topic to slow down a pipeline, I assume... [17:45:21] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Research, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#11433691 (10fkaelin) This html stream (or rather the events table that will be gobblined to the datalake) will be the... [18:34:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598#11433863 (10xcollazo) > In my opinion, in general, it looks potentially a bit dangerous to have a single partition on a topic; for example, if due... [19:34:43] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10GrowthExperiments, and 7 others: mw.track: support for histogram metrics - https://phabricator.wikimedia.org/T383563#11434353 (10Michael) I'm now encountering this issue //again// for our current OKR work on the Revise Tone Structured Task and... [20:51:46] (03PS1) 10Milimetric: Add logic and table for export to fundraising [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1215266 [21:47:44] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), and 2 others: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821 (10Dreamy_Jazz) 03NEW [21:48:00] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434790 (10Dreamy_Jazz) [21:50:07] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434797 (10Dreamy_Jazz) [21:52:53] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434805 (10Dreamy_Jazz) [21:54:11] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434807 (10Dreamy_Jazz) [21:54:42] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434809 (10Dreamy_Jazz) [21:57:15] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434824 (10Dreamy_Jazz) [21:57:25] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434827 (10Dreamy_Jazz) [22:00:20] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11434834 (10Dreamy_Jazz) a:03Dream... [22:00:54] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10GrowthExperiments, and 7 others: mw.track: support for histogram metrics - https://phabricator.wikimedia.org/T383563#11434840 (10colewhite) >>! In T383563#11434353, @Michael wrote: > However, I lack the knowledge about what happens afterward. [... [22:28:03] 06Data-Engineering, 10Data-Platform, 06Wikimedia Enterprise: Provide auth-less access to Enterprise APIs from WMF Analytics cluster - https://phabricator.wikimedia.org/T403298#11434939 (10andrea.denisse) [22:28:16] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 06Wikimedia Enterprise: Provide auth-less access to Enterprise APIs from WMF Analytics cluster - https://phabricator.wikimedia.org/T403298#11434941 (10andrea.denisse) [23:53:09] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 06DBA, 06Product Safety and Integrity (Sprint Mince Pie Dec 1 - Dec 12), 07Schema-change: Add sic_updated_timestamp column and associated indexes to the cusi_case table - https://phabricator.wikimedia.org/T411821#11435125 (10Ladsgroup) LGTM, I ask...