[01:15:03] 06Data-Engineering, 06Data Products, 10Event-Platform: Add schema diffing support to jsonschema-tools and run diff in CI - https://phabricator.wikimedia.org/T321850#10291369 (10Ottomata) [07:50:11] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Data Pipelines, 10Data-Catalog: Upgrade to Spark 3.2 to support Spark lineage for Iceberg tables - https://phabricator.wikimedia.org/T378899#10291596 (10tchin) a:05tchinβ†’03None [09:16:33] 06Data-Engineering, 06Machine-Learning-Team, 06Research, 10Event-Platform: Expose revision revert risk scores in EventStreams - https://phabricator.wikimedia.org/T326179#10291750 (10kostajh) > 1. Create a new mediawiki.page_revert_risk_prediction_change.v1 stream, and produce events to it, either via chang... [09:52:51] 06Data-Engineering, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Request Kerberos identity for jsn.sherman - https://phabricator.wikimedia.org/T378786#10291834 (10MatthewVernon) 05Openβ†’03Resolved a:03MatthewVernon Hi @jsn.sherman this is all done for you now. [10:29:29] 06Data-Engineering, 06Movement-Insights, 10Wikidata, 10Wikidata Analytics, 10Wmfdata-Python: Add linter and formatter to wmfdata-python (and link check) - https://phabricator.wikimedia.org/T348999#10291947 (10AndrewTavis_WMDE) Awesome to hear! Thanks for working on this, @fkaelin! 😊 [10:29:37] (03PS1) 10Joal: Update cassandra_pageview_top_articles_monthly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1087409 [10:37:51] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10Dumps 2.0 (Kanban Board): Identify indicators to inform an SLO for event emission and intake - https://phabricator.wikimedia.org/T345195#10291975 (10gmodena) [10:40:02] 10Data-Engineering (Q2 2024 October 1st - December 31th), 03Discovery-Search (Current work), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Add support for Spark producers in Event Platform - https://phabricator.wikimedia.org/T374341#10291996 (10gmodena) [11:01:33] 06Data-Engineering, 06Data Products, 10MediaWiki-extensions-EventLogging, 10Temporary accounts: Prepare EventLogging for temp accounts - https://phabricator.wikimedia.org/T374812#10292054 (10kostajh) >>! In T374812#10157537, @VirginiaPoundstone wrote: > @Ahoelzl and @lbowmaker this work is part of DPE's te... [11:01:58] 06Data-Engineering, 10Temporary accounts, 10Event-Platform: Prepare EventBus for temp accounts - https://phabricator.wikimedia.org/T374811#10292057 (10kostajh) @gmodena is there an estimated completion date for this task? Thanks! [12:17:13] (03CR) 10Mforns: [C:03+1] "Thank you!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1083192 (https://phabricator.wikimedia.org/T377259) (owner: 10GOlson) [13:23:16] (03PS1) 10Dbrant: Correctly hash IDs in app_patroller_experience stream. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1087453 (https://phabricator.wikimedia.org/T358299) [13:25:19] !log Manually run cassandra_load_pageview_top_articles_monthly for moth 2024-10 to test a fix [13:25:21] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:27:31] (03PS2) 10Joal: Update cassandra_pageview_top_articles_monthly [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1087409 [13:31:41] 06Data-Engineering, 10Temporary accounts, 10Event-Platform: Prepare EventBus for temp accounts - https://phabricator.wikimedia.org/T374811#10292549 (10gmodena) Hey @kostajh >>! In T374811#10292057, @kostajh wrote: > @gmodena is there an estimated completion date for this task? Thanks! AFAIK there's no wor... [14:43:44] (03CR) 10Mforns: [V:03+2 C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1087453 (https://phabricator.wikimedia.org/T358299) (owner: 10Dbrant) [14:46:38] !log Try rerunning dump_month_of_daily_pageviews [14:46:40] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:49:41] 06Data-Engineering, 10Data Products (Data Products (Data Products Sprint 21 πŸͺ‚)), 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10292820 (10Milimetric) current approach is to look at https://modpython.org/ and see if we c... [15:29:47] 06Data-Engineering, 06Data-Platform-SRE, 06SRE, 10SRE-Access-Requests: Request Kerberos identity for jsn.sherman - https://phabricator.wikimedia.org/T378786#10292979 (10jsn.sherman) >>! In T378786#10291834, @MatthewVernon wrote: > Hi @jsn.sherman this is all done for you now. Thanks @MatthewVernon! I w... [16:30:02] 06Data-Engineering, 06Machine-Learning-Team, 06Research, 10Event-Platform: Expose revision revert risk scores in EventStreams - https://phabricator.wikimedia.org/T326179#10293306 (10Ottomata) Hm, yes I think both could be done! However, where should the 'fattening' of recentchange event data stop? Shou... [16:34:22] 06Data-Engineering, 10Data Products (Data Products (Data Products Sprint 21 πŸͺ‚)), 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10293325 (10Ottomata) schema.wikimedia.org is currently hosted using nginx. Could change tho... [16:36:29] 06Data-Engineering, 10Data Products (Data Products (Data Products Sprint 21 πŸͺ‚)), 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10293330 (10Ottomata) >Β The problem will be that jsfh renders the schema file but also render... [16:53:39] 06Data-Engineering, 06Data Products, 10Event-Platform: Add schema diffing support to jsonschema-tools and run diff in CI - https://phabricator.wikimedia.org/T321850#10293452 (10Ottomata) My MR that adds a `jsonschema-tools diff` subcommand is nice. We could configured GitLab CI to run it in the schema repos... [16:56:44] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Privacy Engineering: Add cu_log_event and cu_private_event CheckUser tables to data lake - https://phabricator.wikimedia.org/T376752#10293468 (10Ottomata) Question: > The data in cu_private_event is needed to do proper analysis for T37... [17:10:19] (03CR) 10Mforns: [V:03+2 C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1087409 (owner: 10Joal) [17:10:27] 06Data-Engineering, 10Metrics Platform, 06Wikipedia-Android-App-Backlog: Revert Metrics Platform changes that were made for the Recommended Content feature - https://phabricator.wikimedia.org/T379031#10293574 (10JTannerWMF) [17:12:12] 14Analytics, 06Data-Engineering-Icebox, 10Observability-Logging, 10SRE Observability (FY2024/2025-Q2): Augment NEL reports with a computed timestamp-of-generation - https://phabricator.wikimedia.org/T266886#10293595 (10lmata) [19:12:42] (03CR) 10Mforns: Add New Edit Hourly HQL for Temp Account Change (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1084873 (https://phabricator.wikimedia.org/T377767) (owner: 10Jennifer Ebe) [19:53:55] 06Data-Engineering, 10Temporary accounts, 10Event-Platform: Prepare EventBus for temp accounts - https://phabricator.wikimedia.org/T374811#10294169 (10kostajh) 05Openβ†’03Resolved >>! In T374811#10292549, @gmodena wrote: > Hey @kostajh > >>>! In T374811#10292057, @kostajh wrote: >> @gmodena is there... [21:06:58] (03PS1) 10Jdlrobson: Clear repository to make it clearer it has moved [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1087573 (https://phabricator.wikimedia.org/T366836) [22:29:41] 10Data-Engineering (Q2 2024 October 1st - December 31th), 03Discovery-Search (Current work), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Add support for Spark producers in Event Platform - https://phabricator.wikimedia.org/T374341#10294667 (10Ahoelzl) [22:31:01] 10Data-Engineering (Q2 2024 October 1st - December 31th), 03Discovery-Search (Current work), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Add support for Spark producers in Event Platform - https://phabricator.wikimedia.org/T374341#10294673 (10Ahoelzl) @gmodena @pfischer can you please add a definition... [23:42:46] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Privacy Engineering: Add cu_log_event and cu_private_event CheckUser tables to data lake - https://phabricator.wikimedia.org/T376752#10294789 (10Ahoelzl) @Dreamy_Jazz @Tgr thanks for providing your input. do we agree on not moving forwa...