[07:08:53] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Product-Analytics: dbt-jobs backfill: PP3 API hourly and known clients aggregate jobs - https://phabricator.wikimedia.org/T429341#12039216 (10KCVelaga_WMF) [07:08:54] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Request for backfill of webrequest with is_api_request and ip_provenance columns - https://phabricator.wikimedia.org/T427474#12039218 (10KCVelaga_WMF) 05Open→03Declined a:05amastilovic→03None Note: We have decided not to backfill webrequest. The... [07:30:55] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Bug fix: populate event_user_groups_historical for revisions and pages - https://phabricator.wikimedia.org/T428928#12039281 (10JAllemandou) Data has been backfilled, we now have information for most events. {T429753} wi... [09:27:58] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Product-Analytics: dbt-jobs backfill: PP3 API hourly and known clients aggregate jobs - https://phabricator.wikimedia.org/T429341#12039807 (10amastilovic) [09:33:40] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History, 07Epic, 13Patch-For-Review: Incremental MediaWiki History Phase I - https://phabricator.wikimedia.org/T424350#12039835 (10APizzata-WMF) [09:36:34] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Fix inconsistent use of wmf_log in dbt-jobs - https://phabricator.wikimedia.org/T429771 (10amastilovic) 03NEW [10:19:22] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12039983 (10Mwex) 05Open→03In progress [10:24:20] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Create a data product of IP range to owner/provenance label - https://phabricator.wikimedia.org/T418466#12039992 (10GGoncalves-WMF) @CDanis and I briefly talked about this one, and how it relates to T427068 which is nearly complete and has the edge send IP p... [12:32:21] 06Data-Engineering, 06Data-Engineering-Icebox, 06DBA, 13Patch-For-Review: Move Mostcategories computation to Hadoop - https://phabricator.wikimedia.org/T413362#12040460 (10JAllemandou) Hey @Zabe , the problem comes from the sensor config ([[ https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/... [12:38:52] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12040537 (10GGoncalves-WMF) [14:13:30] 06Data-Engineering, 10Event-Platform: Sanitize stream name in all EventGate metrics - https://phabricator.wikimedia.org/T429799 (10phuedx) 03NEW [14:33:21] 06Data-Engineering, 10Event-Platform: Sanitize stream name in all EventGate metrics - https://phabricator.wikimedia.org/T429799#12040972 (10Ottomata) Hm! So this is just some client sent junk/spam? Interesting! Right, and `applyTransforms` (which includes hoisting logic) is [[ https://gitlab.wikimedia.org/r... [14:56:48] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 10Test Kitchen, 07Essential-Work: Implement/enforce 90 day data retention policy in derived Iceberg tables - https://phabricator.wikimedia.org/T429548#12041109 (10AKhatun_WMF) a:03AKhatun_WMF [15:27:00] !log Test Kitchen experiment (poll 26358) - adds: none; removes: logged-out-retention-test-growthbook-cs; fields: none - TK tips at https://w.wiki/_cvdP [15:27:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:27:20] !log Test Kitchen experiment (poll 26360) - adds: none; removes: logged-in-retention-test-growthbook, logged-out-retention-test-growthbook-ncs; fields: none - TK tips at https://w.wiki/_cvdP [15:27:23] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:30:58] 06Data-Engineering, 06Traffic: Tune refine webrequest data loss threshold to avoid noisy irrelevant alerts. - https://phabricator.wikimedia.org/T429809 (10Ottomata) 03NEW [16:14:11] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Make cusi_user and cusi_case table information available in the data lake - https://phabricator.wikimedia.org/T429703#12041642 (10Ahoelzl) [17:01:03] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06Privacy Engineering, and 2 others: Add global_edit_count to wikireplicas - https://phabricator.wikimedia.org/T344108#12041857 (10sbassett) [17:40:24] 14Analytics, 06Data-Engineering, 06Data-Engineering-Icebox: Rework how mediawiki-history differentiates fake page-create from real ones - https://phabricator.wikimedia.org/T264791#12041920 (10JAllemandou)