[02:57:13] 06Data-Engineering, 06Product-Analytics, 10Event-Platform, 13Patch-For-Review: Enable Event Platform instruments to opt out of collecting User-Agent data - https://phabricator.wikimedia.org/T382173#10469169 (10Ottomata) I opted for Option 1b with the slight tweak: manage defaults fully in EventStreamConfig... [03:54:15] 06Data-Engineering: NEW/CHANGE FEATURE REQUEST: - https://phabricator.wikimedia.org/T383959 (10VirginiaPoundstone) 03NEW [10:06:44] 06Data-Engineering, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 13Patch-For-Review: Create a container image for analytics/refinery to be used with Airflow tasks - https://phabricator.wikimedia.org/T383417#10469481 (10Gehel) [10:06:45] 06Data-Engineering, 07Epic: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#10469482 (10Gehel) [11:02:52] 06Data-Engineering, 10Dumps 2.0, 10Dumps-Generation, 06SRE: Dumps generation cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#10469674 (10MPGuy2824) [11:33:09] 06Data-Engineering, 10EventStreams: eventstreams is hitting memory limits, causing restarts and paging - https://phabricator.wikimedia.org/T383977 (10hnowlan) 03NEW [11:33:47] 06Data-Engineering, 10EventStreams, 06SRE: eventstreams is hitting memory limits, causing restarts and paging - https://phabricator.wikimedia.org/T383977#10469769 (10hnowlan) p:05Triage→03High [13:08:35] 06Data-Engineering, 06Growth-Team, 10GrowthExperiments, 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10470342 (10Urbanecm_WMF) Thanks for the report @daimona.... [13:31:18] 06Data-Engineering, 10MediaWiki-extensions-EventLogging: Allow JavaScript errors to fail CI builds - https://phabricator.wikimedia.org/T318902#10470462 (10VirginiaPoundstone) Chatted with @phuedx, who agrees, but this is unrelated to experiment team work. Who is better suited, maybe Observability? @kostajh & @... [13:36:04] 14Analytics, 06Data-Engineering, 10Event-Platform: EventGate and EventStreams rate limiting - https://phabricator.wikimedia.org/T256891#10470480 (10Ottomata) [13:37:37] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10470503 (10VirginiaPoundstone) {T267251} is an older data catalog task requesting this work. Mentionin... [13:39:39] 06Data-Engineering, 06Growth-Team, 10GrowthExperiments, 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10470521 (10Ladsgroup) Hi, this should have enough inform... [13:40:25] 06Data-Engineering, 06Growth-Team, 10GrowthExperiments, 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10470525 (10Ladsgroup) We should remove boolean from the... [13:42:46] 06Data-Engineering, 06Experimentation Lab, 10Event-Platform: Document instructions for deleting an event stream and its usages - https://phabricator.wikimedia.org/T360210#10470534 (10VirginiaPoundstone) [13:50:52] 06Data-Engineering, 06Data-Engineering-Icebox, 10MediaWiki-extensions-EventLogging, 13Patch-Needs-Improvement: Deprecate/delete the mw.eventLog.Schema class - https://phabricator.wikimedia.org/T305491#10470572 (10VirginiaPoundstone) [13:53:22] 14Analytics-Radar, 06Data-Engineering, 06Experimentation Lab: mw.user.generateRandomSessionId should return a UUID - https://phabricator.wikimedia.org/T266813#10470605 (10phuedx) [14:11:09] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, 07Epic: Deprecate and remove MetricsClient#dispatch() - https://phabricator.wikimedia.org/T352969#10470664 (10VirginiaPoundstone) [14:11:19] 06Data-Engineering, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging: Generate $wgEventLoggingStreamNames from $wgEventStreams - https://phabricator.wikimedia.org/T303602#10470666 (10VirginiaPoundstone) [14:11:29] 14Analytics, 06Data-Engineering, 06Experimentation Lab, 10Event-Platform: Source geolocation directly rather than using IP in schema - https://phabricator.wikimedia.org/T290014#10470667 (10VirginiaPoundstone) [14:13:10] 06Data-Engineering, 06Data-Engineering-Icebox, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging: Determine what UserBucketService::getUserEditCountBucket should return for anons - https://phabricator.wikimedia.org/T329292#10470683 (10phuedx) [14:16:15] 06Data-Engineering, 10MediaWiki-extensions-EventLogging: Develop strategy for mitigating degenerate client timestamps in event data - https://phabricator.wikimedia.org/T259373#10470690 (10VirginiaPoundstone) [14:16:44] 06Data-Engineering, 10MediaWiki-extensions-EventLogging: Develop strategy for mitigating degenerate client timestamps in event data - https://phabricator.wikimedia.org/T259373#10470692 (10VirginiaPoundstone) [14:38:24] 06Data-Engineering, 06Growth-Team, 10GrowthExperiments, 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10470950 (10Daimona) >>! In T383778#10470341, @Urbanecm_W... [14:43:41] (03CR) 10Mforns: [V:03+2 C:03+2] Add event_context to sanitization allowlist for Content Translation events. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1109039 (https://phabricator.wikimedia.org/T373785) (owner: 10KCVelaga) [15:18:59] 06Data-Engineering, 10GrowthExperiments, 10Growth-Team (Current Sprint), 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10471147 (10Urbanecm_WMF) a:03Urbanecm_... [15:28:26] 06Data-Engineering, 10GrowthExperiments, 10Growth-Team (Current Sprint), 13Patch-For-Review, 07Schema-change: The `growthexperiments_mentor_mentee.gemm_mentee_is_active` DB column usees the "boolean" type, which is not supported with postgres - https://phabricator.wikimedia.org/T383778#10471223 (10Urbanec... [15:46:02] 06Data-Engineering, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432#10471396 (10Scott_French) [16:29:28] 06Data-Engineering, 06Data-Engineering-Radar, 10CampaignEvents, 06Data-Persistence, and 5 others: Add "event_is_test_event" field to "campaign_events" table - https://phabricator.wikimedia.org/T381759#10471546 (10Daimona) Nothing to review here. @MHorsey-WMF were you planning to use this task for https://w... [17:04:38] 06Data-Engineering: Epic - update Wikistats dependencies - https://phabricator.wikimedia.org/T384042 (10Milimetric) 03NEW [17:04:48] 06Data-Engineering: Epic - update Wikistats dependencies - https://phabricator.wikimedia.org/T384042#10471756 (10Milimetric) p:05Triage→03Medium [17:06:46] 06Data-Engineering: Migrate from Webpack to Vite - https://phabricator.wikimedia.org/T384045 (10Milimetric) 03NEW [17:08:51] 06Data-Engineering, 07Essential-Work: Migrate from Vue 2 to Vue 3 - https://phabricator.wikimedia.org/T384046 (10Milimetric) 03NEW [17:08:58] 06Data-Engineering, 07Essential-Work: Epic - update Wikistats dependencies - https://phabricator.wikimedia.org/T384042#10471813 (10Milimetric) [17:09:19] 06Data-Engineering, 07Essential-Work: Migrate from Webpack to Vite - https://phabricator.wikimedia.org/T384045#10471814 (10Milimetric) p:05Triage→03Medium [17:10:34] 06Data-Engineering, 07Essential-Work: Migrate from Semantic UI to Codex - https://phabricator.wikimedia.org/T384047 (10Milimetric) 03NEW [17:10:42] 06Data-Engineering, 07Essential-Work: Migrate from Semantic UI to Codex - https://phabricator.wikimedia.org/T384047#10471832 (10Milimetric) p:05Triage→03Medium [17:11:32] 06Data-Engineering, 06Commons, 06Data-Persistence, 10MediaWiki-File-management, and 4 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#10471836 (10Ladsgroup) >>! In T28741#10458681, @TheDJ wrote: >>>! In T28741#1... [17:12:40] 06Data-Engineering, 07Essential-Work: Update tests - https://phabricator.wikimedia.org/T384048 (10Milimetric) 03NEW [17:13:38] !log Deployed latest DAGs for 'analytics' Airflow instance. T366542. [17:13:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:13:41] T366542: Consider renaming columns and/or table to abide by the data modeling guidelines - https://phabricator.wikimedia.org/T366542 [17:14:08] 06Data-Engineering, 07Essential-Work: Update routing - https://phabricator.wikimedia.org/T384049 (10Milimetric) 03NEW [17:15:36] 06Data-Engineering, 07Essential-Work: Update state management - https://phabricator.wikimedia.org/T384050 (10Milimetric) 03NEW [17:16:50] 06Data-Engineering, 07Essential-Work: Update state management - https://phabricator.wikimedia.org/T384050#10471896 (10Milimetric) p:05Triage→03Medium [17:17:00] 06Data-Engineering, 07Essential-Work: Update routing - https://phabricator.wikimedia.org/T384049#10471899 (10Milimetric) p:05Triage→03Medium [17:17:05] 06Data-Engineering, 07Essential-Work: Update tests - https://phabricator.wikimedia.org/T384048#10471900 (10Milimetric) p:05Triage→03Medium [17:17:18] 06Data-Engineering, 07Essential-Work: Migrate from Vue 2 to Vue 3 - https://phabricator.wikimedia.org/T384046#10471902 (10Milimetric) p:05Triage→03Medium [18:07:49] 06Data-Engineering, 06Campaigns-Product-Team, 10Product-Analytics (Kanban): Create ETL pipelines for campaigns-product baseline metrics - https://phabricator.wikimedia.org/T374491#10472038 (10mpopov) [21:37:19] 06Data-Engineering, 10Dumps 2.0 (Kanban Board), 10Event-Platform: [Event Platform] Instrument EventBus with prometheus MW Statslib - https://phabricator.wikimedia.org/T363587#10472707 (10xcollazo) @gmodena: Anything else to do here, or can we close? [22:47:02] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Movement-Insights, 13Patch-For-Review: Test airflow cascading-reruns for projectview-hourly dependent jobs - https://phabricator.wikimedia.org/T383804#10473002 (10Ahoelzl) [23:30:04] 10Data-Engineering (Q2 2024 October 1st - December 31th), 13Patch-For-Review: Migrate and re-deploy eventstreams using service-utils - https://phabricator.wikimedia.org/T361769#10473071 (10Ahoelzl) [23:50:12] 06Data-Engineering, 06Data-Engineering-Icebox: [Spike] Define technology roadmap around Airflow / k8s / ceph - https://phabricator.wikimedia.org/T361509#10473097 (10Ahoelzl) [23:51:20] 06Data-Engineering, 06Data-Engineering-Icebox: [Spike] [Maintenance] Define late arrival event strategy and idem-potent backfilling concept. - https://phabricator.wikimedia.org/T361503#10473101 (10Ahoelzl) [23:55:29] 06Data-Engineering, 10Event-Platform: Orchestrate gobblin ingestion task with Airflow and config store. - https://phabricator.wikimedia.org/T361094#10473105 (10Ahoelzl) Can we close this? I don't think this is relevant anymore. [23:55:52] 06Data-Engineering, 06Data-Engineering-Icebox, 10Event-Platform: Orchestrate gobblin ingestion task with Airflow and config store. - https://phabricator.wikimedia.org/T361094#10473108 (10Ahoelzl) [23:56:04] 06Data-Engineering, 06Data-Engineering-Icebox: [Spike] List out SystemD timers migration targets - https://phabricator.wikimedia.org/T361507#10473110 (10Ahoelzl) [23:56:34] 06Data-Engineering, 06Data-Engineering-Icebox: [Iceberg Migration] Migrate pageview tables to Iceberg - https://phabricator.wikimedia.org/T347690#10473114 (10Ahoelzl) [23:57:38] 06Data-Engineering: [Maintenance] Add a deletion job for `hdfs_usage` data - https://phabricator.wikimedia.org/T348774#10473116 (10Ahoelzl) @JAllemandou hasn't this been addressed already? [23:59:49] 06Data-Engineering: Delete reportupdater jobs data/puppet-code - https://phabricator.wikimedia.org/T358210#10473121 (10Ahoelzl) @JAllemandou hasn't this been completed?