[03:43:22] (03CR) 10Ottomata: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [06:17:34] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11343323 (10Marostegui) [06:31:05] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11343352 (10Marostegui) @Ladsgroup codfw master has been switched at {T409168} [07:54:21] (03CR) 10Joal: "Thank you for the review @nshahquinn@wikimedia.org :) I'll be deploying this today." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) (owner: 10Joal) [07:56:01] (03PS19) 10Joal: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) [07:57:57] (03CR) 10Joal: [C:03+2] "Merging for later deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) (owner: 10Joal) [08:30:12] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11343602 (10Marostegui) [08:46:52] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11343625 (10Marostegui) [08:47:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11343629 (10Marostegui) [08:53:09] 06Data-Engineering, 10CirrusSearch, 06Data-Platform-SRE, 10DPE-Mediawiki-Content, and 4 others: Source the CirrusSearch index dumps from hadoop instead of a MW maintenance script - https://phabricator.wikimedia.org/T366248#11343661 (10brouberol) @EBernhardson if you're happy with these new dumps, do you st... [09:46:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11343856 (10achou) @dcausse Thanks a lot! I... [10:36:15] (03CR) 10Btullis: [V:03+2 C:03+2] Replace an-launcher1002 with an-launcher1003 [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1201581 (https://phabricator.wikimedia.org/T353786) (owner: 10Btullis) [10:41:31] !log deploying refinery to an-launcher1003 after merging https://gerrit.wikimedia.org/r/c/analytics/refinery/scap/+/1201581 for T353786 [10:41:35] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:41:36] T353786: Decommission an-launcher1002 - https://phabricator.wikimedia.org/T353786 [11:03:37] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Update MediaWiki Content History SLO draft for SRE review - https://phabricator.wikimedia.org/T401892#11343987 (10APizzata-WMF) I have created the [[ https://phabricator.wikimedia.org/P84877 | P84877 ]] with the results of the query. I have also change... [11:49:42] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11344206 (10Ladsgroup) Thanks! [12:54:00] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add afl_ip_hex column and afl_var_dump_timestamp index to abuse_filter_log - https://phabricator.wikimedia.org/T396130#11344353 (10Marostegui) [14:14:49] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11344751 (10Marostegui) [14:14:59] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11344752 (10Marostegui) [14:48:47] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 3 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11344958 (10elukey) @RKemper @bking cumin1002 is upgraded with the new spicerack version, please feel free... [15:04:37] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11345062 (10Ladsgroup) 05Open→03Resolved [15:04:58] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11345068 (10Ladsgroup) [15:06:40] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11345086 (10Ottomata) > Yep, we'll use media... [15:19:18] (03CR) 10Joal: [C:03+2] Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) (owner: 10Joal) [15:28:34] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11345218 (10Ladsgroup) Hi, I see the proposed schema... [15:30:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Growth-Team, 10MediaWiki-Page-derived-data, 06Wikipedia-Android-App-Backlog, and 2 others: WE3.3.7 Year in Review and Activity Tab Services - Global Editor Metrics - https://phabricator.wikimedia.org/T403660#11345228 (10Ottomata) Here is produc... [15:33:20] (03Merged) 10jenkins-bot: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) (owner: 10Joal) [15:33:28] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11345245 (10Dreamy_Jazz) >>! In T409093#11345218, @La... [15:39:23] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11345254 (10Dreamy_Jazz) Thanks for looking over this... [16:12:00] (03PS1) 10Joal: Fix mediawiki-history UDF NPE [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202191 (https://phabricator.wikimedia.org/T406000) [16:18:22] (03CR) 10A-pizzata: [C:03+1] Fix mediawiki-history UDF NPE [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202191 (https://phabricator.wikimedia.org/T406000) (owner: 10Joal) [16:20:31] (03CR) 10Joal: [C:03+2] "Bug fix - merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202191 (https://phabricator.wikimedia.org/T406000) (owner: 10Joal) [16:32:09] (03PS1) 10Joal: Update changelog for deloy of v0.3.7 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202196 [16:34:12] (03CR) 10Snwachukwu: [C:03+2] Update changelog for deloy of v0.3.7 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202196 (owner: 10Joal) [16:44:53] (03Merged) 10jenkins-bot: Update changelog for deloy of v0.3.7 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202196 (owner: 10Joal) [16:50:57] (03Abandoned) 10Ottomata: Refine refactor - Use ExplicitSchemaLoader when constructing RefineTarget [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1059429 (https://phabricator.wikimedia.org/T356762) (owner: 10Ottomata) [16:53:43] Starting build #52 for job analytics-refinery-maven-release [17:16:46] Project analytics-refinery-maven-release build #52: 09SUCCESS in 23 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release/52/ [17:32:14] (03CR) 10Ottomata: Maintain Spark schema after Hive DDL operations (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1199473 (https://phabricator.wikimedia.org/T307040) (owner: 10Aqu) [17:42:09] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10Event-Platform, 13Patch-For-Review: mediawiki_event_enrichment should enrich all events for the page_content_change stream - https://phabricator.wikimedia.org/T408850#11345825 (10JMonton-WMF) Merged and deployed. I manually searched for a few e... [17:43:54] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work: Create a dbt Docker container - https://phabricator.wikimedia.org/T406636#11345839 (10JMonton-WMF) a:05amastilovic→03JMonton-WMF [18:01:02] (03CR) 10Xcollazo: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [18:15:03] (03CR) 10Ottomata: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [19:28:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10Data-Platform, 07Essential-Work, 06Movement-Insights (FY25-26 H1): NEWFEATURE REQUEST: Add new referral sources to pageview data - https://phabricator.wikimedia.org/T406531#11346348 (10JAllemandou) The patch has been deployed, new data started... [19:54:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Compare the exported content between File Export and DumpV1 - https://phabricator.wikimedia.org/T407649#11346489 (10xcollazo) From [[ https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/me... [20:00:03] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Compare the exported content between File Export and DumpV1 - https://phabricator.wikimedia.org/T407649#11346517 (10xcollazo) Ran the following to remove all data that is unfortunately wrong due to T4076... [20:12:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Compare the exported content between File Export and DumpV1 - https://phabricator.wikimedia.org/T407649#11346582 (10xcollazo) Cleared and restarted DAGs: ` mw_content_xml_export_current_monthly mw_conten... [20:42:33] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 06MediaWiki-Platform-Team (Radar): Spike: investigate incorrect page_id values in pageview_hourly - https://phabricator.wikimedia.org/T408798#11346685 (10Ottomata) Speaking of redirects: {T89397} :) [20:46:36] 06Data-Engineering, 10MediaWiki-Core-Hooks, 06MW-Interfaces-Team, 06MediaWiki-Platform-Team (Radar): Spike: investigate incorrect page_id values in pageview_hourly - https://phabricator.wikimedia.org/T408798#11346697 (10Ottomata) Also {T159046} [21:22:16] 06Data-Engineering, 06Content-Transform-Team, 10MobileFrontend, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog: Add page_id and namespace to X-Analytics header in Mobile App requests (2025 remake) - https://phabricator.wikimedia.org/T409358 (10Ottomata) 03NEW [21:25:32] 06Data-Engineering, 06Content-Transform-Team, 10MobileFrontend, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog: Add page_id and namespace to X-Analytics header in Mobile App requests (2025 remake) - https://phabricator.wikimedia.org/T409358#11346869 (10Ottomata) [21:51:59] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Growth-Team, 10MediaWiki-Page-derived-data, 06Wikipedia-Android-App-Backlog, and 2 others: WE3.3.7 Year in Review and Activity Tab Services - Global Editor Metrics - https://phabricator.wikimedia.org/T403660#11347022 (10Ottomata) In [[ https://... [21:52:27] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review: Global Editor Metrics - Data Persistence Design Review - https://phabricator.wikimedia.org/T401260#11347026 (10Ottomata) Decision ^ here: T403660#11347022 [22:24:42] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 3 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11347167 (10RKemper) @elukey Brian and I just tested out a couple of operations, and everything looks good.... [23:45:33] (03PS1) 10Aleksandar Mastilovic: Add appropriate NULL checks to computeForTuples [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1202334 (https://phabricator.wikimedia.org/T406000)