[09:57:46] 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Add rc_source_name_timestamp index to recentchanges table in wmf production - https://phabricator.wikimedia.org/T400854#11081890 (10Ladsgroup) [11:55:49] (03PS1) 10Joal: Update webrequest access_method computation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) [12:58:09] (03CR) 10Joal: [C:04-1] "The addition of the previous_access_method field will soon not work because of T214998: using the referrer URL to extract the access metho" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1172085 (https://phabricator.wikimedia.org/T396514) (owner: 10Milimetric) [13:04:08] (03CR) 10Aqu: Update webrequest access_method computation (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [13:11:09] 06Data-Engineering: Request for Kerberos identity for querying SSAC table via statmachines - https://phabricator.wikimedia.org/T401827 (10cmelo) 03NEW [13:16:55] (03PS2) 10Joal: Update webrequest access_method computation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) [13:17:23] (03CR) 10Joal: Update webrequest access_method computation (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [13:17:48] (03CR) 10Xcollazo: "General questions for my understanding:" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [13:56:49] (03CR) 10Joal: "Good question!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [13:59:43] (03CR) 10Xcollazo: [C:03+2] "Fair enough." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1178532 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [14:35:45] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10service-utils, 13Patch-For-Review: Migrate and re-deploy eventgate using new service-utils - https://phabricator.wikimedia.org/T361768#11082839 (10Ottomata) [15:55:40] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform, 13Patch-For-Review: Inconsistent state visibility for Flink deployments on dse-k8s-eqiad - https://phabricator.wikimedia.org/T395984#11083167 (10BTullis) 05In progress→03Resolved [16:02:50] 06Data-Engineering, 10SRE-Access-Requests, 10Data-Platform-SRE (2025.07.26 - 2025.08.15): Request for Kerberos identity for querying SSAC table via statmachines - https://phabricator.wikimedia.org/T401827#11083202 (10Ottomata) [16:02:51] 06Data-Engineering, 10SRE-Access-Requests, 10Data-Platform-SRE (2025.07.26 - 2025.08.15): Request for Kerberos identity for querying SSAC table via statmachines - https://phabricator.wikimedia.org/T401827#11083204 (10Ottomata) `cmelo` is in `analytics-privatedata-users`: https://gerrit.wikimedia.org/r/plugin... [16:06:19] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11083209 (10ops-monitoring-bot) Start pool of db2147 quickly with 2 steps - Repooling - fceratto@cumin1002 [16:07:50] 06Data-Engineering, 06Data-Engineering-Radar, 10SRE-Access-Requests, 10Data-Platform-SRE (2025.07.26 - 2025.08.15): Request for Kerberos identity for querying SSAC table via statmachines - https://phabricator.wikimedia.org/T401827#11083211 (10Ottomata) [16:19:30] 06Data-Engineering, 13Patch-For-Review: Improve spider detection in the webrequest refinery pipeline - https://phabricator.wikimedia.org/T394794#11083267 (10Ottomata) 05Open→03Resolved a:03Ottomata From comments I think this task is done? Please reopen if I am incorrect. TY! [16:20:26] 06Data-Engineering, 10Technical-blog-posts, 10Data-Platform-SRE (2025.07.26 - 2025.08.15): Write a blog post about the recent Airflow migration to Kubernetes - https://phabricator.wikimedia.org/T393603#11083276 (10Ottomata) Cool idea! Let DE know if you need anything from us! [16:21:34] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11083277 (10ops-monitoring-bot) Completed pool of db2147 quickly with 2 steps - Repooling - fceratto@cumin1002 [16:23:34] 06Data-Engineering, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 07Documentation: https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log should be on Wikitech - https://phabricator.wikimedia.org/T387878#11083287 (10Ottomata) I don't love naming things like this after team names, but maybe just Data_Plat... [16:27:36] 06Data-Engineering, 06Data-Engineering-Radar, 06cloud-services-team, 06Data-Persistence, and 2 others: Create wiki replicas views for globaljsonlinks tables - https://phabricator.wikimedia.org/T387419#11083290 (10Ottomata) [16:47:04] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11083336 (10ops-monitoring-bot) Start pool of db2219 gradually with 4 steps - Repooling - fceratto@cumin1002 [17:32:22] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add cl_timestamp_id index to categorylinks table - https://phabricator.wikimedia.org/T399249#11083470 (10ops-monitoring-bot) Completed pool of db2219 gradually with 4 steps - Repooling - fceratto@cumin1002 [17:47:07] 06Data-Engineering, 06Data-Engineering-Icebox, 06Movement-Insights, 10Wmfdata-Python: Support importing Parquet, delimited, and Pandas datasets into the Data Lake using Wmfdata - https://phabricator.wikimedia.org/T273196#11083518 (10nshahquinn-wmf) [17:49:11] 06Data-Engineering, 10Wmfdata-Python, 10Movement-Insights (FY25-26 H1): Support importing Parquet, delimited, and Pandas datasets into the Data Lake using Wmfdata - https://phabricator.wikimedia.org/T273196#11083527 (10nshahquinn-wmf) 05Open→03In progress p:05Low→03Medium Increasing the priority sinc... [17:49:33] 06Data-Engineering, 06Data-Engineering-Icebox, 10Wmfdata-Python, 10Movement-Insights (FY25-26 H1): Support importing Parquet, delimited, and Pandas datasets into the Data Lake using Wmfdata - https://phabricator.wikimedia.org/T273196#11083531 (10nshahquinn-wmf) [17:50:21] 06Data-Engineering, 10Wmfdata-Python: Deprecate the Hive module - https://phabricator.wikimedia.org/T384541#11083533 (10nshahquinn-wmf) [17:50:22] 06Data-Engineering, 06Data-Engineering-Icebox, 10Wmfdata-Python, 10Movement-Insights (FY25-26 H1): Support importing Parquet, delimited, and Pandas datasets into the Data Lake using Wmfdata - https://phabricator.wikimedia.org/T273196#11083534 (10nshahquinn-wmf) [17:55:56] 06Data-Engineering, 10Wmfdata-Python: Deprecate the Hive module - https://phabricator.wikimedia.org/T384541#11083537 (10nshahquinn-wmf) I've decided that we should deprecate, warn, and then remove rather than just silently replacing the Hive functionality with Spark. This won't add much work, although it will... [18:18:35] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Add a YAML linter to airflow-dags GitLab CI/CD - https://phabricator.wikimedia.org/T401854 (10amastilovic) 03NEW [18:27:44] (03PS1) 10Joal: Remove webrequest access_method UDF and core code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1178595 (https://phabricator.wikimedia.org/T401665) [18:39:22] (03CR) 10Xcollazo: [C:03+2] Remove webrequest access_method UDF and core code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1178595 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [18:52:21] (03Merged) 10jenkins-bot: Remove webrequest access_method UDF and core code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1178595 (https://phabricator.wikimedia.org/T401665) (owner: 10Joal) [20:55:13] 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Add rc_source_name_timestamp index to recentchanges table in wmf production - https://phabricator.wikimedia.org/T400854#11084089 (10Ladsgroup) [21:05:42] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Investigate reasons for remaining inconsistencies - https://phabricator.wikimedia.org/T385112#11084123 (10xcollazo) We have successfully run the `all-of-wiki-time` reconciles for a couple months, yet we co... [22:33:37] 06Data-Engineering, 06Data-Engineering-Radar, 10Wmfdata-Python: Convert existing Wmfdata docstrings to a standard format - https://phabricator.wikimedia.org/T380742#11084271 (10nshahquinn-wmf) >>! In T380742#10363937, @Isaac wrote: > Some progress from the offsite: https://gitlab.wikimedia.org/isaacj/wmfdata...