[09:04:55] 06Data-Engineering: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912#11180117 (10phuedx) Not related to the Experimentation Lab product/Experiment Platform work directly. I'll bring this to the attention of #product-analytics and see if they're interested/have thoughts. [09:22:38] (03PS1) 10Gerrit maintenance bot: Add tok.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188291 (https://phabricator.wikimedia.org/T404567) [09:40:50] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop rc_new from recentchanges table in wmf production - https://phabricator.wikimedia.org/T402763#11180259 (10Ladsgroup) [10:24:05] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop rc_new from recentchanges table in wmf production - https://phabricator.wikimedia.org/T402763#11180326 (10Ladsgroup) [10:44:32] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop cl_to and cl_collation from categorylinks in wmf production - https://phabricator.wikimedia.org/T402925#11180372 (10Ladsgroup) [13:33:34] 06Data-Engineering, 06Privacy Engineering, 06Research: Release dataset on top search engine referrers by country, device, and language - https://phabricator.wikimedia.org/T270140#11180865 (10Alexey_Skripnik) @Isaac, thank you for making this dataset possible! I am conducting some research about Yandex pr... [14:16:47] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1186087 (https://phabricator.wikimedia.org/T401666) (owner: 10Joal) [14:17:01] 06Data-Engineering, 06Discovery-Search, 07Epic, 07Essential-Work: [EPIC] Upgrade flink jobs to java 17 - https://phabricator.wikimedia.org/T404340#11181026 (10pfischer) p:05Triage→03High [14:30:18] !log Deploying refinery using scap for unique-devices change [14:30:24] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:33:00] !log Deploying refinery onto HDFS [14:33:03] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:43:01] !log Merging airflow unique-devices change patch [14:43:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:49:21] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 4 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#11181160 (10pfischer) [14:58:02] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 4 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#11181194 (10pfischer) @dcausse, I created T404597 to continue work. Would moving the D... [15:27:42] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10Event-Platform: mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs - https://phabricator.wikimedia.org/T397330#11181309 (10Ahoelzl) a:05Ottomata→03tchin [15:28:32] 06Data-Engineering, 10CirrusSearch, 10Discovery-Search (2025.09.05 - 2025.09.26), 07Essential-Work: SUP: Serde w/o RowTypeInfo - https://phabricator.wikimedia.org/T404597#11181315 (10Gehel) [15:31:06] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Add data-rewrite to airflow iceberg maintenance defaults - https://phabricator.wikimedia.org/T404598 (10JAllemandou) 03NEW [16:54:33] 06Data-Engineering: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912#11181911 (10Isaac) Just a head's up that there is some data available about common user-agent components though I'm not sure if it meets the needs of this task as it's more structured/split as opposed to the... [17:06:13] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 07Essential-Work: OpsWeek: Contact and/or remove dead dump mirrors - https://phabricator.wikimedia.org/T402987#11181940 (10xcollazo) As of now: C3SL -> Working now! PDApps -> Email never responded, deactivated. Center for Research Computing, Universit... [17:06:43] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 07Essential-Work: OpsWeek: Contact and/or remove dead dump mirrors - https://phabricator.wikimedia.org/T402987#11181941 (10xcollazo) 05In progress→03Resolved [17:24:12] (03PS1) 10Joal: Fix unique-devices jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188401 (https://phabricator.wikimedia.org/T401666) [17:47:37] 06Data-Engineering, 06Infrastructure-Foundations: proposal: allow analytics-admins to also `sudo run-puppet-agent` - https://phabricator.wikimedia.org/T404630 (10CDanis) 03NEW [17:58:56] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 13Patch-For-Review: Update unique_devices tables to add the `access_method` field - https://phabricator.wikimedia.org/T401666#11182095 (10JAllemandou) [18:04:54] 06Data-Engineering, 06Infrastructure-Foundations, 13Patch-For-Review: proposal: allow analytics-admins to also `sudo run-puppet-agent` - https://phabricator.wikimedia.org/T404630#11182121 (10taavi) IIRC historically there were concerns that `run-puppet-agent` could leak secrets (if/when code is missing `show... [18:34:30] (03CR) 10Mforns: [C:03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188401 (https://phabricator.wikimedia.org/T401666) (owner: 10Joal) [18:40:07] 06Data-Engineering, 06Infrastructure-Foundations, 13Patch-For-Review: proposal: allow analytics-admins to also `sudo run-puppet-agent` - https://phabricator.wikimedia.org/T404630#11182233 (10CDanis) Thanks @taavi, good points as always. I think `puppet-run` would be fine for this use case (even if not ideal... [20:02:15] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [21:02:44] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): test_produced_by_config SLA miss configured to be too small for upstream dataset run time - https://phabricator.wikimedia.org/T388861#11182777 (10dr0ptp4kt) Belatedly, just wanted to say thanks @xcollazo for the update and @amastilovic noticing that... [21:38:24] 07Analytics-Data-Problem, 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Movement-Insights: NEW BUG REPORT <"Domain" field issue: some domains have trailing dots> - https://phabricator.wikimedia.org/T395963#11182953 (10JAllemandou) 05Open→03Resolved a:03JAllemandou I confirm this bug... [21:42:15] RESOLVED: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [22:58:14] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1188291 (https://phabricator.wikimedia.org/T404567) (owner: 10Gerrit maintenance bot) [23:29:57] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 10Event-Platform: EventGate: Log unparseable X-Experiment-Enrollments headers to an error stream - https://phabricator.wikimedia.org/T396359#11183244 (10dr0ptp4kt) Well, @phuedx reminded me I had already filed {T401198}, so... [23:30:45] 06Data-Engineering, 06Experimentation Lab, 10Event-Platform: Handle empty set regex parsing for MalformedHeaderError edge case - https://phabricator.wikimedia.org/T404352#11183247 (10dr0ptp4kt) →14Duplicate dup:03T401198 [23:30:46] 06Data-Engineering-Radar, 06Experimentation Lab, 10Event-Platform: More strictly validate X-Experiment-Enrollments-Header - https://phabricator.wikimedia.org/T401198#11183249 (10dr0ptp4kt) [23:32:35] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 13Patch-For-Review: Update unique_devices tables to add the `access_method` field - https://phabricator.wikimedia.org/T401666#11183253 (10JAllemandou) [23:36:57] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Experimentation Lab, 10Event-Platform: EventGate: Add Prometheus metric for hoisting errors - https://phabricator.wikimedia.org/T398922#11183281 (10dr0ptp4kt) 05Open→03Resolved Marking this task as resolved. Thank you!