[02:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [06:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [10:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [13:37:04] (03PS4) 10Jennifer Ebe: Edit Geoeditors Daily Monthly to support Temp Account Changes [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098083 (https://phabricator.wikimedia.org/T379728) [14:02:26] (03PS9) 10Gehel: Extraction of RefineHelper [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1080706 [14:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [14:34:55] 06Data-Engineering, 06MediaWiki-Engineering, 10MediaWiki-extensions-WikimediaEvents, 06MediaWiki-Platform-Team, and 5 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10361759 (10Michael) >>! In T355837#10359299, @Krinkle wrote: > @michael The replaceme... [14:44:43] Hi! I have a question for you about adding a field from Wikidata to the Wikidata XML dumps, and the impact that could have on the size/duration of the dumps. [14:44:43] We at WMDE are currently working on this ticket https://phabricator.wikimedia.org/T197090 to add the Wikidata QID for each article to the XML dump - it would increase the size of the dumps of wikipedia by about 9 million rows (1 per article). [14:44:43] It would be great to hear any considerations from your end about how to test or to coordinate the rollout. [14:44:44] I assume that would increase the time it takes to generate the wikipedia dumps - can you tell us if that would cause any issues with their running or any dependencies on them? [14:44:47] Iā€™m Suzie, and have recently started at Wikimedia Deutschland in the Wikidata for Wikimedia Projects team. Feel free to contact me at suzanne.wood@wikimedia.de [14:57:44] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10361875 (10Dreamy_Jazz) I'd like to ask if... [14:59:52] 06Data-Engineering, 06MediaWiki-Engineering, 10MediaWiki-extensions-WikimediaEvents, 06MediaWiki-Platform-Team, and 5 others: Add Prometheus support to statsd.js via mw.track() - https://phabricator.wikimedia.org/T355837#10361879 (10Krinkle) p:05Triageā†’03High [15:17:32] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10361976 (10Ladsgroup) Sorry, we are short o... [15:22:01] 06Data-Engineering, 10Data Products (Data Products Sprint 22), 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10362031 (10apaskulin) Although I haven't tested it, I also came across https://github.com/tomcollins/json-sche... [15:35:36] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10362122 (10Ladsgroup) [15:38:50] a-team [15:39:24] I have sent some questions above about XML dumps, feel free to contact me at suzanne.wood@wikimedia.de [15:52:28] suzannewoodWMDE2: Thanks for getting in touch. I've shared your message with the https://www.mediawiki.org/wiki/Data_Platform_Engineering team, who are responsible for the dumps now. [15:55:29] I'm part of that team too, but I'm not able to give you any categorical answers on whether or not this feature is feasible to implement. Sorry. [15:56:31] What I do know is that the current XML/SQL dumps system is generally considered to be in maintenance-only mode, while we work on the new event-driven Dumps 2.0 system. https://phabricator.wikimedia.org/project/profile/6651/ [15:58:52] thanks! [17:58:15] (03PS1) 10Mforns: Revert removal of bin/refinery-dump-status-webrequest-partitions [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098577 [17:58:56] (03CR) 10Mforns: [V:03+2 C:03+2] "Self merging to unbreak production job" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098577 (owner: 10Mforns) [18:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [18:44:30] 06Data-Engineering, 10Data Pipelines, 06Product-Analytics: Add TikTok's in-app browser to ua-parser library - https://phabricator.wikimedia.org/T325611#10363011 (10Isaac) I'm not sure who the right person to answer your questions is @Cpetrillo but I'll throw some thoughts in the ring. > Do we still need Tik... [18:46:34] 06Data-Engineering: Add "did edit" field to pageview_actor - https://phabricator.wikimedia.org/T277785#10363017 (10Isaac) 05Openā†’03Declined Declining this task as I promised long ago :) A more reasonable approach might instead be using `action=submit` but I think that would be a separate investigation. S... [18:57:40] 06Data-Engineering: The cleanup_tmpdumps service fails when the file to delete doesn't exist - https://phabricator.wikimedia.org/T381026 (10mforns) 03NEW [19:20:28] !log deployed airflow analytics as part of the regular deployment train [19:20:30] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:23:21] 06Data-Engineering, 06Data-Platform, 10Dumps-Generation: The cleanup_tmpdumps service fails when the file to delete doesn't exist - https://phabricator.wikimedia.org/T381026#10363227 (10xcollazo) [19:39:41] !log re-ran wikidata_wikitext_history for 2024-08 [19:39:42] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [21:55:02] 06Data-Engineering, 10Wmfdata-Python: Convert existing Wmfdata docstrings to a standard format - https://phabricator.wikimedia.org/T380742#10363937 (10Isaac) Some progress from the offsite: https://gitlab.wikimedia.org/isaacj/wmfdata-python/-/commit/073e7c3b991e4c9b34e17a7c22e0be14cdddce3d If someone is motiv... [22:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent