[02:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [06:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [09:01:52] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Refine Simplification] Remove Schema Merging in Refine Process by Enforcing Backward Compatibility - https://phabricator.wikimedia.org/T381072 (10Antoine_Quhen) 03NEW [09:02:37] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Refine Simplification] Remove Schema Merging in Refine Process by Enforcing Backward Compatibility - https://phabricator.wikimedia.org/T381072#10364770 (10Antoine_Quhen) [09:02:38] 06Data-Engineering, 10Data Pipelines: Refine jobs should be scheduled by Airflow - https://phabricator.wikimedia.org/T307505#10364771 (10Antoine_Quhen) [09:17:51] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Airflow Optimization] Reduce Overhead in Refine DAG by Precomputing Parameters - https://phabricator.wikimedia.org/T381073 (10Antoine_Quhen) 03NEW [09:18:28] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Airflow Optimization] Reduce Overhead in Refine DAG by Precomputing Parameters - https://phabricator.wikimedia.org/T381073#10364799 (10Antoine_Quhen) [09:18:29] 06Data-Engineering, 10Data Pipelines: Refine jobs should be scheduled by Airflow - https://phabricator.wikimedia.org/T307505#10364800 (10Antoine_Quhen) [09:46:05] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.x and Hive to version 3.1.x - https://phabricator.wikimedia.org/T379385#10364839 (10BTullis) Hive version 4.0.1 has been added to bigtop in this patch: https://github.com/apache/bigtop/commit/d8459aabfcc8d64bc3121bedc1b2c55ca37882... [09:46:08] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Refine DAG Improvement] Add Parameter to Reduce Spark Driver Logs in Skein Log Collection - https://phabricator.wikimedia.org/T381074 (10Antoine_Quhen) 03NEW [09:46:31] 10Data-Engineering (Q2 2024 October 1st - December 31th): [Refine DAG Improvement] Add Parameter to Reduce Spark Driver Logs in Skein Log Collection - https://phabricator.wikimedia.org/T381074#10364851 (10Antoine_Quhen) [09:46:32] 06Data-Engineering, 10Data Pipelines: Refine jobs should be scheduled by Airflow - https://phabricator.wikimedia.org/T307505#10364852 (10Antoine_Quhen) [09:50:03] (03PS3) 10Aqu: Add an option to ignore missing input folders in Refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088346 (https://phabricator.wikimedia.org/T369845) [10:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [10:42:02] (03PS1) 10Gerrit maintenance bot: Add id.wikivoyage to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098904 (https://phabricator.wikimedia.org/T381080) [10:43:03] (03PS1) 10Gehel: build: add sdkman configuration [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1098908 (https://phabricator.wikimedia.org/T346611) [11:04:41] 06Data-Engineering, 06Data-Platform-SRE, 07Epic: Upgrade Hadoop to version 3.3.6 and Hive to version 4.0.1 - https://phabricator.wikimedia.org/T379385#10365088 (10BTullis) [11:06:46] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10365108 (10Ladsgroup) [11:10:25] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10365116 (10Ladsgroup) [11:10:57] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10365135 (10Ladsgroup) [12:03:00] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10365299 (10Ladsgroup) [14:02:04] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10365511 (10Ladsgroup) @Dreamy_Jazz So this... [14:03:10] (03PS1) 10Joal: Use WMF parent pom and fix accordingly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 [14:05:38] (03CR) 10CI reject: [V:04-1] Use WMF parent pom and fix accordingly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 (owner: 10Joal) [14:06:00] (03PS1) 10Joal: Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 [14:08:19] (03CR) 10CI reject: [V:04-1] Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 (owner: 10Joal) [14:18:40] (03PS1) 10Gmodena: gobblin: webrequest_frontend: fix time partitioning [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098959 [14:19:57] (03PS2) 10Gmodena: gobblin: webrequest_frontend: fix time partitioning [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098959 [14:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [14:41:17] (03CR) 10Gmodena: "CI is complaining, but the logic LGTM." [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 (owner: 10Joal) [14:44:33] (03CR) 10Gmodena: Use WMF parent pom and fix accordingly (032 comments) [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 (owner: 10Joal) [14:50:46] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098959 (owner: 10Gmodena) [14:53:33] (03CR) 10Joal: Fix failure scenario when no timestamp available (031 comment) [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 (owner: 10Joal) [14:56:40] (03CR) 10Joal: Use WMF parent pom and fix accordingly (032 comments) [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 (owner: 10Joal) [14:58:03] (03PS2) 10Joal: Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 [14:58:35] (03PS2) 10Joal: Use WMF parent pom and fix accordingly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 [14:59:50] (03PS3) 10Joal: Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 [15:00:02] (03CR) 10CI reject: [V:04-1] Use WMF parent pom and fix accordingly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 (owner: 10Joal) [15:01:26] (03CR) 10CI reject: [V:04-1] Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 (owner: 10Joal) [15:07:05] (03CR) 10Gmodena: [V:03+2 C:03+2] gobblin: webrequest_frontend: fix time partitioning [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1098959 (owner: 10Gmodena) [15:59:04] (03PS3) 10Joal: Use WMF parent pom and fix accordingly [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098954 [15:59:37] (03PS4) 10Joal: Fix failure scenario when no timestamp available [analytics/gobblin-wmf] - 10https://gerrit.wikimedia.org/r/1098955 [16:29:19] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 06Movement-Insights: Modify the automated traffic detection to be applied at the project family level - https://phabricator.wikimedia.org/T377257#10366020 (10Hghani) I've looked at the additional analysis questions mentioned in the la... [17:06:22] 10Data-Engineering (Q2 2024 October 1st - December 31th), 10CheckUser, 06Data Products, 06DBA, 07Schema-change-in-production: Remove cuc_actiontext, cuc_only_for_read_old, and cuc_private from cu_changes on WMF wikis - https://phabricator.wikimedia.org/T370903#10366101 (10Dreamy_Jazz) >>! In T370903#1036... [17:45:20] (03PS1) 10Snwachukwu: Update Druid Geoeditors monthly to Support Temp Accounts. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1099006 (https://phabricator.wikimedia.org/T379772) [17:51:07] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 06Movement-Insights: Modify the automated traffic detection to be applied at the project family level - https://phabricator.wikimedia.org/T377257#10366143 (10JAllemandou) Thank you @Hghani for the analysis. I'm happy with the evidence... [18:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [19:56:46] 06Data-Engineering, 06Research, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#10366282 (10XiaoXiao-WMF) [22:28:16] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent