[01:38:30] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [05:38:30] FIRING: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [06:12:49] 06Data-Engineering, 06SRE, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112#12093090 (10elukey... [06:43:15] RESOLVED: HdfsCapacityRemainingPercent: Alarmingly low free space on the analytics-hadoop HDFS cluster. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Capacity_Remaining - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=106&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsCapacityRemainingPercent [09:53:54] 06Data-Engineering, 06SRE, 06Data-Platform-SRE (2026-07-03 - 2026-07-31), 07Sustainability (Incident Followup): The webrequest_sampled_live data pipeline and its query tools have become mission-critical and require re-engineering for resilience - https://phabricator.wikimedia.org/T431112#12093585 (10hnowlan... [10:07:37] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Traffic, 13Patch-For-Review: Add X-Provenance data to webrequest_sampled_live - https://phabricator.wikimedia.org/T427068#12093643 (10Fabfur) 05Open→03Resolved I think this can be closed [11:50:58] (03PS1) 10Joal: Update filerevision sqoop and table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308087 (https://phabricator.wikimedia.org/T431318) [12:54:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Fix `filerevision` sqoop job - https://phabricator.wikimedia.org/T431318#12094359 (10JAllemandou) [12:55:21] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Personal Hive Database Username - https://phabricator.wikimedia.org/T431412#12094361 (10Urbanecm_WMF) AFAIK, this should be self served – just run `CREATE DATABASE catherinekelsey` to work with it. See https://wikitech.wikimedia.org/wiki/Data_Platform/Systems... [13:04:55] (03CR) 10Aqu: [C:03+2] Update filerevision sqoop and table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308087 (https://phabricator.wikimedia.org/T431318) (owner: 10Joal) [13:07:44] (03PS1) 10Joal: Add x_provenance to webrequest tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308104 (https://phabricator.wikimedia.org/T431256) [13:08:22] (03CR) 10A-pizzata: [C:03+2] Update filerevision sqoop and table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308087 (https://phabricator.wikimedia.org/T431318) (owner: 10Joal) [13:09:32] (03CR) 10Joal: [V:03+2 C:03+2] "Merging for later deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308087 (https://phabricator.wikimedia.org/T431318) (owner: 10Joal) [14:16:52] (03CR) 10CDanis: Add x_provenance to webrequest tables (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308104 (https://phabricator.wikimedia.org/T431256) (owner: 10Joal) [14:29:12] (03PS3) 10Aqu: Add GSC site/url impression Hive tables [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1305857 (https://phabricator.wikimedia.org/T427457) [14:30:03] !log Test Kitchen experiment (poll 154742) - adds: none; removes: attribution-research-2026-06-run; fields: none - TK tips at https://w.wiki/_cvdP [14:30:06] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:30:06] (03CR) 10Mforns: [C:03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308104 (https://phabricator.wikimedia.org/T431256) (owner: 10Joal) [14:40:18] 06Data-Engineering: HDFS under 20% of free space - https://phabricator.wikimedia.org/T431391#12094969 (10brouberol) {F92277602} We're now at 25% free space (+5.5%). [14:41:51] 06Data-Engineering, 06Data-Platform-SRE, 06Research: Enriching the webrequest stream with GeoIP/AS metadata - https://phabricator.wikimedia.org/T431433#12094975 (10BTullis) This ticket was created due to an interesting [[https://docs.google.com/document/d/1B4vEy5N4fo9iumfo3KUlxNDC-JYeVTxPtB_Q_Ryg13g/edit?dis... [14:50:31] !log Deploying Refinery at 7d8dc71f for change 1308087 / T431318 - update filerevision table sqoop and table [14:50:35] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:50:35] T431318: Fix `filerevision` sqoop job - https://phabricator.wikimedia.org/T431318 [14:51:28] (03PS1) 10Elukey: druid: add two new dimensions to the webrequest sampled's indexing [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308121 (https://phabricator.wikimedia.org/T427068) [15:10:58] (03CR) 10Elukey: "Checked in the webrequest_sampled topic and I see the new fields to index!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308121 (https://phabricator.wikimedia.org/T427068) (owner: 10Elukey) [18:08:07] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096392 (10CMyrick-WMF) [18:18:38] 06Data-Engineering, 10DPE-Mediawiki-Content, 10Dumps-Generation: Recently edited pages are not included in MediaWiki Content File Exports - https://phabricator.wikimedia.org/T431339#12096429 (10JeffDoozan) [18:19:49] 06Data-Engineering, 06Data-Persistence, 06Traffic: Data Lake - mediarequests - adapt HQL query to include counting mediarequests from webrequest_text - https://phabricator.wikimedia.org/T431475#12096445 (10Ottomata) [18:24:07] 06Data-Engineering, 06Commons, 06Data-Persistence, 07OKR-Work: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476 (10Ottomata) 03NEW [18:24:14] 06Data-Engineering, 06Commons, 06Data-Persistence, 07OKR-Work: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12096467 (10Ottomata) [18:25:35] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096469 (10CMyrick-WMF) [18:28:13] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096483 (10CMyrick-WMF) [18:28:29] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096484 (10CMyrick-WMF) [18:28:42] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096485 (10CMyrick-WMF) [18:28:46] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096486 (10CMyrick-WMF) Update: ` 2020-07 onwards for base_moderator_actions_history base_moderator_actions_logging base_moderator_actions_reverts base_mo... [18:29:41] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): dbt-jobs backfill: all base models for moderator actions - https://phabricator.wikimedia.org/T429995#12096490 (10CMyrick-WMF) [18:30:27] 06Data-Engineering, 10MediaWiki-extensions-Wikibase-Client, 10Wikidata: Data Lake - sqoop of wikidata_entity_usage needs to adapt to move to x1 MariaDB cluster - https://phabricator.wikimedia.org/T431477 (10Ottomata) 03NEW [19:35:30] (03PS1) 10Astein: add new hql [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308190 [20:13:11] 06Data-Engineering: please merge hql/fr_tech/banner_activity_minutely into refinery - https://phabricator.wikimedia.org/T431494 (10AStein-WMF) 03NEW [20:13:13] (03PS2) 10Astein: add new hql [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1308190 (https://phabricator.wikimedia.org/T431494) [20:28:11] 06Data-Engineering, 06Data-Engineering-Radar, 10ServiceOps-SharedInfra, 06ServiceOps new (Next quarter): Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#12097093 (10Ottomata) > 'headless visual editor' - upcoming use case for computing and serving edit s... [20:29:11] 06Data-Engineering, 06Data-Engineering-Radar, 10ServiceOps-SharedInfra, 06ServiceOps new (Next quarter): Standard helm chart for simple service-utils nodejs apps - https://phabricator.wikimedia.org/T428174#12097100 (10Ottomata) BTW, I just created {T431497} [20:43:39] 07Analytics-Data-Problem, 06Data-Engineering, 06Data-Engineering-Radar, 10Pageviews-API: Missed pageview data over API - https://phabricator.wikimedia.org/T370108#12097154 (10EbenezerRao) a:03EbenezerRao Hi everyone, I am a developer with WikiClub Tech working with backend and SQL data architectures. I... [21:41:02] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12097300 (10nshahquinn-wmf) 05Open→03Resolved [21:41:10] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12097302 (10nshahquinn-wmf) 05Resolved→03Open [21:41:28] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12097304 (10amastilovic) [21:42:52] 06Data-Engineering (Q4 FS25/26 April 1st - June 30st): Dbt backfill: editor_month and base_account_registration - https://phabricator.wikimedia.org/T430602#12097307 (10amastilovic) [21:43:52] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Polish signal_driven_handling_bad_faith_accounts.sql model - https://phabricator.wikimedia.org/T431331#12097310 (10amastilovic) This is now completed.