[00:55:57] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Backfill wmf_objective_metrics.heartbeat_active_editors_weekly - https://phabricator.wikimedia.org/T437118#12310645 (10nshahquinn-wmf) @APizzata-WMF thank you for working on this! The week of 2026-06-01 still needs...
[02:23:04] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Radar, 06Research: mwparserfromhtml errors - https://phabricator.wikimedia.org/T437669 (10AKhatun_WMF) 03NEW
[02:27:00] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): User Acceptance tests in staging - https://phabricator.wikimedia.org/T437670 (10AKhatun_WMF) 03NEW
[05:43:56] <wikibugs>	 06Data-Engineering: Hive partition missing for webrequest upload for 2026-09-09 hour-13 - https://phabricator.wikimedia.org/T437675 (10KCVelaga_WMF) 03NEW
[05:44:05] <wikibugs>	 06Data-Engineering: Hive partition missing for webrequest upload for 2026-09-09 hour-13 - https://phabricator.wikimedia.org/T437675#12310847 (10KCVelaga_WMF) p:05Triage→03High
[06:08:45] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Add a new `is_api` field to webrequest_sampled in druid - https://phabricator.wikimedia.org/T437638#12310853 (10JAllemandou) >>! In T437638#12309808, @Ottomata wrote: > Hm, how urgent is this?  We are working with RDS to redefine th...
[06:29:30] <wikibugs>	 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12310862 (10KCVelaga_WMF) That works for me - keeps it simple! From various Slack threads, I wasn't sure so having it is confirmed here is helpful. We can close this once M...
[06:45:45] <wikibugs>	 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12310901 (10KCVelaga_WMF) > However, if we add permission management back in, I agree a separate database sounds better.  I was mostly struggling with permissions with need...
[07:06:05] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Analyze and correctly classify iMessage User-Agent - https://phabricator.wikimedia.org/T435110#12310935 (10KCVelaga_WMF) Tangential discussion related to this UA: {T432373}
[07:10:45] <wikibugs>	 06Data-Engineering: Hive partition missing for webrequest upload for 2026-09-09 hour-13 - https://phabricator.wikimedia.org/T437675#12310954 (10KCVelaga_WMF) 05Open→03Resolved a:03KCVelaga_WMF @JAllemandou restarted the upstream Airflow DAG that processes webrequest-upload. Downstream jobs are unblocke...
[07:10:46] <wikibugs>	 06Data-Engineering: Hive partition missing for webrequest upload for 2026-09-09 hour-13 - https://phabricator.wikimedia.org/T437675#12310958 (10KCVelaga_WMF) a:05KCVelaga_WMF→03None
[07:55:31] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Backfill wmf_objective_metrics.heartbeat_active_editors_weekly - https://phabricator.wikimedia.org/T437118#12311023 (10APizzata-WMF) weird, from my previous comment the data is there for the 2026-06-01, can you tell...
[09:23:50] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Add a new `is_api` field to webrequest_sampled in druid - https://phabricator.wikimedia.org/T437638#12311287 (10Clement_Goubert) This is a stopgap measure until {T436223} can be implemented to avoid crumbling Druid through Turnilo w...
[09:24:01] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Add a new `is_api` field to webrequest_sampled in druid - https://phabricator.wikimedia.org/T437638#12311289 (10Clement_Goubert)
[09:32:28] <wikibugs>	 (03CR) 10Joal: [C:03+1] "LGTM! Nice findings @xcollazo@wikimedia.org" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1338971 (https://phabricator.wikimedia.org/T425734) (owner: 10Xcollazo)
[10:40:15] <wikibugs>	 14Data-Engineering-Planning, 10Data Pipelines: [Iceberg] Debianize and install iceberg support for Spark, Presto, and optionally Hive - https://phabricator.wikimedia.org/T311738#12311525 (10BTullis)
[11:19:38] <wikibugs>	 14Data-Engineering (Q3 FY25/26 January 1st - March 31th): Sqlufluff on Stat hosts - https://phabricator.wikimedia.org/T417171#12311714 (10GGoncalves-WMF)
[11:19:39] <wikibugs>	 06Data-Engineering, 07Epic: Epic: dbt Backlog - https://phabricator.wikimedia.org/T437619#12311713 (10GGoncalves-WMF)
[11:19:41] <wikibugs>	 14Data-Engineering (Q3 FY25/26 January 1st - March 31th): Sqlfluff Rules for dbt - https://phabricator.wikimedia.org/T417152#12311715 (10GGoncalves-WMF)
[11:19:46] <wikibugs>	 10Data-Engineering-Roadmap, 06Data-Platform-SRE, 06Movement-Insights, 07Epic, 13Patch-For-Review: Provide a dbt-core development environment and production setup in the data-platform - https://phabricator.wikimedia.org/T406764#12311716 (10GGoncalves-WMF)
[11:21:15] <wikibugs>	 10Data-Engineering-Roadmap, 06Data-Platform-SRE, 06Movement-Insights, 07Epic, 13Patch-For-Review: Provide a dbt-core development environment and production setup in the data-platform - https://phabricator.wikimedia.org/T406764#12311731 (10GGoncalves-WMF) Is this and its child {T410017} complete or obsole...
[11:21:46] <wikibugs>	 14Data-Engineering (Q3 FY25/26 January 1st - March 31th): Sqlufluff on Stat hosts - https://phabricator.wikimedia.org/T417171#12311738 (10GGoncalves-WMF)
[11:21:47] <wikibugs>	 14Data-Engineering (Q3 FY25/26 January 1st - March 31th): Sqlfluff Rules for dbt - https://phabricator.wikimedia.org/T417152#12311739 (10GGoncalves-WMF)
[11:21:48] <wikibugs>	 10Data-Engineering-Roadmap, 07Epic: Epic: dbt DPE work - https://phabricator.wikimedia.org/T416679#12311737 (10GGoncalves-WMF)
[11:21:49] <wikibugs>	 14Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Movement-Insights (FY25-26 H2): dbt repository structure (Milestone 3) - https://phabricator.wikimedia.org/T416672#12311740 (10GGoncalves-WMF)
[11:21:56] <wikibugs>	 10Data-Engineering-Roadmap, 06Data-Platform-SRE, 06Movement-Insights, 07Epic, 13Patch-For-Review: Provide a dbt-core development environment and production setup in the data-platform - https://phabricator.wikimedia.org/T406764#12311741 (10GGoncalves-WMF)
[11:22:23] <wikibugs>	 06Data-Engineering, 07Epic: Epic: dbt Roadmap - https://phabricator.wikimedia.org/T437618#12311745 (10GGoncalves-WMF)
[11:22:24] <wikibugs>	 14Data-Engineering (Q4 FS25/26 April 1st - June 30st), 06Movement-Insights (FY25-26 H2): dbt repository structure (Milestone 3) - https://phabricator.wikimedia.org/T416672#12311744 (10GGoncalves-WMF)
[11:31:09] <wikibugs>	 06Data-Engineering: CI checks don't cover is_incremental() path - https://phabricator.wikimedia.org/T437701 (10GGoncalves-WMF) 03NEW
[12:25:34] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Consider classifying www.wikipedia.org from "internal" referrer - https://phabricator.wikimedia.org/T422584#12311910 (10GGoncalves-WMF) Thanks for the detailed report!  To make sure I understand it, let me try to restate the proposal: in the case of port...
[12:26:39] <wikibugs>	 (03PS2) 10Joal: Add transform to kafka-druid webrequest_sampled [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1339067 (https://phabricator.wikimedia.org/T437638)
[12:42:18] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Consider classifying www.wikipedia.org from "internal" referrer - https://phabricator.wikimedia.org/T422584#12311950 (10RHo) Adding @Maryana and @Khantstop for visibility and input as this change would have some impact on our measurements for IR1+IR2, an...
[12:49:11] <wikibugs>	 (03CR) 10A-pizzata: [C:03+1] MWHistoryDeltaWriter: align fields with mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1338971 (https://phabricator.wikimedia.org/T425734) (owner: 10Xcollazo)
[12:54:54] <wikibugs>	 (03CR) 10Clément Goubert: [C:03+1] Add transform to kafka-druid webrequest_sampled [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1339067 (https://phabricator.wikimedia.org/T437638) (owner: 10Joal)
[13:41:03] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Data-Platform, 06Growth-Team, 05FY 2026-27 DE1.3.2 Impact + Dashboard, and 2 others: DE1.3.2: Implement "Sliding 30 Day Top-K Pages for Editor" - https://phabricator.wikimedia.org/T437710 (10GGoncalves-WMF) 03NEW
[14:01:43] <wikibugs>	 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Capture relevant mediawiki tags changes in event stream(s) - https://phabricator.wikimedia.org/T434143#12312220 (10tchin) I updated the patch to implement `EntityTagsUpdatedEvent`, `RevisionTagsUpdatedEvent`, and...
[14:05:35] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Impact Module Metrics - Sliding 30 Day Top-K Pages for Editor - https://phabricator.wikimedia.org/T436368#12312228 (10Snwachukwu) Deleting this ticket because there is {T437710} which is addresses the same task.
[14:05:57] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Impact Module Metrics - Sliding 30 Day Top-K Pages for Editor - https://phabricator.wikimedia.org/T436368#12312230 (10Snwachukwu) 05Open→03Invalid
[14:28:15] <wikibugs>	 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12312304 (10xcollazo) My two cents:  * For `wmf_traffic` to work correctly in Presto, it should only ever hold Iceberg datasets. Most folks would do Iceberg these days, but...
[14:41:18] <wikibugs>	 (03CR) 10Xcollazo: [C:03+2] MWHistoryDeltaWriter: align fields with mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1338971 (https://phabricator.wikimedia.org/T425734) (owner: 10Xcollazo)
[14:46:28] <wikibugs>	 10Analytics-Canonical-Data, 10MediaWiki-Configuration, 10MediaWiki-extensions-Wikibase-Client, 10MediaWiki-extensions-Wikibase-Repo, and 3 others: Completely distinguish various concepts including "global site ID", "global site key", "site DB name", "site ... - https://phabricator.wikimedia.org/T329424#12312411
[14:56:39] <wikibugs>	 (03Merged) 10jenkins-bot: MWHistoryDeltaWriter: align fields with mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1338971 (https://phabricator.wikimedia.org/T425734) (owner: 10Xcollazo)
[15:07:04] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History, 13Patch-For-Review: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12312480 (10xcollazo) **Notice for users of `wmf_mediawiki.mediaw...
[15:24:11] <wikibugs>	 (03CR) 10Xcollazo: "> But the target now is to pull the current event table into the destination table sinc eevent table only keeps 90 days of data" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1319873 (https://phabricator.wikimedia.org/T426347) (owner: 10Snwachukwu)
[15:44:49] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12312618 (10Isaac) Not sure how much this matters or if it's a simple fix but FYI page...
[16:10:06] <wikibugs>	 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12312697 (10mforns) I'm OK that wmf_traffic is used for both core and non-core traffic datasets, provided they are all Iceberg tables. However, it seems inconvenient that d...
[16:42:20] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12312801 (10xcollazo) > Not sure how much this matters or if it's a simple fix but FYI...
[16:58:58] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Quality verification for mediawiki_history_incremental_v1 using Iceberg time travel - https://phabricator.wikimedia.org/T425734#12312864 (10Ottomata) Ya. I think, but am not sure, that page protections are possibly...
[17:03:12] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Figure out the biggest HDFS consumers over time - https://phabricator.wikimedia.org/T437605#12312897 (10xcollazo)
[17:52:11] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: migrate dbt_jobs pipelines to use MWH incremental - https://phabricator.wikimedia.org/T429770#12313023 (10nshahquinn-wmf)
[18:01:43] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12313076 (10xcollazo) 05Open→03In progress
[18:03:43] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Commons, 06Data-Persistence, 07OKR-Work, 13Patch-For-Review: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12313086 (10Milimetric) Picking this up now....
[18:34:37] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Commons, 06Data-Persistence, 07OKR-Work, 13Patch-For-Review: Data Lake - sqoop of commons link tables needs to adapt to move to dedicated MariaDB cluster - https://phabricator.wikimedia.org/T431476#12313189 (10Milimetric) Right now we're doing...
[18:47:15] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Radar, 06Research: mwparserfromhtml errors - https://phabricator.wikimedia.org/T437669#12313212 (10Isaac) Thanks for opening this @AKhatun_WMF ! Thinking through the bugs and please chime in if you have thoughts/opinions:  > Barack_Obama page has infobox, but library re...
[19:08:12] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): User and team Hive database cleanup tracking for Sept 2026 - https://phabricator.wikimedia.org/T437733#12313257 (10xcollazo)
[19:11:38] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Figure out the biggest HDFS consumers over time - https://phabricator.wikimedia.org/T437605#12313261 (10xcollazo) `wmf_mediawiki.mediawiki_history_incremental_v1` holds 0.5 TB of live data but occupies about 211 TB on HDFS. The Iceberg snapshot window is...
[20:16:13] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 13Patch-For-Review: Figure out the biggest HDFS consumers over time - https://phabricator.wikimedia.org/T437605#12313459 (10xcollazo) @KCVelaga_WMF, `wmf_traffic.api_requests_refined` holds about 144.6 TB on-disk. It is the second largest new dataset i...
[20:59:04] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12313636 (10Isaac) > Meaning, it is impossible for us to know what MW's stored version of user_editcount (and fields like...
[22:04:58] <icinga-wm>	 PROBLEM - statsv Varnishkafka log producer on cp5020 is CRITICAL: PROCS CRITICAL: 3 processes with args /usr/bin/varnishkafka -S /etc/varnishkafka/statsv.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[22:05:58] <icinga-wm>	 RECOVERY - statsv Varnishkafka log producer on cp5020 is OK: PROCS OK: 1 process with args /usr/bin/varnishkafka -S /etc/varnishkafka/statsv.conf https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka
[23:52:32] <wikibugs>	 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Backfill wmf_objective_metrics.heartbeat_active_editors_weekly - https://phabricator.wikimedia.org/T437118#12313950 (10nshahquinn-wmf) @APizzata-WMF it's querying `wmf_objective_metrics.heartbeat_active_editors_week...