[07:51:01] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Make Airflow Druid infrastructure more failproof and resilient - https://phabricator.wikimedia.org/T437310#12301728 (10brouberol) @amastilovic can the refinery `org.wikimedia.analytic... [07:56:21] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Make Airflow Druid infrastructure more failproof and resilient - https://phabricator.wikimedia.org/T437310#12301740 (10brouberol) Looks like it's not the case, as both `HiveToDruid` a... [08:53:49] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Make Airflow Druid infrastructure more failproof and resilient - https://phabricator.wikimedia.org/T437310#12301897 (10brouberol) I have defined the `druid-analytics.svc.eqiad.wmnet`... [09:49:52] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12302077 (10GGoncalves-WMF) [09:51:05] 10Data-Engineering-Roadmap, 07Epic: [Epic] KAPOW: The next generation of bot detection in the Data Platform. - https://phabricator.wikimedia.org/T425661#12302090 (10GGoncalves-WMF) [10:10:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Data-Platform-SRE (2026-08-28 - 2026-09-18), 13Patch-For-Review: Make Airflow Druid infrastructure more failproof and resilient - https://phabricator.wikimedia.org/T437310#12302172 (10brouberol) 05In progress→03Resolved [10:40:56] 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12302319 (10GGoncalves-WMF) No problem, it seems tidier to organize the schemas by business domain, and record support status elsewhere (e.g. Datahub). So let's go with `wm... [12:44:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Use the new co_id to replace KAPOW fuzzy join - https://phabricator.wikimedia.org/T437440 (10mforns) 03NEW [13:09:14] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Create new mediawiki.tags_change event stream - https://phabricator.wikimedia.org/T434143#12302963 (10tchin) >>! In T434143#12299923, @Ottomata wrote: >Can the same TagsChange event apply to both a log and revisi... [14:01:38] 06Data-Engineering: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454 (10GGoncalves-WMF) 03NEW [14:13:45] 06Data-Engineering, 06Data-Platform-SRE: Implement brownout logic for dumps v1 on dumps web interface - https://phabricator.wikimedia.org/T437461 (10GGoncalves-WMF) 03NEW [14:13:56] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE: Implement brownout logic for dumps v1 on dumps web interface - https://phabricator.wikimedia.org/T437461#12303410 (10GGoncalves-WMF) [14:25:10] 06Data-Engineering: Include v2 dumps into mirrors rsync configuration - https://phabricator.wikimedia.org/T437463 (10GGoncalves-WMF) 03NEW [14:50:18] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 07Epic: [Tech evaluation] Table management options for the S3 data lake - https://phabricator.wikimedia.org/T437472 (10JAllemandou) 03NEW [16:10:33] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304099 (10JAllemandou) [16:11:08] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304103 (10JAllemandou) [16:30:38] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics, 13Patch-For-Review: dbt: Weekly scheduling dependency checks only check the first date of the week - https://phabricator.wikimedia.org/T436744#12304241 (10amastilovic) I've identified the issue and created the MR to fix it. The r... [16:46:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [17:04:01] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Create new mediawiki.tags_change event stream - https://phabricator.wikimedia.org/T434143#12304397 (10Ottomata) @xcollazo @JAllemandou: Do we use any tags for things other than revisions in MWH? Do we use log ta... [17:06:53] (03CR) 10Joal: [C:03+1] Switch pageview_actor consumers to use webrequest_v2 instead. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1318283 (https://phabricator.wikimedia.org/T425668) (owner: 10Mforns) [17:07:14] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304415 (10Ottomata) [17:07:53] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304420 (10Ottomata) [17:19:13] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Create new mediawiki.tags_change event stream - https://phabricator.wikimedia.org/T434143#12304466 (10xcollazo) >>! In T434143#12304397, @Ottomata wrote: > @xcollazo @JAllemandou: Do we use any tags for things ot... [17:31:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [17:42:17] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 07Epic: [Tech evaluation] Table management options for the S3 data lake - https://phabricator.wikimedia.org/T437472#12304529 (10Ottomata) Thank you. Only a few comments about task description atm: > Judge components, not projects. The parts that m... [17:43:44] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304533 (10Isaac) re: global edit count and `no: this is MW's 'rough' count, not the accurate count.` You should ask the actual stakeholders but I suspect a che... [17:57:20] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10Dumps-Generation: mediawiki_content_current exports silently omit all pages edited between the snapshot cutoff and dump generation time - https://phabricator.wikimedia.org/T431872#12304587 (10JeffDoozan) Hi, @xcollazo I can confirm that this has fi... [17:58:36] 06Data-Engineering, 10Data-Platform, 06Data-Platform-SRE, 07Epic: [Tech evaluation] Table management options for the S3 data lake - https://phabricator.wikimedia.org/T437472#12304590 (10Ottomata) We'd like to think on this Table Management task as we think about {T437080} and {T437079}. I've discussed thi... [18:02:46] 06Data-Engineering: Investigate using --partial-parse for further isolating dbt models in different scheduling groups - https://phabricator.wikimedia.org/T436834#12304618 (10Ahoelzl) @GGoncalves-WMF can you create a parent epic for tracking and link this one. Thanks! [18:03:07] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304620 (10Ottomata) [18:03:17] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304621 (10Ottomata) a:03Ottomata [18:03:51] 06Data-Engineering, 06Data-Engineering-Radar, 06Product-Analytics: Airflow DAG: refine_api_requests_hourly - use artifact approach for rules.json - https://phabricator.wikimedia.org/T437058#12304624 (10Ottomata) [18:06:34] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics, 06Product Safety and Integrity (Sprint Heather (Aug 31 - Sept 18)): Dbt backfill: signal_driven_handling_bad_faith_accounts - https://phabricator.wikimedia.org/T437095#12304666 (10Ottomata) [18:06:53] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Product-Analytics, 06Product Safety and Integrity (Sprint Heather (Aug 31 - Sept 18)): Dbt backfill: signal_driven_handling_bad_faith_accounts - https://phabricator.wikimedia.org/T437095#12304667 (10Ahoelzl) a:03amastilovic [18:10:13] 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12304675 (10Ottomata) [18:11:26] 06Data-Engineering, 06Product-Analytics: Create a new database for WMF traffic related metrics - https://phabricator.wikimedia.org/T436678#12304690 (10Ottomata) Okay! @KCVelaga_WMF @mforns if that works for you all, and KC's jobs can write to `wmf_traffic`, then let's resolve this task. [18:13:24] 06Data-Engineering, 10Pageviews-Anomaly: "Nahui Ollin" is enwiki's top-viewed article in April 2026 - https://phabricator.wikimedia.org/T425600#12304695 (10Ottomata) 05Open→03Declined Not sure what we can do with this atm. Hopefully {T425661} will help. Declining for now, please reopen if this was inc... [18:20:40] 06Data-Engineering, 06Data-Engineering-Radar, 10Data-Platform, 06Growth-Team, and 6 others: Image Suggestions uses AI-generated images from Commons when adding images on English Wikipedia - https://phabricator.wikimedia.org/T422513#12304716 (10Ottomata) @GGoncalves-WMF who is the product owner of image sug... [18:22:27] 06Data-Engineering, 10Dumps-Generation, 10Wikidata: Json wikidata dumps incomplete - https://phabricator.wikimedia.org/T422303#12304734 (10Ottomata) 05Open→03Declined Declining, please reopen if there is something actionaable. Sorry @Jaredk8 ! Cc also #wikidata_platform_team [18:23:08] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 10MediaWiki-Special-pages, and 2 others: Move MediaWiki QueryPages computation to Hadoop - https://phabricator.wikimedia.org/T309738#12304739 (10Ottomata) [18:25:04] 06Data-Engineering: Make cusi_user and cusi_case table information available in the data lake - https://phabricator.wikimedia.org/T429703#12304745 (10Ahoelzl) @APizzata-WMF can you help where we left off on this? [18:25:11] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Make cusi_user and cusi_case table information available in the data lake - https://phabricator.wikimedia.org/T429703#12304747 (10Ahoelzl) [18:27:44] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Consider classifying www.wikipedia.org from "internal" referrer - https://phabricator.wikimedia.org/T422584#12304777 (10Ahoelzl) a:03Ahoelzl [18:27:47] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th): Consider classifying www.wikipedia.org from "internal" referrer - https://phabricator.wikimedia.org/T422584#12304780 (10Ottomata) cc @Ahoelzl @mforns @GGoncalves-WMF can you help understand priority around this? [18:28:18] 10Data-Engineering-Roadmap, 06cloud-services-team, 10Data-Services, 06tools-platform-team, 07Epic: Ability to create new views for tools and analytic purposes - https://phabricator.wikimedia.org/T424524#12304782 (10Ahoelzl) [18:29:19] 10Data-Engineering-Roadmap, 06cloud-services-team, 06Data-Platform-SRE, 10Data-Services, and 2 others: Ability to create new views for tools and analytic purposes - https://phabricator.wikimedia.org/T424524#12304786 (10Ottomata) [18:29:28] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 06Research: Add new referrer-class for AI chatbots/LLMs to clickstream dataset - https://phabricator.wikimedia.org/T428136#12304787 (10Ahoelzl) a:03Ahoelzl [18:50:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [18:55:54] 06Data-Engineering, 06Data-Engineering-Radar, 10Data-Platform, 06Growth-Team, and 6 others: Image Suggestions uses AI-generated images from Commons when adding images on English Wikipedia - https://phabricator.wikimedia.org/T422513#12304921 (10KStoller-WMF) >>! In T422513#12304715, @Ottomata wrote: > @GGon... [19:04:25] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Create new mediawiki.tags_change event stream - https://phabricator.wikimedia.org/T434143#12304948 (10Ottomata) Okay, given ^: @tchin, I think our goal here should be: 1. A new `mediawiki.revision_tags_change.v1... [19:08:06] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Capture relevant mediawiki tags changes in event stream(s) - https://phabricator.wikimedia.org/T434143#12304953 (10Ottomata) [19:10:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [19:14:26] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 10DPE-MediaWiki-Incremental-History: Incremental MediaWiki History v2 - data requests - https://phabricator.wikimedia.org/T437051#12304984 (10Ottomata) Yeah, we were just discussing this. Fields like local and global user edit counts in events are MW'... [19:40:13] FIRING: HaproxyKafkaNoMessages: Unexpected rate of produced HaproxyKafka messages by cp3072 - https://wikitech.wikimedia.org/wiki/HAProxyKafka#HaproxyKafkaNoMessages - https://grafana.wikimedia.org/d/d3e4e37c-c1d9-47af-9aad-a08dae2b3fd5/haproxykafka?orgId=1&var-site=esams&var-instance=cp3072 - https://alerts.wikimedia.org/?q=alertname%3DHaproxyKafkaNoMessages [19:55:13] RESOLVED: HaproxyKafkaNoMessages: Unexpected rate of produced HaproxyKafka messages by cp3072 - https://wikitech.wikimedia.org/wiki/HAProxyKafka#HaproxyKafkaNoMessages - https://grafana.wikimedia.org/d/d3e4e37c-c1d9-47af-9aad-a08dae2b3fd5/haproxykafka?orgId=1&var-site=esams&var-instance=cp3072 - https://alerts.wikimedia.org/?q=alertname%3DHaproxyKafkaNoMessages [20:59:34] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Capture relevant mediawiki tags changes in event stream(s) - https://phabricator.wikimedia.org/T434143#12305334 (10xcollazo) > There must have been a reason I created this ticket other than 'better data model' (... [21:00:12] 06Data-Engineering, 07Epic: Dumps v1 sunset - https://phabricator.wikimedia.org/T437518 (10Ahoelzl) 03NEW [21:00:33] 06Data-Engineering: Include v2 dumps into mirrors rsync configuration - https://phabricator.wikimedia.org/T437463#12305347 (10Ahoelzl) [21:00:34] 06Data-Engineering, 07Epic: Dumps v1 sunset - https://phabricator.wikimedia.org/T437518#12305348 (10Ahoelzl) [21:00:53] 06Data-Engineering: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12305352 (10Ahoelzl) [21:00:54] 06Data-Engineering, 07Epic: Dumps v1 sunset - https://phabricator.wikimedia.org/T437518#12305353 (10Ahoelzl) [21:01:45] 06Data-Engineering, 07Epic: XML Dumps v1 sunset - https://phabricator.wikimedia.org/T437518#12305355 (10xcollazo) [21:01:46] 06Data-Engineering: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12305357 (10Ahoelzl) [21:01:49] 06Data-Engineering: Implement 7z compression for v2 history dumps - https://phabricator.wikimedia.org/T437454#12305360 (10Ahoelzl) [21:01:56] 06Data-Engineering (Q1 FS26/27 July 1st - September 30th), 07Epic: XML Dumps v1 sunset - https://phabricator.wikimedia.org/T437518#12305362 (10Ahoelzl) [21:07:49] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Capture relevant mediawiki tags changes in event stream(s) - https://phabricator.wikimedia.org/T434143#12305380 (10CMyrick-WMF) Was it related to {T419434}? [21:08:52] 06Data-Engineering, 10DPE-MediaWiki-Incremental-History, 10Event-Platform, 13Patch-For-Review: Capture relevant mediawiki tags changes in event stream(s) - https://phabricator.wikimedia.org/T434143#12305383 (10CMyrick-WMF) And/or getting log action tags included in the MWH `revision_tags`? [21:12:06] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [21:52:06] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=000000026&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly