[00:10:16] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11296695 (10Eevans) >>! In T401021#11286092,... [00:12:07] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11296703 (10Eevans) [01:57:25] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work: Global Editor Metrics - Druid mediawiki_history_reduced changes - https://phabricator.wikimedia.org/T406069#11296779 (10amastilovic) @Ottomata so it sounds like we are ready to accept the `mediawiki_histor... [06:50:17] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=codfw%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [06:55:17] FIRING: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [07:10:17] RESOLVED: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [07:20:32] FIRING: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [08:05:32] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=codfw%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [11:09:29] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Wikimedia Enterprise, 10Wikimedia Enterprise - Content Integrity, 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07Essential-Work: Implement an Airflow operator for moving data from point A t... - https://phabricator.wikimedia.org/T405360#11297803 [11:19:55] !log migrated refine_netflow from an-launcher1002 to an-launcher1003 for T402943 [11:19:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:19:59] T402943: Repeated failures to resolve an-master100[3-4] from an-launcher1002 - resulting in pipeline failures - https://phabricator.wikimedia.org/T402943 [11:30:46] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work, 13Patch-For-Review: Add dbt related packages to conda-analytics - https://phabricator.wikimedia.org/T406766#11297869 (10BTullis) >>! In T406766#11296633, @amastilovic wrote: > @BTullis wo... [11:36:03] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work, 13Patch-For-Review: Add dbt related packages to conda-analytics - https://phabricator.wikimedia.org/T406766#11297887 (10BTullis) I pushed out the version 0.0.39 package to the test-cluste... [12:18:17] FIRING: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=codfw%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [12:23:17] RESOLVED: EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-logging-external in codfw. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?orgId=1&refresh=1m&var-dc=codfw%2Bprometheus/k8s&var-service=eventgate-logging-external - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly [12:24:32] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work: Create a dbt Docker container - https://phabricator.wikimedia.org/T406636#11298031 (10JMonton-WMF) The Merge Request https://gitlab.wikimedia.org/repos/data-engineering/dbt/-/merge_requests... [13:23:08] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Certain languages on wmf_raw.mediawiki_project_namespace_map appear wrong - https://phabricator.wikimedia.org/T407902#11298196 (10xcollazo) From what I can tell from P84208, this is the list of wikis that have a bad language code: ` advisors board boar... [13:52:55] 06Data-Engineering, 10Dumps-Generation, 06Wikibase Reuse Team, 10Wikidata, and 3 others: No Wikidata dumps for Week 40 of 2025 (recurring issue) - https://phabricator.wikimedia.org/T406429#11298318 (10hoo) This reminds me of T389199 (which ended up not being reproducible, just noting for reference). [14:14:36] (03PS1) 10Xcollazo: Fix various bugs on MW Dumper code. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) [14:46:32] (03CR) 10Joal: [C:03+1] "LGTM! One question" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) (owner: 10Xcollazo) [15:11:27] (03CR) 10Xcollazo: Fix various bugs on MW Dumper code. (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) (owner: 10Xcollazo) [15:25:45] (03PS2) 10Xcollazo: Fix various bugs on MW Dumper code. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) [15:29:26] (03CR) 10Xcollazo: Fix various bugs on MW Dumper code. (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) (owner: 10Xcollazo) [15:39:27] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06SRE: Move Druid realtime configuration out of Refinery into standalone repo on GitLab - https://phabricator.wikimedia.org/T407994 (10amastilovic) 03NEW [15:59:41] (03CR) 10Xcollazo: [C:03+2] Fix various bugs on MW Dumper code. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) (owner: 10Xcollazo) [16:00:21] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997 (10Dreamy_Jazz) 03NEW [16:01:49] 06Data-Engineering, 06Data-Engineering-Radar, 10AbuseFilter, 06Product Safety and Integrity (Sprint Mint Choc Chip Ice Cream (Oct 20 - Nov 7)), and 2 others: AbuseFilter abuse_filter_log table: Store IP addresses as hex values - https://phabricator.wikimedia.org/T395612#11299065 (10Dreamy_Jazz) [16:02:30] 06Data-Engineering, 06Data-Engineering-Radar, 10AbuseFilter, 06Product Safety and Integrity (Sprint Mint Choc Chip Ice Cream (Oct 20 - Nov 7)), and 2 others: AbuseFilter abuse_filter_log table: Store IP addresses as hex values - https://phabricator.wikimedia.org/T395612#11299068 (10Dreamy_Jazz) 05Open... [16:15:11] (03Merged) 10jenkins-bot: Fix various bugs on MW Dumper code. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198080 (https://phabricator.wikimedia.org/T407902) (owner: 10Xcollazo) [16:21:27] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11299160 (10Marostegui) a:03Marostegui [16:21:31] 06Data-Engineering, 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11299162 (10Marostegui) p:05Triage→03Medium [16:48:35] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work, 13Patch-For-Review: Add dbt related packages to conda-analytics - https://phabricator.wikimedia.org/T406766#11299238 (10BTullis) We got this working with spark in session mode, using the... [16:53:19] !log migrated refine_sanitize from an-launcher1002 to an-launcher1003 for T402943 [16:53:22] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:53:23] T402943: Repeated failures to resolve an-master100[3-4] from an-launcher1002 - resulting in pipeline failures - https://phabricator.wikimedia.org/T402943 [17:25:59] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work, 13Patch-For-Review: Add dbt related packages to conda-analytics - https://phabricator.wikimedia.org/T406766#11299444 (10BTullis) Since I had rebuilt version 0.0.39 of conda-analytics, I u... [17:32:43] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 3 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11299544 (10RKemper) Looks like Brian already covered it, but just to reiterate, the one cumin test host ap... [17:39:35] 14Analytics-Radar, 06Data-Engineering, 06Data-Engineering-Icebox, 10XAnalytics, 10MobileFrontend (Tracking): MobileFrontend should use XAnalytics extension - https://phabricator.wikimedia.org/T217859#11299607 (10Jdlrobson-WMF) [17:53:41] 06Data-Engineering, 10Dumps-Generation, 06Wikibase Reuse Team, 10Wikidata, and 3 others: No Wikidata dumps for Week 40 of 2025 (recurring issue) - https://phabricator.wikimedia.org/T406429#11299682 (10BTullis) >>! In T406429#11298318, @hoo wrote: > This reminds me of T389199 (which ended up not being repro... [19:03:34] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11299916 (10Eevans) [19:09:49] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06SRE: Move Druid realtime configuration out of Refinery into standalone repo on GitLab - https://phabricator.wikimedia.org/T407994#11299948 (10amastilovic) [19:49:56] 06Data-Engineering, 07Essential-Work: Check what projects, if any, are still using gitlab-docker-runner - https://phabricator.wikimedia.org/T408019 (10xcollazo) 03NEW [19:53:17] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11300119 (10Eevans) [20:14:45] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11300177 (10Eevans) [20:32:25] (03PS1) 10Xcollazo: Add utility to create SHA256 fingerprints of the files of a particular HDFS folder. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198152 (https://phabricator.wikimedia.org/T401022) [22:04:11] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11300657 (10Eevans) [22:05:30] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11300658 (10Eevans) [23:18:19] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11300939 (10Eevans) [23:33:37] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.10.17 - 2025.11.07), 07OKR-Work: Add dbt related packages to conda-analytics - https://phabricator.wikimedia.org/T406766#11300966 (10amastilovic) >>! In T406766#11297869, @BTullis wrote: > Yes, that's true. But conda-analy... [23:55:18] 06Data-Engineering, 10Data Pipelines: Add user_central_id to mediawiki_history and mediawiki_history_reduced Hive tables - https://phabricator.wikimedia.org/T365648#11300977 (10amastilovic) a:03amastilovic