[04:56:35] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11426734 (10Marostegui)
[04:56:44] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11426735 (10Marostegui)
[05:58:25] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426803 (10Marostegui) Thank you both. After @Krinkle's message above, I am not sure whether I can proceed deleting them (on both parsercache and ms*) or I should w...
[05:59:19] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426804 (10Marostegui) For the ALTER I will just proceed on msX for now.
[06:05:28] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426820 (10Ladsgroup) >>! In T411497#11426803, @Marostegui wrote: > Thank you both. After @Krinkle's message above, I am not sure whether I can proceed deleting the...
[06:13:35] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426821 (10Marostegui) Of course this needs depooling for each alter as the table is around 50G
[06:15:16] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426824 (10ops-monitoring-bot) Depooled pc1011.eqiad.wmnet and pc2011.codfw.wmnet Schema change - marostegui@cumin1003 - T411497
[06:15:38] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426825 (10ops-monitoring-bot) Depooled pc1011.eqiad.wmnet and pc2011.codfw.wmnet Schema change - marostegui@cumin1003 - T411497
[06:22:57] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop modtoken and flags from cache tables - https://phabricator.wikimedia.org/T411497#11426847 (10Ladsgroup)
[08:47:30] <wikibugs>	 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10mwcli, 13Patch-For-Review: Update eventlogging image for MWCLI - https://phabricator.wikimedia.org/T406317#11427021 (10xSavitar) After some local testing, I noticed the following: - Setting `EVENTLOGGING_IMAGE=docker-registry.wikimedia.org/repos/da...
[08:55:07] <wikibugs>	 (03PS3) 10Joal: Add compute_mediawiki_revision_history_reduced HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1214023 (https://phabricator.wikimedia.org/T411484)
[09:43:18] <wikibugs>	 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10mwcli, 13Patch-For-Review: Update eventlogging image for MWCLI - https://phabricator.wikimedia.org/T406317#11427221 (10SuzanneWood-WMDE) That worked, thanks! : )
[09:43:47] <wikibugs>	 06Data-Engineering, 10MediaWiki-extensions-EventLogging, 10mwcli, 13Patch-For-Review: Update eventlogging image for MWCLI - https://phabricator.wikimedia.org/T406317#11427222 (10Addshore) Great, I'll look at merging this and including it in the next release too :)
[10:41:47] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Platform-SRE (2025.11.07 - 2025.11.28), 07OKR-Work, 13Patch-For-Review: Provide a Spark production access for dbt with Airflow - https://phabricator.wikimedia.org/T410017#11427436 (10BTullis) a:05JAllemandou→03BTullis I'm taking a slig...
[10:50:10] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598 (10JMonton-WMF) 03NEW
[10:56:13] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Increase partitions of mediawiki.content_history_reconcile.v1 - https://phabricator.wikimedia.org/T411598#11427516 (10JMonton-WMF) A question we haven't solved yet is: How many partitions should we use?   It looks like one partition was decided at some...
[11:08:09] <wikibugs>	 14Analytics-Radar, 06Data-Engineering, 06Data-Platform-SRE, 06serviceops-radar, and 2 others: Configuration Management for Kafka settings - https://phabricator.wikimedia.org/T276088#11427590 (10JMonton-WMF) Another option: I worked in the past with https://github.com/devshawn/kafka-gitops to manage all top...
[11:26:06] <wikibugs>	 (03PS3) 10Aqu: Add Spur datasets HQLs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285)
[13:32:34] <wikibugs>	 (03CR) 10Aqu: "I've converted the tables to Iceberg." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285) (owner: 10Aqu)
[14:30:03] <wmftkbot>	 !log Test Kitchen edge-unique experiments (poll 55140) - adds: none; removes: fy2025-26-we3.1-image-browsing-ab-test; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD
[14:30:06] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[14:36:16] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Research, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#11428447 (10tchin) @fkaelin How urgent is the need for this stream? We're considering moving off of PyFlink and this...
[14:39:00] <wikibugs>	 06Data-Engineering, 06Data-Engineering-Radar, 10Citoid, 10Page Content Service, and 6 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#11428470 (10Krinkle)
[15:06:50] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop ar_sha1 from archive table in wmf production - https://phabricator.wikimedia.org/T411163#11428630 (10Marostegui)
[15:06:58] <wikibugs>	 06Data-Engineering, 06DBA, 07Schema-change-in-production: Drop rev_sha1 from revision table in wmf production - https://phabricator.wikimedia.org/T411164#11428631 (10Marostegui)
[15:29:15] <jinxer-wm>	 FIRING: HdfsRpcQueueLength: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength
[16:09:15] <jinxer-wm>	 RESOLVED: HdfsRpcQueueLength: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength
[16:20:34] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Research, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#11429119 (10xcollazo) >>! In T360794#11428447, @tchin wrote: > @fkaelin How urgent is the need for this stream? We're...
[16:49:54] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10MediaWiki-Page-derived-data, 07OKR-Work, 13Patch-For-Review: Create new Druid datasource based on the `mediawiki_revision_history_v1` table - https://phabricator.wikimedia.org/T411484#11429249 (10JAllemandou) I have added the aggregated `events...
[17:31:29] <wikibugs>	 06Data-Engineering, 06Reader Growth Team, 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog, and 3 others: Add page_id and namespace to X-Analytics header in Mobile App requests (2025 remake) - https://phabricator.wikimedia.org/T409358#11429462 (10HSwan-WMF) We are unclear what the Reader Growth...
[17:32:05] <jinxer-wm>	 FIRING: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate  - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly
[17:37:05] <jinxer-wm>	 RESOLVED: [2x] EventgateProduceRateAnomaly: Significant produce rate deviation (+-25%) on eventgate-analytics. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate  - https://alerts.wikimedia.org/?q=alertname%3DEventgateProduceRateAnomaly
[18:53:34] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Review and productionize the WME differential privacy data set - https://phabricator.wikimedia.org/T409601#11429826 (10Snwachukwu) Thanks @Htriedman for all your response. I'm about creating the tables in the newly created database. I was wondering, Do...
[19:30:13] <wikibugs>	 06Data-Engineering (Q2 FY25/26 October 1st - December 31th): Review and productionize the WME differential privacy data set - https://phabricator.wikimedia.org/T409601#11430008 (10Snwachukwu) > To point (1) — in WME datasets, the page_id is referred to as identifier (inside a JSON object). Because this dataset i...
[20:39:18] <wikibugs>	 (03CR) 10Aqu: [C:03+2] Add Spur datasets HQLs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285) (owner: 10Aqu)
[20:39:33] <wikibugs>	 (03CR) 10Aqu: [V:03+2 C:03+2] Add Spur datasets HQLs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285) (owner: 10Aqu)
[20:42:16] <wikibugs>	 (03PS4) 10Aqu: Add Spur datasets HQLs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285)
[20:43:57] <wikibugs>	 (03CR) 10Aqu: [V:03+2 C:03+2] Add Spur datasets HQLs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1210696 (https://phabricator.wikimedia.org/T410285) (owner: 10Aqu)
[21:08:32] <wikibugs>	 06Data-Engineering, 10EventStreams, 10Wikidata, 10Wikidata-Query-Service: `rdf:type` triples for references missing in messages from Wikidata update stream - https://phabricator.wikimedia.org/T406436#11430387 (10BTracy-WMF) 05Open→03Resolved a:03BTracy-WMF Resolving this request and tracking the...