[07:05:58] 10Quarry: Quarry test suite is not being run anymore - https://phabricator.wikimedia.org/T392385#10759427 (10taavi) 05Open→03Resolved [07:10:24] 10Quarry: Remove gerrit git from quarry puppet - https://phabricator.wikimedia.org/T348748#10759437 (10taavi) a:05rook→03taavi [07:10:40] 10Quarry: Remove gerrit git from quarry puppet - https://phabricator.wikimedia.org/T348748#10759438 (10taavi) 05Open→03Resolved a:05taavi→03rook [07:50:54] 10Quarry, 06cloud-services-team: Update quarry redis deployment - https://phabricator.wikimedia.org/T392141#10759674 (10taavi) 05Open→03Resolved [08:19:49] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10759771 (10FCeratto-WMF) [08:40:13] (03CR) 10Filippo Giunchedi: Add Prometheus stats push (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [09:54:50] 06Data-Engineering, 06cloud-services-team, 10Cloud-VPS, 07IPv6: Add new WMCS IP ranges to analytics - https://phabricator.wikimedia.org/T392468 (10taavi) 03NEW [09:55:01] hello, I created T392468 but have no idea if I tagged it correctly for your side [09:55:01] T392468: Add new WMCS IP ranges to analytics - https://phabricator.wikimedia.org/T392468 [13:36:31] 10Data-Engineering (Q4 2025 April 1st - June 30th): NEW BUG REPORT significantly increased edit revert rate for 2025-03 edits; Android, iOS, Mobile Web, Other - https://phabricator.wikimedia.org/T391708#10760761 (10xcollazo) Coppy pasting from [[ https://wikimedia.slack.com/archives/C05R01RBX29/p1745262075670869... [13:52:31] milimetric: I've just noticed that my viz at https://codepen.io/Krinkle/pen/OJoVqXm for browser data stopped working. After some debugging I learned that this is because https://analytics.wikimedia.org/published/datasets/periodic/reports/metrics/browser/all_sites_by_browser_family_and_major.tsv now uses \n instead of \r\n as line seperator. I'll fix the tool now, but I'm curious if this is known/intended. [13:54:00] (03CR) 10Lucas Werkmeister (WMDE): Add Prometheus stats push (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1136417 (https://phabricator.wikimedia.org/T389344) (owner: 10Hasan Akgün (WMDE)) [13:54:32] @Krinkle: not known, not intended. I can't imagine why... nothing in that pipeline changed. I'll let some folks know [13:55:34] may've happened after the "Redacted" change. It's been a while I built that, so might be from around then, or more recently. [13:55:35] Thanks! [13:57:03] Yeah, that shouldn't have changed the output files though, just the intermediate dataset [14:00:24] Krinkle, milimetric: this has changed when we migrated the job generating the data from ReportUpdater to airflow. We have normalized output files to use \n without thinking it would be an issue. Sorry for not communicating better :S [14:00:55] Aha, nice, ok, good to know [14:05:57] you can see the CodePen source but basically: await fetch('....tsv').text(); data.split('\r\n\). foreach, line.split('\t') [14:06:16] anyway, I'll split by \n and trim any whitespace sometime later in case there was an \r somewhere, so it'll work both ways [14:08:07] thanks so much for adapting on your side Krinkle [15:14:49] (03PS1) 10Gerrit maintenance bot: Add rki.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1138395 (https://phabricator.wikimedia.org/T392499) [15:56:05] 10Data-Engineering (Q4 2025 April 1st - June 30th): Analyze impact for webrequest and unique devices pipelines to derive access_method without m-dot domain - https://phabricator.wikimedia.org/T389696#10761580 (10Jdlrobson-WMF) > Maybe, if we go for the x-analytics approach, we can make sure mobile-fronted is lo... [16:55:07] 06Data-Engineering, 10Event-Platform: [EventBus] Stableize EventSerializer and related classes - https://phabricator.wikimedia.org/T392516 (10Ottomata) 03NEW [16:55:29] 06Data-Engineering, 10Event-Platform, 07Technical-Debt: [EventBus] Stableize EventSerializer and related classes - https://phabricator.wikimedia.org/T392516#10761936 (10Ottomata) [16:55:32] 06Data-Engineering, 10Event-Platform, 07Technical-Debt: [EventBus] Stableize EventSerializer and related classes - https://phabricator.wikimedia.org/T392516#10761937 (10Ottomata) p:05Triage→03Low [17:40:13] 10Data-Engineering (Q4 2025 April 1st - June 30th): Facilitate automatic artifact cache warming for airflow-dags artifacts - https://phabricator.wikimedia.org/T392244#10762257 (10Ottomata) [17:40:14] 06Data-Engineering, 14Data-Engineering-Kanban, 10Data Pipelines: [Airflow] Automate sync'ing archiva packages to HDFS - https://phabricator.wikimedia.org/T294024#10762258 (10Ottomata) [17:50:13] 06Data-Engineering, 06Product-Analytics: Allow curl commands from Airflow BashOperator - https://phabricator.wikimedia.org/T392288#10762334 (10Ottomata) :D Direct use of BashOperator and PythonOperator in this way should be discouraged :/ This makes the Airflow scheduler do job work (and potentially have job... [17:53:29] 14Data-Engineering (Q3 2025 January 1st - March 31th), 06Experimentation Lab: NEW/CHANGE FEATURE REQUEST: Documentation for v1 Enterprise endpoint deprecation - https://phabricator.wikimedia.org/T389542#10762354 (10Ottomata) @jberkel FYI also related: - {T380874} - {T360794} [17:54:53] 06Data-Engineering, 06cloud-services-team, 10Cloud-VPS, 07IPv6: Add new WMCS IP ranges to analytics - https://phabricator.wikimedia.org/T392468#10762365 (10Ottomata) @taavi thanks! What's the timeline for this? [18:52:59] 06Data-Engineering, 06Product-Analytics: Allow curl commands from Airflow BashOperator - https://phabricator.wikimedia.org/T392288#10762504 (10xcollazo) >In the new kubernetes world, @brouberol can hopefully advise on how to use something like BashOperator in a k8s pod to do this! Oh, that was my assumption:... [19:18:09] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Create table and pyspark job to produce wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T391282#10762594 (10xcollazo) Ran the following: ` hostname -f an-launcher1002.eqiad.wmnet sud... [19:19:16] !log CREATEd table wmf_content.mediawiki_content_current_v1. T391282. [19:19:18] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:19:19] T391282: Create table and pyspark job to produce wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T391282 [20:41:45] 06Data-Engineering, 06Data-Engineering-Radar, 10BDC-Implementation, 06Data-Platform-SRE, and 2 others: [Trino] Develop procedure and scripting for Trino cluster maintenance. - https://phabricator.wikimedia.org/T386391#10762924 (10Jgreen) 05Open→03Resolved With API permissions deployed, trino-top wo... [20:51:13] 14Data-Engineering (Q3 2025 January 1st - March 31th), 06Experimentation Lab: NEW/CHANGE FEATURE REQUEST: Documentation for v1 Enterprise endpoint deprecation - https://phabricator.wikimedia.org/T389542#10763021 (10HShaikh) Hey @jberkel yes accessing these through the Enterprise APIs does require a separa... [21:40:42] 10Data-Engineering (Q4 2025 April 1st - June 30th): NEW BUG REPORT significantly increased edit revert rate for 2025-03 edits; Android, iOS, Mobile Web, Other - https://phabricator.wikimedia.org/T391708#10763120 (10Ahoelzl) There is no evidence of upstream data corruption, also the observed duplication and incon... [21:52:01] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Add data quality metrics to mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T392494#10763146 (10Ahoelzl)