[00:56:57] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list April 2025 - https://phabricator.wikimedia.org/T392868#10778712 (10GFontenelle_WMF) 05Open→03Resolved Thank you so much! 👏🏻 [08:54:30] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users for madalina - https://phabricator.wikimedia.org/T392893#10779403 (10tappof) Hi @Madalina, While I was checking, I noticed that you've already been added to the group. ` root@...:~#... [08:56:36] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users for madalina - https://phabricator.wikimedia.org/T392893#10779409 (10tappof) [09:17:21] (03PS1) 10Aqu: Refine deterministic transform deduplication [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) [09:36:18] (03PS2) 10Aqu: Refine deterministic transform deduplication [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) [09:36:26] (03CR) 10CI reject: [V:04-1] Refine deterministic transform deduplication [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [09:42:26] (03PS3) 10Aqu: Refine deterministic transform deduplication [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) [09:44:54] (03PS1) 10Aqu: Backport: Refine deterministic transform deduplication [analytics/refinery/source] (0.2.49) - 10https://gerrit.wikimedia.org/r/1140141 (https://phabricator.wikimedia.org/T369845) [09:48:10] FYI, there's a new KDC and I've merged a patch for Kerberos clients (like Hadoop) to also use it. It's the first KDC using Debian 12. In my tests it worked fine, but if you run into any auth issues with kerberised services, please let me know [09:50:00] (03PS4) 10Aqu: Refine deterministic transform deduplication [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) [13:04:56] !log Deploying Refinery at 1136103: Add mad.wikisource to pageview allowlist | https://gerrit.wikimedia.org/r/c/analytics/refinery/+/1136103 T391767 [13:04:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:04:59] T391767: Post-creation work for madwikisource - https://phabricator.wikimedia.org/T391767 [13:05:07] !log deploying refinery at 1138395: Add rki.wikipedia to pageview allowlist | https://gerrit.wikimedia.org/r/c/analytics/refinery/+/1138395 T392499 [13:05:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:05:12] T392499: Post-creation work for rkiwiki - https://phabricator.wikimedia.org/T392499 [13:52:27] (03CR) 10Mforns: [C:03+1] "Makes sense to me!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [13:53:13] (03CR) 10Mforns: [C:03+1] "Makes sense to me!" [analytics/refinery/source] (0.2.49) - 10https://gerrit.wikimedia.org/r/1140141 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [13:55:20] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012 (10xcollazo) 03NEW [13:55:34] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10780441 (10xcollazo) [13:55:36] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 07Epic: Dumps 2.0 Phase III: Production level dumps - https://phabricator.wikimedia.org/T366752#10780442 (10xcollazo) [13:56:12] 06Data-Engineering-Radar, 10BDC-Implementation, 07Epic: [MinIO] Investigate load-balancing approaches to eliminate SPOF - https://phabricator.wikimedia.org/T392249#10780446 (10BTullis) [14:02:44] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10780474 (10FCeratto-WMF) The schema change is running on DC masters; updating summary as I progress. [14:03:27] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10780475 (10FCeratto-WMF) [14:12:20] 06Data-Engineering, 10observability, 10Observability-Logging, 10Event-Platform: Data Platform, SRE Observability, overlaps, use cases, and potential - https://phabricator.wikimedia.org/T390323#10780502 (10lmata) [14:23:31] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10780568 (10FCeratto-WMF) [14:38:19] 06Data-Engineering, 10Community-Wishlist-Wishes, 06Data-Persistence, 10MediaWiki-Page-derived-data, and 3 others: Add page_is_redirect/page_namespace/page_title index - https://phabricator.wikimedia.org/T387537#10780663 (10KSiebert) [14:55:38] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10780741 (10FCeratto-WMF) [15:50:24] (03CR) 10Mforns: [V:03+2] "I've tested the create table statements, the sqoop library changes and the hql changes successfully. The generated data looks good." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1139117 (owner: 10Mforns) [16:07:20] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Commons-Impact-Metrics, 13Patch-For-Review: Update Commons Impact Metrics to account for new File table - https://phabricator.wikimedia.org/T389800#10781042 (10mforns) I've tested the changes: - create statements work well - sqoop library works well - mod... [16:11:42] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1140134 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [16:12:07] (03CR) 10Joal: [C:03+1] "LGTM!" [analytics/refinery/source] (0.2.49) - 10https://gerrit.wikimedia.org/r/1140141 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [16:14:44] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users for madalina - https://phabricator.wikimedia.org/T392893#10781063 (10MMiller_WMF) I am Madalina's manager and I approve her access to this data and these tools. [16:30:21] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 07Schema-change-in-production: Drop afl_patrolled_by from abuse_filter_log in production - https://phabricator.wikimedia.org/T391056#10781174 (10FCeratto-WMF) [16:51:44] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Document wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T392948#10781323 (10xcollazo) 05Open→03In progress [16:57:04] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Experimentation Lab, 13Patch-For-Review: FY 24-25 SDS 2.4.9 CDN Synthetic Beacon: EventGate & Varnish: update to receive events from beacon event v2 - https://phabricator.wikimedia.org/T391959#10781335 (10phuedx) >>! In T391959#10768610, @Ottomata wrote:... [17:11:50] 10Data-Engineering (Q4 2025 April 1st - June 30th), 06Traffic, 10DPE HAProxy Migration: Add HAproxy termination field to webrequest - https://phabricator.wikimedia.org/T387454#10781416 (10Ahoelzl) [17:40:15] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Document wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T392948#10781524 (10xcollazo) Documented at https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Content/Mediawiki_content_current_v1. I als... [17:41:18] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Document wmf_content.mediawiki_content_current_v1 - https://phabricator.wikimedia.org/T392948#10781525 (10xcollazo) [17:41:40] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10Commons-Impact-Metrics, 13Patch-For-Review: [CIM] Skewed ranking with the top Editors monthly API - https://phabricator.wikimedia.org/T370470#10781526 (10Ahoelzl) 05Open→03Resolved [18:18:19] 06Data-Engineering, 06Data-Engineering-Radar, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users for madalina - https://phabricator.wikimedia.org/T392893#10781722 (10Madalina) @tappof I had an access is denied error before. Everything seems ok now, thank you! [19:59:35] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10781930 (10xcollazo) While preparing to do the work for this ticket, we noticed a couple discrepancies in our content table... [19:59:56] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10781931 (10xcollazo) 05Open→03In progress [20:06:29] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10781942 (10xcollazo) Prepare: ` ssh an-launcher1002.eqiad.wmnet sudo -u analytics bash kerberos-run-command analytics spa... [20:07:41] !log Enabled merge-on-read on mw-content-history table and other TBLPROPERTIES changes. T393012. [20:07:44] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:07:44] T393012: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012 [20:19:55] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10781960 (10xcollazo) [20:21:52] 10Data-Engineering (Q4 2025 April 1st - June 30th), 10DPE-Mediawiki-Content, 13Patch-For-Review: Enable merge-on-read for wmf_content.mediawiki_content_history_v1 - https://phabricator.wikimedia.org/T393012#10781964 (10xcollazo) `merge-on-read` is now enabled on `wmf_content.mediawiki_content_history_v1`. W... [23:23:44] 06Data-Engineering, 06Data-Engineering-Radar, 10MediaWiki-Blocks, 10Multiblocks, and 7 others: Race condition in DatabaseBlockStore::acquireTarget() - https://phabricator.wikimedia.org/T389028#10782429 (10tstarling) 05Open→03Resolved