[02:45:53] (03PS5) 10Ecarg: Create HQL for new join table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) [02:53:01] (03PS2) 10David Martin: Add wikilambda_zobject_join query to sqoop [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1041815 (https://phabricator.wikimedia.org/T363434) [02:59:38] (03CR) 10David Martin: "The table creation patch was successfully deployed and operating correctly on production. Would be very helpful to get this patch landed " [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) (owner: 10Ecarg) [03:00:36] (03CR) 10David Martin: "The table creation patch was successfully deployed and is operating correctly on production. Would be very helpful to get this patch land" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1041815 (https://phabricator.wikimedia.org/T363434) (owner: 10David Martin) [03:28:30] (03PS6) 10David Martin: Create wikilambda_zobject_join table in HQL [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) (owner: 10Ecarg) [04:49:32] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9938084 (10Marostegui) [07:40:21] 06Data-Engineering, 06Data-Platform-SRE: Design a suitable DAG deployment method - https://phabricator.wikimedia.org/T368033#9938391 (10brouberol) I agree that option 2 is probably a good default. The repo is small, we have a dedicated Ceph cluster. I'm not forseeing any issue, and we can always pivot if we do. [08:32:08] (03PS1) 10Aqu: Make TransformFunction Deduplication Deterministic [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1051061 (https://phabricator.wikimedia.org/T356762) [08:46:59] (03PS2) 10Aqu: Make TransformFunction Deduplication Deterministic [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1051061 (https://phabricator.wikimedia.org/T356762) [09:04:26] (03CR) 10CI reject: [V:04-1] Make TransformFunction Deduplication Deterministic [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1051061 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [09:31:22] (03CR) 10Gmodena: Make TransformFunction Deduplication Deterministic (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1051061 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [09:40:26] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Grant Access to analytics-privatedata-users for cwylo - https://phabricator.wikimedia.org/T368027#9938810 (10Volans) [09:42:19] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Grant Access to analytics-privatedata-users for cwylo - https://phabricator.wikimedia.org/T368027#9938825 (10Volans) Pending @leila 's approval. [09:44:14] 14Analytics-Radar, 06Data-Engineering-Icebox, 10Data-Services: Discuss labsdb visibility of rev_text_id and ar_comment - https://phabricator.wikimedia.org/T158166#9938830 (10fnegri) p:05Triage→03Low [10:22:19] 14Analytics-Radar, 06Data-Engineering, 06Discovery-Search, 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#9939007 (10gmodena) Alertmanager has been firing again for... [12:40:39] (03CR) 10Ottomata: Make TransformFunction Deduplication Deterministic (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1051061 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [13:11:41] 06Data-Engineering: Request for Kerb credentials for Ariel Glenn - https://phabricator.wikimedia.org/T368911 (10ArielGlenn) 03NEW [13:44:58] 06Data-Engineering: Request for Kerb credentials for Ariel Glenn - https://phabricator.wikimedia.org/T368911#9939728 (10Ottomata) Approved. [13:45:17] 06Data-Engineering, 10Dumps 2.0 (Kanban Board), 10Event-Platform: [Dumps 2] Spike: Figure root causes of missing rows when doing reconciliation - https://phabricator.wikimedia.org/T368176#9939731 (10Milimetric) p:05Triage→03Medium a:03Milimetric [14:23:47] 06Data-Engineering, 10Cassandra: Create keyspace and table for Knowledge Gaps - https://phabricator.wikimedia.org/T340494#9939909 (10lbowmaker) Including @VirginiaPoundstone for AQS 2 questions. [14:28:14] 06Data-Engineering, 10Temporary accounts, 10Data-Platform-SRE (2024.06.17 - 2024.07.07): Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#9939924 (10lbowmaker) @BTullis this seems good for now. We can just re-run nearer the time - last... [14:54:17] 06Data-Engineering, 06Discovery-Search, 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#9940078 (10sbassett) [15:20:21] 06Data-Engineering, 03Discovery-Search (Current work), 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#9940213 (10Gehel) [15:22:27] 06Data-Engineering, 10CirrusSearch, 06Data Products, 10MediaWiki-extensions-EventLogging, and 2 others: Error: Call to a member function getPageAsLinkTarget() on null - https://phabricator.wikimedia.org/T368543#9940243 (10Gehel) [15:24:06] 06Data-Engineering, 10Dumps 2.0 (Kanban Board), 10Event-Platform: [Dumps 2] Spike: Figure root causes of missing rows when doing reconciliation - https://phabricator.wikimedia.org/T368176#9940247 (10Milimetric) My first hunch, that the revisions were coming from only specific pages, is wrong: ` spark-sql (d... [15:25:12] 06Data-Engineering: [Epic] Migrate Data Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927 (10Ottomata) 03NEW [15:25:14] 06Data-Engineering: [Epic] Migrate Data Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940260 (10Ottomata) [15:25:16] 06Data-Engineering, 10Event-Platform: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614#9940261 (10Ottomata) [15:25:47] (03CR) 10Mforns: "Ah! I fully understood your idea now." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1049281 (https://phabricator.wikimedia.org/T342267) (owner: 10Milimetric) [15:26:57] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and validation that deploy... - https://phabricator.wikimedia.org/T367405#9940279 [15:26:58] 06Data-Engineering: [Epic] Migrate Data Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940280 (10Ottomata) [15:27:25] 06Data-Engineering: [Epic] Migrate Data Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940292 (10Ottomata) [15:27:27] 10Data-Engineering (Q4 2024 April 1st - June 30th): Migrate refinery HQL files to CI/CD supported GitLab repository - https://phabricator.wikimedia.org/T362832#9940293 (10Ottomata) [15:33:41] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and validation that deploy... - https://phabricator.wikimedia.org/T367405#9940364 [15:39:31] 06Data-Engineering, 03Discovery-Search (Current work), 07Wikimedia-production-error: '.event.pageViewId' should be string, '.event.subTest' should be string, '.event.searchSessionId' should be string - https://phabricator.wikimedia.org/T286814#9940442 (10pfischer) We skipped estimation of this ticket for now... [15:42:02] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and validation that deploy... - https://phabricator.wikimedia.org/T367405#9940447 [15:44:52] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and validation that deploy... - https://phabricator.wikimedia.org/T367405#9940453 [15:47:15] 06Data-Engineering: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940474 (10Ottomata) [15:48:06] 06Data-Engineering: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940488 (10Ottomata) [15:49:21] 06Data-Engineering: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940491 (10Ottomata) [15:50:20] 06Data-Engineering, 06Data-Platform-SRE: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940494 (10Ottomata) [15:51:20] 06Data-Engineering, 06Data-Platform-SRE: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9940501 (10Ottomata) @Ahoelzl here's my first attempt at pulling this together. Note that most of the work is around migrating Java lib repos to GitLab.... [15:56:35] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Grant Access to analytics-privatedata-users for cwylo - https://phabricator.wikimedia.org/T368027#9940509 (10leila) approved. thanks! [17:07:10] 14Analytics-Radar, 06Data-Engineering-Icebox, 10Data-Services: Discuss labsdb visibility of rev_text_id and ar_comment - https://phabricator.wikimedia.org/T158166#9940977 (10fnegri) I've marked this as "Low" priority as there was no activity on the task since 2019. If someone is still interested in having th... [18:08:36] 06Data-Engineering, 06Data-Platform-SRE: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9941438 (10Ottomata) [18:10:42] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available, and validation that deploy... - https://phabricator.wikimedia.org/T367405#9941443 [18:19:21] 06Data-Engineering, 10Dumps 2.0 (Kanban Board), 10Event-Platform: [Dumps 2] Spike: Figure root causes of missing rows when doing reconciliation - https://phabricator.wikimedia.org/T368176#9941481 (10Ottomata) @Milimetric I'm especially concerned about the sha1 mismatches. Maybe check into those first? Is t... [18:34:48] 06Data-Engineering, 06Data-Platform-SRE: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9941632 (10Gehel) >>! In T368927#9940491, @Ottomata wrote: > Note that most of the work is around migrating Java lib repos to GitLab. These are tracked i... [19:20:34] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9941815 (10apaskulin) [19:21:04] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9941817 (10apaskulin) [19:24:46] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9941840 (10apaskulin) [20:00:54] 06Data-Engineering, 06Data-Platform-SRE: [Epic] Migrate Data Platform Engineering maintained git repos to GitLab - https://phabricator.wikimedia.org/T368927#9942044 (10Ottomata) Yes sorry, phab ticket # copy/paste fail. I meant to link {T367405}. Just checking, did I interpret T367405 wrong then? Is that on... [21:30:06] 06Data-Engineering, 06Tech-Docs-Team, 07Epic, 05Goal: Redesign Data Platform docs on Wikitech - https://phabricator.wikimedia.org/T350911#9942411 (10TBurmeister) 05In progress→03Resolved Final status update! * Finished categorizing / recategorizing all the pages with [[ https://wikitech.wikimedia.o...