[12:33:43] 06Data-Engineering, 06Growth-Team, 10GrowthExperiments, 06Structured-Data-Backlog, and 2 others: structured_data.commons_entity stuck at 2025-01-20 - https://phabricator.wikimedia.org/T387470#10590590 (10mfossati) CC @Ahoelzl @xcollazo . [13:10:43] 06Data-Engineering, 06Data-Persistence, 10MediaWiki-Page-derived-data, 07Schema-change: Add page_is_redirect/page_namespace/page_title index - https://phabricator.wikimedia.org/T387537#10590702 (10Ladsgroup) I suggest doing this backwards, the queries should happen on `redirect` table instead. Maybe even... [13:26:54] 06Data-Engineering, 10Data-Platform-SRE (2025.03.01 - 2025.03.21), 07Epic: HDFS capacity needs FY24/25 - https://phabricator.wikimedia.org/T384098#10590756 (10Gehel) [13:27:02] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.03.01 - 2025.03.21), 07Essential-Work, 13Patch-For-Review: DAG failing due to failure to acquire lock on wmf_data_ops.data_quality_metrics table - https://phabricator.wikimedia.org/T386114#10590764 (1... [13:27:20] 06Data-Engineering, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.03.01 - 2025.03.21), 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10590770 (10Gehel) [13:28:29] 06Data-Engineering, 06Data-Engineering-Radar, 10Cassandra, 10Data Pipelines, and 2 others: Create puppet resource for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#10590808 (10Gehel) [13:28:37] 06Data-Engineering, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Draft a project plan for the Hadoop version 3 upgrade - https://phabricator.wikimedia.org/T379748#10590816 (10Gehel) [13:31:17] 06Data-Engineering, 10Data-Engineering-Jupyter, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Cannot spawn a Jupyter server on stat1010 - https://phabricator.wikimedia.org/T385647#10590880 (10Gehel) [13:32:05] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Data-Platform-SRE (2025.03.01 - 2025.03.21), 07Essential-Work: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10590891 (10Gehel) [13:32:11] 06Data-Engineering, 06Data-Engineering-Radar, 10Data-Platform-SRE (2025.03.01 - 2025.03.21), 13Patch-For-Review: Unable to trigger dag with config - https://phabricator.wikimedia.org/T384805#10590888 (10Gehel) [13:32:35] 06Data-Engineering, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Airflow UI sometimes shows no response for a DAG run task with many mapped tasks - https://phabricator.wikimedia.org/T381479#10590899 (10Gehel) [13:33:23] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 3 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10590913 (10Gehel) [13:33:29] 06Data-Engineering, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Test if an existing conda environment with Spark 3.1.2 clients works fine with Spark 3.5.3 - https://phabricator.wikimedia.org/T380417#10590919 (10Gehel) [13:33:59] 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Some wikibase tables not available in commonswiki_p - https://phabricator.wikimedia.org/T298452#10590930 (10Gehel) [13:34:37] 06Data-Engineering, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.03.01 - 2025.03.21): Consider writing Spark files to Ceph (S3) instead of Hadoop - https://phabricator.wikimedia.org/T384500#10590940 (10Gehel) [13:37:34] 06Data-Engineering, 06Java-Scala-Standardization, 10Discovery-Search (2025.03.01 - 2025.03.21): Create Gitlab CI templates for JVM packages - https://phabricator.wikimedia.org/T386406#10590995 (10Gehel) [13:38:04] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 3 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10591003 (10Gehel) [13:39:34] 06Data-Engineering, 06Data-Platform-SRE, 06Java-Scala-Standardization, 10Discovery-Search (2025.03.01 - 2025.03.21): Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available,... - https://phabricator.wikimedia.org/T367405#10591036 [13:39:49] 14Analytics, 06Data-Engineering, 06Data-Engineering-Icebox, 10EventStreams, and 4 others: [EPIC] Expose rdf-streaming-updater.mutation content through EventStreams - https://phabricator.wikimedia.org/T294133#10591037 (10Gehel) [13:39:52] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Discovery-Search (2025.03.01 - 2025.03.21), 07Epic, 13Patch-For-Review: EPIC: Update flink jobs to support Flink 1.20 - https://phabricator.wikimedia.org/T376812#10591047 (10Gehel) [13:42:43] 06Data-Engineering, 06Research: Research airflow instance can not reach out to gitlab.wikimedia.org - https://phabricator.wikimedia.org/T386933#10591063 (10xcollazo) [14:16:54] (03PS1) 10Ebrahim: Enable Persian language [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1123658 [14:17:14] (03PS2) 10Ebrahim: Enable Persian language [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/1123658 [16:14:38] 10Data-Engineering (Q3 2024 January 1st - March 31th), 03Abstract Wikipedia Fix-It tasks, 10Abstract Wikipedia team (25Q3 (Jan–Mar)): Update wikifunctions Grafana dashboard for service-utils - https://phabricator.wikimedia.org/T387360#10591545 (10DSantamaria) >>! In T387360#10585355, @tchin wrote: > Somethin... [16:46:53] 06Data-Engineering, 10AQS2.0, 10Commons-Impact-Metrics, 13Patch-For-Review: Get more data to populate CIM time series datasets for the AQS local test environment - https://phabricator.wikimedia.org/T372989#10591642 (10mforns) @EChukwukere-WMF I think the changes you pasted belong to [[ https://gitlab.wikim... [16:48:08] 06Data-Engineering, 10AQS2.0, 10Commons-Impact-Metrics, 13Patch-For-Review: Get more data to populate CIM time series datasets for the AQS local test environment - https://phabricator.wikimedia.org/T372989#10591645 (10EChukwukere-WMF) @mforns Thanks for clarying this, I will move this ticket back to in pro... [16:49:37] 06Data-Engineering, 10Commons-Impact-Metrics, 13Patch-For-Review: [CIM] Skewed ranking with the top Editors monthly API - https://phabricator.wikimedia.org/T370470#10591648 (10mforns) For the record, @EChukwukere-WMF and I managed to make the test environment work, and we checked that the issue pointed out i... [17:35:20] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10591824 (10daniel) >>! In T379935#10580653, @Ottomata wrote: > (This comment is a half r... [17:59:44] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10591917 (10Ottomata) Thank you daniel! Your config examples make a lot of sense! This... [18:14:38] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10592031 (10Ottomata) One more example for you to verify I've got this right. Let's say... [18:17:15] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10592040 (10Ottomata) Idea: In `$wgEventStreams`, all streams declare their schema (titl... [18:54:39] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10592138 (10daniel) > Let's say we want to support receiving events that don't have a spe... [18:57:08] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10592143 (10daniel) > Alternatively, instead of determining $eventType from config, it co... [18:59:24] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10592144 (10daniel) >>! In T379935#10592040, @Ottomata wrote: > E.g. If JsonCodec impleme... [20:17:44] 06Data-Engineering, 10Commons-Impact-Metrics, 10Commons-Impact-Metrics-Requests: Update Commons Impact Metrics allow-list February 2025 - https://phabricator.wikimedia.org/T387592 (10GFontenelle_WMF) 03NEW [20:45:24] 10Data-Engineering (Q3 2024 January 1st - March 31th), 03Abstract Wikipedia Fix-It tasks, 10Abstract Wikipedia team (25Q3 (Jan–Mar)), 13Patch-For-Review: Update wikifunctions Grafana dashboard for service-utils - https://phabricator.wikimedia.org/T387360#10592393 (10tchin) >>! In T387360#10585355, @tchin w... [20:58:30] 10Data-Engineering (Q3 2024 January 1st - March 31th), 03Abstract Wikipedia Fix-It tasks, 10Abstract Wikipedia team (25Q3 (Jan–Mar)), 13Patch-For-Review: Update wikifunctions Grafana dashboard for service-utils - https://phabricator.wikimedia.org/T387360#10592423 (10ecarg) Thank you, @tchin! I was querying... [22:32:14] 10Data-Engineering (Q3 2024 January 1st - March 31th), 07Essential-Work: Analyze Dumps Usage Through Apache Logs - https://phabricator.wikimedia.org/T383175#10592595 (10VirginiaPoundstone) @HCoplin-WMF did you get the LDAP access? @Ahoelzl when do you expect these additional view of the data to be added to th...