[02:36:19] 06Data-Engineering, 06Java-Scala-Standardization: Resolve conflict between GitLab CI automated package deployment token variable names - https://phabricator.wikimedia.org/T386056 (10Ottomata) 03NEW [02:42:08] 06Data-Engineering, 06Java-Scala-Standardization: Resolve conflict between GitLab CI automated package deployment token variable names - https://phabricator.wikimedia.org/T386056#10537586 (10Ottomata) I like `CI_RELEASE_TOKEN` better, it is more descriptive. It is a token, and it will be used by CI to make rel... [09:46:04] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.01.11 - 2025.01.31), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10537950 (10brouberol) 05Open→03In... [10:03:25] 06Data-Engineering, 06Data-Engineering-Radar, 10BDC-Implementation, 10fundraising-tech-ops, and 2 others: TLS connection for hive-standalone-metaserver with minio - https://phabricator.wikimedia.org/T385031#10537981 (10Gehel) [10:04:52] 06Data-Engineering, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Upgrade Spark to a version with long term Iceberg support, and with fixes to support Dumps 2.0 - https://phabricator.wikimedia.org/T338057#10538017 (10Gehel) [10:05:11] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10538021 (10Gehel) [10:05:23] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Draft a project plan for the Hadoop version 3 upgrade - https://phabricator.wikimedia.org/T379748#10538035 (10Gehel) [10:05:58] 06Data-Engineering, 06Data-Engineering-Radar, 10Cassandra, 10Data Pipelines, and 2 others: Create puppet resource for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#10538046 (10Gehel) [10:06:20] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Consider writing Spark files to Ceph (S3) instead of Hadoop - https://phabricator.wikimedia.org/T384500#10538062 (10Gehel) [10:06:34] 06Data-Engineering, 06Data-Persistence, 10Dumps-Generation, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Switch dumps 1.0 processes to use the analytics MariadB replicas (dbstore100[7-9]) - https://phabricator.wikimedia.org/T382947#10538058 (10Gehel) [10:06:50] 06Data-Engineering, 06Data-Engineering-Icebox, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Some wikibase tables not available in commonswiki_p - https://phabricator.wikimedia.org/T298452#10538076 (10Gehel) [10:34:59] 06Data-Engineering, 06Data-Persistence, 10Dumps-Generation, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Switch dumps 1.0 processes to use the analytics MariadB replicas (dbstore100[7-9]) - https://phabricator.wikimedia.org/T382947#10538265 (10BTullis) 05Open→03Resolved [10:55:25] 14Analytics, 10Data-Engineering-Wikistats: WikiStats: rounding is too rough - https://phabricator.wikimedia.org/T386075 (10RonnieV) 03NEW [11:22:26] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): The 'trigger_release' pipeline is not working from workflow_utils - https://phabricator.wikimedia.org/T386082 (10BTullis) 03NEW [11:40:16] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10538481 (10brouberol) I've opened [a... [12:09:34] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): The 'trigger_release' pipeline is not working from workflow_utils - https://phabricator.wikimedia.org/T386082#10538561 (10BTullis) a:03BTullis [12:17:08] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): The 'trigger_release' pipeline is not working from workflow_utils - https://phabricator.wikimedia.org/T386082#10538571 (10BTullis) 05Open→03Resolved I created a new access token called: `worfklow_utils_trigger_release` and assigned it t... [12:18:02] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): The 'trigger_release' pipeline is not working from workflow_utils - https://phabricator.wikimedia.org/T386082#10538574 (10BTullis) p:05Triage→03High [13:08:44] 06Data-Engineering, 06DBA, 10Charts (Sprint 15), 07Schema-change-in-production: Deploy patch-gjl_namespace_text.sql on x1.commonswiki for JsonConfig - https://phabricator.wikimedia.org/T385917#10538685 (10Ladsgroup) If it'd be possible, let's go this way to reduce the pain in the long-term. [14:01:53] 06Data-Engineering, 06Machine-Learning-Team, 06Research, 10Event-Platform: Expose revision revert risk scores in EventStreams - https://phabricator.wikimedia.org/T326179#10538901 (10Ottomata) FYI, this was just done for the article country predictions in {T382295}. [14:02:09] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 07Epic: HDFS capacity needs FY24/25 - https://phabricator.wikimedia.org/T384098#10538908 (10Gehel) [14:09:46] 14Data-Engineering (Q2 2024 October 1st - December 31th), 06SRE, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Streamline Data Platform access approvals for WMF staff - https://phabricator.wikimedia.org/T370424#10538953 (10Gehel) [14:18:48] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 3 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#10539007 (10Gehel) [14:21:32] 06Data-Engineering, 06Data-Platform-SRE, 06Java-Scala-Standardization, 10Discovery-Search (2025.02.10 - 2025.02.28): Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all dependencies are available,... - https://phabricator.wikimedia.org/T367405#10539055 [14:22:01] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Dumps 2.0, 10Discovery-Search (2025.02.10 - 2025.02.28), 07Epic, 13Patch-For-Review: EPIC: Update flink jobs to support Flink 1.20 - https://phabricator.wikimedia.org/T376812#10539061 (10Gehel) [14:22:33] 14Analytics, 06Data-Engineering, 06Data-Engineering-Icebox, 10EventStreams, and 4 others: [EPIC] Expose rdf-streaming-updater.mutation content through EventStreams - https://phabricator.wikimedia.org/T294133#10539073 (10Gehel) [14:32:08] 06Data-Engineering, 06Machine-Learning-Team, 06Research, 10Event-Platform: Expose revision revert risk scores in EventStreams - https://phabricator.wikimedia.org/T326179#10539149 (10Ottomata) [14:52:45] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Upgrade eventgate-wikimedia to node20 - https://phabricator.wikimedia.org/T383814#10539239 (10Ottomata) Attempted to deploy eventgate-analytics staging today, but it looks like staging k8s is out of IP addresses? `... [15:00:21] 06Data-Engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): The 'trigger_release' pipeline is not working from workflow_utils - https://phabricator.wikimedia.org/T386082#10539287 (10Ottomata) Interesting! I was just able to create an access token without an expiration date and use it to make CI pus... [15:05:32] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10539320 (10xcollazo) >>! In T379676#... [15:08:50] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Event-Platform, 13Patch-For-Review: Upgrade eventgate-wikimedia to node20 - https://phabricator.wikimedia.org/T383814#10539347 (10Ottomata) Created {T386107} [15:09:51] (03CR) 10Jforrester: "check experimental" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/1061964 (https://phabricator.wikimedia.org/T371706) (owner: 10Lucas Werkmeister (WMDE)) [15:17:32] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10539354 (10Ottomata) @brouberol nice... [15:23:02] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10539367 (10brouberol) @ottomata Sure... [15:23:50] 14Data-Engineering (Q1 2024 July 1st - September 30th), 10CirrusSearch, 10Event-Platform, 07Wikimedia-production-error: PHP Warning: Stats: Cannot associate label keys with label values: Not all initialized labels have an assigned value. - https://phabricator.wikimedia.org/T373086#10539372 (10Gehel) [15:26:28] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10539416 (10Ottomata) > Do we want al... [15:27:50] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10539437 (10Ottomata) Actually, if we... [15:36:17] 14Analytics, 10Event-Platform: swift_upload.py events not making it into kafka - https://phabricator.wikimedia.org/T260743#10539560 (10Gehel) [15:36:55] 14Analytics-Clusters, 06SRE, 10vm-requests: VM request for Analytics -> Elastic Search ML models update - https://phabricator.wikimedia.org/T258189#10539568 (10Gehel) [15:38:33] 14Analytics, 10Cloud-Services, 10Elasticsearch: Create partitioned CirrusSearchElasticaWrite topic - https://phabricator.wikimedia.org/T239135#10539591 (10Gehel) The #Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/8... [15:45:58] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 05FY2024-25 KR 5.2 Simplify feature development, 07OKR-Work: Design and document new Domain Events feature in MediaWiki core - https://phabricator.wikimedia.org/T379959#10539714 (10Ottomata) Now that we... [15:54:41] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10539898 (10BTullis) The [[https://wikitech.wikimedia.org/wiki/Dumps/Adds-changes... [15:55:32] 06Data-Engineering, 10Dumps-Generation: Some dumps are not available since mid may 2024 - https://phabricator.wikimedia.org/T366043#10539907 (10Gehel) [15:55:52] 14Analytics, 06Product-Analytics: Reportupdater cant run on stat1007 because "No module named pymysql" - https://phabricator.wikimedia.org/T240002#10539916 (10Gehel) [16:01:25] 14Analytics, 10CirrusSearch, 10Cognate, 10FlaggedRevs, and 14 others: Replace TitleMoveComplet(e|ing) hooks - https://phabricator.wikimedia.org/T250023#10540002 (10Gehel) [16:23:11] 14Data-Engineering (Q1 2024 July 1st - September 30th), 06Java-Scala-Standardization, 13Patch-For-Review: Update parent pom to disable fetching dependencies from Archiva and use Gitlab instead - https://phabricator.wikimedia.org/T367404#10540357 (10Gehel) [16:24:48] 06Data-Engineering, 07IPv6, 13Patch-For-Review: Some Search clusters have inconsistent AAAA DNS records for the primary IPv6 of the hosts - https://phabricator.wikimedia.org/T312555#10540389 (10Gehel) [16:29:18] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10540514 (10BTullis) Attempting the [[https://wikitech.wikimedia.org/wiki/Dumps/W... [16:30:43] 06Data-Engineering, 06DBA, 06MediaWiki-Platform-Team, 10MediaWiki-ResourceLoader, 07Schema-change: Drop module_deps table in WMF prod - https://phabricator.wikimedia.org/T385997#10540536 (10Krinkle) [16:35:55] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10540668 (10BTullis) Looks good. ` dumpsgen@deployment-snapshot05:/home/btullis$... [16:40:24] 06Data-Engineering, 07Epic, 13Patch-For-Review: [Maintenance] Safeguard VarnishKafka to HAProxy analytics transition - https://phabricator.wikimedia.org/T354694#10540836 (10JAllemandou) I've gone through more data validation (`webrequest_text` only, 2025-02-06T06:00) and found some things to discuss with @Fa... [16:44:43] 06Data-Engineering, 07Epic, 13Patch-For-Review: [Maintenance] Safeguard VarnishKafka to HAProxy analytics transition - https://phabricator.wikimedia.org/T354694#10540870 (10JAllemandou) [16:51:07] 06Data-Engineering, 10Research-engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Requesting Ceph S3 credentials for research - https://phabricator.wikimedia.org/T385608#10540887 (10bking) We have approval from the team, so we'll get to work on this as time allows. [17:01:32] (03CR) 10Pppery: "This was done in https://gerrit.wikimedia.org/r/c/analytics/refinery/+/1118538 instead." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1115418 (https://phabricator.wikimedia.org/T385185) (owner: 10Gerrit maintenance bot) [17:23:26] 06Data-Engineering, 10Dumps 2.0, 07Epic, 14Mediawiki Content: Dumps 2.0 Phase II: Production intermediate table milestone - https://phabricator.wikimedia.org/T358877#10541050 (10Ahoelzl) [17:24:13] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10541058 (10BTullis) Moving on to [[https://wikitech.wikimedia.org/wiki/Dumps/Cat... [17:43:22] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10541156 (10BTullis) Attempting a [[https://wikitech.wikimedia.org/wiki/Dumps/Oth... [17:56:20] (03CR) 10Xcollazo: "Do we also need to modify `Params` to document the new flag? Or was it there already?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1115441 (https://phabricator.wikimedia.org/T384383) (owner: 10Peter Fischer) [18:01:11] (03CR) 10Xcollazo: Adapt table/column names (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1115440 (https://phabricator.wikimedia.org/T384385) (owner: 10Peter Fischer) [18:10:13] (03CR) 10Xcollazo: Rewrite MediawikiDumper partitioning implementation (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1101892 (https://phabricator.wikimedia.org/T381016) (owner: 10Peter Fischer) [18:15:07] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10541250 (10BTullis) This content translation dump ran very quickly, but looks OK... [18:34:52] 06Data-Engineering, 06Data-Platform-SRE, 10Dumps-Generation, 05MW-1.39-notes, and 2 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10541322 (10BTullis) The media titles dumps seems to be ok, too: https://wikitech... [18:38:14] 06Data-Engineering, 06Data-Engineering-Radar, 06Data-Platform-SRE, 10Dumps-Generation, and 3 others: WE 5.4 KR - Hypothesis 5.4.6 - Q3 FY24/25 - Validate Dumps 1.0 compatibility with PHP 8.1 - https://phabricator.wikimedia.org/T382484#10541330 (10Ottomata) [18:40:06] 06Data-Engineering, 06Data-Engineering-Radar, 10Research-engineering, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Requesting Ceph S3 credentials for research - https://phabricator.wikimedia.org/T385608#10541333 (10Ottomata) [18:40:58] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 06MediaWiki-Platform-Team, and 2 others: Drop module_deps table in WMF prod - https://phabricator.wikimedia.org/T385997#10541336 (10Ottomata) [18:41:32] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 07Essential-Work: Update canary_events DAG to use an internal domain and/or the service mesh to obtain its eventstream config - https://phabricator.wikimedia.org/T384329#10541341 (10Ottomata) [18:42:06] 06Data-Engineering-Radar, 10BDC-Implementation, 06Data-Platform-SRE, 07Epic: [Trino] Connect to Google Sheets - https://phabricator.wikimedia.org/T385748#10541343 (10Ottomata) [18:45:02] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 10Charts (Sprint 15), 07Schema-change-in-production: Deploy patch-gjl_namespace_text.sql on x1.commonswiki for JsonConfig - https://phabricator.wikimedia.org/T385917#10541347 (10Ottomata) [18:47:34] 06Data-Engineering, 10Dumps 2.0, 10Dumps-Generation, 06SRE, 07Epic: Dumps generation cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#10541350 (10Ottomata) [18:47:58] 06Data-Engineering, 06Data-Engineering-Radar, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Unable to trigger dag with config - https://phabricator.wikimedia.org/T384805#10541354 (10Ottomata) [18:48:19] 06Data-Engineering, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Consider writing Spark files to Ceph (S3) instead of Hadoop - https://phabricator.wikimedia.org/T384500#10541355 (10Ottomata) [18:49:12] 06Data-Engineering, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Consider writing Spark files to Ceph (S3) instead of Hadoop - https://phabricator.wikimedia.org/T384500#10541358 (10Ottomata) @xcollazo can you add an appropriate parent task? [18:50:22] 06Data-Engineering, 06Java-Scala-Standardization: Resolve conflict between GitLab CI automated package deployment token variable names - https://phabricator.wikimedia.org/T386056#10541365 (10Ottomata) p:05Triage→03Medium [18:51:30] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Data-Platform-SRE, 10Wikidata, 07Epic: EPIC: WDQS categories migration - https://phabricator.wikimedia.org/T375520#10541371 (10Ahoelzl) [18:51:39] 06Data-Engineering, 10Dumps 2.0, 10Discovery-Search (2025.02.10 - 2025.02.28), 07Epic, 13Patch-For-Review: EPIC: Update flink jobs to support Flink 1.20 - https://phabricator.wikimedia.org/T376812#10541374 (10Ahoelzl) [18:51:49] 06Data-Engineering, 06Data-Platform-SRE, 10Wikidata, 07Epic: EPIC: WDQS categories migration - https://phabricator.wikimedia.org/T375520#10541376 (10Ahoelzl) [18:58:03] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Drop event_variant column from echo_event - https://phabricator.wikimedia.org/T385645#10541390 (10Ottomata) [18:59:14] 06Data-Engineering, 06DBA, 07Schema-change-in-production: Add normalization columns to categorylinks table - https://phabricator.wikimedia.org/T384592#10541393 (10Ottomata) @mforns to verify that this schema change is okay for Commons Impact Metrics. [18:59:28] 06Data-Engineering, 06Commons, 06Data-Persistence, 10MediaWiki-File-management, and 3 others: Update Data Pipelines with new image table schema - https://phabricator.wikimedia.org/T386133 (10Milimetric) 03NEW [18:59:50] 06Data-Engineering, 06Data-Persistence, 10MediaWiki-File-management, 10MediaWiki-Platform-Team (Radar): Update Data Pipelines with new image table schema - https://phabricator.wikimedia.org/T386133#10541408 (10Ottomata) [19:00:37] 06Data-Engineering, 06Data-Persistence, 10MediaWiki-File-management, 10MediaWiki-Platform-Team (Radar): Update Data Pipelines with new image table schema - https://phabricator.wikimedia.org/T386133#10541410 (10Ottomata) a:05Ladsgroup→03None [19:01:10] 06Data-Engineering, 06Commons, 06Data-Persistence, 10MediaWiki-File-management, and 4 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#10541414 (10VirginiaPoundstone) @Ladsgroup do you have a timeline for this mi... [19:01:42] 06Data-Engineering, 06Data-Persistence, 10MediaWiki-File-management, 10MediaWiki-Platform-Team (Radar): Update Data Pipelines with new image table schema - https://phabricator.wikimedia.org/T386133#10541421 (10VirginiaPoundstone) @mforns any conflict for #commons-impact-metrics? [19:01:51] 10Data-Engineering-Roadmap, 07Epic: [Epic] Migrate Data Engineering maintained NodeJS repositories to GitLab - https://phabricator.wikimedia.org/T366614#10541423 (10Ahoelzl) [19:02:34] 06Data-Engineering, 10Data-Engineering-Jupyter, 06Data-Platform-SRE: Cannot spawn a Jupyter server on stat1010 - https://phabricator.wikimedia.org/T385647#10541427 (10Ottomata) [19:02:55] 10Data-Engineering-Roadmap, 06Machine-Learning-Team, 06Wikimedia Enterprise, 07Epic, 10Event-Platform: [Event Platform] Implement PoC Event-Driven Data Pipeline for Revert Risk Model Scores using Event Platform Capabilities - https://phabricator.wikimedia.org/T338792#10541428 (10Ahoelzl) [19:05:17] 06Data-Engineering, 10Data-Engineering-Dashiki: Shut down the Page Creation dashboard - https://phabricator.wikimedia.org/T384410#10541438 (10Ottomata) [19:05:24] 06Data-Engineering, 10Data-Engineering-Dashiki: Shut down the Vital Signs dashboard - https://phabricator.wikimedia.org/T384411#10541440 (10Ottomata) [19:05:37] 06Data-Engineering, 10Data-Engineering-Dashiki: Shut down the Flow Reportcard - https://phabricator.wikimedia.org/T384413#10541441 (10Ottomata) [19:06:24] 06Data-Engineering, 10Dumps 2.0: Put together a DPE Deep Dive session on learnings from Dumps 2 XML generation code - https://phabricator.wikimedia.org/T384392#10541447 (10Ottomata) [19:07:18] 10Data-Engineering-Roadmap, 10DPE Temporary Accounts (Sprint 1), 07Epic: [Epic] Modify MediaWiki History pipeline for Temp Accounts - https://phabricator.wikimedia.org/T377304#10541450 (10Ottomata) [19:07:50] 06Data-Engineering, 06Commons, 06Data-Persistence, 10MediaWiki-File-management, and 4 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#10541452 (10Ladsgroup) >>! In T28741#10541414, @VirginiaPoundstone wrote: > @... [19:11:43] 06Data-Engineering, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28): Consider writing Spark files to Ceph (S3) instead of Hadoop - https://phabricator.wikimedia.org/T384500#10541485 (10xcollazo) [19:11:43] 06Data-Engineering, 10Dumps 2.0: Productionization of code to dump in XML - https://phabricator.wikimedia.org/T384382#10541486 (10xcollazo) [19:25:05] 10Data-Engineering-Roadmap, 06Product-Analytics, 10Temporary accounts, 06Trust and Safety Product Team, 07Epic: [Epic] Update schemas and instrumentation code for temporary accounts - https://phabricator.wikimedia.org/T374942#10541510 (10Ottomata) [19:26:01] 10Data-Engineering (Q3 2024 January 1st - March 31th): 8 new wikis missing from mediawiki_history - https://phabricator.wikimedia.org/T368788#10541512 (10Ottomata) [19:29:11] 06Data-Engineering, 10AQS2.0: AQS 2.0 Change AQS repository from generated data sets - https://phabricator.wikimedia.org/T352949#10541518 (10Ottomata) [19:32:23] 06Data-Engineering, 06MW-Interfaces-Team, 06Product-Analytics, 13Patch-For-Review: performer struct fields NULL in event_sanitized.mediawiki_revision_tags_change - https://phabricator.wikimedia.org/T352899#10541525 (10Ottomata) @nettrom_WMF I don't have an opinion or a lot of time to form one for this tick... [19:35:15] 10Data-Engineering-Roadmap, 10Dumps 2.0, 10Dumps-Generation, 07Epic: Outreach to producers of "other dumps" to raise awareness about Dumps 2.0 and options for deprecation or migration - https://phabricator.wikimedia.org/T364856#10541530 (10Ottomata) [19:35:31] 10Data-Engineering-Roadmap, 06Data-Platform-SRE, 10Dumps-Generation, 10MW-on-K8s, and 4 others: WE 5.4 KR - Hypothesis 5.4.4 - Q3 FY24/25 - Migrate current-generation dumps to run on kubernetes - https://phabricator.wikimedia.org/T352650#10541532 (10Ottomata) [19:36:28] 06Data-Engineering, 06Data-Engineering-Radar, 10Dumps-Generation, 06MediaWiki-Platform-Team, 06serviceops: Migrate WMF production from PHP 7.4 to PHP 8.1 - https://phabricator.wikimedia.org/T319432#10541535 (10Ottomata) [19:36:49] 06Data-Engineering, 06Data-Engineering-Radar, 06Campaigns-Product-Team, 10Product-Analytics (Kanban): Create ETL pipelines for campaigns-product baseline metrics - https://phabricator.wikimedia.org/T374491#10541537 (10Ottomata) [19:39:28] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 06Web-Team: WebClientError events have version in unexpected format - https://phabricator.wikimedia.org/T383275#10541544 (10Ottomata) [19:40:31] 06Data-Engineering, 06Commons, 06Data-Persistence, 10MediaWiki-File-management, and 4 others: Migrate file tables to a modern layout (image/oldimage; file/filerevision; add primary keys) - https://phabricator.wikimedia.org/T28741#10541545 (10Mitar) Does this mean that img table dump will change after migra... [19:41:00] 06Data-Engineering, 10AQS2.0: Golang scaffolfing service : Clean up golang scaffolding service - https://phabricator.wikimedia.org/T352950#10541546 (10Ottomata) [20:05:39] 06Data-Engineering, 10Data Pipelines, 10Dumps 2.0: Generate XML dumps for simplewiki - https://phabricator.wikimedia.org/T346147#10541613 (10VirginiaPoundstone) 05Open→03Resolved a:03VirginiaPoundstone [20:10:09] 06Data-Engineering, 10Data Pipelines, 10Observability-Metrics, 10SRE Observability (FY2024/2025-Q3), 07Technical-Debt: migrate Data Platform Engineering maintained metrics from graphite to prometheus - https://phabricator.wikimedia.org/T372855#10541618 (10VirginiaPoundstone) @lmata what is the timeline f... [20:59:52] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10541716 (10Ottomata) The design doc has [[ https://docs.google.com/document/d/1AeYeQFtzs... [21:33:42] 06Data-Engineering: Create a GitLab CI/CD Component project for WMF CI/CD templates and components - https://phabricator.wikimedia.org/T382430#10541845 (10amastilovic) a:03amastilovic [21:49:15] 10Data-Engineering (Q3 2024 January 1st - March 31th), 06Discovery-Search, 10Dumps 2.0, 10Data-Platform-SRE (2025.02.10 - 2025.02.28), 13Patch-For-Review: Add relevant kafka clusters to defined airflow connections in puppet - https://phabricator.wikimedia.org/T379676#10541985 (10xcollazo) >>! In T379676#... [22:01:17] 10Data-Engineering (Q3 2024 January 1st - March 31th), 10MediaWiki-DomainEvents, 06MW-Interfaces-Team, 07Epic: DomainEvents - Broadcasting and receiving cross-process events - https://phabricator.wikimedia.org/T379935#10542046 (10Ottomata) For the data re-use use cases I care about, beyond wiki (easy AKA E...