[07:25:36] 06Data-Engineering, 06Project-Admins: Consider archiving `Q1 FY2025/26 July 1st - September 30th` - https://phabricator.wikimedia.org/T401018#11063442 (10Aklapper) [07:26:23] 06Data-Engineering, 06Project-Admins: Consider archiving `Q1 FY2025/26 July 1st - September 30th` - https://phabricator.wikimedia.org/T401018#11063445 (10Aklapper) [07:27:45] 06Data-Engineering, 06Data-Engineering-Radar, 06Growth-Team, 10GrowthExperiments-NewcomerTasks, and 3 others: Error for mediawiki.cirrussearch-request: '' should NOT have additional properties - https://phabricator.wikimedia.org/T399965#11063450 (10pfischer) [07:29:36] 06Data-Engineering, 06Data-Platform-SRE, 06Java-Scala-Standardization, 10Discovery-Search (2025.07.25 - 2025.08.15), 13Patch-For-Review: Migrate existing Java packages to deploying to Gitlab, including new version of parent pom, validation that all depen... - https://phabricator.wikimedia.org/T367405#11063471 [07:30:32] 06Data-Engineering, 06Java-Scala-Standardization, 10Discovery-Search (2025.07.25 - 2025.08.15): Create Gitlab CI templates for JVM packages - https://phabricator.wikimedia.org/T386406#11063501 (10pfischer) [07:30:36] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Discovery-Search (2025.07.25 - 2025.08.15), 13Patch-For-Review: SUP: Use flink 1.20.1 - https://phabricator.wikimedia.org/T398159#11063505 (10pfischer) [07:30:40] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 4 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#11063503 (10pfischer) [07:31:08] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15), 13Patch-For-Review: Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11063517 (10pfischer) [10:05:18] (03CR) 10Joal: [C:03+1] "LGTM! Merge at will" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [10:56:58] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 10EventStreams, 10Event-Platform, 13Patch-For-Review: EventStreams: duplicate events from double compute (wdqs/rdf) streams - https://phabricator.wikimedia.org/T396564#11064464 (10dcausse) Preferred to add a new settings, turns out you can't really... [11:25:16] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15), 13Patch-For-Review: Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11064520 (10BTullis) The flink-operator has been upgraded... [11:29:33] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15), 13Patch-For-Review: Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11064525 (10BTullis) I will attempt to test the `mw-page-c... [12:02:05] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15): Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11064716 (10BTullis) The first test is to make sure that the operator restarts i... [12:11:05] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15): Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11064749 (10BTullis) I'm happy that the `flinkdeployment` object was recreated c... [12:59:42] 10Data-Engineering-Roadmap, 10DPE-Mediawiki-Content, 10Data-Platform-SRE (2025.07.26 - 2025.08.15), 10Discovery-Search (2025.07.25 - 2025.08.15): Flink: Update k8s operator to 1.12.0 - https://phabricator.wikimedia.org/T398162#11065054 (10BTullis) 05Open→03Resolved Upgraded on codfw and eqiad and d... [13:02:54] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Review Image Suggestion pipeline SLOs - https://phabricator.wikimedia.org/T400282#11065086 (10GGoncalves-WMF) I met with @mfossati to discuss this. The main takeaways are: - Image Suggestions depends on MediaWiki Content Current (MWCC), which is produce... [13:03:50] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Review Image Suggestion pipeline SLOs - https://phabricator.wikimedia.org/T400282#11065091 (10GGoncalves-WMF) 05Open→03Resolved [13:47:54] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Change RefererClassifier to capture malformed but internal referer strings as internal (wmf.pageview_actor) - https://phabricator.wikimedia.org/T400799#11065315 (10JAllemandou) I have looked at the code and the change, while not very difficult, is not st... [14:07:27] 06Data-Engineering, 10MediaWiki-Page-history, 10Temporary accounts, 06Trust and Safety Product Team: mediawiki_history - account for temp accounts in mediawiki_user_history_check_error - https://phabricator.wikimedia.org/T401325 (10Ottomata) 03NEW [14:47:48] 10Analytics-Canonical-Data: Request for a new request dataset for caching research - https://phabricator.wikimedia.org/T401331 (10yazhuoz) 03NEW [15:10:05] (03CR) 10Aleksandar Mastilovic: "I think I need another +1 for Code-Review and +2 for "Verified"" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [15:18:28] (03CR) 10Joal: [C:03+1] "LGTM! Thank you :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [15:19:07] (03CR) 10Aleksandar Mastilovic: [V:03+2 C:03+2] Bring back the deprecated cuc_ip field in cu_changes table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1175918 (https://phabricator.wikimedia.org/T399958) (owner: 10Aleksandar Mastilovic) [15:19:38] (03CR) 10Aleksandar Mastilovic: [V:03+2 C:03+2] Update Sqoop's schema for the categorylinks table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1174563 (https://phabricator.wikimedia.org/T397923) (owner: 10Aleksandar Mastilovic) [15:55:40] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform: Flink images should be build on top of openjdk - https://phabricator.wikimedia.org/T400296#11065871 (10bking) Also note that [[ https://github.com/jattach/jattach | jattach ]] is available from the Debian repos.... [16:04:12] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Platform-SRE, 10Event-Platform: Flink images should be build on top of openjdk - https://phabricator.wikimedia.org/T400296#11065895 (10EBernhardson) I would also be interested in seeing curl included in the image. It's part of a solution to a... [16:10:38] 06Data-Engineering-Radar, 06Experimentation Lab, 10Event-Platform: More strictly validate X-Experiment-Enrollments-Header - https://phabricator.wikimedia.org/T401198#11065902 (10Ottomata) Groomed today, and we think this one could be handled by ExP. DE can help with review and deployment. TY! [16:12:12] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Project-Admins: Consider archiving `Q1 FY2025/26 July 1st - September 30th` - https://phabricator.wikimedia.org/T401018#11065906 (10Ottomata) a:03Ahoelzl [16:13:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Project-Admins: Consider archiving `Q1 FY2025/26 July 1st - September 30th` - https://phabricator.wikimedia.org/T401018#11065920 (10Ottomata) a:05Ahoelzl→03Milimetric [16:13:37] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Project-Admins: Consider archiving `Q1 FY2025/26 July 1st - September 30th` - https://phabricator.wikimedia.org/T401018#11065921 (10Ottomata) p:05Triage→03Low [16:15:58] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Implement the data layout, UI, and documentation for the XML file export - https://phabricator.wikimedia.org/T401022#11065924 (10Milimetric) [16:15:59] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Optimize metrics computation for the MW Content Pipeline - https://phabricator.wikimedia.org/T401010#11065925 (10Milimetric) [16:16:02] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): mediawiki_history - account for temp accounts in mediawiki_user_history_check_error - https://phabricator.wikimedia.org/T401325#11065926 (10Milimetric) [16:16:03] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): HaproxyKafkaDeliveryErrors AlertLintProblem - https://phabricator.wikimedia.org/T395539#11065927 (10Milimetric) [16:30:28] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Persistence: Data Persistence Design Review: Year in Review (YiR) - https://phabricator.wikimedia.org/T401260#11065986 (10Ottomata) a:03mforns [16:30:31] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Data-Persistence: Data Persistence Design Review: Year in Review (YiR) - https://phabricator.wikimedia.org/T401260#11065988 (10Ottomata) [16:50:09] 14Analytics-Radar, 10observability, 10Vector 2022, 10Wikimedia-Logstash, 07Epic: Client side error logging production launch - https://phabricator.wikimedia.org/T226986#11066040 (10dr0ptp4kt) @Krinkle nowadays do the items in T226986#6467370 still apply? We're considering implementing a new URL path... [17:21:07] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Movement-Insights: mediawiki_history - account for temp accounts in mediawiki_user_history_check_error - https://phabricator.wikimedia.org/T401325#11066125 (10Mayakp.wiki) [17:25:36] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 4 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#11066154 (10Ottomata) I somehow missed the big about producing to kafka main from Data... [17:26:06] (03PS1) 10Btullis: Remove obsolete scap target [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1176289 (https://phabricator.wikimedia.org/T390941) [17:26:59] 06Data-Engineering, 06Data-Engineering-Radar, 10CirrusSearch, 10Structured Data Engineering, and 4 others: Migrate image recommendation to use page_weighted_tags_changed stream - https://phabricator.wikimedia.org/T372912#11066160 (10Ottomata) Ah, I see your latest comment: T372912#10857120 talks about thro... [17:27:01] !log Deploy refinery with Aleksander for sqoop updates [17:27:05] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:06:43] (03PS1) 10CDanis: druid: webrequest_sampled_live: remove client_port [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1176294 (https://phabricator.wikimedia.org/T398236) [18:09:02] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 13Patch-For-Review: Manage druid `webrequest_sampled_live` data size - https://phabricator.wikimedia.org/T398236#11066310 (10CDanis) @JAllemandou uploaded some patches, please take a look :) [18:16:42] (03PS1) 10CDanis: Add wmfuniq X-Analytics sub-field [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1176298 (https://phabricator.wikimedia.org/T400753) [18:55:50] cdanis: I noticed you 2 PR for druid ingestion - I'll merge them tomorrow [18:56:17] thanks joal <3 should I have benthos stop generating the field I'm deleting before or after you do that? [19:01:49] (no rush, we can figure that out tomorrow too) [21:17:57] (03CR) 10Aleksandar Mastilovic: [C:03+2] "LGTM! Thank you for a quick turnaround." [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1176289 (https://phabricator.wikimedia.org/T390941) (owner: 10Btullis) [22:07:10] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th): Enable forced cache warmup option for airflow-dags blunderbuss integration - https://phabricator.wikimedia.org/T400411#11066826 (10amastilovic) 05Open→03Resolved [22:08:55] 10Data-Engineering (Q1 FY25/26 July 1st - September 30th), 06Movement-Insights: mediawiki_history - account for temp accounts in mediawiki_user_history_check_error - https://phabricator.wikimedia.org/T401325#11066828 (10amastilovic) >>! In T401325#11065897, @Ottomata wrote: > Hi @amastilovic, could you look in...