[10:41:11] 10Data-Engineering, 10Event-Platform Value Stream: We should provide utilities for local experimentation and unit testing of Python streaming services - https://phabricator.wikimedia.org/T324951 (10gmodena) [10:48:09] 10Data-Engineering, 10Event-Platform Value Stream (Sprint 05): We should provide utilities for local experimentation and unit testing of Python streaming services - https://phabricator.wikimedia.org/T324951 (10gmodena) [10:48:42] 10Data-Engineering, 10Event-Platform Value Stream (Sprint 05): We should provide utilities for local development and unit testing of Python streaming services - https://phabricator.wikimedia.org/T324951 (10gmodena) a:03gmodena [10:50:06] 10Data-Engineering, 10Event-Platform Value Stream: [NEEDS GROOMING] Integrate Flink Table API in eventutils-python - https://phabricator.wikimedia.org/T324953 (10gmodena) [10:50:27] 10Data-Engineering, 10Event-Platform Value Stream: [EPIC] Streaming and event driven Python services - https://phabricator.wikimedia.org/T324689 (10gmodena) [10:58:00] 10Data-Engineering, 10CheckUser, 10MW-1.38-notes (1.38.0-wmf.26; 2022-03-14), 10MW-1.39-notes (1.39.0-wmf.23; 2022-08-01), and 3 others: Update CheckUser for actor and comment table - https://phabricator.wikimedia.org/T233004 (10Dreamy_Jazz) [10:59:18] 10Data-Engineering, 10CheckUser, 10MW-1.38-notes (1.38.0-wmf.26; 2022-03-14), 10MW-1.39-notes (1.39.0-wmf.23; 2022-08-01), and 3 others: Update CheckUser for actor and comment table - https://phabricator.wikimedia.org/T233004 (10Dreamy_Jazz) [12:15:38] 10Data-Engineering-Planning, 10SRE, 10SRE-Access-Requests, 10WMF-Communications: LDAP access for Sondes to access Matomo - https://phabricator.wikimedia.org/T324696 (10LSobanski) [12:39:56] 10Data-Engineering, 10Equity-Landscape: Access input metrics - https://phabricator.wikimedia.org/T324968 (10KCVelaga_WMF) [12:40:22] 10Data-Engineering, 10Equity-Landscape: Access input metrics - https://phabricator.wikimedia.org/T324968 (10KCVelaga_WMF) @ntsako please share the query for access inputs [14:01:14] (03PS5) 10Snwachukwu: [WIP] Refactor and Expand External referer classification [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) [14:06:08] (03CR) 10CI reject: [V: 04-1] [WIP] Refactor and Expand External referer classification [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) (owner: 10Snwachukwu) [14:41:30] 10Data-Engineering, 10Event-Platform Value Stream, 10Epic: Event Driven Data Pipelines should be generated from a template - https://phabricator.wikimedia.org/T324980 (10lbowmaker) [14:53:04] 10Data-Engineering, 10Equity-Landscape: Access input metrics - https://phabricator.wikimedia.org/T324968 (10ntsako) Please run this in a Jupyter notebook ` java_function = "CREATE TEMPORARY FUNCTION array_avg AS 'org.wikimedia.analytics.refinery.hive.ArrayAvgUDF' USING JAR 'hdfs://analytics-hadoop/wmf/refine... [14:55:59] (03CR) 10Joal: [WIP] Refactor and Expand External referer classification (037 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) (owner: 10Snwachukwu) [15:03:59] (03PS1) 10Aqu: Extract to a namenode the creation of the raw FSImage [analytics/refinery] - 10https://gerrit.wikimedia.org/r/867185 [15:04:28] (03PS2) 10Aqu: Extract to a namenode the creation of the raw FSImage [analytics/refinery] - 10https://gerrit.wikimedia.org/r/867185 (https://phabricator.wikimedia.org/T324850) [15:59:23] 10Data-Engineering, 10Data Pipelines: Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10odimitrijevic) @JArguello-WMF ideally before end of year. I would like to add it to the unexpected work in the current sprint. [16:36:21] 10Data-Engineering, 10serviceops, 10Discovery-Search (Current work), 10Event-Platform Value Stream (Sprint 05), 10Patch-For-Review: Flink on Kubernetes Helm charts - https://phabricator.wikimedia.org/T324576 (10Gehel) [16:42:30] 10Data-Engineering, 10serviceops, 10Discovery-Search (Current work), 10Event-Platform Value Stream (Sprint 05), 10Patch-For-Review: Flink on Kubernetes Helm charts - https://phabricator.wikimedia.org/T324576 (10Ottomata) > Ah, but the upstream helm chart does not work with this feature because of its use... [16:42:35] (03PS3) 10Aqu: Extract to a NameNode the creation of the raw FSImage [analytics/refinery] - 10https://gerrit.wikimedia.org/r/867185 (https://phabricator.wikimedia.org/T324850) [16:47:54] 10Analytics-Wikistats, 10Data-Engineering: Stats page - https://phabricator.wikimedia.org/T324993 (10Neechalkaran) [16:57:45] 10Data-Engineering, 10Event-Platform Value Stream, 10Epic: Event Driven Data Pipelines should be generated from a template - https://phabricator.wikimedia.org/T324980 (10Ottomata) I have a preference for keeping as much as we can in the library code. Ideally the scaffolding would be just for placeholders.... [17:00:10] 10Data-Engineering, 10Event-Platform Value Stream, 10Epic: Event Driven Data Pipelines should be generated from a template - https://phabricator.wikimedia.org/T324980 (10Ottomata) [17:00:34] 10Data-Engineering, 10Event-Platform Value Stream, 10Epic: Event Driven Data Pipelines should be generated from a template - https://phabricator.wikimedia.org/T324980 (10Ottomata) [17:06:13] (03CR) 10Joal: [C: 03+1] "LGTM :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/867185 (https://phabricator.wikimedia.org/T324850) (owner: 10Aqu) [17:07:33] 10Data-Engineering-Planning: Include EU Registered Country in the maxmind country import - https://phabricator.wikimedia.org/T324995 (10odimitrijevic) [17:07:54] 10Data-Engineering-Planning, 10Data Pipelines: Include EU Registered Country in the maxmind country import - https://phabricator.wikimedia.org/T324995 (10odimitrijevic) [17:16:46] 10Data-Engineering-Planning, 10SRE, 10SRE-Access-Requests, 10WMF-Communications: LDAP access for Sondes to access Matomo - https://phabricator.wikimedia.org/T324696 (10jhathaway) @Varnent apologies for missing this last week, it wasn't on the board we usually work off of and I missed the phabricator ping.... [17:25:54] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterhub on conda-analytics - https://phabricator.wikimedia.org/T321088 (10Ottomata) [17:26:04] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterhub on conda-analytics - https://phabricator.wikimedia.org/T321088 (10Ottomata) [17:26:22] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterhub on conda-analytics - https://phabricator.wikimedia.org/T321088 (10Ottomata) [17:44:51] 10Data-Engineering: Document how to show your work in phabricator and/or elsewhere - https://phabricator.wikimedia.org/T324796 (10Ottomata) Draft here: https://wikitech.wikimedia.org/wiki/User:Ottomata/Show_your_work#Realtime_and_email [17:46:44] 10Data-Engineering, 10Data Pipelines: Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) [17:50:06] 10Data-Engineering-Planning, 10Data Pipelines: Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JAllemandou) [17:54:55] 10Data-Engineering, 10Data Pipelines: Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) [17:57:57] 10Data-Engineering-Planning, 10Data Pipelines: Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JArguello-WMF) [17:58:39] 10Data-Engineering-Planning, 10Data Pipelines: Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JArguello-WMF) [17:59:28] 10Data-Engineering, 10Data Pipelines: Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) [19:11:29] 10Data-Engineering, 10Event-Platform Value Stream, 10Epic: Event Driven Data Pipelines should be generated from a template - https://phabricator.wikimedia.org/T324980 (10lbowmaker) >>! In T324980#8461043, @Ottomata wrote: > I have a preference for keeping as much as we can in the library code. Ideally the s... [19:51:42] 10Data-Engineering, 10Data Pipelines (Sprint 05-06): Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) [19:53:56] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06): Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JArguello-WMF) [19:54:27] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06): Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JArguello-WMF) p:05Triage→03High [19:55:52] 10Data-Engineering, 10Data Pipelines (Sprint 05-06): Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) a:03Antoine_Quhen [19:56:26] 10Data-Engineering, 10Data Pipelines (Sprint 05-06): Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) p:05Triage→03High [20:08:33] 10Data-Engineering, 10Data Pipelines (Sprint 05-06): Update Automated Traffic Detection Documentation - https://phabricator.wikimedia.org/T324777 (10JArguello-WMF) [21:44:49] RECOVERY - Check systemd state on an-tool1005 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [21:53:44] 10Data-Engineering, 10serviceops, 10Discovery-Search (Current work), 10Event-Platform Value Stream (Sprint 05), 10Patch-For-Review: Flink on Kubernetes Helm charts - https://phabricator.wikimedia.org/T324576 (10Ottomata) [22:10:04] 10Data-Engineering, 10Equity-Landscape: Grants input metric - https://phabricator.wikimedia.org/T309276 (10JAnstee_WMF) Yes, signing off on inputs [23:17:46] 10Data-Engineering-Planning, 10SRE, 10SRE-Access-Requests, 10WMF-Communications: LDAP access for Sondes to access Matomo - https://phabricator.wikimedia.org/T324696 (10Varnent) @jhathaway - Apologies - have added links to that template to our documentation on Office Wiki. Will update ticket with above. I t... [23:19:19] 10Data-Engineering-Planning, 10LDAP-Access-Requests, 10SRE, 10SRE-Access-Requests, 10WMF-Communications: Grant Access to staff LDAP group for Sbenchagra - https://phabricator.wikimedia.org/T324696 (10Varnent) [23:54:43] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink + Event Platform integration for writing into streams via Table API - https://phabricator.wikimedia.org/T324114 (10tchin) Trying to create different functionalities when sourcing/sinking is like trying to fit a square peg into a roun...