[00:45:47] 06Data-Engineering, 10Projects-Cleanup, 10Wikimedia-GitHub: Clean up wikimetrics - https://phabricator.wikimedia.org/T318193#9873995 (10Pppery) →14Duplicate dup:03T219334 [00:45:51] 14Analytics-Radar, 14Analytics-Wikimetrics, 06Data-Engineering-Icebox, 06Diffusion-Repository-Administrators, and 2 others: Archive analytics-wikimetrics (deprecated by Event Metrics) - https://phabricator.wikimedia.org/T219334#9873997 (10Pppery) [00:46:00] 06Data-Engineering, 10Projects-Cleanup, 10Wikimedia-GitHub: Clean up wikimetrics - https://phabricator.wikimedia.org/T318193#9873999 (10Pppery) When a project is so abandoned people try to archive it twice ... [07:42:56] 10Data-Engineering-Operations, 10Cloud-VPS, 06DC-Ops, 07Documentation: Request: documentation of Wikimedia Analytics Cluster and Cloud VPS environmental costs - https://phabricator.wikimedia.org/T367027 (10awight) 03NEW [07:43:35] 10Data-Engineering-Operations, 10Cloud-VPS, 06DC-Ops, 07Documentation: Request: documentation of Wikimedia Analytics Cluster and Cloud VPS environmental costs - https://phabricator.wikimedia.org/T367027#9874381 (10awight) [08:14:13] 10Data-Engineering-Operations, 10Cloud-VPS, 06DC-Ops, 07Documentation: Request: documentation of Wikimedia Analytics Cluster and Cloud VPS environmental costs - https://phabricator.wikimedia.org/T367027#9874410 (10Novem_Linguae) [08:34:33] FYI, dse-k8s-etcd1003 will go down to a few minutes due to a reboot of the underlying Ganeti node [08:37:37] 10Data-Engineering-Operations, 10Cloud-VPS, 06DC-Ops, 07Documentation: Request: documentation of Wikimedia Analytics Cluster and Cloud VPS environmental costs - https://phabricator.wikimedia.org/T367027#9874455 (10awight) [09:53:56] joal: would you have a bit of time for me today? [09:59:14] Hi gehel! I'll have time for you today, yes! sorry for last friday [10:01:59] No emergency, but I want to make sure I catch you before you disappear! [10:02:06] What time works for you? [10:03:35] would 2pm work for you gehel? [10:04:03] Works for me [10:04:20] sending an invite [10:17:16] FYI, dse-k8s-etcd1002 will go down to a few minutes due to a reboot of the underlying Ganeti node [10:18:59] ack thanks [10:44:37] gehel: actually, would 2:30 work for you? it's easier for me if possible [10:45:09] Works for me too [10:45:35] ack, updating invite [10:46:09] Thank you! [12:29:51] (03PS1) 10Gehel: fix(maxmind): fix NPE in creation of RefineryISPDatabaseResponse [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1041087 (https://phabricator.wikimedia.org/T365197) [13:02:31] (03PS2) 10Gehel: fix(maxmind): fix NPE in creation of RefineryISPDatabaseResponse [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1041087 (https://phabricator.wikimedia.org/T365197) [13:02:57] joal: ^ as discussed [13:08:48] reviewing [13:28:15] (03CR) 10Joal: [C:03+2] "LGTM! Thank you @gehel for following up on this :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1041087 (https://phabricator.wikimedia.org/T365197) (owner: 10Gehel) [13:43:56] 06Data-Engineering, 10Metrics Platform Backlog, 10Data Products (Data Products Sprint 15): [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9875390 (10xcollazo) > whereas in Hive we have to explicitly tell it what the... [13:45:03] (03Merged) 10jenkins-bot: fix(maxmind): fix NPE in creation of RefineryISPDatabaseResponse [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1041087 (https://phabricator.wikimedia.org/T365197) (owner: 10Gehel) [15:03:34] 06Data-Engineering, 10Metrics Platform Backlog, 10Data Products (Data Products Sprint 15): [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9875744 (10Ottomata) > If the INSERTs would typically include data for multip... [15:18:57] 10Data-Engineering (Q4 2024 April 1st - June 30th), 06Release-Engineering-Team, 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9875801 (10Ottomata) @lbowmaker can we clarify the user story / requ... [15:27:14] 10Data-Engineering (Q4 2024 April 1st - June 30th), 06Release-Engineering-Team, 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9875834 (10lbowmaker) @Ottomata the intention was to avoid the weekly... [15:33:39] 06Data-Engineering, 10Data-Engineering-Wikistats, 06Data Products, 10MediaWiki-Special-pages: Statistics - https://phabricator.wikimedia.org/T365952#9875857 (10VirginiaPoundstone) @Filipovic_Zoran is the change you are looking for renaming "active users" to "editors"? Or have I misunderstood? [15:38:46] 06Data-Engineering, 06Data Products, 10Dumps-Generation, 10MediaWiki-Core-Snapshots, 10Temporary accounts: Provide attribute to indicate that user is temporary account in exported content - https://phabricator.wikimedia.org/T365693#9875883 (10VirginiaPoundstone) @lbowmaker this should be included in the... [15:46:52] 06Data-Engineering, 06Data Products: Add user Central ID to mediawiki_history table in Hive - https://phabricator.wikimedia.org/T365648#9875962 (10VirginiaPoundstone) CC @lbowmaker Downstream work (AQS & Dumps) could get ignored, but that would limit the value. [15:56:47] 10Data-Engineering (Q4 2024 April 1st - June 30th), 06Release-Engineering-Team, 07Spike: [Developer Experience] [SPIKE] Investigate process to automate deployment of folders and artifacts to HDFS - https://phabricator.wikimedia.org/T360968#9876013 (10Ottomata) Discussed in meeting: It is mostly up to us. S... [15:57:13] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876016 (10Ottomata) [15:58:25] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876021 (10Ottomata) ^ patch to do this once approved. [16:00:21] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876024 (10Ottomata) @thcipriani for group approver [16:31:35] 06Data-Engineering, 10Metrics Platform Backlog, 10Data Products (Data Products Sprint 15): [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9876215 (10xcollazo) >>! In T366627#9875744, @Ottomata wrote: >> If the INSER... [16:46:26] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876336 (10Ahoelzl) Approved. [17:01:36] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876424 (10Ottomata) [17:02:42] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876421 (10ttaylor) Approving in @thcipriani 's place since he is on vacation. [17:10:31] 06Data-Engineering, 10Metrics Platform Backlog, 10Data Products (Data Products Sprint 15): [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#9876441 (10Ottomata) > How will these dashboards be served? Via Presto? Not... [17:25:23] 06Data-Engineering, 10Data Products (Data Products Sprint 14), 10Web-Team-Backlog (FY2023-24 Q4 Sprint 5): Follow-Up Ticket for QA: Validate Sample Rate Adjustments - https://phabricator.wikimedia.org/T365489#9876565 (10SToyofuku-WMF) a:03SToyofuku-WMF Steph to sign off [17:37:14] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876692 (10amastilovic) Merged and applied - done [17:42:29] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to Kubernetes deployment for amastilovic - https://phabricator.wikimedia.org/T367073#9876716 (10Ottomata) 05Open→03Resolved a:03Ottomata [18:16:16] 06Data-Engineering, 10Dumps 2.0: [Iceberg Migration] Implement mechanism for automatic Iceberg data deletion and optimization - https://phabricator.wikimedia.org/T338065#9876879 (10WDoranWMF) [18:17:14] 06Data-Engineering, 10Dumps 2.0 (Sprint 1): [Iceberg Migration] Implement mechanism for automatic Iceberg data deletion and optimization - https://phabricator.wikimedia.org/T338065#9876881 (10lbowmaker) [18:20:02] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Dumps 2.0, 07Spike: [Status Store] [SPIKE] Investigate and document approach for Iceberg Sensors - https://phabricator.wikimedia.org/T360922#9876900 (10lbowmaker) [18:20:21] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Dumps 2.0 (Sprint 1), 07Spike: [Status Store] [SPIKE] Investigate and document approach for Iceberg Sensors - https://phabricator.wikimedia.org/T360922#9876902 (10lbowmaker) [18:21:06] 06Data-Engineering, 10Dumps 2.0, 10Event-Platform: [Data Quality] [SPIKE] Can we identify indicators to inform an SLO for event emission and intake? - https://phabricator.wikimedia.org/T345195#9876906 (10lbowmaker) [18:22:03] 10Data-Engineering (Q4 2024 April 1st - June 30th), 10Dumps 2.0: [Data Quality] Update data_quality schemas to be compatible with Iceberg tables - https://phabricator.wikimedia.org/T356866#9876928 (10lbowmaker) [19:09:51] 06Data-Engineering, 10Data-Platform-SRE (2024.05.27 - 2024.06.16), 13Patch-For-Review: ISPDatabaseReader null pointer exception - https://phabricator.wikimedia.org/T365197#9877086 (10Gehel) 05Open→03Resolved [19:35:28] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting permissions for analytics-privatedata-users (with kerberos) for Mareike Heuer - https://phabricator.wikimedia.org/T364715#9877189 (10herron) 05In progress→03Resolved Resolving as the access looks to have been provisioned, please reopen if a... [20:10:27] 06Data-Engineering, 06Data-Platform-SRE, 10Event-Platform: mw-page-content-change-enrich flink app is missing in k8s staging - https://phabricator.wikimedia.org/T367116 (10Ottomata) 03NEW [20:11:42] 06Data-Engineering, 06Data-Platform-SRE, 10Event-Platform: mw-page-content-change-enrich flink app is missing in k8s staging - https://phabricator.wikimedia.org/T367116#9877320 (10Ottomata) [20:46:37] 06Data-Engineering, 06Tech-Docs-Team, 05Goal: Redesign Data Platform docs on Wikitech - https://phabricator.wikimedia.org/T350911#9877442 (10TBurmeister) As part of replacing the outdated Data Engineering docs on Wikitech, I have made the following revisions after consultation with the DPE team/stakeholders:... [21:24:11] 06Data-Engineering, 06Data-Platform-SRE, 10Event-Platform: mw-page-content-change-enrich flink app is missing in k8s staging - https://phabricator.wikimedia.org/T367116#9877581 (10amastilovic) a:03gmodena [21:48:36] Starting build #1 for job wikimedia-event-utilities-maven-release [21:53:34] Project wikimedia-event-utilities-maven-release build #1: 09SUCCESS in 4 min 58 sec: https://integration.wikimedia.org/ci/job/wikimedia-event-utilities-maven-release/1/ [23:45:34] 10Data-Engineering (Q4 2024 April 1st - June 30th): [Refine Refactoring] Integrate Refine workflow configuration into ESC - https://phabricator.wikimedia.org/T367134 (10Ahoelzl) 03NEW