[00:24:46] 07Analytics-Data-Problem, 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform, 06Movement-Insights: NEW BUG REPORT Mediawiki_history contains duplicate rows for some revisions - https://phabricator.wikimedia.org/T369851#10013401 (10Snwachukwu) I tried to rerun the job for one of the sma... [00:38:42] 07Analytics-Data-Problem, 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data-Platform, 06Movement-Insights: NEW BUG REPORT Mediawiki_history contains duplicate rows for some revisions - https://phabricator.wikimedia.org/T369851#10013449 (10Snwachukwu) So I digged further by looking at the airfl... [07:27:09] 06Data-Engineering, 10Data-Platform-SRE (2024.07.08 - 2024.07.28): Design a suitable DAG deployment method - https://phabricator.wikimedia.org/T368033#10013702 (10hashar) The discussion was on Friday July 12th and can be found in https://wm-bot.wmflabs.org/libera_logs/%23wikimedia-releng/20240712.txt The ques... [09:51:28] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06Privacy Engineering: Raw IPs of logged-out users disclosed in wiki-replicas - https://phabricator.wikimedia.org/T284948#10013983 (10kostajh) It's worth noting that even after temporary accounts are deployed to all wikis, we'll continue to have I... [12:44:39] (03PS28) 10Aqu: Refactor Refine to be triggerd by Airflow [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1016808 (https://phabricator.wikimedia.org/T356762) [12:55:02] 06Data-Engineering, 10Data-Platform-SRE (2024.07.08 - 2024.07.28): Design a suitable DAG deployment method - https://phabricator.wikimedia.org/T368033#10014480 (10Ottomata) > an authenticated human still has to run the DAG manually I don't think this is true. Maybe a new DAG needs to be manually launched (I d... [13:28:38] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform: Event Platform schemas should not support type changes to structs as array element or map value types - https://phabricator.wikimedia.org/T366487#10014625 (10tchin) a:03tchin [13:29:21] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Event-Platform: [Event Platform] - Add schema CI test that array ensures properties with object types also enumerate object properties - https://phabricator.wikimedia.org/T366562#10014638 (10tchin) a:03tchin [13:49:38] (03PS1) 10Milimetric: Add cs.wikivoyage to allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1056947 [13:50:26] (03CR) 10Milimetric: [V:03+2 C:03+2] Add cs.wikivoyage to allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1056947 (owner: 10Milimetric) [14:04:26] 06Data-Engineering, 06cloud-services-team, 10Data-Services, 06Privacy Engineering: Raw IPs of logged-out users disclosed in wiki-replicas - https://phabricator.wikimedia.org/T284948#10014828 (10sguebo_WMF) 05Stalled→03Declined [14:05:41] 06Data-Engineering, 06Movement-Insights, 10Wikidata, 10Wmfdata-Python, 10Wikidata Analytics (Kanban): Add linter and formatter to wmfdata-python (and link check) - https://phabricator.wikimedia.org/T348999#10014833 (10Manuel) p:05Triage→03Low [14:05:57] 06Data-Engineering, 10Wikidata, 10Wmfdata-Python, 10Wikidata Analytics (Kanban): Add testing framework to wmfdata-python - https://phabricator.wikimedia.org/T349531#10014834 (10Manuel) p:05Triage→03Low [14:06:52] 14Data-Engineering (Q4 2024 April 1st - June 30th): [Bug?] Can't find wikidatawiki on wmf.mediawiki_wikitext_history - https://phabricator.wikimedia.org/T364045#10014837 (10Milimetric) oof, I just realized this is for the month BEFORE. I see that's still in-process: ` "metahistorybz2dump": {"status": "in-progr... [14:08:02] 14Data-Engineering (Q4 2024 April 1st - June 30th), 10Wikidata Analytics: [Bug?] Can't find wikidatawiki on wmf.mediawiki_wikitext_history - https://phabricator.wikimedia.org/T364045#10014859 (10Manuel) [14:21:15] 06Data-Engineering, 10Temporary accounts (Blockers to pilot wiki deployment): Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#10014913 (10kostajh) >>! In T347510#9949406, @lbowmaker wrote: > @kostajh we have changes to make to a lot of... [15:32:20] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#10015322 (10Ottomata) Okay, here are some dummy results for ya. I used `event.mediawiki_client_sessi... [15:32:54] 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#10015323 (10Ottomata) a:05phuedx→03Ottomata [15:33:27] 10Data-Engineering (Q1 2024 July 1st - September 30th), 06Data Products, 10Metrics Platform Backlog: [MPIC] Analyse risk of potential performance issues with static approach to stream configuration - https://phabricator.wikimedia.org/T366627#10015331 (10Ottomata) [16:36:02] 10Data-Engineering (Q1 2024 July 1st - September 30th): Replace service runner with a simplified library to better support metrics and debugging: service-utils - https://phabricator.wikimedia.org/T360924#10015601 (10tchin) [17:21:08] 10Data-Engineering (Q1 2024 July 1st - September 30th), 10Data Pipelines, 10Data-Catalog: Spike: Integrate Spark with DataHub - https://phabricator.wikimedia.org/T306896#10015723 (10lbowmaker) a:03tchin [21:26:46] 06Data-Engineering, 10Event-Platform: Implement stream of HTML content on mw.page_change event - https://phabricator.wikimedia.org/T360794#10016391 (10Isaac) > I think we could try and get to this or the batch ingestion one next quarter (starting October). > This is something I really want to do but it’s hard... [23:27:17] 14Analytics, 06Data-Engineering-Icebox: Mediawiki-history release - Backlog - https://phabricator.wikimedia.org/T221828#10016703 (10nshahquinn-wmf) 05Open→03Declined I suspect that this tracking task is no longer useful.