[06:43:23] 06Data-Engineering, 13Patch-For-Review: ISPDatabaseReader null pointer exception - https://phabricator.wikimedia.org/T365197#9807727 (10Gehel) This is related to a recent refactoring that missed a null pointer check (see attached patch). I'm not sure what the impact has been on refine jobs. @JAllemandou : coul... [06:55:03] (03PS4) 10Gehel: fix(ISPDatabaseReader): avoid null pointer exception when reading MaxMind [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032562 [07:13:12] (03PS5) 10Gehel: fix(ISPDatabaseReader): avoid null pointer exception when reading MaxMind [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032562 (https://phabricator.wikimedia.org/T365197) [07:44:46] (03PS6) 10Gehel: fix(ISPDatabaseReader): avoid null pointer exception when reading MaxMind [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032562 (https://phabricator.wikimedia.org/T365197) [07:48:40] (03PS7) 10Gehel: fix(ISPDatabaseReader): avoid null pointer exception when reading MaxMind [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032562 (https://phabricator.wikimedia.org/T365197) [08:12:36] 10Data-Engineering (Q4 2024 April 1st - June 30th), 13Patch-For-Review: [Refine refactoring] Extract refine schema management into a dedicated tool - https://phabricator.wikimedia.org/T356762#9807857 (10Antoine_Quhen) [09:01:30] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data-Platform-SRE (2024.05.06 - 2024.05.26): Migrate Dumps Snapshot hosts from Buster to Bullseye - https://phabricator.wikimedia.org/T325228#9807924 (10ops-monitoring-bot) Host rebooted by btullis@cumin1002 with reason: Rebooting to pick up new kernel [09:09:05] (03PS1) 10Gerrit maintenance bot: Add dtp.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1032729 (https://phabricator.wikimedia.org/T365230) [09:17:38] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data-Platform-SRE (2024.05.06 - 2024.05.26): Migrate Dumps Snapshot hosts from Buster to Bullseye - https://phabricator.wikimedia.org/T325228#9808043 (10ops-monitoring-bot) Host rebooted by btullis@cumin1002 with reason: Rebooting to pick up new kernel [09:35:11] 14Data-Engineering-Kanban, 10Cassandra, 06Data-Platform-SRE: Properly add aqsloader user (w/ secrets) - https://phabricator.wikimedia.org/T305600#9808070 (10BTullis) [09:36:08] 06Data-Engineering, 10EventStreams, 10Event-Platform: Implement server side filtering for EventStreams (if we should) - https://phabricator.wikimedia.org/T152731#9808071 (10BTullis) [09:47:32] 10Data-Engineering (Q4 2024 April 1st - June 30th), 13Patch-For-Review: [Refine refactoring] Extract refine schema management into a dedicated tool - https://phabricator.wikimedia.org/T356762#9808079 (10BTullis) >>! In T356762#9789320, @Antoine_Quhen wrote: > I've encountered an issue with our current producti... [11:13:30] 06Data-Engineering, 10Data-Engineering-Wikistats, 06Data Products: Add azerbaijani language to Wikistats - https://phabricator.wikimedia.org/T365209#9808268 (10lbowmaker) [11:30:18] 06Data-Engineering: Add MW History data quality checks output to Superset dashboard - https://phabricator.wikimedia.org/T365238 (10lbowmaker) 03NEW [11:30:20] 06Data-Engineering: Add MW History data quality checks output to Superset dashboard - https://phabricator.wikimedia.org/T365238#9808299 (10lbowmaker) [11:31:42] 06Data-Engineering: Airflow DAG (hdfs_usage_weekly) failed with no details in the application log - https://phabricator.wikimedia.org/T364487#9808309 (10lbowmaker) 05Open→03Resolved [12:34:55] 06Data-Engineering, 10Data-Platform-SRE (2024.05.06 - 2024.05.26), 13Patch-For-Review: ISPDatabaseReader null pointer exception - https://phabricator.wikimedia.org/T365197#9808421 (10Gehel) [12:46:33] 06Data-Engineering: [Data Quality] Implement wiki completeness check for MediaWiki History - https://phabricator.wikimedia.org/T365203#9808450 (10CMyrick-WMF) An idea for an automated quality check step: Using @Hghani 's **[[ https://gitlab.wikimedia.org/repos/research/incubator-data-exploration/-/blob/main/02_... [13:12:43] 06Data-Engineering, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team-WIP, and 9 others: Enable ipv6 in mesh used in PCS k8s deployment - https://phabricator.wikimedia.org/T365250 (10Jgiannelos) 03NEW [13:30:46] 06Data-Engineering: [Data Quality] Implement wiki completeness check for MediaWiki History - https://phabricator.wikimedia.org/T365203#9808686 (10mpopov) [13:37:40] 06Data-Engineering, 10Citoid, 06Content-Transform-Team-WIP, 10CX-cxserver, and 7 others: Upgrade mobileapps to node 18 - https://phabricator.wikimedia.org/T363168#9808721 (10Lucas_Werkmeister_WMDE) [13:37:45] 06Data-Engineering, 10Citoid, 06Content-Transform-Team-WIP, 10CX-cxserver, and 7 others: Enable ipv6 in mesh used in PCS k8s deployment - https://phabricator.wikimedia.org/T365250#9808722 (10Lucas_Werkmeister_WMDE) [13:41:16] (03PS2) 10Mforns: Improve Commons Impact Metrics base pipeline and dumps [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) [13:47:15] (03CR) 10Mforns: "Sorry for commons_pageviews_per_media_file_monthly.hql. I don't know why git didn't catch the changes... I did a git mv! But somehow, it d" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [13:54:37] 06Data-Engineering: [Data Quality] Implement wiki completeness check for MediaWiki History - https://phabricator.wikimedia.org/T365203#9808802 (10CMyrick-WMF) [14:35:53] (03CR) 10Xcollazo: [C:03+2] "> I don't know why git didn't catch the changes... I did a git mv!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [15:11:39] 10Quarry, 10Data-Services, 10cloud-services-team (FY2023/2024-Q3-Q4): Create db user for Quarry with readonly access to public ToolsDB databases - https://phabricator.wikimedia.org/T348407#9809037 (10fnegri) 05Open→03In progress [15:34:13] (03CR) 10Milimetric: [C:03+2] "no opinion on renaming snapshot, all the code looks really great. I'm amazed at how much simpler some of the queries became. Great work." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [15:36:28] (03PS3) 10Mforns: Improve Commons Impact Metrics base pipeline and dumps [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) [15:39:18] (03CR) 10Mforns: "AAaaahh... thanks!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [16:01:15] (03PS4) 10Mforns: Improve Commons Impact Metrics base pipeline and dumps [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) [16:07:39] (03CR) 10Mforns: Improve Commons Impact Metrics base pipeline and dumps (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [16:22:23] 10Quarry, 10Data-Services, 10cloud-services-team (FY2023/2024-Q3-Q4): Create db user for Quarry with readonly access to public ToolsDB databases - https://phabricator.wikimedia.org/T348407#9809374 (10fnegri) I finally started working on this. As a first proof of concept, I tried creating a new user on ToolsD... [16:44:51] (03PS5) 10Mforns: Improve Commons Impact Metrics base pipeline and dumps [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) [18:28:33] (03CR) 10Xcollazo: [C:03+2] Improve Commons Impact Metrics base pipeline and dumps [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1027525 (https://phabricator.wikimedia.org/T362892) (owner: 10Mforns) [18:36:44] 06Data-Engineering, 06Movement-Insights: Add MW History data quality checks output to Superset dashboard - https://phabricator.wikimedia.org/T365238#9809805 (10Mayakp.wiki) [18:38:24] (03PS1) 10CDanis: Include subdivision ISO code in the geo response [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032847 [18:44:02] (03CR) 10CI reject: [V:04-1] Include subdivision ISO code in the geo response [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032847 (owner: 10CDanis) [18:45:23] (03PS2) 10CDanis: Include subdivision ISO code in the geo response [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1032847 [18:58:31] 06Data-Engineering, 10Data-Engineering-Wikistats: Add azerbaijani language to Wikistats - https://phabricator.wikimedia.org/T365209#9809870 (10VirginiaPoundstone) [18:58:36] 06Data-Engineering, 10Data-Engineering-Wikistats: Add azerbaijani language to Wikistats - https://phabricator.wikimedia.org/T365209#9809866 (10VirginiaPoundstone) Thank you for this request @NMW03. Because of our journalistic safety-base policies, we would not be able to include active editors or top-per-count... [22:36:10] (03PS1) 10Clare Ming: Add agent device_family to app fragment [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032887 (https://phabricator.wikimedia.org/T363700) [22:37:26] (03CR) 10Clare Ming: "whoops - forgot i had to update schemas" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032887 (https://phabricator.wikimedia.org/T363700) (owner: 10Clare Ming) [22:42:47] (03PS1) 10Clare Ming: Update app base schema to include updated fragment [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032890 (https://phabricator.wikimedia.org/T363700) [22:53:05] (03PS1) 10Clare Ming: Bump app base version and article_toc_interaction version [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032891 (https://phabricator.wikimedia.org/T363700) [22:56:56] (03PS1) 10Clare Ming: Bump app base version and article_find_in_page_interaction version [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032892 (https://phabricator.wikimedia.org/T363700) [23:06:28] (03CR) 10Cooltey: [C:03+1] "Looks good to me." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032891 (https://phabricator.wikimedia.org/T363700) (owner: 10Clare Ming) [23:08:14] (03CR) 10Cooltey: [C:03+1] Bump app base version and article_find_in_page_interaction version [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/1032892 (https://phabricator.wikimedia.org/T363700) (owner: 10Clare Ming)