[04:03:45] (03CR) 10Ottomata: Add HQL for edit_per_editor_per_page_daily and pageview_per_editor_per_page_daily (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1196892 (https://phabricator.wikimedia.org/T407559) (owner: 10Ottomata) [04:37:23] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11338627 (10Ottomata) > But it's not about p... [06:14:08] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11338710 (10Marostegui) [07:21:18] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11338743 (10Marostegui) [07:39:35] 06Data-Engineering, 10CampaignEvents, 06DBA, 06Connection-Team (Connection-Current-Sprint), 07Schema-change-in-production: Apply ce_address cleanup schema changes in production (x1) - https://phabricator.wikimedia.org/T409101#11338782 (10Marostegui) @Daimona thanks - let us know when this can proceed [07:50:38] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11338800 (10Marostegui) @Ladsgroup can you double che... [08:29:27] 06Data-Engineering, 06Data-Engineering-Radar, 06Discovery-Search, 06Infrastructure-Foundations, and 3 others: Elasticsearch dependency upgrade in spicerack - https://phabricator.wikimedia.org/T390860#11338853 (10elukey) The spicerack release is currently blocked by the release of python3-conftool, hopefull... [08:35:29] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Add dpogorzelski to ML and Data Platform posix groups - https://phabricator.wikimedia.org/T408579#11338856 (10elukey) I got in touch with @calbon on slack and got the approval from him, I think he didn't have time to follow up so I am go... [08:40:11] 06Data-Engineering, 06SRE, 10SRE-Access-Requests, 13Patch-For-Review: Add dpogorzelski to ML and Data Platform posix groups - https://phabricator.wikimedia.org/T408579#11338867 (10elukey) ` elukey@krb1002:~$ sudo manage_principals.py get dpogorzelski get_principal: Principal does not exist while retrieving... [08:49:06] 06Data-Engineering, 06Data-Engineering-Radar, 06Experimentation Lab, 10MediaWiki-extensions-EventLogging, and 2 others: Deprecate and remove MetricsClient#dispatch() - https://phabricator.wikimedia.org/T352969#11338926 (10Sfaci) p:05High→03Triage [08:55:11] 06Data-Engineering, 06Experimentation Lab, 10Experimentation Lab Roadmap, 10MediaWiki-extensions-EventLogging, 07Epic: [EPIC] Deprecate mw.eventLog.logEvent() - https://phabricator.wikimedia.org/T317874#11338934 (10Sfaci) p:05High→03Triage [08:55:13] 06Data-Engineering, 06Experimentation Lab, 10Experimentation Lab Roadmap, 06Growth-Team, and 4 others: [EPIC] Deprecate EventLogging::logEvent() - https://phabricator.wikimedia.org/T318263#11338932 (10Sfaci) p:05High→03Triage [08:55:16] 06Data-Engineering, 06Experimentation Lab, 10Experimentation Lab Roadmap, 10MediaWiki-extensions-EventLogging, 07Epic: [EPIC] Deprecate EventLogging::schemaValidate() - https://phabricator.wikimedia.org/T317793#11338936 (10Sfaci) p:05High→03Triage [09:59:52] (03PS1) 10Btullis: Update scap targets for hdfs-tools [analytics/hdfs-tools/deploy] - 10https://gerrit.wikimedia.org/r/1201559 (https://phabricator.wikimedia.org/T353786) [10:00:18] (03CR) 10Btullis: [V:03+2 C:03+2] Update scap targets for hdfs-tools [analytics/hdfs-tools/deploy] - 10https://gerrit.wikimedia.org/r/1201559 (https://phabricator.wikimedia.org/T353786) (owner: 10Btullis) [10:02:26] !log deployed hdfs-tools with scap after updating targets [10:02:27] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:05:14] FYI, I'm roll-restarting kafka-test to smoke-test the latest Java 8 forward port of the u472 security release [10:44:26] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 06Data-Persistence, 10Data-Persistence-Design-Review, 06Growth-Team, and 3 others: Data Persistence Design Review: Improve Tone Suggested Edits newcomer task - https://phabricator.wikimedia.org/T401021#11339203 (10Michael) >>! In T401021#11334869... [11:14:01] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11339286 (10Marostegui) [11:16:35] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339301 (10Dreamy_Jazz) [11:33:36] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339358 (10Dreamy_Jazz) [11:37:50] (03PS1) 10Btullis: Replace an-launcher1002 with an-launcher1003 [analytics/refinery/scap] - 10https://gerrit.wikimedia.org/r/1201581 (https://phabricator.wikimedia.org/T353786) [11:38:45] (03PS11) 10Joal: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) [11:41:47] (03CR) 10CI reject: [V:04-1] Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) (owner: 10Joal) [11:42:09] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339392 (10Marostegui) p:05Triage→03Medium @Ladsgroup do you want me to take over? [11:45:44] (03PS12) 10Joal: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) [12:01:48] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339479 (10Dreamy_Jazz) [12:02:04] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339481 (10Dreamy_Jazz) [12:02:17] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339482 (10Dreamy_Jazz) [12:13:29] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339527 (10Dreamy_Jazz) [12:14:13] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339530 (10Dreamy_Jazz) [12:14:33] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339532 (10Dreamy_Jazz) [12:16:01] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339541 (10Dreamy_Jazz) [12:18:34] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339563 (10Dreamy_Jazz) [12:18:55] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339569 (10Dreamy_Jazz) [12:23:47] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339588 (10Dreamy_Jazz) [12:24:25] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339595 (10Dreamy_Jazz) [12:25:54] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339611 (10Ladsgroup) It just need primary switchovers, if you want to do it, I'd be grateful. [12:28:50] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339625 (10Marostegui) >>! In T403362#11339611, @Ladsgroup wrote: > It just need primary switchovers, if you want to do it,... [12:29:41] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339651 (10Marostegui) [12:29:49] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339655 (10Dreamy_Jazz) [12:29:56] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339657 (10Marostegui) [12:35:31] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339673 (10Ladsgroup) Thanks <3 [12:35:53] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339674 (10Dreamy_Jazz) [12:37:04] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339688 (10Dreamy_Jazz) [12:38:06] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Add dpogorzelski to ML and Data Platform posix groups - https://phabricator.wikimedia.org/T408579#11339690 (10calbon) Approved [12:39:10] (03CR) 10Snwachukwu: [C:03+2] Add Data quality check for Pageview Human-Bot ratio anomaly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1199485 (https://phabricator.wikimedia.org/T407239) (owner: 10Snwachukwu) [12:39:34] (03CR) 10Snwachukwu: [V:03+2 C:03+2] Add Data quality check for Pageview Human-Bot ratio anomaly [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1199485 (https://phabricator.wikimedia.org/T407239) (owner: 10Snwachukwu) [12:51:09] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11339759 (10Marostegui) @Ladsgroup eqiad master has been switched - I am repooling it but if you want me to depool it and you... [13:22:16] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339938 (10Dreamy_Jazz) [13:23:01] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339941 (10Dreamy_Jazz) >>! In T409093#11337025, @Dr... [13:23:48] 06Data-Engineering, 10CheckUser-SuggestedInvestigations, 10ConfirmEdit (CAPTCHA extension), 06Data-Persistence, and 3 others: Add columns to store associated log ID or revision ID that caused a signal to match - https://phabricator.wikimedia.org/T409093#11339943 (10Dreamy_Jazz) (This task is now awaiting r... [13:28:35] (03PS13) 10Joal: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) [13:47:01] RECOVERY - Check unit status of refinery-download-project-namespace-map on an-launcher1003 is OK: OK: Status of the systemd unit refinery-download-project-namespace-map https://wikitech.wikimedia.org/wiki/Monitoring/systemd_unit_state [14:01:26] 06Data-Engineering: Fix iceberg table location in hive metastore - https://phabricator.wikimedia.org/T408939#11340151 (10JAllemandou) My understanding is that `DELETE` always removes data whether on managed or `EXTERNAL` tables, while `DROP TABLE` deletes data only on managed tables, not on `EXTERNAL` ones. [14:16:14] (03PS14) 10Joal: Update referer classification patterns [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1198313 (https://phabricator.wikimedia.org/T406531) [14:23:54] (03CR) 10Joal: [V:03+2 C:03+2] "Merging and applying to production" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1200087 (owner: 10CDanis) [14:34:02] cdanis: Heya - the patch has been applied to Druid --^ [14:34:08] thanks!! [15:03:40] 06Data-Engineering, 06Data-Engineering-Radar, 06DBA, 13Patch-For-Review, 07Schema-change-in-production: Change row format of cx_corpora - https://phabricator.wikimedia.org/T403362#11340337 (10Marostegui) It is fully repooled now [15:06:30] 06Data-Engineering (Q2 FY25/26 October 1st - December 31th), 10AbuseFilter, 06DBA, 07Schema-change-in-production: Drop the afl_ip column and the afl_ip_timestamp index from the abuse_filter_log table - https://phabricator.wikimedia.org/T407997#11340350 (10Marostegui)