[01:26:06] 06Data-Engineering, 10Data Pipelines, 10Data-Platform-SRE (2024.05.27 - 2024.06.16), 13Patch-For-Review: Upgrade Airflow to 2.9.1 - https://phabricator.wikimedia.org/T365449#9896574 (10Stevemunene) >>! In T365449#9888004, @Stevemunene wrote: > Manual changes to packages are the mentioned [[ https://wikitec... [04:53:32] 10Quarry: [bug] Quarry queries not completing - https://phabricator.wikimedia.org/T367464#9896698 (10Liz) Well, maybe it depends on the database, queries aren't running on enwiki_p. But I've noticed that editors have just given up on running Quarry queries since about 2 days ago when it stopped working [08:20:15] naive q: should Quarry-related bugs/activity appear here/ [08:20:17] ? [08:25:27] I'm going to rolling-restart kafka-test-eqiad to upgrade kernels [09:03:51] (03CR) 10Joal: [C:03+2] "LGTM - Thanks gehel" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1043781 (https://phabricator.wikimedia.org/T365197) (owner: 10Gehel) [09:21:59] (03Merged) 10jenkins-bot: test: Add unit tests for Refinery*DatabaseResponse [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1043781 (https://phabricator.wikimedia.org/T365197) (owner: 10Gehel) [09:38:24] brouberol: I'm not sure why quarry notifies here either. [09:41:18] Seems to have done so since 2017: https://gerrit.wikimedia.org/r/plugins/gitiles/labs/tools/wikibugs2/+/ca957aaf87fe64aa3a06e0d885c3fdd255c52475/channels.yaml#197 [09:42:10] I think we could probably remove it. Quarry is more within the WMCS realm, and they're working to replace it with superset.wmcloud.org anyway. [09:49:47] I can make a change but I'm not sure how to redeploy it though [09:51:56] I've never done it either. In fact, I 've never deployed any toolforge job. :-) The instructions are here: https://www.mediawiki.org/wiki/Wikibugs#Deploying_changes - I recognise [09:53:35] It says that if it is just the channel list it should get deployed automatically but, if I remember correctly, the last time I did it, the change didn't get picked up automatically. I think that a.mir1 investigated why and redeployed, but it's probably something that we ought to know how to do as well. [09:57:07] let's try to make the change and see whether that's the case? [09:59:39] Ack. Good thinking. [10:00:31] https://gitlab.wikimedia.org/toolforge-repos/wikibugs2/-/merge_requests/46 [10:01:17] Unrelated. I'm going to perform a slow rolling-reboot of kafka-jumbo-eqiad [10:03:54] Nice, thanks. 👍 [10:22:00] merged. The CD _seems_ to be tasked to deploy [10:36:09] the change *should* be deployed and applied [12:35:52] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-CentralAuth, and 3 others: Apply schema change to add type column on GlobalRenameQueue table to the live databases - https://phabricator.wikimedia.org/T367495#9898277 (10SimoneThisDot) [12:37:31] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-CentralAuth, and 3 others: Apply schema change to add type column on GlobalRenameQueue table to the live databases - https://phabricator.wikimedia.org/T367495#9898291 (10SimoneThisDot) [12:55:19] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-CentralAuth, and 3 others: Apply schema change to add type column on GlobalRenameQueue table to the live databases - https://phabricator.wikimedia.org/T367495#9898428 (10ABran-WMF) a:03Ladsgroup [13:00:44] !log Deploy analytics-airflow for webrequest jar version bump [13:00:46] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:01:44] 10Quarry: Deduplicate config load - https://phabricator.wikimedia.org/T349135#9898459 (10github-toolforge-bot) siddharthvp closed https://github.com/toolforge/quarry/pull/55 [13:11:05] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-CentralAuth, and 3 others: Apply schema change to add type column on GlobalRenameQueue table to the live databases - https://phabricator.wikimedia.org/T367495#9898495 (10Ladsgroup) I need to figure out where is this table exactly and or the... [13:20:25] 06Data-Engineering, 06Data Products, 06DBA, 10MediaWiki-extensions-CentralAuth, and 3 others: Apply schema change to add type column on GlobalRenameQueue table to the live databases - https://phabricator.wikimedia.org/T367495#9898560 (10Ladsgroup) 05Open→03Resolved I only found it on `centralauth`.... [13:29:09] 06Data-Engineering, 10Observability-Logging, 06Traffic: Upgrade ulsfo hosts to haproxy 2.8.10 - https://phabricator.wikimedia.org/T367756 (10Fabfur) 03NEW [13:47:37] gehel: Webrequest with new geolocation map-field succeeded - looks like we've won this NPE war. I'll be preping for the one, waiting for your call! :) [13:48:02] (03CR) 10Ottomata: Refactor Refine to be triggerd by Airflow (0317 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1016808 (https://phabricator.wikimedia.org/T356762) (owner: 10Aqu) [13:48:27] joal: \o/ [13:53:52] hm, actually gehel, while triple checking, I realized my change had not yet been applied :S I'll have the answer in about 1h, hopping that I have not completely jinxed it [13:56:37] !log deleting old canary event job now that the new one has a new name [13:56:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [15:07:15] 06Data-Engineering, 10Data Pipelines, 10Data-Platform-SRE (2024.06.17 - 2024.07.07), 13Patch-For-Review: Upgrade Airflow to 2.9.1 - https://phabricator.wikimedia.org/T365449#9899230 (10Gehel) [15:09:06] 06Data-Engineering, 10Cassandra, 10Data Pipelines, 10Data-Platform-SRE (2024.06.17 - 2024.07.07): Create puppet defined type for adding/updating/deleting secrets or other small files on HDFS - https://phabricator.wikimedia.org/T323692#9899273 (10Gehel) [15:09:18] 06Data-Engineering, 10Temporary accounts, 10Data-Platform-SRE (2024.06.17 - 2024.07.07): Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#9899276 (10Gehel) [15:09:24] 14Data-Engineering-Kanban, 10Cassandra, 10Data-Platform-SRE (2024.06.17 - 2024.07.07): Properly add aqsloader user (w/ secrets) - https://phabricator.wikimedia.org/T305600#9899277 (10Gehel) [15:29:22] 06Data-Engineering, 06Data-Platform-SRE, 06Discovery-Search, 06Java-Scala-Standardization: Migrate Python projects that depend on Archiva for deployment - https://phabricator.wikimedia.org/T367406#9899431 (10bking) [15:42:36] 06Data-Engineering, 10probenet: Include geocoded subdivision ISO code in webrequest table - https://phabricator.wikimedia.org/T365594#9899491 (10JAllemandou) Here we are! we have a new `subdivision_code` map key in our `wmf.webrequest` table in `geocoded_data` field :) [15:43:38] Hi _joe_ - would you have any info for me on T355120 please? [15:44:12] <_joe_> uh let me see what that task is :) [15:45:12] <_joe_> joal: are you sure that's the right task number? [15:45:29] <_joe_> ah yea sorry I flipped two numbers [15:45:57] <_joe_> joal: and apologies, I was on sabbatical, so I missed your pings :) [15:46:34] _joe_: no worries, I had guessed something like that. I saw people greeting you back lately and thought I'd ask again :) [15:47:41] <_joe_> and yeah, that was early enough in my sabbatical I didn't see the notification email in the ocean I had when I came back [15:48:08] I guess the notion of 'ocean' reflects well the reality here :) [15:49:07] <_joe_> I think I had 1400 unread email threads from phab [15:49:15] <_joe_> something like that [15:49:38] O_o [15:49:56] <_joe_> I get subscribed to a lot of tasks [15:51:04] _joe_: I can imagine [16:10:49] 06Data-Engineering, 10Cassandra: Create keyspace and table for Knowledge Gaps - https://phabricator.wikimedia.org/T340494#9899670 (10fkaelin) Thanks. Is this now using AQS 2? It has been a moment, can you point to a current/good example job that writes to a AQS cassandra dataset from airflow? [16:11:21] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Delete deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781 (10matej_suchanek) 03NEW [16:11:21] (03PS4) 10Ecarg: Create HQL for new join table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) [16:11:43] (03CR) 10Ecarg: Create HQL for new join table (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) (owner: 10Ecarg) [16:28:19] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#9899772 (10Ladsgroup) [16:28:50] (03CR) 10David Martin: [C:03+1] Create HQL for new join table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1043205 (https://phabricator.wikimedia.org/T363436) (owner: 10Ecarg) [16:43:46] 07Analytics-Data-Problem, 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 15), 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9899857 (10mforns) @Krinkle I think... [16:53:53] 07Analytics-Data-Problem, 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 15), 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9899870 (10Milimetric) Made up some s... [17:49:59] 10Data-Engineering (Q4 2024 April 1st - June 30th): [Datasets Config][Spike] Understand and document the details and conflicts between Datasets Config, Refine refactor, Dynamic EventStreamConfig, and Metrics Platform Instrumentation Configurator - https://phabricator.wikimedia.org/T361853#9900250 (10Ottomata) [19:58:23] 10Quarry: [bug] Quarry queries not completing - https://phabricator.wikimedia.org/T367464#9900722 (10Teslaton) If you look through //Execution time// column on// Recent queries// list, it actually seems like that results of virtually //any// query with execution time longer that ~120s will never make it back, cu... [20:13:44] 07Analytics-Data-Problem, 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 15), 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9900888 (10Krinkle) >>! In T342267#98... [20:26:14] 06Data-Engineering, 06Tech-Docs-Team, 05Goal: Redesign Data Platform docs on Wikitech - https://phabricator.wikimedia.org/T350911#9900961 (10TBurmeister) Epic status update: - I moved 456 pages from /Analytics and /Data_Engineering to their new homes as outlined above in https://phabricator.wikimedia.org/T35... [23:42:50] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9901731 (10apaskulin) [23:45:00] 14Analytics, 10AQS2.0, 06Tech-Docs-Team, 10Data Products (Epics Timeline), and 2 others: AQS 2.0 user documentation - https://phabricator.wikimedia.org/T288664#9901741 (10apaskulin)