[00:44:49] 10Data-Engineering (Q2 2024 October 1st - December 31th), 03Discovery-Search (Current work), 10Dumps 2.0 (Kanban Board), 13Patch-For-Review: Add support for Spark producers in Event Platform - https://phabricator.wikimedia.org/T374341#10298214 (10xcollazo) >>! In T374341#10298065, @gerritbot wrote: > Chang... [08:04:53] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10298544 (10ABran-WMF) [08:06:31] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10298550 (10ABran-WMF) db2219 has been done, description updated. db2155 has been resumed, lets see if it drains properly [08:12:48] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10298594 (10ABran-WMF) dangling connexion is this one: ` 2024-11-07 08:09:19.568761 db-mysql db2155 -N -e "SELECT * FROM... [09:11:04] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10298689 (10ABran-WMF) same thing happened: ` Depool timed out, repooling... [09:14:07] (03PS1) 10Mforns: Update MediaWiki History to support Temp Accounts [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088232 (https://phabricator.wikimedia.org/T379230) [09:20:09] (03CR) 10CI reject: [V:04-1] Update MediaWiki History to support Temp Accounts [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088232 (https://phabricator.wikimedia.org/T379230) (owner: 10Mforns) [09:27:16] (03PS2) 10Mforns: Update MediaWiki History to support Temp Accounts [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088232 (https://phabricator.wikimedia.org/T379230) [09:31:36] <_joe_> my question is probably extremely dumb, but say I want to experiment with pydruid, is there a host where it's installed? [09:32:56] (03CR) 10CI reject: [V:04-1] Update MediaWiki History to support Temp Accounts [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088232 (https://phabricator.wikimedia.org/T379230) (owner: 10Mforns) [09:35:52] _joe_: not according to debmonitor but I don't see it neither on our apt or upstream debian, so I guess a venv in one of stats hosts might be a way [09:36:59] <_joe_> volans: ack, thanks [09:50:59] (03CR) 10Joal: [C:03+1] gobblin: add webrequest_frontend pull job. (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [10:49:54] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#10298891 (10Marostegui) >>! In T367856#10288606, @Marostegui wrote: > Thank you @BTullis - I have started the schema ch... [10:58:01] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10298912 (10Marostegui) If that is the only host pending, you can always depool it manually, run it (with no bin_log=0)... [11:13:07] 06Data-Engineering, 10[DEPRECATED] wdwb-tech, 10Citoid, 06Content-Transform-Team, and 9 others: Migrate node-based services in production to node18 - https://phabricator.wikimedia.org/T349118#10298941 (10Mvolz) [11:47:43] (03CR) 10Gmodena: [C:03+2] gobblin: add webrequest_frontend pull job. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [11:53:28] 06Data-Engineering, 06Data-Platform-SRE, 10observability, 06serviceops-radar, and 3 others: Upgrade Kafka to from 1.x to later version - https://phabricator.wikimedia.org/T300102#10299074 (10jijiki) [12:28:07] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#10299200 (10Lucas_Werkmeister_WMDE) >>! In T367856#10287933, @Ladsgroup wrote: > I explicitly skipped sanitarium master... [12:33:13] (03CR) 10Joal: [V:03+2 C:03+1] gobblin: add webrequest_frontend pull job. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1082432 (https://phabricator.wikimedia.org/T377931) (owner: 10Gmodena) [12:33:35] !log Deploy refinery with scap [12:33:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:46:09] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#10299295 (10Marostegui) Thank you @Lucas_Werkmeister_WMDE - as we spoke on IRC, it looks good to me. [13:04:46] 10Quarry, 10PAWS: PR usually not posting to phabricator - https://phabricator.wikimedia.org/T373134#10299398 (10rook) 05Open→03Declined [13:37:33] !log Deploy refinery onto HDFS [13:37:35] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:02:41] !log Deploying airflow analytics dags [14:02:43] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:06:45] 14Analytics, 06Data-Engineering, 10EventStreams, 10Wikidata, and 3 others: Expose rdf-streaming-updater.mutation content through EventStreams - https://phabricator.wikimedia.org/T294133#10299656 (10Sina_az_9) Hello, I have a question maybe you can help. I am trying to simulate updates happening in Wikidat... [14:08:49] Heya btullis - would you have minute to merge/deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/1082434 [14:08:56] please? [14:11:00] joal: o/ [14:11:10] I can take care of it if Ben is not around [14:11:41] I am around. [14:11:53] ah hello! Sorry then I am driving away :) [14:12:12] it was an excuse to work with a french that I like a lot :) [14:12:21] No need to apologise. We all do. [14:13:49] elukey: <3 [14:23:35] (03CR) 10Joal: [C:03+2] Add items for event sanitization regarding iOS edit, update keep to hash. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1083192 (https://phabricator.wikimedia.org/T377259) (owner: 10GOlson) [14:23:37] (03CR) 10Joal: [V:03+2 C:03+2] Add items for event sanitization regarding iOS edit, update keep to hash. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1083192 (https://phabricator.wikimedia.org/T377259) (owner: 10GOlson) [14:30:24] 10Data-Engineering (Q2 2024 October 1st - December 31th), 13Patch-For-Review: [Refine Refactoring] Refine jobs should be scheduled by Airflow: deployment - https://phabricator.wikimedia.org/T369845#10299730 (10Antoine_Quhen) Discussing the use of convertedTo / merge schemas in refine: Old Refine Process • Th... [14:50:39] 10Data-Engineering (Q2 2024 October 1st - December 31th), 13Patch-For-Review: [Refine Refactoring] Refine jobs should be scheduled by Airflow: deployment - https://phabricator.wikimedia.org/T369845#10299849 (10Ottomata) Thanks for this write up Antoine! This will make it much easier to remember this stuff in... [14:56:23] 10Quarry: unused dns proxies? - https://phabricator.wikimedia.org/T373528#10299898 (10rook) 05Open→03Resolved a:03rook [15:10:05] 10Quarry: update build-and-push - https://phabricator.wikimedia.org/T378978#10299955 (10rook) https://github.com/toolforge/quarry/pull/71 [15:11:15] 10Quarry: update build-and-push - https://phabricator.wikimedia.org/T378978#10299956 (10rook) 05Open→03Resolved a:03rook [15:12:22] 14Analytics, 06Data-Engineering, 10EventStreams, 10Wikidata, and 3 others: Expose rdf-streaming-updater.mutation content through EventStreams - https://phabricator.wikimedia.org/T294133#10299948 (10dcausse) >>! In T294133#10299656, @Sina_az_9 wrote: > Hello, > I have a question maybe you can help. Hey, s... [15:22:42] (03PS2) 10Snwachukwu: Update coeditors_metrics hql file to support temp account changes. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1084282 (https://phabricator.wikimedia.org/T377352) [15:33:24] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#10300138 (10Marostegui) [16:46:16] 06Data-Engineering: Make it possible to select the DAG deployment method - https://phabricator.wikimedia.org/T379279 (10brouberol) 03NEW [17:19:58] (03PS1) 10Mforns: Modify MediaWiki History queries to support Temp Accounts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1088342 (https://phabricator.wikimedia.org/T379230) [17:33:20] (03PS1) 10Aqu: Add an option to ignore missing input folders in Refine [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088346 (https://phabricator.wikimedia.org/T369845) [18:29:36] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06DBA, 07Schema-change-in-production: Drop deprecated abuse filter fields on wmf wikis - https://phabricator.wikimedia.org/T367781#10301163 (10Ladsgroup) When it can't drain, there is a maint script still attaching, here it is from `10.194.147.226`. Y... [18:58:32] (03CR) 10Ottomata: Add an option to ignore missing input folders in Refine (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1088346 (https://phabricator.wikimedia.org/T369845) (owner: 10Aqu) [19:10:53] 06Data-Engineering, 10Data Products (Data Products Sprint 22), 07Documentation, 10Event-Platform: Render human-readable schemas on schema.wikimedia.org - https://phabricator.wikimedia.org/T376841#10301330 (10Milimetric) [23:51:13] 10Data-Engineering (Q2 2024 October 1st - December 31th), 06Data-Platform, 06Traffic, 10Data Products (Data Products (Data Products Sprint 21 🪂)): NEW BUG REPORT - Issues in calculation logic for unique devices tables - https://phabricator.wikimedia.org/T375527#10302470 (10VirginiaPoundstone) 05Open→...