[00:50:32] 07Analytics-Data-Problem, 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 15), 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9919766 (10Milimetric) Great question... [00:53:59] (03PS1) 10Milimetric: [WIP] Implement new way to aggregate browser statistics [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1049281 (https://phabricator.wikimedia.org/T342267) [02:19:03] FIRING: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [02:19:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=codfw.mediawiki.cirrussearch.page_rerender.v1&viewPanel=4 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [02:35:35] 14Analytics-Radar, 06Data-Engineering-Icebox: Decide on JSON validation library - https://phabricator.wikimedia.org/T147137#9919922 (10Pppery) Removing #mediawiki-extensions-collaborationkit as that repo is being archived. This should probably be closed as declined as it has no remaining active components. [03:19:03] RESOLVED: GobblinKafkaRecordsExtractedNotEqualRecordsExpected: Gobblin job event_default ingested an unexpected number of records for a Kafka topic partition. ... [03:19:04] - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Gobblin - https://grafana.wikimedia.org/d/pAQaJwEnk/gobblin?orgId=1&var-gobblin_job_name=event_default&var-kafka_topic=codfw.mediawiki.cirrussearch.page_rerender.v1&viewPanel=4 - https://alerts.wikimedia.org/?q=alertname%3DGobblinKafkaRecordsExtractedNotEqualRecordsExpected [05:00:21] (03PS5) 10KCVelaga: Add Automoderator metrics to WMF Product jobs [analytics/wmf-product/jobs] - 10https://gerrit.wikimedia.org/r/1049106 (https://phabricator.wikimedia.org/T362615) [05:00:44] (03CR) 10KCVelaga: "Ready for your review." [analytics/wmf-product/jobs] - 10https://gerrit.wikimedia.org/r/1049106 (https://phabricator.wikimedia.org/T362615) (owner: 10KCVelaga) [05:21:09] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9920133 (10Marostegui) [06:05:22] 06Data-Engineering, 06Data Products, 06DBA, 07Schema-change-in-production: Cleanup revision table schema - https://phabricator.wikimedia.org/T367856#9920169 (10Marostegui) [07:01:07] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Grant Access to analytics-privatedata-users for DMburugu - https://phabricator.wikimedia.org/T367872#9920235 (10SLyngshede-WMF) @odimitrijevic / @Ottomata / @WDoranWMF Would either of you approve? [07:51:27] brouberol, btullis - as FYI https://phabricator.wikimedia.org/T363191 for dse-k8s-worker1001 - when DSE is migrated to Bookworm, let's remember to remove the option to deploy ROCm packages on dse-k8s-worker1001 [07:51:53] I cannot do it now since one package that we need for monitoring, rocm-smi, is only available for Bookworm [07:52:07] noted, thank you! [07:52:26] (this worker is the only one with a GPU in that k8s cluster, I take it?) [07:53:17] exactly yes [07:54:01] the GPU packages are only needed on containers, so we can skip several GBs of usage on k8s workers [07:54:44] and also we don't have a ROCm version on k8s bare metal and another one on the container (shipped with pytorch for example), that is very confusing [08:03:32] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 15), 13Patch-For-Review: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9920355 (10Joe) >>! In T368098#9918924, @xcollazo wrote: > Ok after ob... [08:10:32] I trust you on that TBH. I'm very much estranged to the ML world [08:13:57] <3 [08:19:20] 14Analytics-Radar, 06Data-Engineering-Icebox: Decide on JSON validation library - https://phabricator.wikimedia.org/T147137#9920400 (10Jdforrester-WMF) In practice we still have two PHP JSON Schema implementations live in production (plus at least one other in JavaScript); I think we should probably keep this... [08:25:25] elukey: That's great. Thanks for the heads-up. [08:40:50] 06Data-Engineering-Icebox: Deprecation (if possible) of the #central channel on irc.wikimedia.org - https://phabricator.wikimedia.org/T242712#9920490 (10elukey) The only users that I can see are in `#central`: ` [ BanBot] [ masti] [ op-56495] ` No contact info under `/whois` afaics :( [10:18:04] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Requesting access to analytics-privatedata-users for WBrown (WMF) - https://phabricator.wikimedia.org/T368260#9920991 (10Dreamy_Jazz) @Dzahn I am requesting membership with access to Kerberos. [11:42:45] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 15), 13Patch-For-Review: Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9921290 (10Ladsgroup) The trigger seems to be a duress imposed on s4:... [14:37:02] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 15): Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9921990 (10Ladsgroup) >>! In T368098#9921528, @jcrespo wrote: > Question, what went wrong wi... [14:43:51] 06Data-Engineering, 06SRE, 10SRE-Access-Requests: Grant Access to analytics-privatedata-users for DMburugu - https://phabricator.wikimedia.org/T367872#9922006 (10Ottomata) Approved! [15:05:07] (03PS1) 10Snwachukwu: Update column definition for uniqueness check. [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/1049580 (https://phabricator.wikimedia.org/T354692) [17:26:43] 06Data-Engineering: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435 (10Dzahn) 03NEW [17:26:49] I /win 25 [17:26:51] er [17:27:22] 06Data-Engineering: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435#9922997 (10Dzahn) [17:27:35] 06Data-Engineering: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435#9923006 (10Dzahn) p:05Triage→03Medium [17:42:31] 06Data-Engineering: add approvers to analytics-research-admins - https://phabricator.wikimedia.org/T368435#9923095 (10Ottomata) > In T367757#9905191 it was suggested to add Xiao Xiao and Fabian Kaelin from research as approvers. For this, I think Research team approval is all that is needed. Data-Engineering an... [17:43:09] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 15): Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9923096 (10xcollazo) >>! In T368098#9921287, @Ladsgroup wrote: >... >That's around 100M hit... [17:44:43] 06Data-Engineering, 10Dumps-Generation, 06SRE, 10Data Products (Data Products Sprint 15): Dumps generation without prefetch cause disruption to the production environment - https://phabricator.wikimedia.org/T368098#9923108 (10xcollazo) >>! In T368098#9921990, @Ladsgroup wrote: >... > - Replicas in dump gr... [19:04:28] 06Data-Engineering, 10Event-Platform: Event Platform schemas should not support type changes to structs as array element or map value types - https://phabricator.wikimedia.org/T366487#9923439 (10Ottomata) [20:11:02] (03CR) 10Btullis: [C:03+1] "Looks good to me." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1048496 (https://phabricator.wikimedia.org/T368038) (owner: 10Xcollazo) [20:13:16] (03CR) 10Xcollazo: [V:03+2 C:03+2] "Thanks @btullis@wikimedia.org! Merging.." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1048496 (https://phabricator.wikimedia.org/T368038) (owner: 10Xcollazo) [20:15:18] (03CR) 10Xcollazo: "Superseded by 1048496." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1048026 (https://phabricator.wikimedia.org/T368067) (owner: 10Gerrit maintenance bot) [20:15:27] (03Abandoned) 10Xcollazo: Add btm.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1048026 (https://phabricator.wikimedia.org/T368067) (owner: 10Gerrit maintenance bot) [20:33:35] 06Data-Engineering, 10Event-Platform: [Event Platform] - Add schema CI test that array ensures properties with object types also enumerate object properties - https://phabricator.wikimedia.org/T366562#9923747 (10Ottomata)