[00:30:38] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [01:14:20] 10Data-Engineering, 10Product-Analytics: Add TikTok's in-app browser to ua-parser library - https://phabricator.wikimedia.org/T325611 (10nshahquinn-wmf) [06:44:51] 10Data-Engineering-Radar, 10MW-on-K8s, 10serviceops, 10Patch-For-Review: IPInfo MediaWiki extension depends on presence of maxmind db in the container/host - https://phabricator.wikimedia.org/T288375 (10Joe) After a discussion on IRC: * We're not happy to add another 500 MB to the docker image * It would... [06:56:56] (03PS8) 10Snwachukwu: [WIP] Refactor and Expand External referer classification [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) [08:42:41] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Investigate wikimedia and wikidata unique devices per-project-family overcount offset - https://phabricator.wikimedia.org/T301403 (10JAllemandou) >>! In T301403#8479701, @odimitrijevic wrote: > I believe this is the same probl... [08:43:19] 10Data-Engineering: Odd behavior in unique device counts - https://phabricator.wikimedia.org/T276472 (10JAllemandou) [08:43:21] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Investigate wikimedia and wikidata unique devices per-project-family overcount offset - https://phabricator.wikimedia.org/T301403 (10JAllemandou) [10:02:16] (03CR) 10Matthias Mullie: [C: 03+2] Modify SearchPreview action to align with requirements [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/868187 (https://phabricator.wikimedia.org/T321069) (owner: 10Simone Cuomo) [10:02:50] (03Merged) 10jenkins-bot: Modify SearchPreview action to align with requirements [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/868187 (https://phabricator.wikimedia.org/T321069) (owner: 10Simone Cuomo) [10:53:12] (VarnishkafkaNoMessages) firing: (2) varnishkafka on cp2033 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [10:58:12] (VarnishkafkaNoMessages) resolved: (2) varnishkafka on cp2033 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [11:10:14] PROBLEM - Check systemd state on matomo1002 is CRITICAL: CRITICAL - degraded: The following units failed: prometheus_puppet_agent_stats.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [11:18:10] RECOVERY - Check systemd state on matomo1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [11:22:56] PROBLEM - Check systemd state on matomo1002 is CRITICAL: CRITICAL - degraded: The following units failed: prometheus_puppet_agent_stats.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [11:34:04] RECOVERY - Check systemd state on matomo1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [12:57:09] 10Data-Engineering, 10Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-05)): Create partman recipe for cephosd servers - https://phabricator.wikimedia.org/T324670 (10BTullis) I've checked the HTTPS management interface for cephosd1001 and all looks good. * The 12 x 16 TB HDDs are detected first, with IDs:... [13:34:30] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure (EQ2 Kanban (Sprints 04-05)): Add an-presto10[06-15] to the presto cluster - https://phabricator.wikimedia.org/T323783 (10Stevemunene) This change has been reverted and we are back to the original an-presto100[1-5] due to and incident where Superset: Pr... [14:12:35] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Spark Streaming Dumps POC: Backfill content table - https://phabricator.wikimedia.org/T323641 (10Milimetric) [15:11:13] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Investigate wikimedia and wikidata unique devices per-project-family overcount offset - https://phabricator.wikimedia.org/T301403 (10JAllemandou) Another finding: the `WMF-Last-Access-Global` cookies are not set for `wikimedia... [15:12:46] 10Data-Engineering-Planning, 10Data Pipelines: Update refinery-source PageviewDefinition to better handle `Special:` pages - https://phabricator.wikimedia.org/T325544 (10JAllemandou) >>! In T325544#8479711, @odimitrijevic wrote: > @Jallemandou can you please provide the delta between the two totals (per family... [15:36:01] 10Data-Engineering, 10Event-Platform Value Stream, 10Patch-For-Review: Design Schema for page state and page state with content (enriched) streams - https://phabricator.wikimedia.org/T308017 (10Ottomata) [15:50:47] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Investigate wikimedia and wikidata unique devices per-project-family overcount offset - https://phabricator.wikimedia.org/T301403 (10Isaac) > This makes it even clearer: we shouldn't use project-family numbers for the wikimedi... [15:52:58] (03CR) 10Joal: [WIP] Refactor and Expand External referer classification (0310 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) (owner: 10Snwachukwu) [16:04:19] 10Data-Engineering, 10Product-Analytics: Add TikTok's in-app browser to ua-parser library - https://phabricator.wikimedia.org/T325611 (10Isaac) Thanks for opening this! We probably want folks from TikTok to do this as they'll be aware of edge-cases etc. and maybe more likely to update if they make changes? The... [16:48:07] (03CR) 10Milimetric: "nice work! My main comment is on the accuracy of the regexes. I don't think we can get them perfect but they can avoid some easy false p" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) (owner: 10Snwachukwu) [16:49:56] (03PS9) 10Snwachukwu: Refactor and Expand External referer classification [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/864772 (https://phabricator.wikimedia.org/T309769) [17:08:55] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06): Include EU Registered Country in the canonical country database - https://phabricator.wikimedia.org/T324995 (10JArguello-WMF) [18:13:19] 10Data-Engineering, 10Product-Analytics: Add TikTok's in-app browser to ua-parser library - https://phabricator.wikimedia.org/T325611 (10kzimmerman) @Maryana @MMiller_WMF I'm curious what your thoughts are regarding Isaac's comment above and this request to update the upstream ua-parser library? [18:44:37] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink SQL queries should access Kafka topics from a Catalog - https://phabricator.wikimedia.org/T322022 (10tchin) Perhaps the end goal would have a user experience like: `lang=sql CREATE CATALOG wmfeventcatalog WITH ( 'type' = 'wmfeventc... [19:09:36] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink SQL queries should access Kafka topics from a Catalog - https://phabricator.wikimedia.org/T322022 (10Ottomata) Yes I like it! > CREATE TABLE `test.event.example` Should we make sure that CREATED 'custom' tables don't use the same... [19:10:43] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink SQL queries should access Kafka topics from a Catalog - https://phabricator.wikimedia.org/T322022 (10Ottomata) > ALTER TABLE `mediawiki.page-create` SET ('schema'='0.0.1'); BTW, it technically should be fine to always use the latest... [21:10:02] 10Quarry: [feedback] Advanced acces - https://phabricator.wikimedia.org/T325683 (10Dusan_Krehel) [21:15:07] 10Quarry: [feedback] Advanced acces - https://phabricator.wikimedia.org/T325683 (10Aklapper) @Dusan_Krehel: Thanks for reporting this. For future reference, please use the feature request form (linked from the top of the task creation page) to create feature requests, and fill in the sections in the template. Th... [21:15:29] 10Quarry: Allow downloading output via CLI - https://phabricator.wikimedia.org/T325683 (10Aklapper) [21:15:43] 10Quarry: Allow downloading output via CLI - https://phabricator.wikimedia.org/T325683 (10Aklapper) [21:39:16] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform Value Stream, 10MediaWiki-Recent-changes, 10Technical-Debt (Deprecation process): Remove deprecated RCFeedEngine support - https://phabricator.wikimedia.org/T250628 (10Umherirrender) [22:04:37] 10Quarry: Allow downloading output via CLI - https://phabricator.wikimedia.org/T325683 (10Dusan_Krehel) Aklapper: I proceeded according to: https://quarry.wmcloud.org/query/69880 -> Feedback (bottom menu). [22:22:35] 10Data-Engineering: Provide aggregated user device data per-country - https://phabricator.wikimedia.org/T325306 (10mpopov) Tagging @Htriedman who has been working on releasing another dataset aggregated per-country and @Niharika as the PM for Anti-Harassment Tools.