[00:18:42] 10Data-Engineering, 10Data-Services, 10Privacy Engineering, 10cloud-services-team (Kanban): Increased visibility in wiki-replicas for volunteers fighting vandals - https://phabricator.wikimedia.org/T284944 (10odimitrijevic) [00:18:46] 10Data-Engineering, 10Data-Services: Expose more properties to the user_properties_anon table on Wiki Replicas - https://phabricator.wikimedia.org/T226162 (10odimitrijevic) [00:18:52] 10Data-Engineering: Add ignore success flags option to pageview monthly dumps - https://phabricator.wikimedia.org/T283593 (10odimitrijevic) [00:18:56] 10Data-Engineering, 10Data-Engineering-Kanban: Data structuring guidance request - https://phabricator.wikimedia.org/T287402 (10odimitrijevic) [00:19:00] 10Data-Engineering, 10Anti-Harassment, 10CheckUser, 10Privacy Engineering, and 2 others: SPIKE: consider all problems that might happen when we handle Google's privacy changes - https://phabricator.wikimedia.org/T265057 (10odimitrijevic) [00:19:06] 10Data-Engineering: Make data quality stats alert only if anomalous metrics change - https://phabricator.wikimedia.org/T263030 (10odimitrijevic) [00:19:11] 10Data-Engineering: pagecounts-ez of month 2020-08 is incomplete - https://phabricator.wikimedia.org/T262141 (10odimitrijevic) [00:19:15] 10Data-Engineering: explicitily exclude mobile-app traffic from unique devices per domain computation - https://phabricator.wikimedia.org/T258288 (10odimitrijevic) [00:19:19] 10Data-Engineering: actor_signature_per_project_family does not work for apps - https://phabricator.wikimedia.org/T258101 (10odimitrijevic) [00:19:25] 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics, 10Structured-Data-Backlog, and 4 others: Create a Commons equivalent of the wikidata_entity table in the Data Lake - https://phabricator.wikimedia.org/T258834 (10odimitrijevic) [00:19:29] 10Data-Engineering, 10SRE, 10Traffic-Icebox: Add backend field to webrequest Hive table - https://phabricator.wikimedia.org/T257354 (10odimitrijevic) [00:19:39] 10Data-Engineering: MW REST API Historical Data Endpoint Needs - https://phabricator.wikimedia.org/T240387 (10odimitrijevic) [00:19:43] 10Data-Engineering, 10Product-Analytics: Keep canonical_data.wikis updated - https://phabricator.wikimedia.org/T241741 (10odimitrijevic) [00:19:47] 10Data-Engineering, 10Product-Analytics: Get "edits hourly" on a daily basis - https://phabricator.wikimedia.org/T231938 (10odimitrijevic) [00:19:51] 10Data-Engineering, 10Platform Engineering: Update mediawiki-history to use new Multi-Content-Revision tables - https://phabricator.wikimedia.org/T239591 (10odimitrijevic) [00:19:59] 10Data-Engineering: Improve pageview automated traffic detection heuristics - https://phabricator.wikimedia.org/T280565 (10odimitrijevic) [00:20:03] 10Data-Engineering, 10Research: Adding data from centralauth to the lake and the mediawiki_history dataset - https://phabricator.wikimedia.org/T282657 (10odimitrijevic) [00:20:07] 10Data-Engineering, 10Product-Analytics: Top read repeats - https://phabricator.wikimedia.org/T280011 (10odimitrijevic) [00:20:15] 10Data-Engineering, 10WMCZ-Stats: Review request: New datasets for WMCZ published under analytics.wikimedia.org - https://phabricator.wikimedia.org/T279567 (10odimitrijevic) [00:20:19] 10Data-Engineering: Odd behavior in unique device counts - https://phabricator.wikimedia.org/T276472 (10odimitrijevic) [00:20:23] 10Data-Engineering: Filter out .wikidata requests from pageview definition - https://phabricator.wikimedia.org/T273215 (10odimitrijevic) [00:20:27] 10Data-Engineering: Add "did edit" field to pageview_actor - https://phabricator.wikimedia.org/T277785 (10odimitrijevic) [00:20:31] 10Analytics, 10Data-Engineering, 10Event-Platform, 10EventStreams: Expose mediawiki/revision/tags-change in stream.wikimedia.org - https://phabricator.wikimedia.org/T294391 (10odimitrijevic) [00:20:35] 10Data-Engineering, 10Privacy Engineering, 10Research: Release dataset on top search engine referrers by country, device, and language - https://phabricator.wikimedia.org/T270140 (10odimitrijevic) [00:20:39] 10Data-Engineering, 10Product-Analytics, 10Epic: Readership Retention: New vs. Returning Unique devices - https://phabricator.wikimedia.org/T269815 (10odimitrijevic) [00:20:43] 10Data-Engineering, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: Evaluate a differentially private solution to release wikipedia's project-title-country data - https://phabricator.wikimedia.org/T267283 (10odimitrijevic) [00:20:47] 10Data-Engineering, 10Product-Analytics: Add timestamps of important revision events to mediawiki_history - https://phabricator.wikimedia.org/T266375 (10odimitrijevic) [00:20:51] 10Data-Engineering, 10All-and-every-Wiktionary: Add editors per country data for Wiktionary projects - https://phabricator.wikimedia.org/T266643 (10odimitrijevic) [00:20:55] 10Data-Engineering: Gather all data-purge into a single job - https://phabricator.wikimedia.org/T262201 (10odimitrijevic) [00:20:59] 10Data-Engineering, 10Product-Analytics: Analyze differences between checksum-based and revert-tag based reverts in mediawiki_history - https://phabricator.wikimedia.org/T266374 (10odimitrijevic) [00:21:03] 10Data-Engineering: Add `is_last_page_event` and `is_last_user_event` fields in mediawiki_history - https://phabricator.wikimedia.org/T255890 (10odimitrijevic) [00:21:07] 10Data-Engineering, 10Product-Analytics: Add editors_monthly data to Druid - https://phabricator.wikimedia.org/T256719 (10odimitrijevic) [00:21:11] 10Data-Engineering: Mediawiki History dumps unique editors feature request - https://phabricator.wikimedia.org/T254234 (10odimitrijevic) [00:21:17] 10Data-Engineering, 10Wikidata: Make Wikidata item_page_link table available publicly - https://phabricator.wikimedia.org/T258514 (10odimitrijevic) [00:21:21] 10Data-Engineering: Unusual unknown external webrequest referers - https://phabricator.wikimedia.org/T241295 (10odimitrijevic) [00:21:25] 10Data-Engineering: Anomaly detection alarms for the edit event stream - https://phabricator.wikimedia.org/T250845 (10odimitrijevic) [00:21:35] 10Data-Engineering: Output schema with mediawiki_history snapshots dumps - https://phabricator.wikimedia.org/T238668 (10odimitrijevic) [00:21:39] 10Data-Engineering, 10SRE, 10Traffic-Icebox: Increased number of webrequest sequence-numbers alarms (mostly) on upload webrequest-source - https://phabricator.wikimedia.org/T225786 (10odimitrijevic) [00:21:45] 10Data-Engineering: Rename event_sanitized to event_longterm - https://phabricator.wikimedia.org/T225751 (10odimitrijevic) [00:21:49] 10Data-Engineering, 10CheckUser, 10Patch-For-Review, 10Platform Team Workboards (Clinic Duty Team), 10Schema-change: Schema changes for `cu_changes` and `cu_log` table - https://phabricator.wikimedia.org/T233004 (10odimitrijevic) [00:21:53] 10Data-Engineering: Include user group expiry events in mediawiki history reconstruction - https://phabricator.wikimedia.org/T225249 (10odimitrijevic) [00:21:57] 10Data-Engineering: Use historical fields in history reduced dataset - https://phabricator.wikimedia.org/T222278 (10odimitrijevic) [00:22:01] 10Data-Engineering, 10Product-Analytics: Identify imported revisions in mediawiki_history - https://phabricator.wikimedia.org/T221482 (10odimitrijevic) [00:22:05] 10Data-Engineering, 10Product-Analytics: Update geo-editors job to use tags and report desktop/mobile edits - https://phabricator.wikimedia.org/T223444 (10odimitrijevic) [00:22:09] 10Data-Engineering, 10Product-Analytics: A few alterblocks events have event_timestamps from before 2001 - https://phabricator.wikimedia.org/T218824 (10odimitrijevic) [00:22:13] 10Data-Engineering: Move FR banner-impression jobs to events - https://phabricator.wikimedia.org/T215636 (10odimitrijevic) [00:22:17] 10Data-Engineering: Some event data (like the one that comes from mediawiki events such us revision create) should not get sanitized - https://phabricator.wikimedia.org/T217271 (10odimitrijevic) [00:22:21] 10Data-Engineering: Mediawiki History: moves counted twice in Revision - https://phabricator.wikimedia.org/T189044 (10odimitrijevic) [00:22:25] 10Data-Engineering, 10Research-Backlog: [Open question] Improve bot identification at scale - https://phabricator.wikimedia.org/T138207 (10odimitrijevic) [00:22:29] 10Data-Engineering: Create new table for 'referer' aggregated data - https://phabricator.wikimedia.org/T112284 (10odimitrijevic) [00:22:33] 10Data-Engineering, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231 (10odimitrijevic) [00:22:37] 10Data-Engineering, 10Datasets-Webstatscollector: Add alarms for high volume of views to pages with replacement characters - https://phabricator.wikimedia.org/T117945 (10odimitrijevic) [00:22:42] 10Data-Engineering: Fill holes in pageview-complete dumps using pageview-count-raw - https://phabricator.wikimedia.org/T290060 (10odimitrijevic) [00:22:45] 10Data-Engineering, 10Product-Analytics, 10Research: Use Hive/Spark timestamps in Refined event data - https://phabricator.wikimedia.org/T278467 (10odimitrijevic) [00:22:50] 10Data-Engineering: page_id is null where it shouldn't be in mediawiki history - https://phabricator.wikimedia.org/T259823 (10odimitrijevic) [00:22:54] 10Analytics, 10Data-Engineering, 10Pageviews-API: Provide weekly top pageviews stats - https://phabricator.wikimedia.org/T133575 (10odimitrijevic) [00:22:58] 10Data-Engineering, 10Product-Analytics: History: mismatched historical and latest values - https://phabricator.wikimedia.org/T258967 (10odimitrijevic) [00:23:02] 10Data-Engineering: Join slot, content, revision, and page once on load - https://phabricator.wikimedia.org/T240413 (10odimitrijevic) [00:23:06] 10Data-Engineering: Set entropy alarm in editors per country per wiki - https://phabricator.wikimedia.org/T227809 (10odimitrijevic) [00:23:10] 10Data-Engineering, 10incubator.wikimedia.org: Create dashiki dashboard / small tool to track statistics about incubated wikis - https://phabricator.wikimedia.org/T237389 (10odimitrijevic) [00:23:14] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering, 10Product-Analytics: Support including edits to deleted pages in editing metrics - https://phabricator.wikimedia.org/T295212 (10odimitrijevic) [00:23:18] 10Data-Engineering: Aggregate pageviews to Wikidata entities - https://phabricator.wikimedia.org/T215438 (10odimitrijevic) [00:23:22] 10Data-Engineering, 10Product-Analytics: Sqoop more tables for mediawiki in monthly schedule - https://phabricator.wikimedia.org/T198983 (10odimitrijevic) [00:23:26] 10Data-Engineering: Release wikimedia history dumps sorted by user ID and page ID - https://phabricator.wikimedia.org/T232844 (10odimitrijevic) [00:23:30] 10Data-Engineering: Add legacy per-article pagecounts data (prior to 2015) - https://phabricator.wikimedia.org/T193759 (10odimitrijevic) [00:23:34] 10Data-Engineering, 10Epic: Sanitize pageview_hourly - https://phabricator.wikimedia.org/T114675 (10odimitrijevic) [00:23:42] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Add "Top used photos" metric - https://phabricator.wikimedia.org/T220485 (10odimitrijevic) [00:23:46] 10Analytics, 10Data-Engineering, 10Event-Platform, 10User-Elukey: Create EventStream's equivalent to irc.wikimedia.org's #central channel - https://phabricator.wikimedia.org/T240182 (10odimitrijevic) [00:25:48] 10Data-Engineering, 10Stewards-and-global-tools: Collect information about users affected by blocks - https://phabricator.wikimedia.org/T297051 (10odimitrijevic) [00:26:01] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Readers-Web-Backlog, 10Patch-For-Review: WikipediaPortal Event Platform Migration - https://phabricator.wikimedia.org/T282012 (10odimitrijevic) [00:26:07] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Metrics-Platform, 10Browser-Support-Microsoft-Edge: Problem with delay caused by intake-analytics.wikimedia.org - https://phabricator.wikimedia.org/T295427 (10odimitrijevic) [00:26:10] 10Data-Engineering, 10Infrastructure-Foundations: Netflow data pipeline - https://phabricator.wikimedia.org/T257554 (10odimitrijevic) [00:26:12] 10Data-Engineering: Transform EventLoggingToDruid job to read schemas to ingest from an allowlist and process them all - https://phabricator.wikimedia.org/T202312 (10odimitrijevic) [00:26:15] 10Data-Engineering: Implement digest-only mediawiki_history_reduced dataset in spark - https://phabricator.wikimedia.org/T181703 (10odimitrijevic) [00:26:17] 10Analytics, 10Data-Engineering, 10Event-Platform: Automate EventGate validation error reporting - https://phabricator.wikimedia.org/T268027 (10odimitrijevic) [00:26:19] 10Analytics, 10Data-Engineering, 10Event-Platform, 10SRE, 10Sustainability (Incident Followup): Pool eventgate-main in both datacenters (active/active) - https://phabricator.wikimedia.org/T296699 (10odimitrijevic) [00:26:23] 10Analytics, 10Data-Engineering, 10Event-Platform: Mediaviewer views should be reworked to be an eventlogging event - https://phabricator.wikimedia.org/T239630 (10odimitrijevic) [00:28:04] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Running into errors while adding Hive table to Superset dataset - https://phabricator.wikimedia.org/T284604 (10odimitrijevic) [00:28:07] 10Data-Engineering: Allow users to differentiate their JupyterHub logs in Logstash - https://phabricator.wikimedia.org/T293243 (10odimitrijevic) [00:28:09] 10Data-Engineering, 10Inuka-Team, 10Product-Analytics, 10Superset: Superset timeouts for KaiOS dashboard - https://phabricator.wikimedia.org/T277320 (10odimitrijevic) [00:28:11] 10Data-Engineering, 10Superset: Remove support for the (deprecated) Druid datasources (in favor of Druid Tables) on Superset - https://phabricator.wikimedia.org/T263972 (10odimitrijevic) [00:28:13] 10Data-Engineering, 10Product-Analytics: Remove COUNT(*) from datasets when not useful in Superset & Turnilo - https://phabricator.wikimedia.org/T255725 (10odimitrijevic) [00:28:15] 10Data-Engineering: Easy dimensional data visualization - https://phabricator.wikimedia.org/T280029 (10odimitrijevic) [00:28:17] 10Data-Engineering: Turnilo "Display Druid query" gives "general error" - https://phabricator.wikimedia.org/T273685 (10odimitrijevic) [00:28:54] 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of christinedk - https://phabricator.wikimedia.org/T297461 (10odimitrijevic) [00:29:06] 10Data-Engineering, 10Epic: Traffic anomaly alarms - https://phabricator.wikimedia.org/T267355 (10odimitrijevic) [00:29:29] 10Analytics-Kanban, 10Data-Engineering, 10Airflow: SPIKE - Will Hadoop 3 container support help us for Airflow deployment pipelines? - https://phabricator.wikimedia.org/T288247 (10odimitrijevic) [00:29:32] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review, and 2 others: Migrate analytics cluster alerts from Icinga to AlertManager - https://phabricator.wikimedia.org/T293399 (10odimitrijevic) [00:29:34] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Conda's CPPFLAGS may not be correct when pip installing a package that needs c/cpp compilation - https://phabricator.wikimedia.org/T292699 (10odimitrijevic) [00:29:38] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Automate kerberos credential creation and management to ease the creation of testing infrastructure - https://phabricator.wikimedia.org/T292389 (10odimitrijevic) [00:29:40] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: dbstore1007 is swapping heavilly, potentially soon killing mysql services due to OOM error - https://phabricator.wikimedia.org/T290841 (10odimitrijevic) [00:29:43] 10Data-Engineering, 10Product-Analytics: Fix presto kerberos support for system users - https://phabricator.wikimedia.org/T292072 (10odimitrijevic) [00:29:44] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Standardize the stats system user uid - https://phabricator.wikimedia.org/T291384 (10odimitrijevic) [00:29:47] 10Data-Engineering: Create aggregate alarms for Hadoop daemons running on worker nodes - https://phabricator.wikimedia.org/T287027 (10odimitrijevic) [00:29:49] 10Data-Engineering: Use inclusive language in code for private analytics infrastructure - https://phabricator.wikimedia.org/T280268 (10odimitrijevic) [00:29:58] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:(Need By: TBD) rack/setup/install an-worker11[42-48].eqiad.wmnet - https://phabricator.wikimedia.org/T293922 (10odimitrijevic) [00:30:02] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics, and 2 others: wmfdata.mariadb relies on analytics-mysql being available - https://phabricator.wikimedia.org/T292479 (10odimitrijevic) [00:30:08] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Un-fork analytics/gobblin - https://phabricator.wikimedia.org/T292396 (10odimitrijevic) [00:30:12] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Move the Analytics/DE testing infrastructure to Pontoon - https://phabricator.wikimedia.org/T292388 (10odimitrijevic) [00:30:16] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Observability-Alerting: Apparent latency warning in 90th centile of eventgate-logging-external - https://phabricator.wikimedia.org/T294911 (10odimitrijevic) [00:30:20] 10Data-Engineering: Deploy an-test-launcher1002 as a Ganeti VM to test high-availability of scheduled jobs - https://phabricator.wikimedia.org/T288767 (10odimitrijevic) [00:30:24] 10Data-Engineering: Use corosync and pacemaker for presto coordinator active/standby configuration - https://phabricator.wikimedia.org/T287967 (10odimitrijevic) [00:30:28] 10Data-Engineering: Analytics coordinator failover improvements - https://phabricator.wikimedia.org/T280905 (10odimitrijevic) [00:30:32] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Refactor analytics-meta MariaDB layout to use an-db100[12] - https://phabricator.wikimedia.org/T284150 (10odimitrijevic) [00:30:36] 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Add a presto query logger - https://phabricator.wikimedia.org/T269832 (10odimitrijevic) [00:30:40] 10Data-Engineering: Productionize HDFS fsimage data analysis job - https://phabricator.wikimedia.org/T261283 (10odimitrijevic) [00:30:44] 10Data-Engineering, 10Observability-Logging: Indexing errors / malformed logs for aqs on cassandra timeout - https://phabricator.wikimedia.org/T262920 (10odimitrijevic) [00:30:48] 10Data-Engineering: LVS in Analytics VLANs - https://phabricator.wikimedia.org/T288750 (10odimitrijevic) [00:30:52] 10Data-Engineering, 10SRE: Trash cleanup cron spams on an-test hosts - https://phabricator.wikimedia.org/T286442 (10odimitrijevic) [00:30:58] 10Data-Engineering: SLF4J logspam when using hadoop command-line clients - https://phabricator.wikimedia.org/T276240 (10odimitrijevic) [00:31:58] 10Analytics, 10Data-Engineering, 10Event-Platform, 10EventStreams: Expose mediawiki/revision/tags-change in stream.wikimedia.org - https://phabricator.wikimedia.org/T294391 (10odimitrijevic) [00:32:09] 10Analytics, 10Data-Engineering, 10Pageviews-API: Provide weekly top pageviews stats - https://phabricator.wikimedia.org/T133575 (10odimitrijevic) [00:32:11] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering, 10Product-Analytics: Support including edits to deleted pages in editing metrics - https://phabricator.wikimedia.org/T295212 (10odimitrijevic) [00:32:30] 10Analytics, 10Data-Engineering, 10Event-Platform, 10User-Elukey: Create EventStream's equivalent to irc.wikimedia.org's #central channel - https://phabricator.wikimedia.org/T240182 (10odimitrijevic) [00:32:32] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Add "Top used photos" metric - https://phabricator.wikimedia.org/T220485 (10odimitrijevic) [00:32:45] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Improve Refine bad data handling - https://phabricator.wikimedia.org/T289003 (10odimitrijevic) [00:32:47] 10Data-Engineering: Analytics-test-hadoop Spark3 package upgrade - https://phabricator.wikimedia.org/T291465 (10odimitrijevic) [00:32:49] 10Data-Engineering: Analytics-hadoop Spark3 package upgrade (production) - https://phabricator.wikimedia.org/T291466 (10odimitrijevic) [00:32:52] 10Data-Engineering: Refinery python code should use anaconda-wmf - https://phabricator.wikimedia.org/T286743 (10odimitrijevic) [00:32:54] 10Data-Engineering: Refine: Use Spark SQL instead of Hive JDBC - https://phabricator.wikimedia.org/T209453 (10odimitrijevic) [00:32:56] 10Data-Engineering, 10Airflow: Alarms for virtualpageview should exist (probably in oozie) for jobs that have been idle too long - https://phabricator.wikimedia.org/T213716 (10odimitrijevic) [00:32:58] 10Data-Engineering, 10Patch-For-Review: Upgrade Refinery Jobs to Spark 3 - https://phabricator.wikimedia.org/T291386 (10odimitrijevic) [00:33:19] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Refine drops $schema field values - https://phabricator.wikimedia.org/T255818 (10odimitrijevic) [00:33:40] 10Data-Engineering: Reduce number of missing revisions from mediawiki-history - https://phabricator.wikimedia.org/T294578 (10odimitrijevic) [00:33:42] 10Data-Engineering: [Session length] Apply different sample rates per wiki - https://phabricator.wikimedia.org/T291693 (10odimitrijevic) [00:33:44] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Add gobblin metrics per kafka-topic-partition - https://phabricator.wikimedia.org/T294422 (10odimitrijevic) [00:33:46] 10Data-Engineering: [EventGate] Failures when getting stream config from MediaWiki API - https://phabricator.wikimedia.org/T286793 (10odimitrijevic) [00:33:48] 10Data-Engineering: Fix gobblin not writing _IMPORTED flags when runs don't overlap hours - https://phabricator.wikimedia.org/T286343 (10odimitrijevic) [00:33:50] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Push Gobblin import metrics to Prometheus and add alerts on some critical imports - https://phabricator.wikimedia.org/T286503 (10odimitrijevic) [00:33:56] 10Analytics-EventLogging, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, and 3 others: Determine which remaining legacy EventLogging schemas need to be migrated or decommissioned - https://phabricator.wikimedia.org/T282131 (10odimitrijevic) [00:33:58] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Epic, 10Patch-For-Review: Replace Camus by Gobblin - https://phabricator.wikimedia.org/T271232 (10odimitrijevic) [00:34:02] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform, and 4 others: Migrate WikibaseTermboxInteraction EventLogging Schema to new EventPlatform thingy - https://phabricator.wikimedia.org/T290303 (10odimitrijevic) [00:34:07] 10Data-Engineering: Change sqoop project list config so that content sqoop doesn't fail - https://phabricator.wikimedia.org/T239589 (10odimitrijevic) [00:34:09] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10odimitrijevic) [00:34:11] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Product-Analytics, 10MW-1.36-notes (1.36.0-wmf.18; 2020-11-17): Migrate legacy metawiki schemas to Event Platform - https://phabricator.wikimedia.org/T259163 (10odimitrijevic) [00:34:13] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Update refinery gobblin jars to use new gobblin-wmf jars and update puppet gobblin jobs - https://phabricator.wikimedia.org/T297939 (10odimitrijevic) [00:34:15] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Add Jenkins job for gobblin-wmf jar release to archiva - https://phabricator.wikimedia.org/T297938 (10odimitrijevic) [00:34:17] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Engineering, 10tech-decision-forum: MediaWiki Events as a Source of Truth - Problem Statement - https://phabricator.wikimedia.org/T291120 (10odimitrijevic) [00:34:19] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Send some existing Gobblin metrics to prometheus - https://phabricator.wikimedia.org/T294420 (10odimitrijevic) [00:34:23] 10Data-Engineering: Data Loss Check always shows false positives - https://phabricator.wikimedia.org/T288496 (10odimitrijevic) [00:34:39] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Users should run explicit commands to materialize schema versions, rather than using magic git hooks - https://phabricator.wikimedia.org/T290074 (10odimitrijevic) [00:34:54] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Observability-Logging, and 3 others: Integrate Event Platform and ECS logs - https://phabricator.wikimedia.org/T291645 (10odimitrijevic) [00:34:59] 10Analytics, 10Analytics-EventLogging, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10odimitrijevic) [00:35:02] 10Analytics, 10Data-Engineering, 10Discovery, 10Event-Platform, and 2 others: Avoid accepting Kafka messages with whacky timestamps - https://phabricator.wikimedia.org/T282887 (10odimitrijevic) [00:35:06] 10Analytics, 10Data-Engineering, 10Event-Platform: Discussion of Event Driven Systems - https://phabricator.wikimedia.org/T290203 (10odimitrijevic) [00:35:14] 10Data-Engineering: Sqoop image metadata - https://phabricator.wikimedia.org/T285783 (10odimitrijevic) [00:35:16] 10Data-Engineering: Add wikitech (labswiki) to the sqoop list - https://phabricator.wikimedia.org/T217792 (10odimitrijevic) [00:35:19] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10SRE: ~1 request/minute to intake-logging.wikimedia.org times out at the traffic/service interface - https://phabricator.wikimedia.org/T264021 (10odimitrijevic) [00:35:41] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Automate creation of sqoop list of wikis to import data for from sitematrix - https://phabricator.wikimedia.org/T190700 (10odimitrijevic) [00:35:46] 10Analytics, 10Data-Engineering, 10DBA, 10Event-Platform, 10WMF-Architecture-Team: Consistent MediaWiki state change events | MediaWiki events as source of truth - https://phabricator.wikimedia.org/T120242 (10odimitrijevic) [00:35:54] 10Data-Engineering, 10Patch-For-Review: HiveExtensions.convertToSchema does not properly convert arrays of structs - https://phabricator.wikimedia.org/T259924 (10odimitrijevic) [00:35:56] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Data-Engineering-Kanban: Autocomplete is very slow (unusable) in Newpyter - https://phabricator.wikimedia.org/T290008 (10odimitrijevic) [00:35:58] 10Data-Engineering, 10Data-Services, 10cloud-services-team (Kanban): Plan a replacement for wiki replicas that is better suited to typical OLAP use cases than the MediaWiki OLTP schema - https://phabricator.wikimedia.org/T215858 (10odimitrijevic) [00:36:06] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Infrastructure-Foundations, and 2 others: Allow login to JupyterHub via CAS - https://phabricator.wikimedia.org/T260386 (10odimitrijevic) [00:36:16] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Product-Analytics: conda list does not show all packages in environment - https://phabricator.wikimedia.org/T294368 (10odimitrijevic) [00:36:22] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Product-Analytics: Internal nbviewer instance for sharing notebooks among 'wmf' and 'nda' members - https://phabricator.wikimedia.org/T290693 (10odimitrijevic) [00:36:26] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Product-Analytics: Functionality to share & view notebooks - https://phabricator.wikimedia.org/T156934 (10odimitrijevic) [00:36:29] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering: Notebook machine to double as RStudio Server? - https://phabricator.wikimedia.org/T190769 (10odimitrijevic) [00:41:59] 10Analytics, 10Data-Engineering, 10Event-Platform, 10EventStreams: Expose mediawiki/revision/tags-change in stream.wikimedia.org - https://phabricator.wikimedia.org/T294391 (10odimitrijevic) [00:42:08] 10Analytics, 10Data-Engineering, 10Pageviews-API: Provide weekly top pageviews stats - https://phabricator.wikimedia.org/T133575 (10odimitrijevic) [00:42:11] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering, 10Product-Analytics: Support including edits to deleted pages in editing metrics - https://phabricator.wikimedia.org/T295212 (10odimitrijevic) [00:42:27] 10Analytics, 10Data-Engineering, 10Event-Platform, 10User-Elukey: Create EventStream's equivalent to irc.wikimedia.org's #central channel - https://phabricator.wikimedia.org/T240182 (10odimitrijevic) [00:42:31] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Add "Top used photos" metric - https://phabricator.wikimedia.org/T220485 (10odimitrijevic) [00:46:37] 10Data-Engineering, 10Code-Health-Objective, 10Platform Engineering Roadmap, 10User-Eevans: Dashboards for AQS 2.0 - https://phabricator.wikimedia.org/T288667 (10odimitrijevic) [00:46:41] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: Create k8s deployment of AQS 2.0 - https://phabricator.wikimedia.org/T288661 (10odimitrijevic) [00:46:43] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement geoeditors endpoints - https://phabricator.wikimedia.org/T288305 (10odimitrijevic) [00:46:45] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Internet-Archive, 10The-Wikipedia-Library: Page-links-change stream doesn't capture duplicated links - https://phabricator.wikimedia.org/T216492 (10odimitrijevic) [00:46:47] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement mediarequests endpoints - https://phabricator.wikimedia.org/T288303 (10odimitrijevic) [00:46:49] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement wikistats 2 endpoints - https://phabricator.wikimedia.org/T288301 (10odimitrijevic) [00:46:53] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement the unique devices endpoints - https://phabricator.wikimedia.org/T288298 (10odimitrijevic) [00:46:56] 10Data-Engineering, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Implement pageviews endpoints - https://phabricator.wikimedia.org/T288296 (10odimitrijevic) [00:46:58] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Repair and reload cassandra2 mediarequest_per_file data table - https://phabricator.wikimedia.org/T291470 (10odimitrijevic) [00:47:17] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform, and 4 others: page-links-change stream is assigning template propagation events to the wrong edits - https://phabricator.wikimedia.org/T216504 (10odimitrijevic) [00:48:23] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Internet-Archive, 10The-Wikipedia-Library: Page-links-change stream doesn't capture duplicated links - https://phabricator.wikimedia.org/T216492 (10odimitrijevic) [00:48:54] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Allow disabling/enabling configured streams via wgEventStreams config - https://phabricator.wikimedia.org/T259712 (10odimitrijevic) [00:49:01] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Product-Analytics, 10MW-1.36-notes (1.36.0-wmf.18; 2020-11-17): Migrate legacy metawiki schemas to Event Platform - https://phabricator.wikimedia.org/T259163 (10odimitrijevic) [00:49:07] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform, and 4 others: page-links-change stream is assigning template propagation events to the wrong edits - https://phabricator.wikimedia.org/T216504 (10odimitrijevic) [00:49:09] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform, and 4 others: Migrate WikibaseTermboxInteraction EventLogging Schema to new EventPlatform thingy - https://phabricator.wikimedia.org/T290303 (10odimitrijevic) [00:49:16] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Engineering, 10tech-decision-forum: MediaWiki Events as a Source of Truth - Problem Statement - https://phabricator.wikimedia.org/T291120 (10odimitrijevic) [00:49:36] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Users should run explicit commands to materialize schema versions, rather than using magic git hooks - https://phabricator.wikimedia.org/T290074 (10odimitrijevic) [00:49:48] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Observability-Logging, and 3 others: Integrate Event Platform and ECS logs - https://phabricator.wikimedia.org/T291645 (10odimitrijevic) [00:49:58] 10Analytics, 10Analytics-EventLogging, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10odimitrijevic) [00:50:00] 10Analytics, 10Data-Engineering, 10Discovery, 10Event-Platform, and 2 others: Avoid accepting Kafka messages with whacky timestamps - https://phabricator.wikimedia.org/T282887 (10odimitrijevic) [00:50:02] 10Analytics, 10Data-Engineering, 10Event-Platform: Discussion of Event Driven Systems - https://phabricator.wikimedia.org/T290203 (10odimitrijevic) [00:50:35] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Automate creation of sqoop list of wikis to import data for from sitematrix - https://phabricator.wikimedia.org/T190700 (10odimitrijevic) [00:50:41] 10Analytics, 10Data-Engineering, 10DBA, 10Event-Platform, 10WMF-Architecture-Team: Consistent MediaWiki state change events | MediaWiki events as source of truth - https://phabricator.wikimedia.org/T120242 (10odimitrijevic) [00:59:23] 10Analytics, 10Data-Engineering, 10Event-Platform: Mediaviewer views should be reworked to be an eventlogging event - https://phabricator.wikimedia.org/T239630 (10odimitrijevic) [01:00:07] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Event-Platform, and 4 others: page-links-change stream is assigning template propagation events to the wrong edits - https://phabricator.wikimedia.org/T216504 (10odimitrijevic) [01:00:26] 10Analytics, 10Data-Engineering, 10Event-Platform: Mediaviewer views should be reworked to be an eventlogging event - https://phabricator.wikimedia.org/T239630 (10odimitrijevic) [01:02:38] 10Data-Engineering: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912 (10odimitrijevic) [02:12:56] 10Data-Engineering: Access to aggregate User Agent statistics - https://phabricator.wikimedia.org/T298912 (10GeneralNotability) [03:40:15] (EventgateLoggingExternalLatency) firing: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org [03:45:15] (EventgateLoggingExternalLatency) resolved: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org [07:46:35] !log `systemctl reset-failed product-analytics-movement-metrics.service` on stat1007 [07:46:36] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:58:16] (EventgateLoggingExternalLatency) firing: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org [08:03:15] (EventgateLoggingExternalLatency) resolved: Elevated latency for POST events on eventgate-logging-external in eqiad. - https://wikitech.wikimedia.org/wiki/Event_Platform/EventGate - https://grafana.wikimedia.org/d/ZB39Izmnz/eventgate?viewPanel=79&orgId=1&var-service=eventgate-logging-external - https://alerts.wikimedia.org [08:33:20] 10Data-Engineering-Kanban, 10SRE, 10SRE-Access-Requests: Requesting access to the data engineering team resources for Antoine Qu'hen - https://phabricator.wikimedia.org/T298657 (10Antoine_Quhen) Thanks @cmooney My analytics-admin access is working as it should. For example, I can now access an-launcher1002... [09:51:24] btullis: o/ I was about to merge https://gerrit.wikimedia.org/r/c/operations/puppet/+/742747 but then I wondered if you are interested in doing it (not sure how much exposure you had up to now about varnishkafka) [09:51:33] no rush, we can do it anytime, even next week [09:52:17] the rollout that I have in mind is simple: disable puppet on all cp nodes (alerting traffic beforehand), test on a few nodes, check metrics, re-enable puppet [09:59:09] I'd be more than happy to do it, but I've got quite a bit on at the moment. I haven't even got around to T296990, which I said I'd do early this year. [09:59:09] T296990: Move kafka-jumbo to a fixed uid/gid - https://phabricator.wikimedia.org/T296990 [10:15:13] btullis: yes yes anytime, even next week, it can wait :) [10:16:16] I can take care of the fixed uid if you want! [10:17:35] OK, thanks. If you take the fixed uid ticket back, swap it for the varnishkafka certificates. I'll do that one instead. [10:17:42] Is that OK? [10:19:18] btullis: sure sure, to be clear - only if you have time and are interested, I can do both as well. Nothing urgent :) [10:21:07] I am always interested. :-) [10:27:29] ack! [10:27:59] so is it ok if I apply the fixed uids to kafka jumbo now? it will require some stop/start [10:28:14] (+ recover etc..) [10:34:19] Yes, please feel free to go ahead. [10:34:51] (03PS2) 10Awight: Sanitize additional event streams [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) [10:35:08] (03PS3) 10Awight: Sanitize additional event streams [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) [10:36:54] (03CR) 10Awight: "Thanks for challenging the assumptions here, I've pared down the data to a bare minimum and happy to continue if you spot anything else. " [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) (owner: 10Awight) [11:25:13] !log move kafka-jumbo nodes to fixed kafka uid/gid [11:25:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:03:37] I did kafka-jumbo100[6-9], going to lunch and then I'll do 100[1-5] :) [12:04:08] Cool. Thanks elukey. [12:18:34] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] Sanitize additional event streams (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) (owner: 10Awight) [12:19:47] (03CR) 10Awight: Sanitize additional event streams (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) (owner: 10Awight) [12:20:18] (03PS4) 10Awight: Sanitize additional event streams [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) [12:25:26] 10Data-Engineering, 10Data-Engineering-Kanban, 10observability, 10serviceops, 10Patch-For-Review: Move kafka-jumbo to a fixed uid/gid - https://phabricator.wikimedia.org/T296990 (10BTullis) a:05BTullis→03elukey @elukey has offered to carry out this work after all. [12:34:04] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] Sanitize additional event streams (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) (owner: 10Awight) [12:40:18] (03CR) 10Awight: Sanitize additional event streams (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/747065 (https://phabricator.wikimedia.org/T297679) (owner: 10Awight) [13:26:14] 10Data-Engineering-Kanban, 10SRE, 10SRE-Access-Requests: Requesting access to the data engineering team resources for Antoine Qu'hen - https://phabricator.wikimedia.org/T298657 (10cmooney) 05Open→03Resolved Super thanks for confirming! And other problems just let me know :) [13:48:59] 10Data-Engineering, 10observability, 10serviceops, 10Patch-For-Review: Move kafka clusters to fixed uid/gid - https://phabricator.wikimedia.org/T296982 (10elukey) [13:49:11] 10Data-Engineering, 10observability, 10serviceops, 10Patch-For-Review: Move kafka clusters to fixed uid/gid - https://phabricator.wikimedia.org/T296982 (10elukey) [13:49:20] 10Data-Engineering, 10Data-Engineering-Kanban, 10observability, 10serviceops, 10Patch-For-Review: Move kafka-jumbo to a fixed uid/gid - https://phabricator.wikimedia.org/T296990 (10elukey) 05Stalled→03Resolved Cluster running with new gid/uid! [14:01:32] kafka jumbo with fixed uid/gid for kafka [14:01:34] :) [14:06:14] thanks luca! [14:32:58] 10Analytics-Radar, 10WMDE-GeoInfo-FocusArea: Review existing dashboards and metrics for maps - https://phabricator.wikimedia.org/T295315 (10awight) 05Open→03Resolved a:03awight [14:41:40] (03PS1) 10Ladsgroup: Move away from db-eqiad.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753465 (https://phabricator.wikimedia.org/T260297) [14:50:22] (03PS1) 10Ladsgroup: Fix outdated documentation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753468 (https://phabricator.wikimedia.org/T260297) [14:52:28] (03PS1) 10Ladsgroup: Fix outdated documentation [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/753470 (https://phabricator.wikimedia.org/T260297) [14:53:12] (03CR) 10jerkins-bot: [V: 04-1] Fix outdated documentation [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/753470 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [14:54:00] Hi, I have two easy patches for review: https://gerrit.wikimedia.org/r/753468 and https://gerrit.wikimedia.org/r/753470 [14:54:09] It would be great if they get reviewed [14:55:12] (03CR) 10Btullis: [C: 03+1] Fix outdated documentation [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753468 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [14:59:43] Not sure why that jenkins failed on the second one. Perhaps it would work on a 2nd run. [14:59:43] This remonds me that I need to get added to the analytics group in gerrit. I will send a request to rel-eng. [15:07:07] 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics, 10Structured-Data-Backlog, and 3 others: Write an Airflow job converting commons structured data dump to Hive - https://phabricator.wikimedia.org/T299059 (10JAllemandou) [15:37:54] (03CR) 10Lucas Werkmeister (WMDE): [C: 03+2] "Looks like the old file is already gone, so I’ll just believe you that it used to contain whatever db-production now contains :/" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753465 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [15:39:35] (03Merged) 10jenkins-bot: Move away from db-eqiad.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753465 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [15:41:09] (03PS1) 10Ladsgroup: Move away from db-eqiad.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753489 (https://phabricator.wikimedia.org/T260297) [15:41:13] (03CR) 10Ladsgroup: [C: 03+2] Move away from db-eqiad.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753489 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [15:42:34] (03Merged) 10jenkins-bot: Move away from db-eqiad.php [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/753489 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [16:35:05] (03CR) 10Sharvaniharan: [C: 03+2] "Thank you for the review @Ottomata" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747967 (https://phabricator.wikimedia.org/T290920) (owner: 10Sharvaniharan) [16:36:11] (03Merged) 10jenkins-bot: MEP schema for IOS Notification Interaction [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747967 (https://phabricator.wikimedia.org/T290920) (owner: 10Sharvaniharan) [16:38:17] (03CR) 10Sharvaniharan: [C: 03+2] "Thank you for the review @Ottomata" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747226 (https://phabricator.wikimedia.org/T297818) (owner: 10Sharvaniharan) [16:39:11] (03Merged) 10jenkins-bot: Android MEP schema for customizing toolbar [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747226 (https://phabricator.wikimedia.org/T297818) (owner: 10Sharvaniharan) [17:30:51] 10Analytics-Clusters, 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra, and 2 others: Investigate high levels of garbage collection on new AQS nodes - https://phabricator.wikimedia.org/T298516 (10BTullis) I'm starting to work with `mat` and its `ParseHeapDump.sh` script on the existing dump file.... [17:45:27] 10Data-Engineering, 10serviceops, 10Patch-For-Review: Move kafka clusters to fixed uid/gid - https://phabricator.wikimedia.org/T296982 (10herron) [18:35:34] 10Analytics-Radar, 10Anti-Harassment, 10CheckUser, 10Privacy Engineering, and 4 others: Deal with Google Chrome User-Agent deprecation - https://phabricator.wikimedia.org/T242825 (10brion) [19:00:53] (03CR) 10Joal: [V: 03+2 C: 03+2] "Ok for me - merging! Thank you Amir :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753468 (https://phabricator.wikimedia.org/T260297) (owner: 10Ladsgroup) [19:06:05] 10Data-Engineering, 10Data-Engineering-Kanban, 10Airflow: Create a SparkSQL runner for cluster-mode deployment - https://phabricator.wikimedia.org/T297427 (10JAllemandou) 05Open→03Resolved [19:06:21] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Repair and reload cassandra2 mediarequest_per_file data table - https://phabricator.wikimedia.org/T291470 (10JAllemandou) 05In progress→03Resolved [19:06:23] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Epic: Check AQS with cassandra (serving + data) - https://phabricator.wikimedia.org/T290068 (10JAllemandou) [19:08:05] 10Data-Engineering, 10Data-Engineering-Kanban, 10Generated Data Platform, 10Image-Suggestion-API, 10Image-Suggestions: Update HiveToCassandra for variable substitution and HQL from files loading - https://phabricator.wikimedia.org/T297934 (10JAllemandou) a:03JAllemandou [19:11:50] (03CR) 10Joal: Update structured_data dumps parsing job (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747508 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [19:12:07] (03PS5) 10Joal: Update structured_data dumps parsing job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747508 (https://phabricator.wikimedia.org/T258834) [19:18:50] (03PS5) 10Joal: Update wikidata_entity table create and oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740589 (https://phabricator.wikimedia.org/T258834) [19:19:43] (03PS6) 10Joal: Update wikidata_entity table create and oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740589 (https://phabricator.wikimedia.org/T258834) [19:20:44] (03PS5) 10Joal: Add structured_data.commons_entity table create [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740590 (https://phabricator.wikimedia.org/T258834) [19:24:18] (03CR) 10Joal: [C: 03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747508 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [19:32:12] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Add gobblin metrics per kafka-topic-partition - https://phabricator.wikimedia.org/T294422 (10JAllemandou) 05Open→03Invalid the evolution of analysis on gobblin metrics makes this task invalid. The valid one is T294420. [19:32:15] 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Push Gobblin import metrics to Prometheus and add alerts on some critical imports - https://phabricator.wikimedia.org/T286503 (10JAllemandou) [19:33:08] (03Merged) 10jenkins-bot: Update structured_data dumps parsing job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747508 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [19:33:25] (03PS1) 10Joal: Update readme.md to deploy v0.1.24 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/753542 [19:34:24] (03CR) 10Joal: Update refine netflow_augment transform function (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747561 (https://phabricator.wikimedia.org/T263277) (owner: 10Joal) [19:34:47] (03PS3) 10Joal: Update refine netflow_augment transform function [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747561 (https://phabricator.wikimedia.org/T263277) [19:35:06] (03CR) 10Joal: [C: 03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747561 (https://phabricator.wikimedia.org/T263277) (owner: 10Joal) [19:47:17] (03PS1) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [19:48:58] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [19:49:30] (03Merged) 10jenkins-bot: Update refine netflow_augment transform function [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/747561 (https://phabricator.wikimedia.org/T263277) (owner: 10Joal) [19:52:00] (03PS2) 10Joal: Update readme.md to deploy v0.1.24 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/753542 [19:52:39] (03CR) 10Joal: [C: 03+2] "Merging for deploy" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/753542 (owner: 10Joal) [19:54:21] (03PS2) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [19:55:59] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [20:03:34] (03PS3) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [20:04:21] (03Merged) 10jenkins-bot: Update readme.md to deploy v0.1.24 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/753542 (owner: 10Joal) [20:04:55] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [20:09:38] Starting build #101 for job analytics-refinery-maven-release-docker [20:12:42] (03PS4) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [20:16:02] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [20:19:34] (03PS5) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [20:21:16] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [20:28:14] Project analytics-refinery-maven-release-docker build #101: 09SUCCESS in 18 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release-docker/101/ [20:35:23] (03PS6) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [20:35:50] !log refinery-source v0.1.24 released on archiva [20:35:52] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:37:10] Starting build #60 for job analytics-refinery-update-jars-docker [20:37:28] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [20:37:35] (03PS1) 10Maven-release-user: Add refinery-source jars for v0.1.24 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753558 [20:37:35] Project analytics-refinery-update-jars-docker build #60: 09SUCCESS in 25 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars-docker/60/ [20:38:54] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753558 (owner: 10Maven-release-user) [20:39:41] (03PS7) 10Joal: Update wikidata_entity table create and oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740589 (https://phabricator.wikimedia.org/T258834) [20:40:13] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740589 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [20:40:53] (03PS6) 10Joal: Add structured_data.commons_entity table create [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740590 (https://phabricator.wikimedia.org/T258834) [20:41:26] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740590 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [20:42:01] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/740590 (https://phabricator.wikimedia.org/T258834) (owner: 10Joal) [20:43:03] (03PS2) 10Joal: Add network_internal_flows to gobblin netflow job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/748099 (https://phabricator.wikimedia.org/T263277) [20:43:29] (03CR) 10Joal: [V: 03+2 C: 03+2] "Merging for deploy" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/748099 (https://phabricator.wikimedia.org/T263277) (owner: 10Joal) [20:46:10] (03PS7) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [20:46:58] !log Deploying refinery with scap [20:46:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:47:38] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [21:13:11] !log Deploying refinery to HDFS [21:13:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [21:16:37] (03PS8) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [21:17:16] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [21:22:32] (03PS9) 10AGueyte: WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) [21:23:56] (03CR) 10jerkins-bot: [V: 04-1] WIP: Basic ipinfo instrument setup [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/753548 (https://phabricator.wikimedia.org/T296415) (owner: 10AGueyte) [21:50:18] (03PS1) 10Joal: Correct netflow gobblin job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/753563 (https://phabricator.wikimedia.org/T263277)