[00:01:42] 10Analytics-Clusters, 10Data-Engineering: Define a list of exactly which alerts should page the Analytics team in VictorOps - https://phabricator.wikimedia.org/T296552 (10odimitrijevic) p:05Triage→03Medium [00:03:17] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Product-Analytics: Develop comprehensive process, guidelines, and roles for Event Platform stream sanitization - https://phabricator.wikimedia.org/T276955 (10odimitrijevic) [00:04:29] 10Analytics, 10Data-Engineering: Check home/HDFS leftovers of christinedk - https://phabricator.wikimedia.org/T297461 (10odimitrijevic) [00:04:31] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Wikistats feature: run http server in npm run dev - https://phabricator.wikimedia.org/T296890 (10odimitrijevic) [00:04:33] 10Analytics, 10Data-Engineering, 10Stewards-and-global-tools: Collect information about users affected by blocks - https://phabricator.wikimedia.org/T297051 (10odimitrijevic) [00:16:10] RECOVERY - Check unit status of monitor_refine_eventlogging_analytics on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_eventlogging_analytics https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [00:45:10] 10Data-Engineering, 10WMF-NDA: Reindex netflow data before 2020-09-04 to sanitize dimensions - https://phabricator.wikimedia.org/T296206 (10odimitrijevic) [00:51:47] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of christinedk - https://phabricator.wikimedia.org/T297461 (10odimitrijevic) p:05Triage→03High [00:59:47] 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics (Kanban): Test log file and error notification - https://phabricator.wikimedia.org/T295733 (10odimitrijevic) @EChetty putting this on your radar. If I understand the ticket correctly it adds job status observability/notifications to "producti... [01:21:35] 10Analytics, 10Data-Engineering, 10Pageviews-API, 10User-Elukey: Improve user management for AQS Cassandra - https://phabricator.wikimedia.org/T142073 (10odimitrijevic) [01:25:13] 10Analytics, 10Data-Engineering: Use inclusive language in code for private analytics infrastructure - https://phabricator.wikimedia.org/T280268 (10odimitrijevic) [01:27:44] 10Analytics, 10Data-Engineering, 10User-Elukey: Only hdfs (or authenticated user) should be able to run Druid indexing jobs - https://phabricator.wikimedia.org/T192959 (10odimitrijevic) [01:27:45] 10Analytics, 10Data-Engineering: Refinery python code should use anaconda-wmf - https://phabricator.wikimedia.org/T286743 (10odimitrijevic) [01:32:46] 10Analytics, 10Data-Engineering: Use corosync and pacemaker for presto coordinator active/standby configuration - https://phabricator.wikimedia.org/T287967 (10odimitrijevic) [01:34:08] 10Data-Engineering, 10Observability-Metrics, 10SRE, 10Superset: statsd and gunicorn metrics for superset - https://phabricator.wikimedia.org/T293761 (10odimitrijevic) [01:35:47] 10Analytics, 10Data-Engineering, 10Discovery, 10Event-Platform, and 2 others: Avoid accepting Kafka messages with whacky timestamps - https://phabricator.wikimedia.org/T282887 (10odimitrijevic) [01:38:29] 10Analytics, 10Data-Engineering: SLF4J logspam when using hadoop command-line clients - https://phabricator.wikimedia.org/T276240 (10odimitrijevic) [01:43:36] 10Data-Engineering: Prometheus metrics for Spark 3 - https://phabricator.wikimedia.org/T298666 (10odimitrijevic) [01:45:06] 10Analytics: Add Spark metrics - https://phabricator.wikimedia.org/T271513 (10odimitrijevic) We are close to starting Spark 3 upgrade, and I would rather we spent effort making that migration happen. Created https://phabricator.wikimedia.org/T298666 [01:45:15] 10Analytics: Add Spark metrics - https://phabricator.wikimedia.org/T271513 (10odimitrijevic) 05Open→03Declined [01:46:21] 10Analytics: Investigate adding user-friendly testing functionality to Reportupdater - https://phabricator.wikimedia.org/T156523 (10odimitrijevic) 05Open→03Declined Plan is to move away from reportupdater to airflow. [01:46:23] 10Analytics: reportupdater TLC - https://phabricator.wikimedia.org/T193167 (10odimitrijevic) [01:47:49] 10Analytics, 10Data-Engineering: Refine: Use Spark SQL instead of Hive JDBC - https://phabricator.wikimedia.org/T209453 (10odimitrijevic) [01:49:15] 10Analytics, 10Data-Engineering, 10Airflow: Alarms for virtualpageview should exist (probably in oozie) for jobs that have been idle too long - https://phabricator.wikimedia.org/T213716 (10odimitrijevic) [01:52:10] 10Analytics, 10Data-Engineering: Update geocode UDF to NOT lookup some addresses - https://phabricator.wikimedia.org/T271340 (10odimitrijevic) @JAllemandou Can you please add a few more details: which system? Maybe location of the code where this is happening and link to the corresponding logs. Wondering if th... [01:54:11] 10Analytics, 10Data-Engineering, 10SRE: Trash cleanup cron spams on an-test hosts - https://phabricator.wikimedia.org/T286442 (10odimitrijevic) [01:57:13] 10Analytics, 10Data-Engineering: Analytics coordinator failover improvements - https://phabricator.wikimedia.org/T280905 (10odimitrijevic) [02:01:49] 10Analytics-Radar, 10Generated Data Platform, 10Pageviews-API, 10Tool-Pageviews: 429 Too Many Requests hit despite throttling to 100 req/sec - https://phabricator.wikimedia.org/T219857 (10odimitrijevic) [02:03:49] 10Analytics, 10User-Elukey: Add python urllib_kerberos package to analytics clients - https://phabricator.wikimedia.org/T240891 (10odimitrijevic) 05Open→03Declined Since we are moving away from oozie to airflow declining task. [02:06:28] 10Analytics, 10Event-Platform, 10User-Elukey: Create EventStream's equivalent to irc.wikimedia.org's #central channel - https://phabricator.wikimedia.org/T240182 (10odimitrijevic) [02:08:49] 10Analytics, 10Data-Engineering, 10Observability-Logging: Indexing errors / malformed logs for aqs on cassandra timeout - https://phabricator.wikimedia.org/T262920 (10odimitrijevic) [02:10:27] 10Analytics, 10Data-Engineering, 10Superset: Remove support for the (deprecated) Druid datasources (in favor of Druid Tables) on Superset - https://phabricator.wikimedia.org/T263972 (10odimitrijevic) [02:12:21] 10Analytics, 10Data-Engineering: Productionize HDFS fsimage data analysis job - https://phabricator.wikimedia.org/T261283 (10odimitrijevic) [02:14:34] 10Analytics, 10Patch-For-Review: Migrate pagecounts-ez generation to hadoop - https://phabricator.wikimedia.org/T192474 (10odimitrijevic) 05Open→03Declined [02:14:37] 10Analytics, 10Product-Analytics: Creation of canonical pageview dumps for users to download - https://phabricator.wikimedia.org/T251777 (10odimitrijevic) [02:17:54] 10Analytics, 10Data-Engineering, 10Product-Analytics: Sqoop more tables for mediawiki in monthly schedule - https://phabricator.wikimedia.org/T198983 (10odimitrijevic) [02:19:33] 10Analytics, 10Event-Platform: Provide easy way to know the content language of a wiki about which there is an event log line - https://phabricator.wikimedia.org/T65204 (10odimitrijevic) 05Open→03Declined [02:20:52] 10Analytics, 10Data-Engineering: Crunch and delete many old dumps logs - https://phabricator.wikimedia.org/T280678 (10odimitrijevic) [02:20:54] 10Analytics, 10Data-Engineering: Address jackson version security vulnerabilities in refinery-source - https://phabricator.wikimedia.org/T272058 (10odimitrijevic) [02:20:56] 10Analytics, 10Data-Engineering: Add Authentication/Encryption to Kafka Jumbo's clients - https://phabricator.wikimedia.org/T250146 (10odimitrijevic) [02:20:58] 10Analytics, 10Data-Engineering, 10Data-release, 10Privacy Engineering, and 2 others: Apache Beam go prototype code for DP evaluation - https://phabricator.wikimedia.org/T280385 (10odimitrijevic) [02:21:02] 10Analytics, 10Data-Engineering: Add authentication and encryption to Druid Analytics clients - https://phabricator.wikimedia.org/T250484 (10odimitrijevic) [02:21:04] 10Analytics, 10Data-Engineering: Build a process to check permissions when changing datasets from non-PII to PII - https://phabricator.wikimedia.org/T273818 (10odimitrijevic) [02:21:06] 10Analytics, 10Data-Engineering: Defining a better authentication scheme for Druid and Presto - https://phabricator.wikimedia.org/T241189 (10odimitrijevic) [02:21:07] 10Analytics, 10Data-Engineering: Idea: Add 'top X bigger than Y' sanitization method to EL-to-Druid - https://phabricator.wikimedia.org/T251145 (10odimitrijevic) [02:21:10] 10Analytics, 10Data-Engineering, 10Epic: Sanitize pageview_hourly - https://phabricator.wikimedia.org/T114675 (10odimitrijevic) [02:21:12] 10Analytics, 10Data-Engineering, 10Data-release, 10Privacy Engineering, and 2 others: ApacheBeam prototype for DP noise addition with pageview privacy units on top of Spark - https://phabricator.wikimedia.org/T282195 (10odimitrijevic) [02:22:11] 10Data-Engineering-Radar, 10Data-release, 10Privacy Engineering, 10Research, 10Privacy: ApacheBeam prototype for DP noise addition with pageview privacy units on top of Spark - https://phabricator.wikimedia.org/T282195 (10odimitrijevic) [02:33:11] 10Analytics: Consider converting AQS to TypeScript - https://phabricator.wikimedia.org/T206269 (10odimitrijevic) 05Open→03Declined [02:33:13] 10Analytics-Radar: Evaluate using TypeScript on node projects - https://phabricator.wikimedia.org/T206268 (10odimitrijevic) [02:34:21] 10Analytics: Better publishing of Annotations about Data Issues - https://phabricator.wikimedia.org/T142408 (10odimitrijevic) 05Open→03Declined [02:40:54] 10Analytics: Rewrite cassandra loading In spark - https://phabricator.wikimedia.org/T249735 (10odimitrijevic) 05Open→03Declined This has been implemented via a different task. [02:49:57] 10Analytics-Kanban: Browser reports improvements (parent task) - https://phabricator.wikimedia.org/T130405 (10odimitrijevic) [02:50:01] 10Analytics: Browser Reports. Break down "Other" bucket a little more? - https://phabricator.wikimedia.org/T131127 (10odimitrijevic) 05Open→03Declined The deprecation of User Agent header and possible move to User Headers will prompt a reworking of browser categorization. [02:50:25] 10Analytics, 10Data-Engineering, 10Wikidata: Make Wikidata item_page_link table available publicly - https://phabricator.wikimedia.org/T258514 (10odimitrijevic) [02:52:56] 10Analytics, 10Data-Engineering: Fix gobblin not writing _IMPORTED flags when runs don't overlap hours - https://phabricator.wikimedia.org/T286343 (10odimitrijevic) [03:07:36] 10Analytics, 10Data-Engineering, 10Product-Analytics: Add editors_monthly data to Druid - https://phabricator.wikimedia.org/T256719 (10odimitrijevic) [03:07:50] 10Analytics-Clusters, 10Data-Engineering, 10Data-Engineering-Kanban, 10Cassandra, and 2 others: Investigate high levels of garbage collection on new AQS nodes - https://phabricator.wikimedia.org/T298516 (10Eevans) >>! In T298516#7600211, @odimitrijevic wrote: > Could we give the upgrade a try to see if it... [03:12:33] 10Analytics, 10Analytics-Wikistats: "Page views by edition of Wikipedia" for each country - https://phabricator.wikimedia.org/T257071 (10odimitrijevic) p:05Medium→03Low [03:12:59] 10Analytics, 10Analytics-Wikistats: Metrics tooltip in detail page is not localized - https://phabricator.wikimedia.org/T287908 (10odimitrijevic) [03:13:02] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Change state to store project as an array - https://phabricator.wikimedia.org/T283624 (10odimitrijevic) [03:13:07] 10Analytics, 10Analytics-Wikistats, 10Epic: Add ability to compare wikis - https://phabricator.wikimedia.org/T283251 (10odimitrijevic) [03:13:09] 10Analytics, 10Analytics-Wikistats: Wikistats UI: legend for bar graph not visible on mobile UI - https://phabricator.wikimedia.org/T257530 (10odimitrijevic) [03:13:11] 10Analytics, 10Analytics-Wikistats: wikistats UI: language menu not usable after making language selection on mobile - https://phabricator.wikimedia.org/T257529 (10odimitrijevic) [03:13:13] 10Analytics, 10Analytics-Wikistats: Wikistats2 time related bugs - https://phabricator.wikimedia.org/T231248 (10odimitrijevic) [03:13:15] 10Analytics, 10Analytics-Wikistats: Change Wikistats UI language without reloading the page - https://phabricator.wikimedia.org/T251375 (10odimitrijevic) [03:13:17] 10Analytics, 10Analytics-Wikistats, 10I18n: Fixed time range names are forced to capitalized regardless of locale in sidebar - https://phabricator.wikimedia.org/T287910 (10odimitrijevic) [03:13:19] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering, 10Patch-For-Review: Expand Wikiselector to allow more than one wiki - https://phabricator.wikimedia.org/T285050 (10odimitrijevic) [03:13:22] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Wikistats should allow more than one project - https://phabricator.wikimedia.org/T283254 (10odimitrijevic) [03:13:25] 10Analytics, 10Analytics-Wikistats: Automate the deployment procedure of Wikistats 2 to Production - https://phabricator.wikimedia.org/T274126 (10odimitrijevic) [03:13:27] 10Analytics, 10Analytics-Wikistats: Larger system text causes problems on mobile - https://phabricator.wikimedia.org/T258273 (10odimitrijevic) [03:13:29] 10Analytics, 10Analytics-Wikistats: Javascript-less Wikistats - https://phabricator.wikimedia.org/T251979 (10odimitrijevic) [03:13:30] 10Analytics, 10Analytics-Wikistats: Wiki selector: "All wikis" is not translated - https://phabricator.wikimedia.org/T246911 (10odimitrijevic) [03:13:33] 10Analytics, 10Analytics-Wikistats: Metric widget won't stretch when content overflows - https://phabricator.wikimedia.org/T246913 (10odimitrijevic) [03:13:35] 10Analytics, 10Analytics-Wikistats: Use version of Lato that renders non-roman alphabets - https://phabricator.wikimedia.org/T246777 (10odimitrijevic) [03:13:37] 10Analytics, 10Analytics-Wikistats, 10I18n, 10RTL: Support right-to-left languages in Wikistats - https://phabricator.wikimedia.org/T251376 (10odimitrijevic) [03:13:39] 10Analytics, 10Analytics-Wikistats: [Wikistats2] Normalize pageviews per country by population - https://phabricator.wikimedia.org/T242621 (10odimitrijevic) [03:13:41] 10Analytics, 10Analytics-Wikistats: Incorporate Erik's great work on WiViVi into Wikistats 2 - https://phabricator.wikimedia.org/T229665 (10odimitrijevic) [03:13:43] 10Analytics, 10Analytics-Wikistats: Make MapChart load async instead of parts of MapChart - https://phabricator.wikimedia.org/T229382 (10odimitrijevic) [03:13:45] 10Analytics, 10Analytics-Wikistats: Render dashboard graphs with Canvas instead of SVG - https://phabricator.wikimedia.org/T224871 (10odimitrijevic) [03:13:47] 10Analytics, 10Analytics-Wikistats: Bug when toggling Chrome mobile view - https://phabricator.wikimedia.org/T217559 (10odimitrijevic) [03:13:49] 10Analytics, 10Analytics-Wikistats: Wikistats2 metric: top article creators - https://phabricator.wikimedia.org/T210423 (10odimitrijevic) [03:13:51] 10Analytics, 10Analytics-Wikistats: Serve legacy code only to legacy browsers - https://phabricator.wikimedia.org/T207311 (10odimitrijevic) [03:13:53] 10Analytics, 10Analytics-Wikistats: Under construction page in wikistats to take site down - https://phabricator.wikimedia.org/T192847 (10odimitrijevic) [03:13:55] 10Analytics, 10Analytics-Wikistats: wikistats , move to webpack 5 - https://phabricator.wikimedia.org/T188759 (10odimitrijevic) [03:13:57] 10Analytics, 10Analytics-Wikistats: We need better UI addressing when are metrics publicly available - https://phabricator.wikimedia.org/T226403 (10odimitrijevic) [03:13:59] 10Analytics, 10Analytics-Wikistats: Basic Wiki Numbers - https://phabricator.wikimedia.org/T224722 (10odimitrijevic) [03:14:01] 10Analytics, 10Analytics-Wikistats: Increase topojson resolution: Singapore does not appear on wikistats map - https://phabricator.wikimedia.org/T199571 (10odimitrijevic) [03:14:03] 10Analytics, 10Analytics-Wikistats: [Wikistats Design] integrate browser dashboard data into wikistats - https://phabricator.wikimedia.org/T198333 (10odimitrijevic) [03:14:05] 10Analytics, 10Analytics-Wikistats: Gather all constants related to mobile/responsiveness in config - https://phabricator.wikimedia.org/T190339 (10odimitrijevic) [03:14:07] 10Analytics, 10Analytics-Wikistats: Wikistats2 and SEO - https://phabricator.wikimedia.org/T192172 (10odimitrijevic) [03:14:09] 10Analytics, 10Analytics-Wikistats: Easter Egg: wikistats classic style on wikistats 2.0 - https://phabricator.wikimedia.org/T177408 (10odimitrijevic) [03:16:58] 10Analytics, 10Product-Analytics, 10Epic: Data Lake incremental Data Updates - https://phabricator.wikimedia.org/T258511 (10odimitrijevic) 05Open→03Declined This is an important long term goal. I don't think that having a high level aspirational task in phabricator will help us prioritize it. I'll add to... [03:18:04] 10Analytics, 10Analytics-Wikistats: wikistats: montly pageview dumps are not bz2 files - https://phabricator.wikimedia.org/T287684 (10odimitrijevic) 05Open→03Resolved [03:18:57] 10Analytics: spike: import all tables for mediawiki history - https://phabricator.wikimedia.org/T258418 (10odimitrijevic) 05Open→03Declined [03:20:38] 10Analytics, 10Machine-Learning-Team, 10ORES, 10editquality-modeling, 10artificial-intelligence: Generate dump of scored-revisions from 2018-2020 for Wikis except English Wikipedia - https://phabricator.wikimedia.org/T280107 (10odimitrijevic) a:05JAllemandou→03None [03:22:23] 10Analytics, 10Data-Engineering: Add "did edit" field to pageview_actor - https://phabricator.wikimedia.org/T277785 (10odimitrijevic) [03:22:27] 10Analytics, 10Data-Engineering: Fill holes in pageview-complete dumps using pageview-count-raw - https://phabricator.wikimedia.org/T290060 (10odimitrijevic) [03:22:29] 10Analytics, 10Data-Engineering, 10SRE, 10Traffic-Icebox: Add backend field to webrequest Hive table - https://phabricator.wikimedia.org/T257354 (10odimitrijevic) [03:22:33] 10Analytics, 10Data-Engineering, 10Research: Adding data from centralauth to the lake and the mediawiki_history dataset - https://phabricator.wikimedia.org/T282657 (10odimitrijevic) [03:22:37] 10Analytics, 10Data-Engineering, 10WMCZ-Stats: Review request: New datasets for WMCZ published under analytics.wikimedia.org - https://phabricator.wikimedia.org/T279567 (10odimitrijevic) [03:22:41] 10Analytics, 10Data-Engineering, 10Privacy Engineering, 10Research: Release dataset on top search engine referrers by country, device, and language - https://phabricator.wikimedia.org/T270140 (10odimitrijevic) [03:22:45] 10Analytics, 10Data-Engineering, 10Product-Analytics, 10Epic: Readership Retention: New vs. Returning Unique devices - https://phabricator.wikimedia.org/T269815 (10odimitrijevic) [03:22:49] 10Analytics, 10Data-Engineering, 10Product-Analytics: Add timestamps of important revision events to mediawiki_history - https://phabricator.wikimedia.org/T266375 (10odimitrijevic) [03:22:53] 10Analytics, 10Data-Engineering, 10All-and-every-Wiktionary: Add editors per country data for Wiktionary projects - https://phabricator.wikimedia.org/T266643 (10odimitrijevic) [03:22:57] 10Analytics, 10Data-Engineering, 10Data-release, 10Privacy Engineering, and 2 others: Evaluate a differentially private solution to release wikipedia's project-title-country data - https://phabricator.wikimedia.org/T267283 (10odimitrijevic) [03:23:01] 10Analytics, 10Data-Engineering: Mediawiki History dumps unique editors feature request - https://phabricator.wikimedia.org/T254234 (10odimitrijevic) [03:23:05] 10Analytics, 10Data-Engineering, 10Product-Analytics: Update geo-editors job to use tags and report desktop/mobile edits - https://phabricator.wikimedia.org/T223444 (10odimitrijevic) [03:23:09] 10Analytics, 10Data-Engineering: Output schema with mediawiki_history snapshots dumps - https://phabricator.wikimedia.org/T238668 (10odimitrijevic) [03:23:13] 10Analytics, 10Data-Engineering, 10Research-Backlog, 10SRE, and 2 others: Enable layered data-access and sharing for a new form of collaboration - https://phabricator.wikimedia.org/T245833 (10odimitrijevic) [03:23:19] 10Analytics, 10Data-Engineering, 10Data-release: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231 (10odimitrijevic) [03:23:23] 10Analytics, 10Data-Engineering: Create new table for 'referer' aggregated data - https://phabricator.wikimedia.org/T112284 (10odimitrijevic) [03:23:27] 10Analytics, 10Data-Engineering, 10Product-Analytics, 10Research: Use Hive/Spark timestamps in Refined event data - https://phabricator.wikimedia.org/T278467 (10odimitrijevic) [03:23:31] 10Analytics, 10Data-Engineering, 10incubator.wikimedia.org: Create dashiki dashboard / small tool to track statistics about incubated wikis - https://phabricator.wikimedia.org/T237389 (10odimitrijevic) [03:23:36] 10Analytics, 10Data-Engineering, 10Pageviews-API: Provide weekly top pageviews stats - https://phabricator.wikimedia.org/T133575 (10odimitrijevic) [03:23:39] 10Analytics, 10Data-Engineering: Add legacy per-article pagecounts data (prior to 2015) - https://phabricator.wikimedia.org/T193759 (10odimitrijevic) [03:23:44] 10Analytics, 10Data-Engineering: Aggregate pageviews to Wikidata entities - https://phabricator.wikimedia.org/T215438 (10odimitrijevic) [03:34:45] 10Analytics, 10Data-Engineering: Fill holes in pageview-complete dumps using pageview-count-raw - https://phabricator.wikimedia.org/T290060 (10odimitrijevic) p:05Medium→03Low [03:36:51] 10Analytics, 10Data-Engineering, 10Event-Platform, 10User-Elukey: Create EventStream's equivalent to irc.wikimedia.org's #central channel - https://phabricator.wikimedia.org/T240182 (10odimitrijevic) p:05Medium→03Lowest [03:38:25] 10Analytics, 10Data-Engineering: Add "did edit" field to pageview_actor - https://phabricator.wikimedia.org/T277785 (10odimitrijevic) p:05High→03Medium [03:39:15] 10Analytics, 10Data-Engineering: Add legacy per-article pagecounts data (prior to 2015) - https://phabricator.wikimedia.org/T193759 (10odimitrijevic) p:05Low→03Lowest [03:48:11] 10Analytics, 10Data-Engineering: Release wikimedia history dumps sorted by user ID and page ID - https://phabricator.wikimedia.org/T232844 (10odimitrijevic) p:05Low→03Lowest [03:49:57] 10Analytics, 10Analytics-Wikistats: Wikistats feature: run http server in npm run dev - https://phabricator.wikimedia.org/T296890 (10odimitrijevic) [03:55:06] 10Analytics, 10Analytics-Wikistats: Metrics tooltip in detail page is not localized - https://phabricator.wikimedia.org/T287908 (10odimitrijevic) [03:55:10] 10Analytics, 10Analytics-Wikistats: Change routing to accept a list of wikis in URL - https://phabricator.wikimedia.org/T283596 (10odimitrijevic) [03:55:12] 10Analytics, 10Analytics-Wikistats: Wikistats shows 0 views for April when data isn't available yet - https://phabricator.wikimedia.org/T281617 (10odimitrijevic) [03:55:15] 10Analytics, 10Analytics-Wikistats: Wikistats UI: legend for bar graph not visible on mobile UI - https://phabricator.wikimedia.org/T257530 (10odimitrijevic) [03:55:18] 10Analytics, 10Analytics-Wikistats, 10User-Samtar, 10Voice & Tone, 10good first task: Wikistats Bug - easy to understand language for pageviews - https://phabricator.wikimedia.org/T263973 (10odimitrijevic) [03:55:20] 10Analytics, 10Analytics-Wikistats: wikistats UI: language menu not usable after making language selection on mobile - https://phabricator.wikimedia.org/T257529 (10odimitrijevic) [03:55:24] 10Analytics, 10Analytics-Wikistats: [Wikistats] The permanent link is broken - https://phabricator.wikimedia.org/T245445 (10odimitrijevic) [03:55:26] 10Analytics, 10Analytics-Wikistats: Wikimedia Statistics - Confusing number formatting/rounding on the vertical axis scale on the "Page to date" statistics graph - https://phabricator.wikimedia.org/T242790 (10odimitrijevic) [03:55:28] 10Analytics, 10Analytics-Wikistats: Wikistats2 time related bugs - https://phabricator.wikimedia.org/T231248 (10odimitrijevic) [03:55:30] 10Analytics, 10Analytics-Wikistats: There are two entries for Cantonese language - https://phabricator.wikimedia.org/T215139 (10odimitrijevic) [03:55:32] 10Analytics, 10Analytics-Wikistats: Audit and address performance issues - https://phabricator.wikimedia.org/T207197 (10odimitrijevic) [03:55:34] 10Analytics, 10Analytics-Wikistats: Audit Wikistats unit testing - https://phabricator.wikimedia.org/T192836 (10odimitrijevic) [03:55:36] 10Analytics, 10Analytics-Wikistats: Wikistats 2.0 Remaining reports. - https://phabricator.wikimedia.org/T186121 (10odimitrijevic) [03:55:38] 10Analytics, 10Analytics-Wikistats: Offer map and queries by subdivisions of countries - https://phabricator.wikimedia.org/T284294 (10odimitrijevic) [03:55:40] 10Analytics, 10Analytics-Wikistats, 10I18n: Fixed time range names are forced to capitalized regardless of locale in sidebar - https://phabricator.wikimedia.org/T287910 (10odimitrijevic) [03:55:42] 10Analytics, 10Analytics-Wikistats: Automate the deployment procedure of Wikistats 2 to Production - https://phabricator.wikimedia.org/T274126 (10odimitrijevic) [03:55:44] 10Analytics, 10Analytics-Wikistats: Search bar from stats.wikimedia.org displays characters poorly on iOS 12 - https://phabricator.wikimedia.org/T269827 (10odimitrijevic) [03:55:46] 10Analytics, 10Analytics-Wikistats: Javascript-less Wikistats - https://phabricator.wikimedia.org/T251979 (10odimitrijevic) [03:55:48] 10Analytics, 10Analytics-Wikistats: Wikistats New Feature - regioanl and group evaluations - https://phabricator.wikimedia.org/T264512 (10odimitrijevic) [03:55:50] 10Analytics, 10Analytics-Wikistats: Add proper trend numbers to wikistats metrics - https://phabricator.wikimedia.org/T251813 (10odimitrijevic) [03:55:52] 10Analytics, 10Analytics-Wikistats, 10I18n, 10RTL: Support right-to-left languages in Wikistats - https://phabricator.wikimedia.org/T251376 (10odimitrijevic) [03:55:55] 10Analytics, 10Analytics-Wikistats: Metric widget won't stretch when content overflows - https://phabricator.wikimedia.org/T246913 (10odimitrijevic) [03:55:57] 10Analytics, 10Analytics-Wikistats: Wiki selector: "All wikis" is not translated - https://phabricator.wikimedia.org/T246911 (10odimitrijevic) [03:55:59] 10Analytics, 10Analytics-Wikistats: Use version of Lato that renders non-roman alphabets - https://phabricator.wikimedia.org/T246777 (10odimitrijevic) [03:56:01] 10Analytics, 10Analytics-Wikistats: Wikistats New Feature-Country pageview breakdown by language - https://phabricator.wikimedia.org/T250001 (10odimitrijevic) [03:56:03] 10Analytics, 10Analytics-Wikistats: Wikistats New Feature - bot edits / new articles - https://phabricator.wikimedia.org/T241922 (10odimitrijevic) [03:56:05] 10Analytics, 10Analytics-Wikistats: Incorporate Erik's great work on WiViVi into Wikistats 2 - https://phabricator.wikimedia.org/T229665 (10odimitrijevic) [03:56:07] 10Analytics, 10Analytics-Wikistats: [Wikistats2] Normalize pageviews per country by population - https://phabricator.wikimedia.org/T242621 (10odimitrijevic) [03:56:09] 10Analytics, 10Analytics-Wikistats: Make MapChart load async instead of parts of MapChart - https://phabricator.wikimedia.org/T229382 (10odimitrijevic) [03:56:11] 10Analytics, 10Analytics-Wikistats: 'All" time range does not transfer well across metrics - https://phabricator.wikimedia.org/T227038 (10odimitrijevic) [03:56:13] 10Analytics, 10Analytics-Wikistats: Render dashboard graphs with Canvas instead of SVG - https://phabricator.wikimedia.org/T224871 (10odimitrijevic) [03:56:15] 10Analytics, 10Analytics-Wikistats: Siteviews of all Wikipedias per month - https://phabricator.wikimedia.org/T224963 (10odimitrijevic) [03:56:17] 10Analytics, 10Analytics-Wikistats: Add an option to export the current graph into image file - https://phabricator.wikimedia.org/T219969 (10odimitrijevic) [03:56:19] 10Analytics, 10Analytics-Wikistats: Long annotations text being clipped - https://phabricator.wikimedia.org/T218846 (10odimitrijevic) [03:56:21] 10Analytics, 10Analytics-Wikistats: Bug when toggling Chrome mobile view - https://phabricator.wikimedia.org/T217559 (10odimitrijevic) [03:56:23] 10Analytics, 10Analytics-Wikistats: Wikistats2 metric: top article creators - https://phabricator.wikimedia.org/T210423 (10odimitrijevic) [03:56:25] 10Analytics, 10Analytics-Wikistats: Annotations in wikistats2 can't be split on project and language - https://phabricator.wikimedia.org/T208665 (10odimitrijevic) [03:56:27] 10Analytics, 10Analytics-Wikistats: Annotations in wikistats that are only visible on "all" time range get bundled up (probably an issue we cannot resolve until we have a more granular time range) - https://phabricator.wikimedia.org/T200020 (10odimitrijevic) [03:56:29] 10Analytics, 10Analytics-Wikistats: Serve legacy code only to legacy browsers - https://phabricator.wikimedia.org/T207311 (10odimitrijevic) [03:56:31] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Create report for "articles with most contributors" in Wikistats2 - https://phabricator.wikimedia.org/T204965 (10odimitrijevic) [03:56:37] 10Analytics, 10Analytics-Wikistats: Under construction page in wikistats to take site down - https://phabricator.wikimedia.org/T192847 (10odimitrijevic) [03:56:41] 10Analytics, 10Analytics-Wikistats: Organize annotations pages on meta by convention - https://phabricator.wikimedia.org/T194706 (10odimitrijevic) [03:56:45] 10Analytics, 10Analytics-Wikistats: wikistats , move to webpack 5 - https://phabricator.wikimedia.org/T188759 (10odimitrijevic) [03:57:03] 10Analytics, 10Analytics-Wikistats: Wikimedia Statistics - Horizontal (time) axis wrongly formatted when the option "Monthly" is choosen - https://phabricator.wikimedia.org/T290551 (10odimitrijevic) [03:57:56] 10Analytics, 10Analytics-Wikistats: Change state to store project as an array - https://phabricator.wikimedia.org/T283624 (10odimitrijevic) [03:58:05] 10Analytics, 10Analytics-Wikistats: Wikistats should allow more than one project - https://phabricator.wikimedia.org/T283254 (10odimitrijevic) [03:58:07] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Expand Wikiselector to allow more than one wiki - https://phabricator.wikimedia.org/T285050 (10odimitrijevic) [03:58:09] 10Analytics, 10Analytics-Wikistats: Larger system text causes problems on mobile - https://phabricator.wikimedia.org/T258273 (10odimitrijevic) [03:58:17] 10Analytics, 10Analytics-Wikistats: Contextualize wikistats metrics - https://phabricator.wikimedia.org/T187212 (10odimitrijevic) [03:58:19] 10Analytics, 10Analytics-Wikistats: Allow namespace selection on Top Viewed Articles - https://phabricator.wikimedia.org/T182964 (10odimitrijevic) [03:58:21] 10Analytics, 10Analytics-Wikistats: Active Editors metric per project family - https://phabricator.wikimedia.org/T188265 (10odimitrijevic) [03:58:23] 10Analytics, 10Analytics-Wikistats: Trends for editor types, and new editors in particular (in Wikistats 2.0) - https://phabricator.wikimedia.org/T186791 (10odimitrijevic) [03:58:25] 10Analytics, 10Analytics-Wikistats: "Page views by edition of Wikipedia" for each country - https://phabricator.wikimedia.org/T257071 (10odimitrijevic) [03:58:27] 10Analytics, 10Analytics-Wikistats: We need better UI addressing when are metrics publicly available - https://phabricator.wikimedia.org/T226403 (10odimitrijevic) [03:58:29] 10Analytics, 10Analytics-Wikistats: Wikistats Bug - View numbers contradictory - https://phabricator.wikimedia.org/T261565 (10odimitrijevic) [03:58:31] 10Analytics, 10Analytics-Wikistats: Split wikistats metrics out by namespace - https://phabricator.wikimedia.org/T275466 (10odimitrijevic) [03:58:33] 10Analytics, 10Analytics-Wikistats: Basic Wiki Numbers - https://phabricator.wikimedia.org/T224722 (10odimitrijevic) [03:58:35] 10Analytics, 10Analytics-Wikistats: Improve Annotations on Wikistats - https://phabricator.wikimedia.org/T207057 (10odimitrijevic) [03:58:37] 10Analytics, 10Analytics-Wikistats, 10Internet-Archive: Allow WikiStats2 to be archived by the wayback machine - https://phabricator.wikimedia.org/T206836 (10odimitrijevic) [03:58:39] 10Analytics, 10Analytics-Wikistats: [Wikistats 2] Provide metric area headings in the 'Explore Topics' dropdown - https://phabricator.wikimedia.org/T200498 (10odimitrijevic) [03:58:41] 10Analytics, 10Analytics-Wikistats: Increase topojson resolution: Singapore does not appear on wikistats map - https://phabricator.wikimedia.org/T199571 (10odimitrijevic) [03:58:43] 10Analytics, 10Analytics-Wikistats: [Wikistats Design] integrate browser dashboard data into wikistats - https://phabricator.wikimedia.org/T198333 (10odimitrijevic) [03:58:45] 10Analytics, 10Analytics-Wikistats, 10Product-Analytics: Contribution inequality graphs for Wikistats - https://phabricator.wikimedia.org/T195033 (10odimitrijevic) [03:58:47] 10Analytics, 10Analytics-Wikistats: Wikistats2 and SEO - https://phabricator.wikimedia.org/T192172 (10odimitrijevic) [03:58:49] 10Analytics, 10Analytics-Wikistats: Easter Egg: wikistats classic style on wikistats 2.0 - https://phabricator.wikimedia.org/T177408 (10odimitrijevic) [03:58:51] 10Analytics, 10Analytics-Wikistats: Gather all constants related to mobile/responsiveness in config - https://phabricator.wikimedia.org/T190339 (10odimitrijevic) [03:58:53] 10Analytics, 10Analytics-Wikistats: Add flagged revision status statistics to Wikistats 2.0 - https://phabricator.wikimedia.org/T177951 (10odimitrijevic) [03:58:56] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Improve scoping of CSS - https://phabricator.wikimedia.org/T190915 (10odimitrijevic) [03:58:57] 10Analytics, 10Analytics-Wikistats: WikiStats should recognize global bots - https://phabricator.wikimedia.org/T37196 (10odimitrijevic) [04:02:38] 10Analytics, 10Product-Analytics, 10Platform Team Initiatives (Modern Event Platform (TEC2)): Retrofit event pipeline with bot detection code - https://phabricator.wikimedia.org/T263286 (10odimitrijevic) 05Open→03Declined [04:03:14] 10Analytics, 10Analytics-Wikistats: Organize annotations pages on meta by convention - https://phabricator.wikimedia.org/T194706 (10odimitrijevic) 05Open→03Declined [04:03:17] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Beta Release: Wikistats: support annotations in graphs - https://phabricator.wikimedia.org/T178015 (10odimitrijevic) [04:05:50] 10Analytics: Create a tool checking for data presence based on file-size - https://phabricator.wikimedia.org/T256644 (10odimitrijevic) p:05High→03Low [04:10:16] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Automate creation of sqoop list of wikis to import data for from sitematrix - https://phabricator.wikimedia.org/T190700 (10odimitrijevic) [04:17:46] 10Analytics, 10Data-Engineering, 10Patch-For-Review: HiveExtensions.convertToSchema does not properly convert arrays of structs - https://phabricator.wikimedia.org/T259924 (10odimitrijevic) [04:17:48] 10Analytics, 10Data-Engineering: MW REST API Historical Data Endpoint Needs - https://phabricator.wikimedia.org/T240387 (10odimitrijevic) [04:17:52] 10Analytics, 10Data-Engineering, 10Platform Engineering: Update mediawiki-history to use new Multi-Content-Revision tables - https://phabricator.wikimedia.org/T239591 (10odimitrijevic) [04:17:54] 10Analytics, 10Data-Engineering, 10Product-Analytics: Get "edits hourly" on a daily basis - https://phabricator.wikimedia.org/T231938 (10odimitrijevic) [04:17:56] 10Analytics, 10Data-Engineering: Implement digest-only mediawiki_history_reduced dataset in spark - https://phabricator.wikimedia.org/T181703 (10odimitrijevic) [04:17:58] 10Analytics, 10Data-Engineering: Gather all data-purge into a single job - https://phabricator.wikimedia.org/T262201 (10odimitrijevic) [04:18:00] 10Analytics, 10Data-Engineering: Move FR banner-impression jobs to events - https://phabricator.wikimedia.org/T215636 (10odimitrijevic) [04:18:02] 10Analytics, 10Data-Engineering: Transform EventLoggingToDruid job to read schemas to ingest from a whitelist and process them all - https://phabricator.wikimedia.org/T202312 (10odimitrijevic) [04:18:04] 10Analytics, 10Data-Engineering: XML dumps imports into hadoop need to adapt to latest changes done in xml schema for MCR - https://phabricator.wikimedia.org/T209545 (10odimitrijevic) [04:18:06] 10Analytics, 10Data-Engineering, 10Data-Services, 10cloud-services-team (Kanban): Plan a replacement for wiki replicas that is better suited to typical OLAP use cases than the MediaWiki OLTP schema - https://phabricator.wikimedia.org/T215858 (10odimitrijevic) [04:23:48] 10Analytics, 10Data-Engineering: Make data quality stats alert only if anomalous metrics change - https://phabricator.wikimedia.org/T263030 (10odimitrijevic) [04:23:50] 10Analytics, 10Data-Engineering, 10Anti-Harassment, 10CheckUser, and 3 others: SPIKE: consider all problems that might happen when we handle Google's privacy changes - https://phabricator.wikimedia.org/T265057 (10odimitrijevic) [04:23:52] 10Analytics, 10Data-Engineering: explicitily exclude mobile-app traffic from unique devices per domain computation - https://phabricator.wikimedia.org/T258288 (10odimitrijevic) [04:23:55] 10Analytics, 10Data-Engineering, 10Product-Analytics: Keep canonical_data.wikis updated - https://phabricator.wikimedia.org/T241741 (10odimitrijevic) [04:23:57] 10Analytics, 10Data-Engineering: actor_signature_per_project_family does not work for apps - https://phabricator.wikimedia.org/T258101 (10odimitrijevic) [04:23:59] 10Analytics, 10Data-Engineering: Join slot, content, revision, and page once on load - https://phabricator.wikimedia.org/T240413 (10odimitrijevic) [04:24:01] 10Analytics, 10Data-Engineering: Improve pageview automated traffic detection heuristics - https://phabricator.wikimedia.org/T280565 (10odimitrijevic) [04:24:03] 10Analytics, 10Data-Engineering, 10Product-Analytics, 10Pageviews-Anomaly: Too many views to Skathi (moon) on enwiki - https://phabricator.wikimedia.org/T280844 (10odimitrijevic) [04:24:05] 10Analytics, 10Data-Engineering, 10Product-Analytics: Top read repeats - https://phabricator.wikimedia.org/T280011 (10odimitrijevic) [04:24:07] 10Analytics, 10Data-Engineering: Odd behavior in unique device counts - https://phabricator.wikimedia.org/T276472 (10odimitrijevic) [04:24:09] 10Analytics, 10Data-Engineering: Filter out .wikidata requests from pageview definition - https://phabricator.wikimedia.org/T273215 (10odimitrijevic) [04:24:11] 10Analytics, 10Data-Engineering: Add `is_last_page_event` and `is_last_user_event` fields in mediawiki_history - https://phabricator.wikimedia.org/T255890 (10odimitrijevic) [04:24:13] 10Analytics, 10Data-Engineering, 10Product-Analytics: History: mismatched historical and latest values - https://phabricator.wikimedia.org/T258967 (10odimitrijevic) [04:24:15] 10Analytics, 10Data-Engineering, 10Product-Analytics: Analyze differences between checksum-based and revert-tag based reverts in mediawiki_history - https://phabricator.wikimedia.org/T266374 (10odimitrijevic) [04:24:17] 10Analytics, 10Data-Engineering, 10Product-Analytics: Remove COUNT(*) from datasets when not useful in Superset & Turnilo - https://phabricator.wikimedia.org/T255725 (10odimitrijevic) [04:24:19] 10Analytics, 10Data-Engineering: Explore in jupyter notebook whether the raw pageview timeseries can help on outage/censhorsip automatic detection - https://phabricator.wikimedia.org/T249849 (10odimitrijevic) [04:24:21] 10Analytics, 10Data-Engineering: Unusual unknown external webrequest referers - https://phabricator.wikimedia.org/T241295 (10odimitrijevic) [04:24:23] 10Analytics, 10Data-Engineering: Anomaly detection alarms for the edit event stream - https://phabricator.wikimedia.org/T250845 (10odimitrijevic) [04:24:25] 10Analytics, 10Data-Engineering: Rename event_sanitized to event_longterm - https://phabricator.wikimedia.org/T225751 (10odimitrijevic) [04:24:27] 10Analytics, 10Data-Engineering: Include user group expiry events in mediawiki history reconstruction - https://phabricator.wikimedia.org/T225249 (10odimitrijevic) [04:24:29] 10Analytics, 10Data-Engineering, 10SRE, 10Traffic-Icebox: Increased number of webrequest sequence-numbers alarms (mostly) on upload webrequest-source - https://phabricator.wikimedia.org/T225786 (10odimitrijevic) [04:24:32] 10Analytics, 10Data-Engineering, 10CheckUser, 10Patch-For-Review, and 2 others: Schema changes for `cu_changes` and `cu_log` table - https://phabricator.wikimedia.org/T233004 (10odimitrijevic) [04:24:34] 10Analytics, 10Data-Engineering: Use historical fields in history reduced dataset - https://phabricator.wikimedia.org/T222278 (10odimitrijevic) [04:24:36] 10Analytics, 10Data-Engineering, 10Product-Analytics: Identify imported revisions in mediawiki_history - https://phabricator.wikimedia.org/T221482 (10odimitrijevic) [04:24:38] 10Analytics, 10Data-Engineering: Mediawiki History: moves counted twice in Revision - https://phabricator.wikimedia.org/T189044 (10odimitrijevic) [04:24:40] 10Analytics, 10Data-Engineering: Sesssion reconstruction - evaluate privacy threat - https://phabricator.wikimedia.org/T194058 (10odimitrijevic) [04:24:42] 10Analytics, 10Data-Engineering, 10Research-Backlog: [Open question] Improve bot identification at scale - https://phabricator.wikimedia.org/T138207 (10odimitrijevic) [04:24:44] 10Analytics, 10Data-Engineering: Investigate US traffic by state normalized by population - https://phabricator.wikimedia.org/T114469 (10odimitrijevic) [04:25:48] 10Analytics: de-duplicate archive records matching revision records in mediawiki_history - https://phabricator.wikimedia.org/T152546 (10odimitrijevic) p:05Medium→03Low [04:28:46] PROBLEM - Check unit status of monitor_refine_event_sanitized_analytics_immediate on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit monitor_refine_event_sanitized_analytics_immediate https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [04:29:19] 10Analytics: Better redirect handling for pageview API - https://phabricator.wikimedia.org/T121912 (10odimitrijevic) p:05High→03Low [04:46:40] 10Analytics, 10Data-Engineering, 10Data-Engineering-Radar, 10Product-Analytics, 10Pageviews-Anomaly: Too many views to Skathi (moon) on enwiki - https://phabricator.wikimedia.org/T280844 (10odimitrijevic) [04:49:51] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Reindex mediawiki_history_reduced with lookups - https://phabricator.wikimedia.org/T193650 (10odimitrijevic) 05Open→03Declined [04:50:48] 10Analytics, 10Analytics-Wikistats, 10Inuka-Team, 10Language-strategy, and 2 others: Have a way to show the most popular pages per country - https://phabricator.wikimedia.org/T207171 (10odimitrijevic) [04:52:23] 10Analytics, 10Data-Engineering-Radar, 10Product-Analytics, 10Pageviews-Anomaly: Too many views to Skathi (moon) on enwiki - https://phabricator.wikimedia.org/T280844 (10odimitrijevic) [05:02:40] 10Analytics, 10Data-Engineering: Investigate US traffic by state normalized by population - https://phabricator.wikimedia.org/T114469 (10odimitrijevic) 05Open→03Declined Could it be that us-east-1 AWS zone is in north virginia? Since this has been open for over 6 years I'll go ahead and close :) [05:04:24] 10Analytics, 10Data-Engineering: XML dumps imports into hadoop need to adapt to latest changes done in xml schema for MCR - https://phabricator.wikimedia.org/T209545 (10odimitrijevic) 05Open→03Declined [05:05:22] 10Analytics, 10Data-Engineering: Explore in jupyter notebook whether the raw pageview timeseries can help on outage/censhorsip automatic detection - https://phabricator.wikimedia.org/T249849 (10odimitrijevic) 05Open→03Declined [05:09:07] 10Analytics, 10Data-Engineering: Implement digest-only mediawiki_history_reduced dataset in spark - https://phabricator.wikimedia.org/T181703 (10odimitrijevic) p:05High→03Low @JAllemandou Is this still relevant? Is this something that can be marked as a good starter task provided there is benefit to doing it? [05:10:11] 10Analytics, 10Analytics-Wikistats, 10Data-Engineering: Automate creation of sqoop list of wikis to import data for from sitematrix - https://phabricator.wikimedia.org/T190700 (10odimitrijevic) p:05High→03Low [05:12:59] 10Analytics, 10Data-Engineering: Join slot, content, revision, and page once on load - https://phabricator.wikimedia.org/T240413 (10odimitrijevic) p:05High→03Low [05:15:23] 10Analytics, 10Data-Engineering: Transform EventLoggingToDruid job to read schemas to ingest from a whitelist and process them all - https://phabricator.wikimedia.org/T202312 (10odimitrijevic) @mforns I believe that this is already implemented - can the task be closed? If not please triage accordingly and modi... [05:16:10] 10Analytics, 10Data-Engineering, 10Product-Analytics: History: mismatched historical and latest values - https://phabricator.wikimedia.org/T258967 (10odimitrijevic) p:05Medium→03Low [05:29:09] 10Data-Engineering: Deprecate Analytics Tag - https://phabricator.wikimedia.org/T298671 (10odimitrijevic) [06:44:12] 10Analytics, 10Data-Engineering: Aggregate pageviews to Wikidata entities - https://phabricator.wikimedia.org/T215438 (10Sascha) Meanwhile I’ve set up [[ https://qrank.toolforge.org/ | Wikidata QRank ]], which computes this dataset on a weekly basis and offers the results for public download. So, feel free to... [07:36:15] Good morning team - I need to keep Naé home, her teacher being absent - I'll be on-and-off today, sorry for that :S [07:39:07] !log Rerun failed refine_eventlogging_analytics for mobilewikiappiosuserhistory schema, hours 2022-01-05T2[123]:00:00 and 2022-01-06T00:00:00, dropping malformed rows as discussed with schema owner [07:39:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:58:07] !log Rerun refine_event_sanitized_analytics_immediate missing hours after errors from the past days [07:58:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:09:48] Hi btullis - for when you're there - The test hive-server seems down - many failed test jobs lately - would yo mind restarting it please? [08:12:46] !log Rerun failed webrequest-load-wf-text-2022-1-6-7 [08:12:48] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:17:13] joal: bonjour, it is my fault, I am trying to test the new bigtop packages in there :( [08:17:34] Ah! no problem elukey - I just didn't link the two :) [08:17:39] I can restore its state if needed, for some reason the new pkgs fail in there [08:17:47] Mwarf :( [08:18:10] Ben was trying the upgrade and for some reason the hive metastore fails for a weird datanucleus logging issue [08:18:11] no real issue - it'll be good to have a working env when you're done, but no emergency on that [08:19:47] ack - if you have a moment I could use some help in debugging the stacktrace [08:19:54] (even later on of course) [08:20:08] sure elukey - siesta time will be better if ok for you [08:20:16] elukey: Naé is home (no teacher) [08:20:24] ah right :) [08:56:46] I was just reading about https://www.wireguard.com/ and wondering if we considered it as a VPN for our analytics tools and more. I only found one use in our code, disabling it in the kubeadm module [08:57:08] elukey: have you looked into wireguard? ^ [08:59:09] milimetric: hi :) The VPN solution in general was taken into consideration but it is not straightforward to implement in our infr [09:00:07] at the networking level, adding a VPN for production is tricky and difficult to get it right (security wise) [09:08:03] o/ [09:08:26] if anyone objects I'd like to release eventutilities (using https://integration.wikimedia.org/ci/job/wikimedia-event-utilities-maven-release-docker IIUC) [09:17:23] hm, it seemed like WireGuard made that simpler, and there are some interesting "easy" config solutions on top of it like https://docs.netmaker.org/ [09:17:40] dcausse: sorry not sure, but I don't see any reason not to [09:19:11] milimetric: ok, I don't plan to deploy it or anything it's that in its current shape I can't build a project depending on it [09:19:46] because of some annoying guava version conflicts :/ [09:21:49] milimetric: the arch graph in https://docs.netmaker.org/architecture.html is far from simple :D [09:21:54] (first picture that I checked) [09:22:45] with a VPN you'd get direct access to the prod network to visit our sites, something that is difficult to manage from the security point of view [09:33:55] dcausse: I've no objection either. [09:46:17] btullis (o/), joal - I've restored the old hive packages on an-test-coord1001, trying to reproduce the error on docker, so far I can't (usual luck) [09:46:33] I added the stacktrace in the BIGTOP's jira, let's see if anybody chimes in [09:46:51] elukey: Thanks ever so much for your help debugging this. [09:47:16] joal: Apologies for having broken it in the first place and causing the failed jobs. [09:48:00] elukey: Did you say that you had built packages as well and confirmed that they were identical to mine? [09:49:51] btullis: I have uploaded my packages and tested them, and used yours in the docker container, no difference.. I have some ideas about why this happens, but the fact that I cannot repro on docker puzzles me [09:50:28] OK, thanks. Definitely a mystery. [09:50:55] good morning! systemd-timedated.service seems to be flapping on stat1004 per icinga spam in -operations [09:51:39] taavi: ack. I will look at it now. [09:51:48] and thanks. [10:00:55] elukey: would you paste a link to the bigtop task with the stacktrace please? [10:15:13] Hi SandraEbele :) [10:15:32] hello @joal [10:18:13] joal: https://issues.apache.org/jira/browse/BIGTOP-3619 [10:35:13] Thanks elukey [10:53:46] Starting build #12 for job wikimedia-event-utilities-maven-release-docker [10:54:56] Project wikimedia-event-utilities-maven-release-docker build #12: 09SUCCESS in 1 min 10 sec: https://integration.wikimedia.org/ci/job/wikimedia-event-utilities-maven-release-docker/12/ [11:24:53] !log restarting cassandra-a service on aqs1010.eqiad.wmnet in order to troubleshoot logging. [11:24:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:40:06] 10Data-Engineering, 10Data-Engineering-Kanban: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) I have been looking into this, but have yet to find the reason that they are not making it all the way through to the Logstash dashboard. The elements of the logg... [12:22:27] !log restarting cassandra-a service on aqs1004.eqiad.wmnet in order to troubleshoot logging. [12:22:28] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:43:25] elukey: Hi! I'm available to help if you wish [12:43:56] elukey: I took a look at the stack-trace you sent to bigtop, and nothing comes to mind :S [12:50:56] joal: o/ [12:51:30] the funny thing is that I cannot repro on the bigtop environment [12:51:41] this is weird [12:51:55] I tried to adapt our settings (mariadb instead of derby, jvm 8 from our repos, etc..) [12:52:58] elukey: could it be related to changes we made to mitigate log4j issues? [12:53:18] yes yes definitely [12:54:35] I tried to check https://github.com/datanucleus/datanucleus-core/blob/4.1.0/src/main/java/org/datanucleus/util/NucleusLogger.java#L193 and it may be due to the new log4j, but I would have expected a different error in the logs [12:55:10] it is cryptic, it seems a classpath error from a first check, but if so I am not sure why it happens [12:57:13] elukey: it's actually not a simple classpath error - it comes from a "java.lang.reflect.InvocationTargetException" -> the class can't be created from reflection [13:01:04] any idea about the root cause? [13:02:42] that happens only on our dear coordinator node [13:05:46] elukey: except manual changes in package I can't think of anything else :( [13:10:42] elukey: other question - there is no version change in your test, right? As in, the version of the hive software is the same as our current one? [13:11:47] same yes [13:12:21] the patch, adapted from the hive 3.x one, is https://github.com/apache/bigtop/pull/844 [13:12:34] I filed a change to hive upstream as well (for 2.3.10) but no traction [13:13:21] at this point the only thing that I can think of is that on bigtop's container there is a config/lib/whatever that prevents the exception from happening [13:15:31] yeah, I can't think of anything else :S [13:16:13] a different version of log4j for isntance [13:20:34] same java libs installed, just made sure about it [13:25:16] interesting: https://trino.io/blog/2021/12/31/trino-2021-a-year-of-growth.html [14:29:17] 10Data-Engineering, 10Data-Engineering-Kanban: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) On second thoughts, I'm not so sure about this now. It looks like Cassandra on the new hosts isn't sending any traffic to localhost port 11514 when the service re... [14:37:21] (03CR) 10Ottomata: [C: 03+1] MEP schema for IOS Notification Interaction [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747967 (https://phabricator.wikimedia.org/T290920) (owner: 10Sharvaniharan) [14:37:48] (03CR) 10Ottomata: [C: 03+1] Android MEP schema for customizing toolbar [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/747226 (https://phabricator.wikimedia.org/T297818) (owner: 10Sharvaniharan) [14:38:15] 10Data-Engineering, 10Data-Engineering-Kanban: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) Oh, I have something! Compare: ` btullis@aqs1004:/etc/cassandra-a$ ls -lt /srv/deployment/cassandra/logstash-logback-encoder/lib/logstash-logback-encoder-4.2.jar... [14:50:26] Great catch on the logback-encoder btullis!! [15:01:29] 10Data-Engineering, 10Generated Data Platform: Set up regular-repairs for AQS cassandra cluster tables - https://phabricator.wikimedia.org/T297944 (10JAllemandou) Adding my views on the questions from Olja (reordering questions as well): > - What was the nature and percentage of missing data? After loading... [15:01:52] 10Data-Engineering: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10EYener) [15:02:20] joal: Thanks. It took me longer than I expected to find out what it was, but I learnt a lot along the way. [15:04:39] If anyone would like to review this, I'd be grateful: https://gerrit.wikimedia.org/r/c/operations/software/logstash-logback-encoder/+/751948 [15:09:26] 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) I have created a CR to add the aqs_next hosts to the deployment targets for logstash-logback-encoder. Also I've checked that @Eevans is happ... [15:53:39] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10odimitrijevic) p:05Triage→03High @razzi Did we add the sql lab permission to the main (Alpha?) role? If so then an upgrade will reset the permissions on the default superset ro... [16:11:33] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10EYener) Reading @odimitrijevic's comment, I'm going to proactively add @JMando to ask that he also have SQL Lab enabled (I will also add this to the description). Thanks! [16:12:35] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10EYener) [16:17:53] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10BTullis) I have added the sql_lab role to the accounts in superset for @EYener and @JMando . Please could you verify whether or not this works for you now? Do we have a list of wh... [16:26:26] Is anyone around who might be able to help me to troubleshoot a scap issue? I'm running this: [16:26:30] `btullis@deploy1002:/srv/deployment/cassandra/logstash-logback-encoder$ scap deploy --environment aqs -l aqs101[0:5].eqiad.wmnet "Deploying logstash-logback-encoder to production"` [16:26:45] It's failing on `git fat pull` [16:27:46] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10EYener) A temporary inconvenience @BTullis , thank you for the quick fix! I am back up and running and I'll let @JMando confirm when he gets some time. I can now see the SQL Lab dr... [16:28:35] btullis: i'm here! [16:29:30] got a full paste of error? [16:29:31] ottomata: Thanks. I get the feeling that I'm going to have to understand this more anyway. Here's a paste from the canary: https://phabricator.wikimedia.org/P18407 [16:30:12] It's repeatable. I merged this and pulled it on deploy1002 before running it: https://gerrit.wikimedia.org/r/c/operations/software/logstash-logback-encoder/+/751948 [16:30:40] The artifact appears to be in Archiva correctly: https://archiva.wikimedia.org/#artifact/net.logstash.logback/logstash-logback-encoder/4.2 [16:32:37] hm indeed. [16:32:59] https://www.irccloud.com/pastebin/0qyPqb8l/ [16:34:19] That looks ... right? [16:34:22] yes [16:34:28] Matches this: [16:34:35] btullis@aqs1010:/srv/deployment/cassandra/logstash-logback-encoder-cache/revs/fb10de1d231ec4ce65cca2f72ce2a3a4d0e644cc$ cat /srv/deployment/cassandra/logstash-logback-encoder/lib/logstash-logback-encoder-4.2.jar#$# git-fat 71ea0efb1251138d0db33850450170f77cb3d939 735565 [16:34:35] from what i can tell though btullis , git-fat wasn't ever working properly on aqs1010 [16:34:41] the previous deploy doesn't have the artifacts pulled either [16:34:49] ls -l /srv/deployment/cassandra/logstash-logback-encoder-cache/revs/7fdbd6a98d1f77d4f00a4db1468f6a86799e4c70/lib [16:38:57] Yes, I see. But `git fat init` exits with zero from the paste though. Not sure what that really tells us. [16:39:19] oh [16:39:33] hm [16:39:37] i guess it needs to run that every time [16:39:37] hm [16:40:10] It does, doesn't it? https://phabricator.wikimedia.org/P18407$23 [16:41:18] Nothing to do with my shell account is it? Does that scap command fail if you run it? [16:41:33] i tried to do a git fat pull on aqs1010 and got the same thing [16:41:38] so it isn't a scap issue i think. [16:41:42] but i'll try [16:42:06] uhhh [16:42:06] canary deploy successful. Continue? [y]es/[n]o/[c]ontinue all groups: [16:42:12] i'll continue? [16:42:20] Be my guest. [16:42:44] ok, i dunno... [16:42:46] it suceeded [16:42:46] but. [16:42:55] the artifacts def were not pulled [16:43:03] they still have the text git-fat content [16:43:12] but, that looks like what it was doing before too? [16:43:16] so i'm not sure if this ever worked? [16:43:17] You ran scap as otto, not under sudo, right? [16:43:22] right [16:44:19] OK. Maybe I could copy the jars into place manually for now, or do you think that this is a good thing to delve into? [16:45:37] > git fat pull on aqs1010 and got the same thing [16:45:37] Did you do this under `sudo deploy-service`or similar? [16:46:20] yes [16:46:38] btullis: maybe ask eric evans? looks like he set this up originally [16:46:44] probably copying in place is not a good idea [16:47:07] but, you could copy the clone of the repo on e.g. aqs1010 into your home dir and try to troubleshoot git fat that way [16:47:10] git fat pull should just work. [16:47:18] there are no perms on the git-fat rsync module in archiva [16:47:23] you could even do it locally [16:47:25] OK, will do. Many thanks. [16:53:27] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10odimitrijevic) Thanks @BTullis! Can you please assign the role to the product analytics as well as data engineering team members too? [17:00:08] 10Data-Engineering, 10Data-Engineering-Kanban: Connect Atlas to a Data Source - https://phabricator.wikimedia.org/T298710 (10odimitrijevic) [17:00:54] 10Data-Engineering, 10Data-Engineering-Kanban: Connect Atlas to a Data Source - https://phabricator.wikimedia.org/T298710 (10odimitrijevic) [17:00:56] 10Data-Engineering, 10Data-Engineering-Kanban, 10User-razzi: Run Atlas on test cluster - https://phabricator.wikimedia.org/T296670 (10odimitrijevic) [17:01:12] 10Data-Engineering, 10Data-Engineering-Kanban: Connect Atlas to a Data Source - https://phabricator.wikimedia.org/T298710 (10odimitrijevic) p:05Triage→03High [17:06:36] 10Data-Engineering, 10Data-Engineering-Kanban: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) I am having some difficulty with the deploy. There's a paste here P18407 with some useful output. @Ottomata tried a deploy and it appeared successful, but the ja... [17:07:11] ping ottomata ? [17:07:17] joal hello! [17:07:25] standup? [17:07:28] OH [17:07:33] i thought it was no meeting week? [17:07:58] coming... [17:32:23] razzi - I must have dreamt the role-inheritance in superset - I'm ver sorry :S [17:32:33] haha no worries :) [17:32:37] It would be a good feature [17:32:41] yeah for sure [17:32:54] 10Data-Engineering, 10Superset: Create a custom Superset Role to allow for non default permissioning - https://phabricator.wikimedia.org/T298714 (10odimitrijevic) [17:33:11] I wonder if we prefer having a special role for us or adding a second role to everyone S [17:33:58] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10odimitrijevic) After discussing this in Standup, let's add sql_labs back to the Alpha role, and as a follow up setup a custom role: https://phabricator.wikimedia.org/T298714 and in... [17:35:33] !llog add "can menu access on sqllab" and "can sql json on superset" to superset Alpha role permissions [18:02:22] !log btullis@aqs1010:~$ sudo systemctl restart cassandra-a.service [18:02:25] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:05:19] https://usercontent.irccloud-cdn.com/file/HV6qs2NO/image.png [18:07:59] Sorted the deploy with Eric's help. Some weird issue with archiva's link not being created. Not sure what caused this link to be recreated by the timer, but it was and now it works. [18:08:03] https://www.irccloud.com/pastebin/C8tUslyu/ [18:13:53] \o/ [18:19:39] huh btullis i thought i saw the link just fine! [18:22:07] It wasn't the main jar I was looking for: `logstash-logback-encoder-4.2.jar` which had the problem... [18:22:07] It was a dependency: `jackson-core-2.4.0.jar` [18:34:01] 10Data-Engineering, 10Data-Engineering-Kanban: Send cassandra3 (new hosts) logs to logstash - https://phabricator.wikimedia.org/T297460 (10BTullis) I have successfully deployed the logstash-logback-encoder and restarted aqs1010-a. It send the logs through to logstash as expected. {F34909552,width=60%} I will s... [18:35:47] 10Data-Engineering, 10Data-Engineering-Kanban, 10Superset: Superset SQL Lab error - https://phabricator.wikimedia.org/T298699 (10JMando) I am also able to see the SQL Lab drop down menu as well as run queries. Thank you! [18:42:01] The strange thing is, I still can't find that version of the jar in Archiva: https://archiva.wikimedia.org/#artifact-details-download-content/com.fasterxml.jackson.core/jackson-core/2.4.0 [18:42:01] There is a pom and a bundle, but no jar shown here. Then for some reason after browsing archiva, the timer that runs every 5 minutes found a copy of the jar on the file system and linked it, which made the deploy work. Very odd. [19:04:15] 10Data-Engineering, 10Data-Engineering-Kanban, 10Data-Services, 10cloud-services-team (Kanban): Recreate views for globaluser table - https://phabricator.wikimedia.org/T298505 (10razzi) 05Open→03Resolved This is done. Took even more manual steps since I made some human errors; let's automate this! http... [19:16:03] 10Data-Engineering, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-Analytics: MobileWikiAppiOSUserHistory sending incompatible data - https://phabricator.wikimedia.org/T298721 (10Ottomata) [19:51:15] huh [21:28:37] 10Data-Engineering, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-Analytics: MobileWikiAppiOSUserHistory sending incompatible data - https://phabricator.wikimedia.org/T298721 (10SNowick_WMF) Hi @Ottomata, this is a new field we are tracking in this schema so we don't expect any old da... [21:36:07] 10Data-Engineering, 10Product-Analytics, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-Analytics: MobileWikiAppiOSUserHistory sending incompatible data - https://phabricator.wikimedia.org/T298721 (10Ottomata) Hm, something must have sent this data with a boolean value initially. That's the only way the t... [22:55:18] 10Analytics-Clusters, 10DC-Ops, 10SRE, 10ops-eqiad, 10Patch-For-Review: (Need By: TBD) rack/setup/install an-test-coord1002 - https://phabricator.wikimedia.org/T293938 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmjohnson@cumin1001 for host an-test-coord1002.eqiad.wmnet wit... [23:24:04] 10Analytics-Clusters, 10DC-Ops, 10SRE, 10ops-eqiad: (Need By: TBD) rack/setup/install an-test-coord1002 - https://phabricator.wikimedia.org/T293938 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage started by cmjohnson@cumin1001 for host an-test-coord1002.eqiad.wmnet with OS buster executed with... [23:29:06] (03PS1) 10EpicPupper: add link to database names [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/752034 [23:29:08] (03CR) 10Welcome, new contributor!: "Thank you for making your first contribution to Wikimedia! :) To learn how to get your code changes reviewed faster and more likely to get" [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/752034 (owner: 10EpicPupper)