[02:03:06] 10Analytics, 10Analytics-SWAP, 10Product-Analytics: conda list does not show all packages in environment - https://phabricator.wikimedia.org/T294368 (10nshahquinn-wmf) Thanks for the context and the workaround! Since the issue is that packages from the base environment aren't list, why does my newly created... [07:02:32] 10Analytics-Clusters, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Upgrade Matomo to latest upstream - https://phabricator.wikimedia.org/T275144 (10elukey) I think that we should open a separate task to fork https://github.com/matomo-org/matomo-package/tree/master/debian in a separate... [07:17:15] joal: bonjour https://www.featurestore.org/feature-store-summit-videos :) [07:49:52] Thanks elukey :) Bonjour! [08:58:25] (03PS1) 10Lucas Werkmeister (WMDE): Remove Facebook, IRC and Twitter social metrics [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/734898 (https://phabricator.wikimedia.org/T294014) [09:19:58] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Push Gobblin import metrics to Prometheus and add alerts on some critical imports - https://phabricator.wikimedia.org/T286503 (10JAllemandou) Summary of yesterday's discussion with @Ottomata and @fgiunched... [09:50:20] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Push Gobblin import metrics to Prometheus and add alerts on some critical imports - https://phabricator.wikimedia.org/T286503 (10fgiunchedi) Thank you for the summary @JAllemandou, looks great! A few point... [10:26:21] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10BTullis) The last message reported to be in logstash with the `program` field of `jupyterhub-conda-singleuser` was timestamped at 09:41 on October 13th. {F347127... [10:26:42] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Send some existing Gobblin metrics to prometheus - https://phabricator.wikimedia.org/T294420 (10JAllemandou) [10:30:35] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Add gobblin metrics per kafka-topic-partition - https://phabricator.wikimedia.org/T294422 (10JAllemandou) [12:34:43] 10Analytics-Kanban: Check abnormal pageviews for XHamster - https://phabricator.wikimedia.org/T158071 (10Aravindkumar) [13:00:47] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Wikidata, and 3 others: Add MCR slot information to revision-create events - https://phabricator.wikimedia.org/T293195 (10dcausse) >>! In T293195#7459268, @Ottomata wrote: > I was about to merge that today but then thought that your suggestion to ensure... [13:01:17] (03PS5) 10DCausse: Spark JsonSchemaConverter - additionalProperties with schema is always a MapType [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) (owner: 10Ottomata) [13:07:15] (03CR) 10DCausse: [C: 03+1] Spark JsonSchemaConverter - additionalProperties with schema is always a MapType [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) (owner: 10Ottomata) [13:10:47] ottomata: o/ You mentioned you might be able to help me track down some missing logstash messages re: https://phabricator.wikimedia.org/T288348#7460944 - Do you think you might have some time to look at it today? [13:36:28] (03CR) 10Ottomata: [C: 03+1] "This change is ready for review." (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [13:40:19] 10Analytics, 10Analytics-SWAP, 10Product-Analytics: conda list does not show all packages in environment - https://phabricator.wikimedia.org/T294368 (10Ottomata) > why does my newly created environment have so many packages in it There might be other reasons, but this is at least one I know of: https://ger... [13:44:35] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10Ottomata) > Do they make as far as to the Kafka topic? To check this, log into kafka-logging1001 and use `kafkacat -C -b localhost:9092 -t -o end`... [13:44:49] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10BTullis) I have verified that there are logs appearing in the kafka topic: `rsyslog-info` ` {"timestamp":"2021-10-21T10:18:45.128895+00:00", "message":"[I 2021-... [13:47:52] (03CR) 10Ottomata: [C: 03+1] Spark JsonSchemaConverter - additionalProperties with schema is always a MapType (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) (owner: 10Ottomata) [13:48:08] joal should we merge https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/629406/ ? [13:49:17] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10Ottomata) K sounds like a logstash filter/ingestor problem them. @colewhite maybe can help? [13:49:34] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10BTullis) Thanks @Ottomata The `-o end` helped there. I was able to identify one refresh of my notebook with a single kafka event. I was browsing from a stat bo... [13:54:54] ottomata: let me triple check [14:00:39] (03CR) 10Joal: [C: 03+2] "Let's move that" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) (owner: 10Ottomata) [14:00:44] (03PS6) 10Ottomata: Spark JsonSchemaConverter - additionalProperties with schema is always a MapType [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) [14:00:53] hahah :) [14:01:05] just changed hte commit message a bit :) [14:01:31] ottomata: it looks like jenkins might be able to merge - let's triple check [14:14:20] (03CR) 10Ottomata: "I will try to find some time to make a release and deploy today." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/629406 (https://phabricator.wikimedia.org/T263466) (owner: 10Ottomata) [15:06:35] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10BTullis) I have found and fixed the problem! The logs were being saved to a different index pattern by the logstash output filter: `index => ecs-1.7.0-5-%{[@met... [15:07:22] 10Analytics, 10Analytics-Kanban: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10BTullis) [15:08:21] 10Analytics, 10Analytics-Kanban, 10Data-Engineering: Snapshot and Reload cassandra2 pageview_per_article data table from all 12 instances - https://phabricator.wikimedia.org/T291472 (10BTullis) The loading of the 7th snapshot is now 61% complete. [15:08:44] (03PS1) 10Ottomata: Fix bug in HDFSCleaner where directories with only directories would always be deleted [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/734993 (https://phabricator.wikimedia.org/T287084) [15:10:51] (03CR) 10Joal: [C: 03+2] "Merging!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/734993 (https://phabricator.wikimedia.org/T287084) (owner: 10Ottomata) [15:14:02] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Patch-For-Review, and 2 others: Migrate analytics cluster alerts from Icinga to AlertManager - https://phabricator.wikimedia.org/T293399 (10BTullis) [15:21:57] (03Merged) 10jenkins-bot: Fix bug in HDFSCleaner where directories with only directories would always be deleted [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/734993 (https://phabricator.wikimedia.org/T287084) (owner: 10Ottomata) [15:28:47] (03PS1) 10Ottomata: Update changelog for JsonSchemaConverter map type change [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/735000 [15:29:01] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Update changelog for JsonSchemaConverter map type change [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/735000 (owner: 10Ottomata) [15:29:57] 10Analytics, 10Analytics-Kanban: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10Ottomata) Nice! [15:30:54] (03PS1) 10Joal: Update jar version of hdfs-cleaner script [analytics/refinery] - 10https://gerrit.wikimedia.org/r/735001 (https://phabricator.wikimedia.org/T287084) [15:31:01] ottomata: --^ [15:31:49] 10Analytics, 10EventStreams: Expose mediawiki/revision/tags-change in stream.wikimedia.org - https://phabricator.wikimedia.org/T294391 (10nettrom_WMF) Noting that @Count_Count also asked for this in T266375#6608351 [15:34:05] (03PS1) 10Clare Ming: Add new schema for web UI scroll tracking. [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/735003 (https://phabricator.wikimedia.org/T292586) [15:35:07] (03CR) 10Jdlrobson: Add new web A/B test schema to track bucketing of users for a given experiment. (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [15:35:41] 10Analytics, 10Analytics-Kanban: Jupyter notebook logs should appear in Logstash - https://phabricator.wikimedia.org/T288348 (10colewhite) Related: T288623 It seems to me important to get the word out that ECS adoption necessitates the use of a separate index pattern. Also, that there is documentation around... [15:39:44] Starting build #97 for job analytics-refinery-maven-release-docker [15:47:35] ottomata: batcave before standup? I had a reminiscence of an old idea :) [15:48:02] joal ok! [15:54:17] Project analytics-refinery-maven-release-docker build #97: 09SUCCESS in 14 min: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-release-docker/97/ [15:58:28] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering, 10Jupyter-Hub: Autocomplete is very slow (unusable) in Newpyter - https://phabricator.wikimedia.org/T290008 (10nshahquinn-wmf) [15:58:33] 10Analytics, 10Analytics-Jupyter, 10Jupyter-Hub, 10PAWS: Add nbextensions to PAWS - https://phabricator.wikimedia.org/T287078 (10nshahquinn-wmf) [15:58:44] 10Analytics, 10Analytics-Jupyter, 10Jupyter-Hub: Issue Connecting to Jupyter - https://phabricator.wikimedia.org/T162759 (10nshahquinn-wmf) [15:58:48] 10Analytics, 10Analytics-Jupyter, 10Jupyter-Hub: Proxy target missing - https://phabricator.wikimedia.org/T95818 (10nshahquinn-wmf) [16:16:46] Starting build #56 for job analytics-refinery-update-jars-docker [16:17:14] (03PS1) 10Maven-release-user: Add refinery-source jars for v0.1.20 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/735022 [16:17:15] Project analytics-refinery-update-jars-docker build #56: 09SUCCESS in 29 sec: https://integration.wikimedia.org/ci/job/analytics-refinery-update-jars-docker/56/ [16:17:39] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Add refinery-source jars for v0.1.20 to artifacts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/735022 (owner: 10Maven-release-user) [16:19:44] (03CR) 10Ottomata: [V: 03+2 C: 03+2] Update jar version of hdfs-cleaner script [analytics/refinery] - 10https://gerrit.wikimedia.org/r/735001 (https://phabricator.wikimedia.org/T287084) (owner: 10Joal) [16:23:54] (03CR) 10MNeisler: talk_page_event schema (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/731333 (https://phabricator.wikimedia.org/T286076) (owner: 10DLynch) [16:35:33] (03CR) 10Fdans: "@milimetric yo" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/714123 (https://phabricator.wikimedia.org/T283254) (owner: 10Fdans) [18:32:02] PROBLEM - Check unit status of refine_event on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [18:53:15] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Wikidata, and 3 others: Add MCR slot information to revision-create events - https://phabricator.wikimedia.org/T293195 (10Ottomata) Done and deployed! [18:53:48] ^^ was me, fixed by using -shaded jar [18:54:03] ack thanks ottomata :) [18:54:12] RECOVERY - Check unit status of refine_event on an-launcher1002 is OK: OK: Status of the systemd unit refine_event https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [18:55:33] (03CR) 10Jdlrobson: [C: 03+1] "Ottomata I'll let this today, and merge when convinced it's working. Let me know if you would rather +2." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [18:56:02] (03CR) 10Ottomata: [C: 03+1] "Merge at will! :)" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [19:08:45] 10Analytics, 10Analytics-Jupyter, 10Jupyter-Hub, 10PAWS: Add nbextensions to PAWS - https://phabricator.wikimedia.org/T287078 (10Chicocvenancio) I am bit worried of us going in the direction of depending more on jupyter notebook classic interface while the jupyter community seems to be [[ https://github.co... [19:28:57] (03PS4) 10Clare Ming: Add new web A/B test schema to track bucketing of users for a given experiment. [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) [19:29:42] (03CR) 10Clare Ming: "updated schema name per https://phabricator.wikimedia.org/T292586#7461441" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [20:33:54] 10Analytics, 10Analytics-Kanban: Check home/HDFS leftovers of jmads - https://phabricator.wikimedia.org/T290715 (10Ottomata) Done. I archived jmads stat1005 home directory (minus a python virtualenv) and put it in HDFS at `/wmf/data/archive/user/jmads/jmads.stat1005.home.T290715.tgz`. ` root@stat1005:/home... [20:34:10] 10Analytics, 10Analytics-Kanban: Check home/HDFS leftovers of jmads - https://phabricator.wikimedia.org/T290715 (10Ottomata) 05Open→03Resolved [20:34:47] 10Analytics, 10Analytics-Kanban, 10Chinese-Sites: Some pageviews data are missing for Oct 21, 2021 - https://phabricator.wikimedia.org/T294193 (10Ottomata) 05Open→03Resolved [20:49:25] 10Analytics, 10Analytics-Kanban, 10Chinese-Sites: Some pageviews data are missing for Oct 21, 2021 - https://phabricator.wikimedia.org/T294193 (10Urbanecm) Yup, looks good now. Thanks! [21:01:46] (03CR) 10Bartosz Dziewoński: [C: 04-1] "(Per MNeisler, I'm assuming that requires changes here)" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/731333 (https://phabricator.wikimedia.org/T286076) (owner: 10DLynch) [21:13:49] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Engineering, 10tech-decision-forum: MediaWiki Events as Source of Truth - Decision Statement Overview - https://phabricator.wikimedia.org/T291120 (10Milimetric) >>! In T291120#7451037, @Joe wrote: > ... [snip] > Also, I would like to see a ref... [21:13:53] 10Analytics, 10Data-Engineering, 10Event-Platform, 10Platform Engineering, 10tech-decision-forum: MediaWiki Events as Source of Truth - Decision Statement Overview - https://phabricator.wikimedia.org/T291120 (10Milimetric) There's a different way to state the higher level requirement, and I'm also curiou... [21:29:03] (03CR) 10Nray: [C: 03+1] "LGTM, just had one question" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/735003 (https://phabricator.wikimedia.org/T292586) (owner: 10Clare Ming) [21:44:34] 10Analytics, 10Analytics-Jupyter, 10Product-Analytics: conda list does not show all packages in environment - https://phabricator.wikimedia.org/T294368 (10nshahquinn-wmf) @Ottomata thanks, that makes sense! For now, I've [documented the workaround on Wikitech](https://wikitech.wikimedia.org/w/index.php?title... [21:57:46] 10Analytics: Use inclusive language - https://phabricator.wikimedia.org/T280268 (10Milimetric) subtask, fixing [21:58:01] 10Analytics: Use inclusive language - https://phabricator.wikimedia.org/T280268 (10Milimetric) [22:09:20] 10Analytics, 10Analytics-Jupyter, 10Jupyter-Hub: Proxy target missing - https://phabricator.wikimedia.org/T95818 (10nshahquinn-wmf) 05Open→03Invalid Since this is 5 years old and #paws has changed so much since then, I assume this is no longer an issue. [22:09:51] 10Analytics, 10Analytics-Jupyter, 10Data-Engineering: Autocomplete is very slow (unusable) in Newpyter - https://phabricator.wikimedia.org/T290008 (10nshahquinn-wmf) [22:13:03] 10Analytics, 10Inuka-Team, 10Product-Analytics: Superset timeouts for KaiOS dashboard - https://phabricator.wikimedia.org/T277320 (10Milimetric) cc @JAllemandou who's thinking about the "faster superset" umbrella of issues. [22:13:29] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, and 2 others: Replace Camus by Gobblin - https://phabricator.wikimedia.org/T271232 (10odimitrijevic) [22:15:34] 10Analytics, 10Data-Engineering, 10Epic: Traffic anomaly alarms - https://phabricator.wikimedia.org/T267355 (10odimitrijevic) [22:17:31] 10Analytics-Clusters, 10Analytics-Kanban: Move the Analytics infrastructure to Debian Buster - https://phabricator.wikimedia.org/T234629 (10odimitrijevic) @elukey are you ok with us closing this task given that the Cassandra upgrade is near completion? [22:17:54] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Analytics Presto improvements - https://phabricator.wikimedia.org/T266639 (10odimitrijevic) [22:28:12] 10Analytics, 10Product-Analytics, 10Epic: Data Lake incremental Data Updates - https://phabricator.wikimedia.org/T258511 (10odimitrijevic) [22:29:47] 10Analytics: Remove support for the (deprecated) Druid datasources (in favor of Druid Tables) on Superset - https://phabricator.wikimedia.org/T263972 (10odimitrijevic) [22:29:49] 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Superset Updates - https://phabricator.wikimedia.org/T211706 (10odimitrijevic) [22:33:37] 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Superset Updates - https://phabricator.wikimedia.org/T211706 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic Closing catch-all task. Added link to ticket from wikitech as it contains history and some possibly useful information. [22:35:41] 10Analytics: Use inclusive language in code for private analytics infrastructure - https://phabricator.wikimedia.org/T280268 (10nshahquinn-wmf) [22:36:47] 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Data quality issues in the mediawiki_history data stream - https://phabricator.wikimedia.org/T204953 (10odimitrijevic) [22:36:49] 10Analytics: Rework how mediawiki-history differentiates fake page-create from real ones - https://phabricator.wikimedia.org/T264791 (10odimitrijevic) [22:37:03] 10Analytics, 10Analytics-Data-Quality, 10Product-Analytics: A few alterblocks events have event_timestamps from before 2001 - https://phabricator.wikimedia.org/T218824 (10odimitrijevic) [22:37:05] 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Data quality issues in the mediawiki_history data stream - https://phabricator.wikimedia.org/T204953 (10odimitrijevic) [22:37:21] 10Analytics: Enhance mediawiki-history page reconstruction with best historical information possible - https://phabricator.wikimedia.org/T179692 (10odimitrijevic) [22:37:23] 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Data quality issues in the mediawiki_history data stream - https://phabricator.wikimedia.org/T204953 (10odimitrijevic) [22:37:54] 10Analytics, 10Product-Analytics, 10Epic: Provide feature parity between the wiki replicas and the Analytics Data Lake - https://phabricator.wikimedia.org/T212172 (10odimitrijevic) [22:38:07] 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Data quality issues in the mediawiki_history data stream - https://phabricator.wikimedia.org/T204953 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic [22:38:52] 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics, 10Tracking-Neverending: Data quality issues in the mediawiki_history data stream - https://phabricator.wikimedia.org/T204953 (10odimitrijevic) Closing tracking never-ending task in favor of short lived epics. [22:39:58] (03PS5) 10Clare Ming: Add new web A/B test schema to track bucketing of users for a given experiment. [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) [22:40:07] 10Analytics, 10Analytics-Kanban: Data Quality Alarms - https://phabricator.wikimedia.org/T198986 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic All child tasks are complete. Closing long standing parent tasks. [23:12:07] (03CR) 10Jdlrobson: [C: 03+2] Add new web A/B test schema to track bucketing of users for a given experiment. [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [23:13:17] (03Merged) 10jenkins-bot: Add new web A/B test schema to track bucketing of users for a given experiment. [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/732089 (https://phabricator.wikimedia.org/T292587) (owner: 10Clare Ming) [23:15:17] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review, 10User-MoritzMuehlenhoff: Improve user experience for Kerberos by creating automatic token renewal service - https://phabricator.wikimedia.org/T268985 (10odimitrijevic) [23:15:19] 10Analytics, 10Analytics-Kanban: Analytics Ops Technical Debt - https://phabricator.wikimedia.org/T240437 (10odimitrijevic) [23:22:08] 10Analytics: Remove support for the (deprecated) Druid datasources (in favor of Druid Tables) on Superset - https://phabricator.wikimedia.org/T263972 (10odimitrijevic) [23:22:10] 10Analytics, 10Analytics-Kanban: Analytics Ops Technical Debt - https://phabricator.wikimedia.org/T240437 (10odimitrijevic) [23:22:33] 10Analytics, 10Patch-For-Review: Use types in Analytics Puppet classes/profiles/etc.. - https://phabricator.wikimedia.org/T252617 (10odimitrijevic) [23:22:37] 10Analytics, 10Analytics-Kanban: Analytics Ops Technical Debt - https://phabricator.wikimedia.org/T240437 (10odimitrijevic) [23:23:39] 10Analytics, 10Analytics-Kanban: Archive /home/ezachte data on stat1007 - https://phabricator.wikimedia.org/T238243 (10odimitrijevic) [23:23:41] 10Analytics, 10Analytics-Kanban: Analytics Ops Technical Debt - https://phabricator.wikimedia.org/T240437 (10odimitrijevic) [23:24:19] 10Analytics, 10Analytics-Kanban: Analytics Ops Technical Debt - https://phabricator.wikimedia.org/T240437 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic Closing long standing catch all task [23:24:43] 10Analytics, 10Epic: Sanitize pageview_hourly - https://phabricator.wikimedia.org/T114675 (10odimitrijevic) [23:25:31] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Epic: Check AQS with cassandra (serving + data) - https://phabricator.wikimedia.org/T290068 (10odimitrijevic) [23:28:34] 10Analytics-Kanban: Analytics Hardware for Fiscal Year 2020/2021 - https://phabricator.wikimedia.org/T255145 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic Marking as closed in favor of opening another Epic for hardware for 21/22. [23:29:15] 10Analytics: Archive /home/ezachte data on stat1007 - https://phabricator.wikimedia.org/T238243 (10odimitrijevic) [23:32:16] 10Analytics, 10Patch-For-Review: Migrate pagecounts-ez generation to hadoop - https://phabricator.wikimedia.org/T192474 (10odimitrijevic) Removing from kanban [23:36:49] 10Analytics-EventLogging, 10Analytics-Kanban, 10Event-Platform, 10Goal, and 2 others: Modern Event Platform - https://phabricator.wikimedia.org/T185233 (10odimitrijevic) 05Open→03Resolved a:03odimitrijevic Added reference to task to wikitech. Closing long standing task. [23:37:11] 10Analytics, 10Analytics-EventLogging, 10Data-Engineering, 10Event-Platform, 10Patch-For-Review: Decommission EventLogging backend components by migrating to MEP - https://phabricator.wikimedia.org/T238230 (10odimitrijevic) [23:37:51] 10Analytics, 10Analytics-EventLogging, 10Data-Engineering, 10Event-Platform, and 2 others: Migrate legacy metawiki schemas to Event Platform - https://phabricator.wikimedia.org/T259163 (10odimitrijevic) [23:38:52] 10Analytics, 10Data-Engineering: pagecounts-ez of month 2020-08 is incomplete - https://phabricator.wikimedia.org/T262141 (10odimitrijevic) [23:40:36] 10Analytics: Change routing to accept a list of wikis in URL - https://phabricator.wikimedia.org/T283596 (10odimitrijevic) [23:41:03] 10Analytics, 10Data-Engineering: Change state to store project as an array - https://phabricator.wikimedia.org/T283624 (10odimitrijevic) [23:43:50] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Add logic to purging scripts that requires admin action if it's about to delete a lot of data - https://phabricator.wikimedia.org/T270433 (10odimitrijevic) [23:43:52] 10Analytics, 10Analytics-Kanban, 10Data-Engineering-Kanban, 10wmfdata-python, 10Product-Analytics (Kanban): wmfdata-python's Hive query output includes logspam - https://phabricator.wikimedia.org/T275233 (10odimitrijevic) [23:46:10] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban: Wikistats should allow more than one project - https://phabricator.wikimedia.org/T283254 (10odimitrijevic) [23:46:34] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Expand Wikiselector to allow more than one wiki - https://phabricator.wikimedia.org/T285050 (10odimitrijevic) [23:46:59] 10Analytics: Productionize HDFS fsimage data analysis job - https://phabricator.wikimedia.org/T261283 (10odimitrijevic) [23:47:47] 10Analytics, 10Product-Analytics: Creation of canonical pageview dumps for users to download - https://phabricator.wikimedia.org/T251777 (10odimitrijevic) [23:50:14] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Snapshot and Reload cassandra2 pageview_per_article data table from all 12 instances - https://phabricator.wikimedia.org/T291472 (10odimitrijevic) [23:50:16] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Repair and reload cassandra2 mediarequest_per_file data table - https://phabricator.wikimedia.org/T291470 (10odimitrijevic) [23:50:18] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban: Improve Refine bad data handling - https://phabricator.wikimedia.org/T289003 (10odimitrijevic) [23:50:20] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, and 3 others: Migrate analytics cluster alerts from Icinga to AlertManager - https://phabricator.wikimedia.org/T293399 (10odimitrijevic) [23:50:22] 10Analytics, 10Analytics-Kanban, 10Data-Engineering-Kanban, 10Patch-For-Review: Purge gobblin files - https://phabricator.wikimedia.org/T287084 (10odimitrijevic) [23:50:24] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, and 2 others: Add SearchSatisfaction to the allowlist - https://phabricator.wikimedia.org/T274607 (10odimitrijevic) [23:50:26] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Data-Engineering, and 4 others: Determine which remaining legacy EventLogging schemas need to be migrated or decommissioned - https://phabricator.wikimedia.org/T282131 (10odimitrijevic) [23:50:28] 10Analytics, 10Analytics-Kanban, 10Data-Engineering-Kanban, 10Event-Platform, and 5 others: Revisions missing from mediawiki_revision_create - https://phabricator.wikimedia.org/T215001 (10odimitrijevic) [23:50:34] 10Analytics-Clusters, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Set hive.warehouse.subdir.inherit.perms to false - https://phabricator.wikimedia.org/T291664 (10odimitrijevic) [23:50:36] 10Analytics, 10Analytics-Kanban, 10Data-Engineering, 10Data-Engineering-Kanban, 10Patch-For-Review: Refactor analytics-meta MariaDB layout to multi instance with failover - https://phabricator.wikimedia.org/T284150 (10odimitrijevic)