[05:55:35] (03CR) 10Joal: "Heya - I don't understand why the proposed patch would change the field value :S Would you mind explaining me please?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/771730 (https://phabricator.wikimedia.org/T300029) (owner: 10Aqu) [06:16:04] 10Data-Engineering, 10Data-Engineering-Kanban, 10ContentTranslation, 10Language-analytics, 10Product-Analytics: Abuse filter analytics dashboard is broken - https://phabricator.wikimedia.org/T302970 (10JAllemandou) Changing the table ownership to `analytics:analytics-privatedata-users` and restrict reade... [06:49:18] 10Data-Engineering, 10Data-Engineering-Kanban, 10ContentTranslation, 10Language-analytics, 10Product-Analytics: Abuse filter analytics dashboard is broken - https://phabricator.wikimedia.org/T302970 (10JAllemandou) a:03JAllemandou [07:00:00] 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of rhuang-ctr - https://phabricator.wikimedia.org/T302194 (10JAllemandou) Hi @CMacholan - Friendly reminder to please let us know if we can delete the data belonging to rhuang-ctr (see previous comment). [07:02:01] 10Analytics, 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of bumeh-ctr - https://phabricator.wikimedia.org/T300607 (10JAllemandou) a:03JAllemandou [07:03:32] 10Data-Engineering, 10Data-Engineering-Kanban: Check home/HDFS leftovers of rhuang-ctr - https://phabricator.wikimedia.org/T302194 (10JAllemandou) a:03JAllemandou [10:08:12] (03CR) 10David Caro: [C: 04-1] Update home to direct to profile (031 comment) [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771352 (https://phabricator.wikimedia.org/T85175) (owner: 10Vivian Rook) [10:10:19] (03PS1) 10David Caro: compose: Add order to the startup [analytics/quarry/web] - 10https://gerrit.wikimedia.org/r/771844 [12:28:24] hola teammm :] [12:28:49] Hi mforns - How's it going? [13:07:30] :] [13:17:41] Is there anything special I should know about querying wmf.webrequest? I'm limiting to 1 day partition, but my queries always seem to hang and there's this strange log message printed before the column headers: "Getting log thread is interrupted, since query is done!" [13:22:39] awight: Hmm, that sounds a bit unusual to me. Can you share a bit more information on how you're querying it please? Is this in a hive cli, or in Jupyter, or using spark, or presto? [13:22:51] awight: I take it you've looked at these sample queries? https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/Webrequest#Sample_queries [13:45:10] btullis: Thanks, yeah my queries are similar. I'm running in beeline on stat1004, and I've pasted some of my attempts here: [13:45:13] https://turnilo.wikimedia.org/#webrequest_sampled_128/4/N4IgbglgzgrghgGwgLzgFwgewHYgFwhpwBGCApiADTjTxKoY4DKZaG2A5lPqAMaYIEcAA5QyAJUwB3bngBmiMQF9qGALZlkOCgQCiaXgHoAqgBUAwlRByICNGQBOsgNqg0AT2E7CEDVYdkcvggAPoh6hSqvhJwnGQuoFBEDmjBAEwADGlpALRZOQCMAGymBWl4GRkVGQB0lRkAWlZk2AAm6Vm5GQDMhQAcpeWV1XWVTUoAuipunt5JDhCc/oHBMAshABaYSVZwvIy4BItWYIgw8fjOIACO5w7uNVIQANYQrehwNZgOHCAT1NhMKl5IoyNNCLNgvNFr9qAEggQYGIH [13:45:20] CE4BwWqlqHsDsFjtRTghzi4QAABJ6vDStCBwQzPOApIEbRzU7A5dHbDYieKGAo1XkAdgABAAKDZsUR4QyGSnU8kQL4/QwAWREUAAlH8AUD8AoEMp/iAoMIkGgEhCvFC0AsltQqRpsFAsIcQEjHKj0dhMYbvsDQPDghsIKarB4LQRkRALraIAF9k7gq14rwWlSbSAvAtMO0CCAVCAkGog/gAKwZcGh7wRKx2lqOnDBMJV6gcByxGBCBYeYIABQKABErFAfTwQP6CI3oiHIQQaw749HYziCHAoMm2jDc9QC0W8Nh2whJtQIxc8K5R4FHC1kwGg9xqHJvmp0COKwnAnB216Cedgpvzd4NBXNYKHBeFL2wa9lw4FsyA4dAyFaEIAiNHAxBCR1kEia [13:45:26] xH2fPAZjDEBEwUT8TjObw/1fAhANgAJcwNYRFmwBC+2iOdUMuejGIQphhwIQNgyUIA [13:45:28] aargh sorry [13:45:31] here: https://phabricator.wikimedia.org/P22830 [14:01:41] awight: 1 day is a lot of data :) [14:02:05] one thing that you can try is select the webrequest partition, afaics you need text only [14:02:10] it will prune a lot of data [14:02:53] (so webrequest_source=text basically) [14:03:49] and maybe start with a smaller range of hours [14:04:20] if it doesn't work, spark sql should be more efficient (instead of beeline) [14:17:42] ty I'll try these tips. My problem at the moment is that I wasn't even sure which hour to look in. [18:30:17] PROBLEM - Check unit status of eventlogging_to_druid_navigationtiming_hourly on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit eventlogging_to_druid_navigationtiming_hourly https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [19:05:49] RECOVERY - Check unit status of eventlogging_to_druid_navigationtiming_hourly on an-launcher1002 is OK: OK: Status of the systemd unit eventlogging_to_druid_navigationtiming_hourly https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [21:40:02] PROBLEM - Check unit status of produce_canary_events on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [22:06:49] RECOVERY - Check unit status of produce_canary_events on an-launcher1002 is OK: OK: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers