[11:13:05] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: 14Change HAProxy log-format to support missing information - 14https://phabricator.wikimedia.org/T358105#9637079 (10Fabfur) 05Open→03Resolved 14This can be closed as the new log format is tested and read... [11:13:11] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9637081 (10Fabfur) [12:01:21] 14Analytics, 06Data-Engineering-Icebox, 10CX-analytics, 10Language-analytics, and 2 others: Special:ContentTranslationStats is slow and getting crowded - https://phabricator.wikimedia.org/T325790#9637288 (10KCVelaga_WMF) Summary of the Slack discussion: Seems like the best option we have is http://superset... [12:23:09] 06Data-Engineering, 10Data-Engineering-Wikistats, 06Data Products, 06Data-Platform, 06Movement-Insights: Wikistats "Active Editors by Country" does not follow definition for active editors - https://phabricator.wikimedia.org/T360073#9637386 (10lbowmaker) [12:42:19] (03PS1) 10Gerrit maintenance bot: Add kus.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1011434 (https://phabricator.wikimedia.org/T360303) [12:42:55] (03PS1) 10Gerrit maintenance bot: Add bew.wikipedia to pageview allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/1011437 (https://phabricator.wikimedia.org/T360310) [13:11:45] I was attempting to run the AQS restart cookbook (to ensure that the libuv security updates gets picked up by nodejs), but there's an alert which currently prevents rolling restarts: [13:11:50] /analytics.wikimedia.org/v1/edits/per-page/{project}/{page-title}/{editor-type}/{granularity}/{start}/{end} (Get daily edits for english wikipedia page 0) is WARNING: Test Get daily edits for english wikipedia page 0 responds with unexpected value at path /items[0]/results => Expected 1 array elements, gotten 0 [13:11:53] known issue? [14:53:33] 10Data-Engineering (Sprint 9), 13Patch-For-Review: [Dataset Config Store] Deploy poc to dse-k8s - https://phabricator.wikimedia.org/T357434#9638206 (10CodeReviewBot) dancy opened https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/65 Add dataset config store project to trusted run... [15:00:21] 10Data-Engineering (Sprint 9), 13Patch-For-Review: [Dataset Config Store] Deploy poc to dse-k8s - https://phabricator.wikimedia.org/T357434#9638247 (10CodeReviewBot) dancy merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/64 Add dataset config store project to trusted run... [15:00:31] 10Data-Engineering (Sprint 9), 13Patch-For-Review: [Dataset Config Store] Deploy poc to dse-k8s - https://phabricator.wikimedia.org/T357434#9638248 (10CodeReviewBot) dancy closed https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/65 Add dataset config store project to trusted run... [16:14:33] 14Analytics, 06Data-Engineering-Icebox, 10CX-analytics, 10Language-analytics, and 2 others: Special:ContentTranslationStats is slow and getting crowded - https://phabricator.wikimedia.org/T325790#9638551 (10Nikerabbit) There is a task for that {T196020} [16:30:09] 06Data-Engineering, 10EventStreams, 10MediaWiki-Page-protection, 10MediaWiki-Revision-deletion, and 3 others: Create Mediawiki "oversightprotect" action that suppresses usernames of all edits of a page - https://phabricator.wikimedia.org/T354577#9638686 (10Sdrqaz) Maybe I've misunderstood this task, but I... [16:41:23] 06Data-Engineering, 10EventStreams, 10MediaWiki-Page-protection, 10MediaWiki-Revision-deletion, and 3 others: Create Mediawiki "oversightprotect" action that suppresses usernames of all edits of a page - https://phabricator.wikimedia.org/T354577#9638758 (10Ladsgroup) I highly doubt this would be enabled in... [16:59:50] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9638903 (10Fabfur) I've fixed some errors (now metadata field name should be correct) in the Benthos configuration. I've repo... [18:05:42] 06Data-Engineering, 10EventStreams, 10MediaWiki-Page-protection, 10MediaWiki-Revision-deletion, and 3 others: Create Mediawiki "oversightprotect" action that suppresses usernames of all edits of a page - https://phabricator.wikimedia.org/T354577#9639315 (10Htriedman) @Ladsgroup is correct about this — this... [18:15:48] 06Data-Engineering, 10EventStreams, 10MediaWiki-Page-protection, 10MediaWiki-Revision-deletion, and 3 others: Create Mediawiki "oversightprotect" action that suppresses usernames of all edits of a page - https://phabricator.wikimedia.org/T354577#9639360 (10stjn) (Per the list in Russian Wikipedia, in more... [18:21:22] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Benthos: better management for unparsable logs - https://phabricator.wikimedia.org/T359627#9639365 (10Fabfur) 05Open→03In progress After some second of live logging I can confirm that the `SSL handshake failu... [18:21:27] 06Data-Engineering, 06Data Products, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Move analytics log from Varnish to HAProxy - https://phabricator.wikimedia.org/T351117#9639367 (10Fabfur) [19:46:05] 06Data-Engineering, 10Data-Engineering-Wikistats, 10Data Pipelines, 06Data Products: geoeditors public version is not available for non-Wikipedia projects - https://phabricator.wikimedia.org/T317040#9639627 (10VirginiaPoundstone) [19:47:58] 06Data-Engineering, 10Data Products (Data Products Sprint 11): project-title-country missing US data in recent data, and double quote escaping - https://phabricator.wikimedia.org/T341139#9639632 (10VirginiaPoundstone) [20:04:45] 06Data-Engineering, 10Data Products (Data Products Sprint 11): Method for per-file cumulative total in mediarequests API - https://phabricator.wikimedia.org/T343947#9639740 (10VirginiaPoundstone) @Dominicbm thank you for this enhancement request. It seems like it could be a good companion to the overall [[ htt... [20:09:22] 06Data-Engineering, 10Data Products (Data Products Sprint 11): NEW BUG REPORT - Pageviews Missing Hourly Partition - https://phabricator.wikimedia.org/T358142#9639772 (10VirginiaPoundstone) @xcollazo or @Milimetric please have a look and make sure the hourly pageviews dumps are available now. [20:16:44] 06Data-Engineering, 10Data-Engineering-Wikistats, 06Data Products: arywiki view stats too low for agent = user? - https://phabricator.wikimedia.org/T359004#9639826 (10VirginiaPoundstone) @Maurusian thank you for reporting this. We will investigate and report back here on what we find. [20:18:23] 06Data-Engineering, 10Data-Engineering-Wikistats, 06Data-Platform, 06Movement-Insights, 10Data Products (Data Products Sprint 11): Wikistats "Active Editors by Country" does not follow definition for active editors - https://phabricator.wikimedia.org/T360073#9639830 (10VirginiaPoundstone) [20:20:59] 06Data-Engineering, 06Data Products, 06Data-Platform, 06Movement-Insights: New Data Pipeline for New and Returning Active Editor metrics by Geo (Country & Region) and wiki - https://phabricator.wikimedia.org/T359646#9639861 (10VirginiaPoundstone) Thanks @Mayakp.wiki! Does this support current or upcoming O... [20:23:25] 06Data-Engineering, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Benthos: better management for unparsable logs - https://phabricator.wikimedia.org/T359627#9639900 (10VirginiaPoundstone) [20:27:10] 06Data-Engineering, 10Observability-Logging, 06Traffic: Check statsv and eventlogging VarnishKafka instances - https://phabricator.wikimedia.org/T359178#9639915 (10VirginiaPoundstone) [20:36:38] 06Data-Engineering, 10Data-Engineering-Dashiki, 06Data Products, 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9639972 (10VirginiaPoundstone) [20:51:21] 06Data-Engineering, 10Data-Engineering-Dashiki, 10Data Products (Data Products Sprint 13), 10MediaWiki-Platform-Team (Radar): Investigate surprising "10% Other" portion of Analytics Browsers report - https://phabricator.wikimedia.org/T342267#9639994 (10VirginiaPoundstone) [21:05:48] 06Data-Engineering, 10Data-Engineering-Dashiki, 06Data Products: Provide visitor stats that can be imported into caniuse.com - https://phabricator.wikimedia.org/T359750#9640039 (10VirginiaPoundstone) @Tgr If I understand your goal, you want access to data that tells you how many readers and editors have acce... [21:19:53] 14Analytics-Radar, 06Data-Engineering, 06Data Products, 10Metrics Platform Backlog: mw.user.generateRandomSessionId should return a UUID - https://phabricator.wikimedia.org/T266813#9640060 (10VirginiaPoundstone) @phuedx ` Does Metrics Platform support logging the MW session ID? ` ? [21:19:58] 06Data-Engineering, 10Observability-Logging, 06Traffic, 13Patch-For-Review: Install new Benthos instance on cp hosts - https://phabricator.wikimedia.org/T358109#9640061 (10VirginiaPoundstone) [21:19:59] 06Data-Engineering, 06Structured-Data-Backlog: Bump memory to enable large artifacts sync on HDFS - https://phabricator.wikimedia.org/T348958#9640062 (10VirginiaPoundstone) [21:21:09] 06Data-Engineering, 06Data Products, 06Data-Platform-SRE, 10Temporary accounts: Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#9640067 (10VirginiaPoundstone) [21:21:22] 06Data-Engineering, 06Data-Platform-SRE, 10Temporary accounts: Generate a list of Superset users affected by changes to IP masking/temp users - https://phabricator.wikimedia.org/T347510#9640068 (10VirginiaPoundstone) [22:14:45] hi! i suddenly lost the ability to query at dbstore1008.eqiad.wmnet (it says `ERROR 2013 (HY000): Lost connection to MySQL server at 'reading authorization packet', system error: 104`). is there some maintenance going on that'd cause this? [22:15:30] 06Data-Engineering, 06MediaWiki-Engineering, 06serviceops, 10WMF-JobQueue, and 2 others: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable" - https://phabricator.wikimedia.org/T249745#9640208 (10Ottomata) > I wonder if JobQueueGroup::lazyPush()/JobQueueEventBus could be rigged... [22:20:30] 06Data-Engineering, 06MediaWiki-Engineering, 06serviceops, 10WMF-JobQueue, and 2 others: Could not enqueue jobs: "Unable to deliver all events: 503: Service Unavailable" - https://phabricator.wikimedia.org/T249745#9640217 (10Ottomata) > This doesn't mean that MediaWiki shoudn't try to improve the situatio... [22:22:50] ottomata: since appear to be online – can you help me out, please? ^ [22:25:57] 06Data-Engineering, 10Observability-Logging, 06Traffic: Check statsv and eventlogging VarnishKafka instances - https://phabricator.wikimedia.org/T359178#9640223 (10Ottomata) @Fabfur I'm really really hoping we can remove varnishkafka-eventlogging after we complete {T353817}, as part of {T238230}. This is re... [22:26:59] urbanecm: hi I'm not sure if I can help you. I'm on an airplane, and I've mostly been on parent leave / sabbatical so am techfu rusty, and I haven't personaly looked at mariadb replicas in probably years. SorrYYy! [22:27:17] oh, sorry. i didn't realize you're flying. safe flight then! [23:22:14] 06Data-Engineering, 06Data Products, 06Data-Platform, 06Movement-Insights: New Data Pipeline for New and Returning Active Editor metrics by Geo (Country & Region) and wiki - https://phabricator.wikimedia.org/T359646#9640403 (10Mayakp.wiki) Hi @VirginiaPoundstone this work supports the **Essential Work** we... [23:29:46] 06Data-Engineering, 10MediaWiki-General, 10Event-Platform, 10MediaWiki-Platform-Team (Radar), 13Patch-For-Review: Create legacy EventLogging proxy HTTP intake (for MediaWikiPingback) endpoint to EventGate - https://phabricator.wikimedia.org/T353817#9640420 (10Krinkle) [23:37:27] 06Data-Engineering, 06Structured-Data-Backlog: Bump memory to enable large artifacts sync on HDFS - https://phabricator.wikimedia.org/T348958#9640432 (10Ottomata) Maybe: https://filesystem-spec.readthedocs.io/en/latest/api.html?highlight=clear%20cache#fsspec.utils.read_block ? [23:56:10] 06Data-Engineering, 06Structured-Data-Backlog: Bump memory to enable large artifacts sync on HDFS - https://phabricator.wikimedia.org/T348958#9640485 (10Ottomata) Or maybe: https://filesystem-spec.readthedocs.io/en/stable/features.html#file-buffering-and-random-access Change the ArtifactSource open method to...