[00:13:02] 10Data-Engineering, 10Product-Analytics, 10wmfdata-python: wmfdata.spark module should provide easy access to pyspark - https://phabricator.wikimedia.org/T293722 (10xcollazo) > Ah, so a plain import pyspark will work? Correct. So this should be solved by T302819. [00:17:17] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [00:20:43] RECOVERY - Check unit status of monitor_refine_eventlogging_analytics on an-launcher1002 is OK: OK: Status of the systemd unit monitor_refine_eventlogging_analytics https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [02:17:22] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0:Wikistats 2 service - https://phabricator.wikimedia.org/T288301 (10BPirkle) Circling back to this finally and reading @Milimetric comment in detail. I mostly agree. Some thoughts after further consideration: I like... [03:30:57] 10Data-Engineering, 10Machine-Learning-Team, 10Observability-Logging, 10observability, 10Event-Platform Value Stream (Sprint 03): Evaluate Benthos as stream processor - https://phabricator.wikimedia.org/T319214 (10lmata) [08:17:53] 10Data-Engineering, 10Machine-Learning-Team, 10Observability-Logging, 10observability, 10Event-Platform Value Stream (Sprint 03): Evaluate Benthos as stream processor - https://phabricator.wikimedia.org/T319214 (10fgiunchedi) >>! In T319214#8321555, @Ottomata wrote: > By the way, any Event Platform produ... [09:22:07] 10Data-Engineering, 10CheckUser, 10MW-1.38-notes (1.38.0-wmf.26; 2022-03-14), 10MW-1.39-notes (1.39.0-wmf.23; 2022-08-01), and 3 others: Update CheckUser for actor and comment table - https://phabricator.wikimedia.org/T233004 (10hashar) This task is about the migration to actor and comment table. T233004#5... [10:13:16] 10Data-Engineering, 10Equity-Landscape: Editorship Output Rank Metrics - https://phabricator.wikimedia.org/T306618 (10KCVelaga_WMF) I can also make the change at output metrics level, but I think it is best if it is handled as input metrics level, as I read from it, which I assume in turn takes the labels from... [10:17:57] 10Data-Engineering-Operations, 10Data Engineering Planning, 10Mail, 10SRE: Add xcollazo@wikimedia.org to the analytics-alerts mailing list - https://phabricator.wikimedia.org/T315486 (10BTullis) Thanks @Ladsgroup - noted. So it seems that we have five topic or team based `-alerts` lists on mailman already:... [10:20:12] 10Data-Engineering, 10Equity-Landscape: Grants input metric - https://phabricator.wikimedia.org/T309276 (10KCVelaga_WMF) regional level metrics can be accessed at ` SELECT geography_level, iso2_country_code, country_name, continent, sub_continent, grants_usd_historical_to_date, gr... [10:20:59] 10Data-Engineering, 10Equity-Landscape: Grants Leadership Output Metrics transformation - https://phabricator.wikimedia.org/T306620 (10KCVelaga_WMF) Transformation code at ` WITH grants_input_metric_hstack AS ( SELECT geography_level, iso2_country_code AS country_code,... [10:23:44] 10Data-Engineering, 10Equity-Landscape: Grants Leadership Output Metrics transformation - https://phabricator.wikimedia.org/T306620 (10KCVelaga_WMF) access output rank metrics at ` SELECT geography_level, country_code, country_name, sub_continent, continent, grantee_presence, gran... [10:24:50] 10Data-Engineering, 10Equity-Landscape: Grants Leadership Output Metrics transformation - https://phabricator.wikimedia.org/T306620 (10KCVelaga_WMF) a:05KCVelaga_WMF→03JAnstee_WMF [11:55:12] (VarnishkafkaNoMessages) firing: varnishkafka on cp2033 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2033%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [12:00:12] (VarnishkafkaNoMessages) resolved: varnishkafka on cp2033 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2033%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [12:28:12] (VarnishkafkaNoMessages) firing: varnishkafka on cp2035 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2035%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [12:33:12] (VarnishkafkaNoMessages) resolved: (3) varnishkafka on cp2033 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [13:11:51] 10Data-Engineering, 10Event-Platform Value Stream (Sprint 03), 10Spike: [SPIKE] Build simple stateless service using Flink SQL - https://phabricator.wikimedia.org/T318856 (10Ottomata) Looks great, so cool that it works. Today in standup we moved this to Done, hope that's okay with you. [14:40:10] 10Analytics-Jupyter, 10Data-Engineering, 10Product-Analytics: Change puppet jupyterhub module to point to conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) [15:13:11] 10Data-Engineering-Operations, 10Data Engineering Planning, 10Mail, 10SRE: Add xcollazo@wikimedia.org to the analytics-alerts mailing list - https://phabricator.wikimedia.org/T315486 (10Ladsgroup) >>! In T315486#8324480, @BTullis wrote: > Thanks @Ladsgroup - noted. So it seems that we have five topic or te... [15:13:45] 10Data-Engineering-Operations, 10Data Engineering Planning, 10Mail, 10SRE: Add xcollazo@wikimedia.org to the analytics-alerts mailing list - https://phabricator.wikimedia.org/T315486 (10Ladsgroup) Another thing. Mailman2 had many many issues but mailman3 (the current infra) is much easier to use and handle. [15:15:49] 10Data-Engineering: Stop ingesting data to the old AQS cluster - https://phabricator.wikimedia.org/T302276 (10Jclark-ctr) [15:15:51] 10Data-Engineering, 10Cassandra, 10Epic, 10Platform Team Workboards (Platform Engineering Reliability): Cassandra3 migration for Analytics AQS - https://phabricator.wikimedia.org/T249755 (10Jclark-ctr) [16:30:04] 10Analytics, 10Data-Engineering, 10Event-Platform Value Stream, 10EventStreams: Old events in the stream - https://phabricator.wikimedia.org/T320558 (10Ottomata) > Simple code from Wikitech example: I just pasted this code (with ! dt.startsWith filter changed to today) into the developer tools console in... [16:35:07] (03PS3) 10Joal: Update mediawiki-history page computation [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/842922 (https://phabricator.wikimedia.org/T318589) [17:12:10] 10Analytics-Jupyter, 10Data-Engineering, 10Product-Analytics, 10Patch-For-Review: Change puppet jupyterhub module to point to conda-analytics - https://phabricator.wikimedia.org/T321088 (10mpopov) Will this break https://gerrit.wikimedia.org/r/plugins/gitiles/analytics/wmf-product/jobs/+/refs/heads/master/... [18:03:34] 10Analytics-Jupyter, 10Data Engineering Planning, 10Product-Analytics, 10Patch-For-Review: Change puppet jupyterhub module to point to conda-analytics - https://phabricator.wikimedia.org/T321088 (10EChetty) [18:03:52] 10Analytics, 10Data Engineering Planning, 10Event-Platform Value Stream, 10EventStreams: Old events in the stream - https://phabricator.wikimedia.org/T320558 (10EChetty) [18:04:04] 10Analytics-Wikistats, 10Data Engineering Planning, 10translatewiki.net, 10I18n: Addition of Slovene language in the WikiStats 2.0 language selector - https://phabricator.wikimedia.org/T320356 (10EChetty) [18:04:51] 10Analytics-Radar, 10API Platform, 10Data Engineering Planning, 10Pageviews-API, 10Tool-Pageviews: 429 Too Many Requests hit despite throttling to 100 req/sec - https://phabricator.wikimedia.org/T219857 (10EChetty) [18:05:01] 10Analytics, 10Data Engineering Planning, 10Event-Platform Value Stream, 10Performance-Team (Radar): Avoid extra HTTPS connections for most Event Platform beacons - https://phabricator.wikimedia.org/T263049 (10EChetty) [18:07:04] 10Analytics-Jupyter, 10Data Engineering Planning, 10Product-Analytics, 10Patch-For-Review: Change puppet jupyterhub module to point to conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) > Will this break https://gerrit.wikimedia.org/r/plugins/gitiles/analytics/wmf-product/jobs/+/refs/... [19:09:53] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:rack/setup/install an-coord100[3,4] & an-mariadb100[1,2] - https://phabricator.wikimedia.org/T321119 (10RobH) [19:10:28] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q2:rack/setup/install an-coord100[3,4] & an-mariadb100[1,2] - https://phabricator.wikimedia.org/T321119 (10RobH) [20:15:15] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0:Wikistats 2 service - https://phabricator.wikimedia.org/T288301 (10VirginiaPoundstone) I made a[[ https://miro.com/app/board/uXjVPMLNmao=/ | map of AQS 2.0 ]] to wrap my head around the naming convention discussion.... [20:53:24] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Patch-For-Review: Change puppet jupyterhub module to point to conda-analytics - https://phabricator.wikimedia.org/T321088 (10mpopov) > In this task we are deprecating `anaconda-wmf`, but we are not removing it. Your script will contin... [21:30:47] 10Data-Engineering-Planning, 10Data Pipelines, 10Pageviews-Anomaly, 10Product-Analytics: Analyze possible bot traffic for frwiki article Cookie (informatique) - https://phabricator.wikimedia.org/T313114 (10odimitrijevic) Improving the bot traffic detection is on the longer term planning horizon, however we... [21:34:57] 10Data-Engineering-Planning, 10Shared-Data-Infrastructure, 10Data Pipelines (Sprint 03): Create Plan for Spark 2 Deprecation - https://phabricator.wikimedia.org/T318367 (10odimitrijevic) Thanks @mforns for creating the list. Would it be helpful to have the risk v complexity column similar to the Oozie migrat... [23:06:40] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10Product-Analytics, 10Growth-Team (Current Sprint), and 3 others: Eventgate error: '' should have required property 'database', '' should have required property 'performer' - https://phabricator.wikimedia.org/T317343 (10Etonkovidova) 05Open→03... [23:58:57] 10Data-Engineering, 10DC-Ops, 10SRE, 10ops-eqiad: Q4:(Need By: TBD) rack/setup/install an-presto10[06-15].eqiad.wmnet - https://phabricator.wikimedia.org/T306835 (10ops-monitoring-bot) Cookbook cookbooks.sre.hosts.reimage was started by cmjohnson@cumin1001 for host an-presto1007.eqiad.wmnet with OS bullseye