[01:48:38] 10Data-Engineering-Radar, 10Cassandra: Bootstrap new Cassandra nodes (eqiad) - https://phabricator.wikimedia.org/T307802 (10Eevans) [01:48:57] 10Data-Engineering-Radar, 10Cassandra: AQS multi-datacenter cluster expansion - https://phabricator.wikimedia.org/T307641 (10Eevans) [01:48:59] 10Data-Engineering-Radar, 10Cassandra: Bootstrap new Cassandra nodes (eqiad) - https://phabricator.wikimedia.org/T307802 (10Eevans) 05Open→03Resolved a:03Eevans Complete. [01:52:36] 10Data-Engineering-Radar, 10Cassandra: AQS multi-datacenter cluster expansion - https://phabricator.wikimedia.org/T307641 (10Eevans) [01:52:57] 10Data-Engineering-Radar, 10Cassandra: AQS multi-datacenter cluster expansion - https://phabricator.wikimedia.org/T307641 (10Eevans) 05Open→03Resolved Complete. [06:51:14] 10Data-Engineering, 10CheckUser, 10MW-1.38-notes (1.38.0-wmf.26; 2022-03-14), 10MW-1.39-notes (1.39.0-wmf.23; 2022-08-01), and 3 others: Update CheckUser for actor and comment table - https://phabricator.wikimedia.org/T233004 (10Marostegui) [08:21:44] 10Data-Engineering, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10Joe) p:05Triage→03Unbreak! [09:32:45] (03PS17) 10Aqu: Add HdfsXMLFsImageConverter to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) [09:36:48] (03CR) 10Aqu: "I have added all the modifications we talked about. The most important ones are:" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) (owner: 10Aqu) [09:43:05] (03CR) 10Aqu: "@Mforns, could you read this change, please?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) (owner: 10Aqu) [09:56:15] 10Data-Engineering, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) a:03BTullis [10:35:20] 10Data-Engineering-Planning, 10SRE, 10Data Pipelines (Sprint 05-06), 10Kubernetes, 10Patch-For-Review: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) Adding various Data Engineering planning and streaming... [10:43:09] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes, 10Patch-For-Review: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) [10:58:49] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes, 10Patch-For-Review: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) I have received guidance from @Clement_Goubert on the st... [11:11:04] !log depooling codfw for eventstreams for T324074 [11:11:06] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:11:07] T324074: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 [11:19:26] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes, 10Patch-For-Review: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) New eventstreams clients in codfw have virtually stopped... [11:31:17] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) eventstreams in codfw has been completely drained. {F35825309,width=60%} Proce... [11:32:14] !log destroying the eventstreams deployment in codfw and reapplying for T324074 [11:32:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:32:16] T324074: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 [11:34:55] !log repooling codfw for eventstreams for T324074 [11:34:58] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:46:11] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) eventreams in codfw is now handling traffic again nicely. {F35825322,width=60%... [11:59:20] !log depooling eqiad for eventstreams for T324074 [11:59:23] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:59:23] T324074: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 [12:05:30] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) Depooled eqiad using the cookbook method, which was new to me. ` btullis@cumin... [12:23:14] ! destroying the eventstreams deployment in eqiad and reapplying for T324074 [12:23:15] T324074: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 [12:29:53] !log repooling eqiad for eventstreams for T324074 [12:29:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [12:29:56] T324074: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 [12:36:58] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) All looks good with eventstreams in eqiad again. {F35825355,width=60%} [12:48:18] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) Proceeding with the deployment. The plan is as follows: - Stop puppet on st... [12:48:51] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10Ottomata) Thank you both! [12:50:23] btullis: <3 [12:54:13] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) ` btullis@cumin1001:~$ sudo cumin O:statistics::explorer 'puppet agent --disabl... [12:59:16] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) Puppet change merged and deployed. This is our debdeploy spec for this change:... [13:02:50] !log deploying conda-analytics 0.0.12 to stat boxes for T321088 [13:02:52] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:02:53] T321088: Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 [13:03:54] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) Pushing out the update to the stat boxes with: ` btullis@cumin1001:~$ sudo debd... [13:33:07] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) Tests are OK, with the exception that all users appear to be able to stop other users' sessions. Rol... [13:34:14] ping btullis - SDI standup! [13:36:46] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) It turns out that all analytics-admins can now access the Admin tab. This was already in the config,... [13:39:24] !log pushing out conda-analytics to all remaining servers `btullis@cumin1001:~$ sudo debdeploy deploy -u 2022-11-30-conda-analytics.yaml -Q P:analytics::conda_analytics` [13:39:25] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:40:05] joal: Apologies for missing standup, was fully absorbed by conda-analytics jupyter deployment. Will add async updates. [13:40:18] np btullis - I saw our message on slack :) [13:40:22] + [13:40:26] +Y [13:40:39] meh - my doesn't work well :( [13:49:36] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) Sanity tests done on `stat1007`: - Create a `conda-analytics` environment thru JupyterHub, and us... [13:51:15] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) Pushed out the updated conda-analytics package to all remaining servers with: ` btullis@cumin1001:~$... [13:54:57] 10Data-Engineering-Planning, 10API Platform (Sprint 02), 10AQS2.0, 10Platform Engineering Roadmap, 10User-Eevans: Obtain security review of uniqueDevices - https://phabricator.wikimedia.org/T320976 (10JArguello-WMF) [13:56:18] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) As per debmonitor, the `conda-analytics` 0.0.12 package has reached all stat and worker nodes: https... [13:56:47] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10BTullis) This was the output from the `debdeploy` command. Notice that `libmariadb3` was also updated on a num... [13:58:37] 10Data-Engineering, 10API Platform (Sprint 02), 10AQS2.0, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Pageviews: Implement Unit Tests - https://phabricator.wikimedia.org/T299735 (10JArguello-WMF) [14:01:04] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) > Notice that libmariadb3 was also updated on a number of hosts, but I believe that was expected. @x... [14:06:11] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) Phew! I think we are finally done with this ticket! Thank you *so* much for all the help @BTullis,... [14:09:06] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink SQL queries should access Kafka topics from a Catalog - https://phabricator.wikimedia.org/T322022 (10JArguello-WMF) [14:11:30] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05): Flink SQL queries should access Kafka topics from a Catalog - https://phabricator.wikimedia.org/T322022 (10tchin) [[ https://gitlab.wikimedia.org/tchin/flink-wmf-event-catalog/-/blob/main/catalog/src/main/java/org/wikimedia/eventplatform/W... [14:22:41] 10Data-Engineering, 10Event-Platform Value Stream: [SPIKE] Use Flink to develop bounded service - https://phabricator.wikimedia.org/T324108 (10lbowmaker) [14:33:30] 10Data-Engineering-Planning, 10Event-Platform Value Stream (Sprint 05), 10Patch-For-Review: Create a shared flink docker image - https://phabricator.wikimedia.org/T316519 (10Ottomata) Writing down some ideas and thoughts from todays talk with @gmodena: - We will only support Application Mode Deployments, no... [14:42:02] 10Data-Engineering, 10API Platform (Sprint 02), 10AQS2.0, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0: Pageviews: Implement Unit Tests - https://phabricator.wikimedia.org/T299735 (10JArguello-WMF) [14:45:25] 10Data-Engineering-Planning, 10API Platform (Sprint 02), 10AQS2.0, 10Platform Engineering Roadmap, 10User-Eevans: Obtain security review of uniqueDevices - https://phabricator.wikimedia.org/T320976 (10JArguello-WMF) p:05Triage→03High [14:46:11] 10Data-Engineering-Planning, 10DC-Ops, 10SRE, 10Shared-Data-Infrastructure, 10ops-eqiad: Q2:rack/setup/install cephosd100[1-5] - https://phabricator.wikimedia.org/T322760 (10Jclark-ctr) installed 6.4tb nvme into servers [14:48:28] 10Data-Engineering, 10Event-Platform Value Stream: Flink + Event Platform integration for writing into streams via Table API - https://phabricator.wikimedia.org/T324114 (10Ottomata) [14:50:31] 10Data-Engineering, 10Event-Platform Value Stream: Flink + Event Platform integration for writing into streams via Table API - https://phabricator.wikimedia.org/T324114 (10Ottomata) [14:51:16] 10Data-Engineering, 10Event-Platform Value Stream: Flink + Event Platform integration for writing into streams via Table API - https://phabricator.wikimedia.org/T324114 (10Ottomata) BTW, there may be other better ways to do this than a custom serialization format. Please comment / update description with findi... [14:59:33] 10Data-Engineering, 10Event-Platform Value Stream: [SPIKE] Use Flink for batch backfilling - https://phabricator.wikimedia.org/T324108 (10Ottomata) [15:21:57] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10EventStreams: EventStreams doesn't show the Wikistories-* streams - https://phabricator.wikimedia.org/T307679 (10SBisson) 05Open→03Resolved a:03SBisson Thanks for looking into it. [15:28:12] 10Data-Engineering, 10Equity-Landscape: Grants input metric - https://phabricator.wikimedia.org/T309276 (10ntsako) Updated query: Affiliates pulls from Grants leadership so it should trickle down the changes. ` WITH grants_countries AS ( SELECT distinct country_code FROM ntsako.grants_input_metric... [15:28:54] 10Data-Engineering, 10Equity-Landscape: Grants input metric - https://phabricator.wikimedia.org/T309276 (10ntsako) a:05ntsako→03KCVelaga_WMF [16:17:22] 10Data-Engineering, 10Event-Platform Value Stream (Sprint 05): Deploy Mediawiki Stream Enrichment on an-launcher1002. - https://phabricator.wikimedia.org/T323914 (10Ottomata) Nice! [16:19:37] !log restarting presto-server on an-coord1001 for T321960 and T321231 [16:19:42] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:19:42] T321960: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 [16:19:43] T321231: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 [16:20:41] !log roll-restarting presto workers for T321960 and T321231 [16:20:45] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:24:40] (03PS1) 10MNeisler: Add the mediawiki_web_ab_test_enrollment stream to the allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/862298 (https://phabricator.wikimedia.org/T323664) [16:27:14] (03Abandoned) 10MNeisler: Add the mediawiki_web_ab_test_enrollment stream to the allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/862298 (https://phabricator.wikimedia.org/T323664) (owner: 10MNeisler) [16:32:43] (03PS2) 10MNeisler: Add the mediawiki_web_ab_test_enrollment stream to the allowlist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/859652 (https://phabricator.wikimedia.org/T323664) [16:33:31] 10Data-Engineering-Planning, 10Data Pipelines, 10Product-Analytics: Improved edit summary data in mediawiki_history - https://phabricator.wikimedia.org/T318010 (10mpopov) > The multilingual edit summaries that Wikidata, Commons structured data and some Growth features use don't contain any interesting data w... [16:34:21] (03PS1) 10Sergio Gimeno: HomepageModule: document impact module link ids [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/862303 (https://phabricator.wikimedia.org/T324087) [16:34:44] (03CR) 10MNeisler: Add the mediawiki_web_ab_test_enrollment stream to the allowlist (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/859652 (https://phabricator.wikimedia.org/T323664) (owner: 10MNeisler) [16:44:58] !log roll-restarting presto workers again for T321960 and T321231 [16:45:02] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:45:02] T321960: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 [16:45:03] T321231: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 [16:45:19] 10Analytics-Jupyter, 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06): Add support for jupyterlab on conda-analytics - https://phabricator.wikimedia.org/T321088 (10xcollazo) Opened tickets for follow up issues: Investigate whether admin privileges on Jupyter are correct -... [16:56:06] (03CR) 10Sergio Gimeno: "Does the schema needs to be bump for a docs change like this one? The values documented are already in use." [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/862303 (https://phabricator.wikimedia.org/T324087) (owner: 10Sergio Gimeno) [17:08:39] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 (10JAllemandou) Hi @nshahquinn-wmf , a fix has been deployed by @BTullis - could you please confirm this is fix... [17:09:52] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 (10JAllemandou) Hi here as well @nshahquinn-wmf - A fix (the same as in the other bug) has been deployed by @BT... [17:11:26] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 (10BTullis) For reference, here is my test query prior to the fix: ` resto> SELECT contribution_attempt_id, COU... [17:13:20] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10BTullis) [17:17:04] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10BTullis) Hopefully this is fixed now. @Milimetric are you able to confirm? Thanks. {F35825524,width=60%} [17:17:33] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10BTullis) Is this still an issue for `hive`? [17:18:47] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 (10nshahquinn-wmf) >>! In T321960#8432515, @BTullis wrote: > ` > Here is the same query after the configuration... [17:21:08] 10Data-Engineering-Planning, 10Product-Analytics, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Presto returns incorrect data for an added field - https://phabricator.wikimedia.org/T321960 (10BTullis) 05Open→03Resolved [17:48:46] 10Data-Engineering, 10Product-Analytics, 10Wmfdata-Python: Wmfdata-Python triggers a Pandas warning during mariadb.run - https://phabricator.wikimedia.org/T324135 (10nshahquinn-wmf) [17:50:17] (03CR) 10Kosta Harlan: [C: 04-1] HomepageModule: document impact module link ids (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/862303 (https://phabricator.wikimedia.org/T324087) (owner: 10Sergio Gimeno) [17:55:43] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10Milimetric) Works! Not an issue in hive either, @BTullis [17:57:23] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10BTullis) \o/ [17:58:02] 10Data-Engineering-Planning, 10Data Pipelines (Sprint 05-06), 10Patch-For-Review: Bug: User History has mismatching order of fields in Parquet vs. Hive - https://phabricator.wikimedia.org/T321231 (10BTullis) 05Open→03Resolved a:03BTullis [17:58:58] 10Data-Engineering, 10Product-Analytics, 10Wmfdata-Python: Create a script that installs Wmfdata-Python in development mode - https://phabricator.wikimedia.org/T294668 (10nshahquinn-wmf) 05Open→03Declined In a `conda-analytics` environment, `pip install -e .` works just fine, so there's no need for an in... [18:00:02] 10Data-Engineering-Planning, 10Event-Platform Value Stream, 10SRE, 10Kubernetes: eventstreams cannot be deployed and its deployments will need to be destroyed and recreated - https://phabricator.wikimedia.org/T324074 (10BTullis) 05Open→03Resolved >>! In T324074#8431657, @Ottomata wrote: > Thank you bot... [18:02:12] 10Data-Engineering, 10Data-Engineering-Kanban, 10Product-Analytics, 10Wmfdata-Python: conda-create-stacked breaks wmfdata.presto - https://phabricator.wikimedia.org/T301734 (10nshahquinn-wmf) 05Open→03Declined The simpler base environment is definitely real now, and in any case I've created a lot of ne... [18:08:54] 10Data-Engineering, 10Product-Analytics, 10Wmfdata-Python: Allow query results to be cached in the filesystem or HDFS - https://phabricator.wikimedia.org/T248739 (10nshahquinn-wmf) 05Open→03Declined I still kind of like this idea, but it would be significant amount of work for a pretty marginal benefit.... [18:48:22] (03PS2) 10Sergio Gimeno: HomepageModule: document impact module link ids [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/862303 (https://phabricator.wikimedia.org/T324087) [18:50:08] (03CR) 10Sergio Gimeno: HomepageModule: document impact module link ids (031 comment) [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/862303 (https://phabricator.wikimedia.org/T324087) (owner: 10Sergio Gimeno) [19:33:18] (03PS5) 10Nmaphophe: GDI Equity Landscape Tables/Scripts [analytics/refinery] - 10https://gerrit.wikimedia.org/r/858370 [19:51:29] PROBLEM - Check systemd state on an-launcher1002 is CRITICAL: CRITICAL - degraded: The following units failed: produce_canary_events.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [19:52:10] (03CR) 10Mforns: "Code looks great! I left a bunch of comments, but all probably very unimportant." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/852315 (https://phabricator.wikimedia.org/T321168) (owner: 10Aqu) [19:56:53] 10Data-Engineering, 10Event-Platform Value Stream: Flink Tables should have a default ROWTIME column. - https://phabricator.wikimedia.org/T324144 (10gmodena) [20:01:47] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state