[03:17:12] (VarnishkafkaNoMessages) firing: varnishkafka on cp2029 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2029%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [03:22:12] (VarnishkafkaNoMessages) resolved: varnishkafka on cp2029 is not sending enough cache_text requests - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Varnishkafka - https://grafana.wikimedia.org/d/000000253/varnishkafka?orgId=1&var-datasource=codfw%20prometheus/ops&var-cp_cluster=cache_text&var-instance=cp2029%3A9132&viewPanel=14 - https://alerts.wikimedia.org/?q=alertname%3DVarnishkafkaNoMessages [08:01:37] 10Analytics-Wikistats, 10Data Engineering Planning, 10Data Pipelines: "Pages to date" not loading with "daily" metric - https://phabricator.wikimedia.org/T312717 (10JAllemandou) Thank you for the troubleshooting and the description @Milimetric! [08:18:40] 10Analytics: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Siko_WMDE) Hi, Kindly asking, if somebody could help us out with this request. Best regards and thank you, Simon [08:20:46] (03CR) 10Joal: [C: 03+1] "Added comments to Marcel's comment - Thank you so much Marcel for being more thorough than I have and suggesting we take the opportunity o" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832336 (https://phabricator.wikimedia.org/T317124) (owner: 10Xcollazo) [08:54:37] (03Abandoned) 10Milimetric: Increase resources for history job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/803551 (https://phabricator.wikimedia.org/T309987) (owner: 10Milimetric) [08:56:20] (03CR) 10Milimetric: [V: 03+2 C: 03+2] "this is just cleanup, should've merged it earlier" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/820776 (owner: 10Milimetric) [08:57:11] (03Abandoned) 10Milimetric: Add base_uri config parameter to Superset source [analytics/datahub] (wmf) - 10https://gerrit.wikimedia.org/r/812023 (https://phabricator.wikimedia.org/T306903) (owner: 10Milimetric) [08:58:36] (03Abandoned) 10Milimetric: [WIP] Restore logic from oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/774535 (https://phabricator.wikimedia.org/T304884) (owner: 10Milimetric) [08:59:19] (03CR) 10Gergő Tisza: [C: 03+2] Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [08:59:35] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Review TODOs [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/804432 (owner: 10Milimetric) [08:59:53] (03CR) 10Gergő Tisza: [C: 04-1] "will run materialize" [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [08:59:57] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Remove unused vega timeseries component [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/804433 (owner: 10Milimetric) [09:00:05] (03CR) 10CI reject: [V: 04-1] Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:00:20] hi all, I'm trying to find the docummentation or a working example for the wmfdata.mariadb package. Does anyone here knows where to find this? [09:00:23] (03CR) 10Phuedx: [C: 04-1] "If this is going to be used with the pre-existing stream (which I think it is?), then this schema will have to be compatible with the sche" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [09:00:54] (03CR) 10Milimetric: [C: 03+2] Fix annotation display [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/763847 (owner: 10Milimetric) [09:02:08] (03Abandoned) 10Milimetric: [WIP] Finding ways to deduplicate users across projects and geographies [analytics/refinery] - 10https://gerrit.wikimedia.org/r/726945 (owner: 10Milimetric) [09:02:31] (03PS2) 10Gergő Tisza: Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:02:57] (03CR) 10CI reject: [V: 04-1] Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:03:50] (03PS3) 10Gergő Tisza: Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:17:29] (03CR) 10Kosta Harlan: [C: 03+2] Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:18:08] (03Merged) 10jenkins-bot: Add Welcome Survey reminder to the module list [schemas/event/secondary] - 10https://gerrit.wikimedia.org/r/833108 (https://phabricator.wikimedia.org/T310320) (owner: 10Nettrom) [09:50:13] 10Analytics: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10JAllemandou) Ping @Ottomata and @BTullis - Can you help please? [09:50:35] 10Data-Engineering-Operations: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10JAllemandou) [09:51:53] PROBLEM - Check systemd state on an-launcher1002 is CRITICAL: CRITICAL - degraded: The following units failed: produce_canary_events.service https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [09:55:17] PROBLEM - Check unit status of produce_canary_events on an-launcher1002 is CRITICAL: CRITICAL: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [10:00:50] (HdfsRpcQueueLength) firing: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:01:13] RECOVERY - Check systemd state on an-launcher1002 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state [10:05:50] (HdfsRpcQueueLength) resolved: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:06:37] RECOVERY - Check unit status of produce_canary_events on an-launcher1002 is OK: OK: Status of the systemd unit produce_canary_events https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [10:08:50] (HdfsRpcQueueLength) firing: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:13:50] (HdfsRpcQueueLength) resolved: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:22:00] 10Quarry: See if query is running or complete in browser tab - https://phabricator.wikimedia.org/T316307 (10github-toolforge-bot) vivian-rook closed https://github.com/toolforge/quarry/pull/7 [10:22:23] 10Quarry: See if query is running or complete in browser tab - https://phabricator.wikimedia.org/T316307 (10rook) 05In progress→03Resolved [10:23:50] (HdfsRpcQueueLength) firing: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:28:50] (HdfsRpcQueueLength) resolved: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:30:50] (HdfsRpcQueueLength) firing: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [10:36:14] 10Data-Engineering, 10Research: Consider adding more namespaces to Clickstream dataset - https://phabricator.wikimedia.org/T296359 (10MGerlach) Another potential expansion could be adding Category-namespace to the clickstream. There was a question to Research how readers use categories ([[ https://meta.wikime... [10:55:50] (HdfsRpcQueueLength) resolved: RPC call queue length on the analytics-hadoop cluster is too high. - https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hadoop/Alerts#HDFS_Namenode_RPC_length_queue - https://grafana.wikimedia.org/d/000000585/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen - https://alerts.wikimedia.org/?q=alertname%3DHdfsRpcQueueLength [11:30:11] (03CR) 10Kosta Harlan: mediawiki/client/error: rename 'tags' to 'error_context' (031 comment) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [12:14:25] 10Analytics, 10Infrastructure-Foundations, 10Mail: kerberos manage_principals.py emails go to spam - https://phabricator.wikimedia.org/T318155 (10jbond) p:05Triage→03Medium [12:53:14] (03CR) 10Phuedx: [C: 03+1] "+Ottomata because I don't have +2 in this repo…" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [12:57:11] (03CR) 10Gergő Tisza: mediawiki/client/error: rename 'tags' to 'error_context' (031 comment) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [13:16:35] PROBLEM - SSH on analytics1077.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [13:45:31] 10Data-Engineering, 10Equity-Landscape: Editorship Input Metrics - https://phabricator.wikimedia.org/T309274 (10KCVelaga_WMF) @JAnstee_WMF I have fixed the permissions. Can you check if you are able to access now? [14:17:51] RECOVERY - SSH on analytics1077.mgmt is OK: SSH OK - OpenSSH_7.4 (protocol 2.0) https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook [14:23:53] (03CR) 10Xcollazo: Move most CREATE statements from hive/ to hql/ (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832336 (https://phabricator.wikimedia.org/T317124) (owner: 10Xcollazo) [14:31:44] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform Value Stream, 10Patch-For-Review: Move Kafka Jumbo's TLS clients to the new bundle - https://phabricator.wikimedia.org/T296064 (10Ottomata) Ping @BTullis @odimitrijevic @emil BTW, @elukey I will likely reach out to you about how to do PKI righ... [14:44:14] (03CR) 10Mforns: Begin sanitizing Wikistories streams (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832383 (https://phabricator.wikimedia.org/T312262) (owner: 10Neil P. Quinn-WMF) [14:45:56] 10Data-Engineering, 10Equity-Landscape: Readership input metrics - https://phabricator.wikimedia.org/T309273 (10ntsako) Hi Jaime, Please note the tables and statements that go with them: ` SELECT country_code, metric_value global_average_devices_column_z, year FROM ntsako.georeadership_inp... [14:53:05] (03CR) 10Mforns: Move most CREATE statements from hive/ to hql/ (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832336 (https://phabricator.wikimedia.org/T317124) (owner: 10Xcollazo) [14:55:25] 10Data-Engineering-Kanban, 10Data Engineering Planning, 10Event-Platform Value Stream, 10Metrics-Platform, 10Patch-For-Review: Remove StreamConfig::INTERNAL_SETTINGS logic from EventStreamConfig and do it in EventLogging client instead - https://phabricator.wikimedia.org/T286344 (10Ottomata) +1 [14:58:53] heya ottomata :] one question please, do you know what those analytics1077.mgmt/SSH alerts could be about? [15:00:58] (03PS3) 10Xcollazo: Move most CREATE statements from hive/ to hql/ [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832336 (https://phabricator.wikimedia.org/T317124) [15:03:08] 10Data-Engineering-Operations, 10SRE-Access-Requests: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Ottomata) [15:04:46] 10Data-Engineering-Operations, 10SRE-Access-Requests: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Ottomata) The rest of the access for this account was given in {T314117}. This request is adding Kerberos only. I approve. Doing now... [15:07:16] 10Data-Engineering-Operations, 10SRE-Access-Requests: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Ottomata) @Aline_Bruenger_WMDE you should have an email with instructions to login and set your kerberos password. Please confirm when you've done so. :) [15:17:43] 10Data-Engineering-Operations, 10SRE-Access-Requests: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Siko_WMDE) Hi @Ottomata , I also would need Kerberos (siko, username: Siko WMDE), the rest of the access for my account was also already given (see https:/... [15:29:58] (03CR) 10Ottomata: "Can you add a changelog.md file to this directory to explain the changes?" [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [15:31:01] (03CR) 10Ottomata: mediawiki/client/error: rename 'tags' to 'error_context' (031 comment) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [15:35:33] 10Data-Engineering-Operations, 10SRE, 10SRE-Access-Requests, 10Patch-For-Review: Requesting Kerberos access for alinebruenger and siko - https://phabricator.wikimedia.org/T316766 (10Ottomata) Ah, sorry missed that. I can do it here. Link to {T315878}. You should have an email now. [15:39:55] (03PS5) 10Gergő Tisza: mediawiki/client/error: rename 'tags' to 'error_context' [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) [15:40:19] (03CR) 10Gergő Tisza: mediawiki/client/error: rename 'tags' to 'error_context' (031 comment) [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [15:41:52] mforns: no but i wouldn't worry about them. the .mgmt interfaces are very backend ones that are used for connecting to the machines console directly, usually only used when installating or reimaging the host [15:42:03] it looks like it recovered, and i see a few more non analytics ones doing that at the same time [15:45:15] !log kill oozie hourly cassandra loading job (1 job) in favor of the airflow one [15:45:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:05:35] (03CR) 10Ottomata: [C: 03+2] mediawiki/client/error: Improves ECS compatibility [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/647025 (owner: 10Jason Linehan) [16:05:47] (03CR) 10Ottomata: mediawiki/client/error: Improves ECS compatibility [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/647025 (owner: 10Jason Linehan) [16:06:04] (03CR) 10Ottomata: [C: 03+2] mediawiki/client/error: rename 'tags' to 'error_context' [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/829295 (https://phabricator.wikimedia.org/T316992) (owner: 10Gergő Tisza) [16:06:25] (03CR) 10CI reject: [V: 04-1] mediawiki/client/error: Improves ECS compatibility [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/647025 (owner: 10Jason Linehan) [16:19:05] mforns, milimetric: I missed the message on the chan earlier during standup - Do any of you remember what it was about? [16:19:19] (I killed the window too fast, just saw m name :S) [16:21:39] 10Data-Engineering, 10Equity-Landscape: Editorship Input Metrics - https://phabricator.wikimedia.org/T309274 (10JAnstee_WMF) @KCVelaga_WMF - Yes, permissions are fixed [16:24:14] 10Data-Engineering, 10Equity-Landscape: Load country data - https://phabricator.wikimedia.org/T310712 (10JAnstee_WMF) thanks, @ntsako Next steps I will take will be: * Report back to classification owners about unclassed to grants and WNF regions * Work to finalize reduced column and row set for production [16:25:23] 10Data-Engineering, 10Equity-Landscape: Load country data - https://phabricator.wikimedia.org/T310712 (10JAnstee_WMF) 05Open→03In progress p:05Triage→03Medium [16:25:25] 10Data-Engineering, 10Equity-Landscape: Extract + Transformation Raw Data into Input Metrics - https://phabricator.wikimedia.org/T306625 (10JAnstee_WMF) [16:27:39] 10Data-Engineering, 10Equity-Landscape: Editorship Input Metrics - https://phabricator.wikimedia.org/T309274 (10JAnstee_WMF) p:05Triage→03High [16:28:02] 10Data-Engineering, 10Equity-Landscape: Readership input metrics - https://phabricator.wikimedia.org/T309273 (10JAnstee_WMF) p:05Triage→03High [16:31:12] (03CR) 10Xcollazo: "Patch set 3 takes care of review comments as of now. I've updated the commit message as well to reflect new state. Please re-review." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832336 (https://phabricator.wikimedia.org/T317124) (owner: 10Xcollazo) [16:37:09] joal: there's nothing during standup except 4 gerrit CR echos from Andrew, about mediawiki/client/error, on schemas/event/primary [16:37:22] ack milimetric - many thanks [17:02:57] hm - there is a bottleneck on the production queue [17:04:00] too many applications launched, many waiting for slots [17:08:43] dcausse: Hi, you have a flink job running in prod queue - I wonder if that is on purpose :) [17:10:08] joal: https://yarn.wikimedia.org/cluster/app/application_1655808530211_306341 is prod but https://yarn.wikimedia.org/cluster/app/application_1654780610649_4378 can be stopped [17:10:24] it's prod because I need to restart it in k8s [17:10:56] stopping application_1654780610649_4378 [17:11:06] Thanks dcausse [17:11:29] I'll sent a patch soon allowing for more concurrent AMs to run on the prod queue [17:14:56] joal: if application_1655808530211_306341 can stay alive til tomorrow this would be great [17:15:06] (03CR) 10Gergő Tisza: "I guess this needs to be 3.0.0 now if we still want to merge it. Also I think still blocked on T265938 (which was done for another schema " [schemas/event/primary] - 10https://gerrit.wikimedia.org/r/647025 (owner: 10Jason Linehan) [17:15:14] otherwise I can try to to move now [17:15:33] application_1654780610649_4378 is now stopped [17:15:51] dcausse: it can wait - I'm gonna try to provide more ressources for AMs as I was saying [17:16:11] dcausse: I talked to ou about the other one cause I thought it might have been different :) [17:16:46] yes sorry about that, I should use my own username for these tests [17:16:55] np dcausse [17:17:00] thanks for the quick actions [17:30:55] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform Value Stream, 10Patch-For-Review: Move Kafka Jumbo's TLS clients to the new bundle - https://phabricator.wikimedia.org/T296064 (10elukey) >>! In T296064#8248225, @Ottomata wrote: > BTW, @elukey I will likely reach out to you about how to do PKI... [17:33:40] 10Analytics-Radar, 10Data-Engineering-Radar, 10Event-Platform Value Stream, 10Patch-For-Review: Move Kafka Jumbo's TLS clients to the new bundle - https://phabricator.wikimedia.org/T296064 (10Ottomata) Oh yeah, and hopefully even KRaft. More context in {T307944} [17:41:45] 10Analytics-Radar, 10Domains, 10SRE, 10Traffic-Icebox, 10WMF-General-or-Unknown: Don't set cookies in traffic layer for non-user facing domains (avoid false third-party cookie warning) - https://phabricator.wikimedia.org/T262996 (10BCornwall) I think this ticket could do with some more clarification: Whi... [18:07:05] (03PS15) 10Joal: Update refine to use Iceberg for event_sanitize [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/811212 (https://phabricator.wikimedia.org/T311739) [19:55:14] !log starting refinery deployment (weekly train) [19:55:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:10:36] !log finished refinery deployment (weekly train) [20:10:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:24:30] 10Data-Engineering, 10SRE, 10serviceops, 10Event-Platform Value Stream (Sprint 01), 10Patch-For-Review: eventstreams chart should use latest common_templates - https://phabricator.wikimedia.org/T310721 (10Ottomata) @akosiaris, good news, Gabriele is working on this!!! @Jelto @JMeybohm, it seems the upgr... [20:27:07] 10Data-Engineering, 10SRE, 10serviceops, 10Event-Platform Value Stream (Sprint 01), 10Patch-For-Review: eventstreams chart should use latest common_templates - https://phabricator.wikimedia.org/T310721 (10Ottomata) BTW, since we merged the helm chart changes, eventstreams is currently undeployable. We r... [20:30:30] hey mforns, are you currently deploying to HDFS? [20:32:38] 10Data-Engineering, 10API Platform, 10Platform Engineering Roadmap, 10User-Eevans: AQS 2.0:Wikistats 2 service - https://phabricator.wikimedia.org/T288301 (10Milimetric) @BPirkle: sorry so late. Only one tiny misunderstanding left, I think. Wikistats 2 pulls data from all the endpoints you have in your s... [20:37:47] joal: it's finished! about 30 mins ago [20:38:16] 10Analytics-Radar: Evaluate using TypeScript on node projects - https://phabricator.wikimedia.org/T206268 (10Milimetric) @Physikerwelt: yes! Our new design system, [[ https://doc.wikimedia.org/codex | codex ]], is using Vue 3 with typescript support. I'm very excited about this and trying to incorporate it int... [20:40:30] 10Data-Engineering, 10Projects-Cleanup: Clean up wikimetrics - https://phabricator.wikimedia.org/T318193 (10Milimetric) [20:41:45] 10Data-Engineering, 10Projects-Cleanup: Clean up wikimetrics - https://phabricator.wikimedia.org/T318193 (10Milimetric) [20:45:54] ack mforns! Thanks :) [20:46:13] 👍 [21:04:07] 10Data-Engineering, 10Projects-Cleanup: Clean up wikimetrics - https://phabricator.wikimedia.org/T318193 (10Milimetric) [21:11:45] 10Data-Engineering: Use inclusive language in code for private analytics infrastructure - https://phabricator.wikimedia.org/T280268 (10Milimetric) The former Analytics and Research, former Analytics Engineering, now Data Engineering team has a number of repositories that need to be searched and updated. There a... [21:15:48] 10Data-Engineering, 10Event-Platform Value Stream, 10SRE, 10Platform Team Workboards (Clinic Duty Team): Avoid accepting Kafka messages with whacky timestamps - https://phabricator.wikimedia.org/T282887 (10Milimetric) This is a nasty bug if Andrew happens to not be around, I just wanna ++ the tech debt val... [21:19:34] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: [SPIKE] Prototype of incremental updates for mediawiki history for simplewiki , including reverts using apache hudi - https://phabricator.wikimedia.org/T258532 (10Milimetric) [21:19:37] 10Data-Engineering, 10Event-Platform Value Stream, 10Platform Engineering Roadmap Decision Making, 10Platform Team Workboards (S&F Workboard): Need for new event-type - `user_create` and `user_rename` - https://phabricator.wikimedia.org/T262205 (10Milimetric) 05Open→03Declined a:05roman-stolar→03Non... [21:21:54] (03PS1) 10Milimetric: Enable Uzbek interface translation [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/833457 [21:22:07] (03CR) 10Milimetric: [C: 03+2] Enable Uzbek interface translation [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/833457 (owner: 10Milimetric) [21:23:19] 10Analytics-Wikistats, 10Data Engineering Planning, 10Data Pipelines: Wikistats in Uzbek - https://phabricator.wikimedia.org/T314477 (10Milimetric) a:03Milimetric @Nataev: I'm so sorry it took me so long to see this. I'm deploying now with uz as the language code. I wasn't sure which "native name" to use... [21:24:32] (03Merged) 10jenkins-bot: Enable Uzbek interface translation [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/833457 (owner: 10Milimetric) [21:29:41] (03PS5) 10Neil P. Quinn-WMF: Begin sanitizing Wikistories streams [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832383 (https://phabricator.wikimedia.org/T312262) [21:30:49] (03CR) 10Neil P. Quinn-WMF: "Thank you for the review, Marcel!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/832383 (https://phabricator.wikimedia.org/T312262) (owner: 10Neil P. Quinn-WMF) [21:46:54] 10Data-Engineering, 10Event-Platform Value Stream, 10Platform Engineering Roadmap Decision Making, 10Platform Team Workboards (S&F Workboard): Need for new event-type - `user_create` and `user_rename` - https://phabricator.wikimedia.org/T262205 (10Ottomata) We have no current plans to provide a user_change... [22:28:07] PROBLEM - SSH on analytics1077.mgmt is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/Dc-operations/Hardware_Troubleshooting_Runbook