[10:10:14] <dcausse>	 drop_old_data_daily is failing because hdfs://analytics-hadoop/wmf/data/exports/cirrus-search-index/ does not exist, wondering if we should create it manually with the right perms? quickly testing analytics-search cannot write to hdfs://analytics-hadoop/wmf/data/exports/
[10:11:03] <dcausse>	 and thus I suspect the new export task to this folder might fail as well
[10:55:44] <dcausse>	 lunch
[12:52:57] <dcausse>	 errand
[13:16:20] <inflatador>	 <o/
[13:42:35] <dcausse>	 o/
[14:02:07] <dcausse>	 wondering where this upload build (https://gitlab.wikimedia.org/repos/search-platform/opensearch-plugins-deb/-/jobs/659050) is pushing files
[14:02:16] <dcausse>	 mainly WMF_DEB_PACKAGE_REGISTRY_URL
[14:03:47] <dcausse>	 ok seems like it's pushing to itself? https://gitlab.wikimedia.org/repos/search-platform/opensearch-plugins-deb/-/packages/1781
[14:20:12] <ebernhardson>	 \o
[14:20:26] <ebernhardson>	 yea it should be pushing to the packages of sam erepo
[14:21:13] <dcausse>	 o/
[14:49:29] <pfischer>	 o/
[15:16:39] <inflatador>	 for the curious, I just imported the upstream OpenSearch exporter's dashboard to use w/OpenSearch on k8s: https://grafana.wikimedia.org/goto/HCJU5URvg?orgId=1 . I'll likely trim it down quite a lot but LMK what you think
[15:33:40] <dcausse>	 thanks!
[15:33:57] <dcausse>	 interesting it's using max to select the value out of all the exporters
[15:35:36] <inflatador>	 I dunno if it's more or less useful than https://github.com/prometheus-community/elasticsearch_exporter (what we currently use) but we can look at that during a future OpenSearch version upgrade if it seems better
[15:36:57] <inflatador>	 I guess "max" is its clever way of dealing with inconsistent info
[15:36:59] <dcausse>	 hard to tell, I think the difficulty is to separate actual node metrics vs cluster wide metrics that may end up duplicated by all exporters
[15:37:38] <dcausse>	 I think the max is to deal with cluster wide metrics reported from all nodes
[15:38:24] <inflatador>	 Understood. I'm not super motivated to change our exporter or anything, although we might look at using a newer version of the same exporter when it comes time to update OS
[15:39:48] <dcausse>	 sure, I think as long as the data is there and we understand what it means I have no problem with this :)
[16:03:09] <inflatador>	 ryankemper we're up DPE standup if you're around
[17:52:21] * ebernhardson decides to find out how much breaks if we update integration testing from node16 to node24
[17:59:11] <dcausse>	 dinner
[18:09:47] <inflatador>	 lunch, back in ~40
[18:36:57] * ebernhardson loves error messages that give half the context.. 'file not found some/relative/path' ... but from what base directory? :S
[18:48:47] <inflatador>	 back
[19:02:24] <inflatador>	 Dr appointment, back in ~90m
[20:26:25] <inflatador>	 back
[21:05:22] <inflatador>	 ryankemper ebernhardson I'm in meet.google.com/eki-rafx-cxi if y'all wanna join for pairing
[21:14:08] <ryankemper>	 inflatador: oops I won’t be back for another 30’
[21:15:15] * ryankemper def thought it was an hour earlier than it was
[21:16:06] <ryankemper>	 when I’m back I’ll send you a couple things to look at tomorrow morning or whenever’s convenient
[21:23:10] <inflatador>	 ryankemper cool, I'll ping ya in Slack with some stuff re: OpenSearch on k8s since we talked about getting you and Steve more involved
[21:24:01] <ryankemper>	 inflatador: excellent. I was looking at the ticket about testing standard operations, was thinking that might be a good place for me to start
[21:56:54] <inflatador>	 OK, pinged in Slack. See ya tomorrow!