[15:23:28] <Amir1>	 hi, let me know when it's okay for you to move these metrics mentioned here so I start process of deploying the metric change
[15:23:29] <Amir1>	 https://phabricator.wikimedia.org/T272128
[15:23:36] <Amir1>	 godog: ^
[15:29:15] <godog>	 Amir1: sure, tomorrow EU morning works for you? I'll need the dumbed-down version of the filesystem paths to move/copy though
[15:29:33] <Amir1>	 sure
[15:29:38] <Amir1>	 sounds awesome
[15:30:50] <godog>	 Amir1: kk, 9 UTC would work, I'll be around
[15:43:44] <Amir1>	 Is graphite up for you? https://graphite.wikimedia.org/ it gives me "upstream connect error or disconnect/reset before headers. reset reason: connection termination"
[15:43:57] <Amir1>	 it can be just the webservice 
[15:47:44] <Amir1>	 godog: This is the straightforward version of what I want https://phabricator.wikimedia.org/T272128#7212524
[15:48:43] <godog>	 Amir1: ack thank you! graphite WFM
[15:49:04] <Amir1>	 meh, I used grafana
[15:49:05] <Amir1>	 thanks!
[17:45:44] <ottomata>	 AHHH i know why!
[17:45:50] <ottomata>	 i know why eventgate- analytics failed
[17:46:01] <ottomata>	 (sorry had to type my eureka somewhere)
[17:46:53] <lmata>	 eurekas are always welcome \o/
[17:50:27] <cwhite>	 ottomata: I've been looking at this too.  What did you find?
[17:50:56] <ottomata>	 cwhite:  i noticed that i got that error when I was runining service-runner with num_workers > 0
[17:51:05] <ottomata>	 i had worked around this in staging by setting num_workers: 0
[17:51:16] <ottomata>	 not sure exactly why, but this might be a bug in prom-client
[17:51:20] <ottomata>	 testing some newer versions of it now
[17:51:55] <ottomata>	 not sure but maybe https://github.com/siimon/prom-client/pull/384 is frelated
[17:52:16] <ottomata>	 but maybe not
[17:52:33] <ottomata>	 cwhite i can repro by setting num_workers: 1 locally
[17:52:37] <ottomata>	 and then curling /metrics endpoing
[17:53:36] <cwhite>	 so strange... we unit test `num_workers: 1` and `num_workers: 2`
[17:54:01] <ottomata>	 hm
[17:54:11] <ottomata>	 could be somethign eventgate code is doing, not sure
[17:54:25] <ottomata>	 eventgate is using the prom registry directly to work with node-rdkafka-prometheus
[17:54:28] <cwhite>	 https://github.com/wikimedia/service-runner/blob/master/test/utils/simple_config_one_worker.yaml
[17:54:34] <ottomata>	 and yeah, not fixed in any newer prom-clients
[17:54:52] <cwhite>	 ottomata: maybe? https://github.com/siimon/prom-client/issues/199#issuecomment-556908200
[17:55:06] <ottomata>	 hmm i don't think so but hm
[17:55:20] <ottomata>	 lemme see
[17:56:15] <ottomata>	 ok i can confirm it is the code in eventgate 
[17:56:18] <ottomata>	 investigating
[18:08:34] <ottomata>	 cwhite: i expect my problem has something to do with this line
[18:08:34] <ottomata>	 https://gerrit.wikimedia.org/r/plugins/gitiles/eventgate-wikimedia/+/refs/heads/master/eventgate-wikimedia.js#1053
[18:17:11] <cwhite>	 Hmm, warrants a closer look.  I gotta step away for a few, bbiab.