[09:47:09] <volans>	 mmmh I can't see any graph on tendril, is it just me or there is some known issue?
[09:48:11] <kormat>	 volans: answer is almost certainly yes
[09:48:26] <kormat>	 volans: so.. what do you mean by 'graph'?
[09:48:29] <kormat>	 https://tendril.wikimedia.org/tree ?
[09:48:41] <volans>	 https://tendril.wikimedia.org/host/view/db1163.eqiad.wmnet/3306
[09:48:48] <volans>	 metrics
[09:49:02] <volans>	 or did we deprecate/removed them?
[09:49:03] <jynus>	 I remember manuel disabling some functionalities not a long time ago
[09:49:11] <jynus>	 but cannot remember the details
[09:49:29] <kormat>	 volans: oh, yeah, use grafana :P
[09:49:31] <volans>	 the linked grafana dashboard too has a lot of missing data
[09:49:32] <jynus>	 in any case, you should have a link to the equivalent prometheues ones
[09:49:34] <volans>	 https://grafana.wikimedia.org/d/000000607/cluster-overview?cluster=mysql&orgId=1&var-datasource=eqiad%20prometheus%2Fops&var-cluster=mysql&var-instance=db1163
[09:49:48] <jynus>	 at "[ Prometheus host metrics | MySQL instance metrics"
[09:49:58] <kormat>	 volans: https://grafana-rw.wikimedia.org/d/000000273/mysql?orgId=1&from=now-1h&to=now&var-server=db1163&forceLogin=true
[09:50:38] <kormat>	 volans: i've never seen the 'cluster overview' dashboard before
[09:50:56] <volans>	 it's what is linked there :D
[09:51:24] <kormat>	 volans: step 1: don't look at tendril. step 2: if you disregard step 1, don't complain about getting useless data
[09:51:35] <volans>	 lol
[09:51:35] <jynus>	 for grafana, you should ping obs team
[09:51:45] <kormat>	 volans: https://orchestrator.wikimedia.org/web/cluster/alias/s1
[09:51:51] <jynus>	 (for the common dashboards, I mean)
[09:52:11] <jynus>	 cluster, host, etc.
[09:52:21] <volans>	 thanks :)
[09:52:51] <volans>	 I need m3 actually but yep
[09:53:17] <jynus>	 but I was thinking of creating a separate host one without the less useful metrics (disk usage) and more useful to me (bytes utilization, iops, etc.)
[09:54:33] <jynus>	 feel free to ping me if you decide to create a new one, we may have common needs!
[09:54:58] <volans>	 me? surely not :)
[09:55:07] <volans>	 I just needed to check some data
[09:55:09] <kormat>	 volans: click the 'clusters' dropdown at the top
[09:55:24] <volans>	 would be nice to have the cluster name in grafana
[09:55:25] <kormat>	 we Fancy now
[09:55:48] <jynus>	 volans, I may also be able to provide detailed size metrics from the backup metrics too
[09:56:03] <jynus>	 but those are not public out of privacy concerns
[11:25:00] <Emperor>	 godog: I expect to finish ms-be2045 tomorrow, FWIW
[11:25:54] <Emperor>	 (so yeah, we can surely do more codfw hw next week - is this adding more be hosts?)
[11:42:57] <godog>	 Emperor: sweet! yeah bringing the new hosts up to full weight in T288458
[11:42:57] <stashbot>	 T288458: Put ms-be20[62-65] in service - https://phabricator.wikimedia.org/T288458
[15:11:20] <Emperor>	 TIL swift-dispersion-report -j doesn't emit valid JSON :(
[15:12:23] <Emperor>	 (specifically, it always outputs "Using storage policy: standard" or whatever before the JSON output, so the result isn't valid JSON)
[15:15:20] <Emperor>	 ...fixed upstream at least
[15:17:11] <Emperor>	 (and in bullseye)
[15:17:44] <kormat>	 Emperor: i had a sad a few months ago when i discovered that gerrit _deliberately_ emits invalid json
[15:18:04] <Emperor>	 😢
[15:18:46] <kormat>	 i don't remember the exact justification, but it had something to do with the web having godawful security
[15:19:05] <kormat>	 and on that note, https://orchestrator.wikimedia.org/ is now running the latest version! ;)
[15:20:01] <Emperor>	 \o/
[15:20:04] <jynus>	 cool!
[15:20:54] <kormat>	 Emperor: thanks again for your help battling the debian-packaging demons who plagued me at the start 💜
[15:22:38] <Emperor>	 YW :)
[15:27:29] * jbond also hit the gerrit json issue https://github.com/wikimedia/puppet/blob/production/utils/pcc.py#L106-L110