[12:17:45] What is going on with db1154? [12:19:32] Did anyone restarted mariadb@s8 at db1154? [12:19:49] ladsgrou pts/0 91.198.174.9 Mon Aug 7 12:16 - 12:18 (00:01) [12:19:51] Amir1: ^? [12:20:14] marostegui: yeah, give it a minute [12:20:19] https://phabricator.wikimedia.org/T341076#9030773 [12:20:36] Yeah, it showed up on the alerts [12:20:44] That's why I thought it was not intended [12:20:57] oh sorry, I thought it wouldn't alert [12:21:09] at least it doesn't page [12:21:33] yeah, no it doesn't page [13:08:37] Amir1: for example, https://github.com/percona/pmm/blob/fba350688bfe047bea6c34057418d7760b115dbe/agent/runner/jobs/mysql_backup_job.go this is a backups script not a "backup solution" [13:10:09] yeah [13:10:36] no monitoring, no integration with cumin, with alerts, etc, which is what would take us more of the time [13:10:55] however, that doesn't mean we cannot "steal" more stuff from that [13:11:32] most advance functionality you see regarding monitoring (query monitoring) is possible, but was shutdown to be enabled for the public prometheus [13:11:57] but we have a lot of disabled metrics (usage, query performance, etc) :-( [13:12:14] as we use the same agent than them (we use their agent!) [13:15:22] e.g. taking their agents for the slow log / performance schema and sending it to a private storage (?) [13:15:45] I am guessing observability would have a saying on that [13:16:22] (I've been asking for an additonal private store that matches my needs for backups, for example) [13:18:21] I wonder if we could have at some point a meeting between the teams to tell them our needs (?) [13:56:57] I am going to merge all pending for rewview pampinus patches (better in than lost) [13:57:07] https://gerrit.wikimedia.org/r/q/project:operations%252Fsoftware%252Fpampinus+status:open [19:01:26] just optimizing extlinks tables in s4 has dropped 90GB https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=db1138&var-datasource=thanos&var-cluster=wmcs&from=1691396777880&to=1691433240738&viewPanel=28