[08:02:28] I am going to reimage dbprov2002, T287454 is happening tomorrow, right? [08:02:37] yep [08:30:30] \o/ [08:43:20] hi [09:20:20] so many new clients :-D [09:33:06] The more, the merrier :) [13:19:17] Emperor: one thing i forgot to mention - php has no way of reusing db connections between queries, so it has to make a fresh connection for every query. this is the main reason (that i'm aware of) why we don't use ssl for mw->db queries, as the ssl negotiation overhead would kill latency/performance [13:35:20] Ah [13:36:45] performance always told us they reuse them to some extent at app layer, e.g.: https://github.com/wikimedia/mediawiki/blob/1d71fd91d687ffd1131c833c44f720062eccce0a/includes/libs/rdbms/loadbalancer/LoadBalancer.php#L807 but I don't really know much about mw internals [14:17:57] jynus: FYI: https://gerrit.wikimedia.org/r/c/operations/puppet/+/710976 adds a cumin `A:db-backup-source` alias, which might be useful for you [14:18:18] ah, that would be indeed! let me see [14:19:08] being dbstores but being named db* is not longer a bug, but a feature :-D [14:19:37] haha [14:20:22] if I can mentiong why that happened- those used to be dbstores (large servers for analytics and backups) [14:21:03] but when we started uniformizing sizes, it made more sense to call them db*s because they could move more easily (had the same hw specs) [14:21:25] eg. moving a backup source to production in an emergency [14:21:35] but the roles kept calling the old name dbstore [14:22:00] * kormat nods [14:22:15] ah, the joy of historical reasons :) [14:22:16] thanks for the addition [14:22:30] and I think the backup cluster name should work already [14:23:00] https://grafana.wikimedia.org/d/000000607/cluster-overview?orgId=1&from=now-24h&to=now&var-site=eqiad&var-cluster=backup&var-instance=All&var-datasource=thanos [14:39:28] jynus: re: T288244, the s7 primary in eqiad is now upgraded, so the next step is yours i believe [14:39:37] let me see [14:40:42] I will work on that next, but I won't be a blocker [14:41:10] ok cool [14:41:14] next week for codfw, kormat ? [14:41:39] seems likely, yeah [14:41:47] i'll look at scheduling tomorrow [14:42:21] no problem, I don't need a specifics yet, it is more to organize the switch in advance [14:42:26] 👍 [14:52:41] I added you to the change for a sanity check: https://gerrit.wikimedia.org/r/c/operations/puppet/+/710977 [14:52:52] to make sure I am changing the right section and dc :-) [14:53:04] and I will prepare the change for next week [14:53:41] +1'd [14:54:37] and as usual, I will leave the old sources around for a week until cleanup [16:08:41] One thing I think kormat mentioned earlier was the hassle of needing to restart prometheus-mysqld-exporter after mysqld is restarted on a host. It's not clear to me why we couldn't make the prometheus service Requisite= and After= the relevant mysqld service? (or maybe even BindsTo=) [16:10:55] Emperor: my reading of the docs for that combination implies that the prom exporter service will just never be automatically started [16:12:34] the language really could be a _lot_ clearer [16:13:24] Emperor, probably something that can be added to https://phabricator.wikimedia.org/T252761 [16:14:02] this was a new behaviour in buster prometheus exporter [16:16:11] btw, semi-related, this issue I reported at: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953040 seems it was fixed [16:19:06] actually, based on the patch, I don't think it was fixed :-( [16:21:17] well, in a way it is [16:22:19] kormat: yeah, I have to re-read it 3 times every time I come back to it [16:24:14] jynus: it looks like they at least _think_ that fixed it [16:24:31] it was confusing because the patch was very different to what I asked [16:24:45] but they just really mean: this is the intended way to authenticate [16:24:57] I will comment on the ticket [16:28:29] let's not put 3 unrelated things into the same task, please