[00:23:26] elukey: ping regarding https://phabricator.wikimedia.org/T290232 [00:24:15] I believe the files are not yet in my homedir, but I may be looking in the wrong place. let me know :) [06:49:53] Krinkle: o/ this week is DSE hackathon for Analytics/DE so they might be slower to respond, worst case scenario next week. If I have time I'll try to move the files (I am helping them when I can) [08:44:55] godog: we want to upgrade grafana on cloudmetrics, do you know where cwhite got the packages for T282863 ? [08:44:55] T282863: Upgrade Grafana to 8.1 - https://phabricator.wikimedia.org/T282863 [08:45:25] (also, I'm open to give a push to upgrade to 8, but you seemed to have some issues to work out) [08:46:32] dcaro: hi, I'm assuming from upstream debian packages and then dpkg -i [08:47:13] godog: yep, moritzm found the debs in cwhite's home xd, I can use those ^.^ [08:48:13] dcaro: SGTM, or https://dl.grafana.com/oss/release/grafana_7.5.11_amd64.deb [08:48:18] can't we add them to our apt? [08:48:54] +1 for that ^ [08:49:02] it seems 8.1.6 is available from debmonitor: https://debmonitor.wikimedia.org/packages/grafana [08:49:13] (though I have a very small knowledge on how to do so) [08:49:15] we do, but upstream IIRC doesn't keep older versions around [08:49:22] volans: yes, but upgrade to 8 requires some investigating [08:49:53] dcaro: thanks re: 8 upgrade, yeah we'll get on that perhaps this quarter [08:50:36] then we might need to downgrade what we have in APT, unless pinned in puppet a new or reimaged host will get 8.1.6 on buster [08:50:39] grafana | 7.4.2 | stretch-wikimedia | amd64 [08:50:41] grafana | 8.1.6 | buster-wikimedia | amd64 [08:51:14] good point [08:51:46] but I'll defer to mor.itz for authoritative directions ;) [08:51:52] ;) [08:56:06] yeah, I think it would be best to move 7.5.11 to apt.w.o to get a clean state and only upload the latest version of 8.x once work towards the update progresses. these servers are unlikely to be reimaged, but e.g. one could accidentally run dist-upgrade and mess up the installation state [09:00:49] thoughts on how to do that with the current upstream repo? should we disconnect first the repo from reprepro 'updates' I'd imagine [09:05:26] we can use a hook like we do for gitlab or elasticsearch already [09:06:01] for gitlab upstream offers multiple streams, but we explicitly jump to new micro branches in updates which are distinct from the weekly security releases [09:06:46] making a patch, the current config doesn't use such a hook [09:08:04] I have to run to the doctor, but those that did attend the outage replay session yesterday please have a look at my proposal at https://gerrit.wikimedia.org/r/c/operations/puppet/+/726857 [09:13:34] godog: https://gerrit.wikimedia.org/r/c/operations/puppet/+/726858/ [09:15:08] cheers moritzm [09:20:53] moritzm: puppet change deployed to apt1001 [09:21:54] moritzm: checkupdate on thirdparty/grafana still returns "Nothing to do found" [09:22:11] should I pass --noskipold? [09:26:04] just a sec, there's a bug in the config [09:26:25] ack [09:34:27] https://gerrit.wikimedia.org/r/c/operations/puppet/+/726860 [09:34:50] and yes, we need --noskipold [09:35:09] I'm not 100% why this option exists, it seems almost always needed :-) [09:35:20] LGTM [09:35:20] so with that patch merged, [09:35:47] reprepro --noskipold --component thirdparty/grafana checkupdate buster-wikimedia [09:35:55] will show the pending 7.5.2 import [09:35:58] and then [09:36:04] reprepro --noskipold --component thirdparty/grafana update buster-wikimedia [09:36:08] will import it for good [09:36:50] ack, running [09:37:34] done :), thanks moritzm! [09:37:42] great :-) [09:37:57] * dcaro going to upgrade our instances [09:38:47] s/our/wmcs/ [09:41:05] good luck :-) [09:42:54] done and done :), everything up and running [09:46:24] nice :-) [18:12:43] hey legoktm - I was wondering if you'd be able to help me with deploying a couple of patches? No one seems to be available [18:13:03] sure