[11:04:04] jynus: do you remember from the top of your head if dumps (m5 in particular) run on Tuesdays? [11:04:05] jynus: do you remember from the top of your head if dumps (m5 in particular) run on Tuesdays? [11:04:06] I've put the stats you asked me about during our meeting on the SRE doc- if you asked them, probably it will be interesting for someone else, too [11:04:07] I've put the stats you asked me about during our meeting on the SRE doc- if you asked them, probably it will be interesting for someone else, too [11:04:35] Ah great! [11:04:35] Ah great! [11:04:35] they run lately (it changed some time ago) on the night of monday to tuesday [11:04:35] they run lately (it changed some time ago) on the night of monday to tuesday [11:04:44] ah great [11:04:45] ah great [11:04:46] thanks [11:04:46] thanks [11:04:58] I want to give them one more run before removing wikitech backups from m5 [11:04:59] I want to give them one more run before removing wikitech backups from m5 [11:05:12] do I need to change something for now, or just to check if runs ok? [11:05:12] do I need to change something for now, or just to check if runs ok? [11:05:23] jynus: maybe you should highlight that point for the SRE meeting [11:05:23] jynus: maybe you should highlight that point for the SRE meeting [11:05:27] I think you are answering my questions before I finish typing them :-) [11:05:28] I think you are answering my questions before I finish typing them :-) [11:05:28] I think it is worth being mentioned there [11:05:28] I think it is worth being mentioned there [11:05:38] which part, sorry? [11:05:38] which part, sorry? [11:05:40] jynus: haha yeah, no need to change anything [11:05:41] jynus: haha yeah, no need to change anything [11:05:47] jynus: the commons backup first pass [11:05:48] jynus: the commons backup first pass [11:06:05] ah, that's what I was telling you while you asked me [11:06:05] ah, that's what I was telling you while you asked me [11:06:07] see the doc [11:06:07] see the doc [11:06:17] yes, but it is not bolded [11:06:17] yes, but it is not bolded [11:06:20] ah, you mean highlight [11:06:21] ah, you mean highlight [11:06:21] mmm [11:06:22] mmm [11:06:23] Yes [11:06:24] Yes [11:06:32] not sure, it is not completed (the OKR) [11:06:32] not sure, it is not completed (the OKR) [11:06:39] It is up to you, but I think it is something to mention [11:06:39] It is up to you, but I think it is something to mention [11:06:44] but I guess you can mention it quickly [11:06:44] but I guess you can mention it quickly [11:07:09] You should do it :) [11:07:09] You should do it :) [11:07:24] nah, no worth [11:07:24] nah, no worth [11:07:30] ok, up to you [11:07:30] ok, up to you [11:07:31] you do both, and only if people have questions [11:07:31] you do both, and only if people have questions [11:07:38] I can answer them [11:07:38] I can answer them [11:07:56] sure [11:07:56] sure [11:08:17] in any case, your original question was how long: 19 days [11:08:17] in any case, your original question was how long: 19 days [11:08:23] 88 million files [11:08:23] 88 million files [11:08:27] 360 TB [11:08:27] 360 TB [11:08:31] aprox [11:08:32] aprox [11:08:38] :) [11:08:38] :) [11:24:28] Emperor, mostly a curiosity, but this is why sha1 is not enough: https://phabricator.wikimedia.org/T269128 [11:24:28] Emperor, mostly a curiosity, but this is why sha1 is not enough: https://phabricator.wikimedia.org/T269128 [11:48:10] :sadpanda: [11:48:10] :sadpanda: [11:48:30] Ceph's versioned buckets are rather better, I suspect :) [11:48:30] Ceph's versioned buckets are rather better, I suspect :) [11:48:55] it is really not an issue with the backend, but mw's file management overally [11:48:55] it is really not an issue with the backend, but mw's file management overally [11:49:18] oh, it's MW doing the renaming? [11:49:19] oh, it's MW doing the renaming? [11:49:26] yeah [11:49:26] yeah [11:51:20] I honestly don't know enough to know about Swift issues/weaknesses, but in my time working on media backups, almost all my headaches were from a less-than ideal mw file workflow, my guess mostly due to historical reasons/technical debt [11:51:20] I honestly don't know enough to know about Swift issues/weaknesses, but in my time working on media backups, almost all my headaches were from a less-than ideal mw file workflow, my guess mostly due to historical reasons/technical debt [11:51:53] although I may be biased because I hardly interacted with Swift- just downloaded from it [11:51:53] although I may be biased because I hardly interacted with Swift- just downloaded from it [11:52:24] after all, I am doing a "logical" kind of backup, so I only used it from a very high level [11:52:24] after all, I am doing a "logical" kind of backup, so I only used it from a very high level [12:01:38] Mmm [12:01:38] Mmm [12:06:38] I sometimes use DBA terminology for things that don't apply, here logic means "in a different format than the original, that is portable" [12:06:39] I sometimes use DBA terminology for things that don't apply, here logic means "in a different format than the original, that is portable" [12:07:06] so I only interact with the Swift download api, not any of the admin tasks [12:07:06] so I only interact with the Swift download api, not any of the admin tasks [15:57:44] some of the files that had failed to upload were: https://commons.wikimedia.org/wiki/Category:Videos_of_Hurtigruten_minutt_for_minutt which were a collection of files so large that I hadn't account a timeout for them [15:57:44] some of the files that had failed to upload were: https://commons.wikimedia.org/wiki/Category:Videos_of_Hurtigruten_minutt_for_minutt which were a collection of files so large that I hadn't account a timeout for them