[04:25:47] TimStarling: closing loop on SiteConfiguration - abandoning https://gerrit.wikimedia.org/r/c/mediawiki/core/+/819225 [04:27:21] yes, that's fine [09:51:36] Krinkle: agreed re: 1d TTL, also what ori said re: resources and you are correct that swift is with data persistence now [15:14:34] Krinkle: > no longer about easy problems for big returns [15:14:41] I never said that about query sorting :) [17:27:46] Fair. It was in my head I suppose. [17:43:00] godog: I've been reading a fair bit Dan Luu recently.. Pulling some numbers out of the air (eg no thumbs in swift at all, and putting 10% of its hw cost to cache_upload). What would we save? Are we talking big money or small optimisations that would merely flatten swift growth for a "short" time maybe pushing back natural growth needs for original by only a few months at most, or something much bigger? [17:50:17] Krinkle: the retail price of 20TB SSDs is ~$300, and the total footprint for thumbs is around 500 TB, which works out to ~$7,500. This doesn't take a lot of things into account (disk lifespan / replacement cost, redundancy, cost of power, etc.) but I'd guess it's still the right order of magnitude. [17:51:57] iow it's not a million dollars or even a 100k [17:57:08] Right. And I guess that's a small enough portion to talk about the swift servers themselves without a disk each cost more than 20TB disk(s). [18:00:16] Just spit balling here but there are quotas in swift right? Maybe a way to derisk thumb growing out of control is just to put a quota on it, thus isolating from originals in terms of competing for space and think through failure scenario, eg thumbor and ATS remain fine serving if they can't store, and ideally we'd have a way to prune one off everything that's older than X even if not regularly done. Although at that point we are just [18:00:16] creeping toward a big disk-based HTTP cache backend.. probably not worth the effort to do that instead of actually moving to ATS with higher TTL and more space if it can manage that [18:00:38] So TDLR: never mind :) [19:13:09] $ curl -s -D - -o/dev/null 'https://en.wikipedia.org/wiki/Science' | grep cache-control [19:13:11] cache-control: private, s-maxage=0, max-age=0, must-revalidate [19:13:39] is that expected? do I need to curl the app server directly in order to see the cdn maxage? [20:04:01] (answering my own question: yes) [20:04:32] ori: yep, we swap it out in VCL [20:05:17] I believe it does allow browsers to perform 304s privately, and back-forward cache. but otherwise, fresh loads indeed.