[16:35:19] <wikibugs>	 06serviceops, 10Thumbor: investigate ThumbnailRender volume 2024-09-20 til 2024-10-04 (thumbnail, thumbor) - https://phabricator.wikimedia.org/T376509#10204641 (10hnowlan) 05Open→03Invalid
[16:44:21] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204646 (10hnowlan) 05Stalled→03Resolved a:03hnowlan >>! In T372470#10203863, @jeremyb wrote: > found T3...
[16:49:28] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204653 (10Zache) This is still reported on Wikimedia Commons - https://commons.wikimedia.org/wiki/Commons:Vil...
[16:57:42] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204657 (10Cremastra) No longer a problem for me at Wikisource, so I think whatever hnowlan did fixed it.
[17:13:52] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204686 (10hnowlan) 05Resolved→03Open I'm seeing recoveries on most of the linked images, but reopening this u...
[17:19:34] <wikibugs>	 06serviceops, 10Thumbor: Per-format monitoring for Thumbor - https://phabricator.wikimedia.org/T376538 (10hnowlan) 03NEW
[17:39:47] <wikibugs>	 06serviceops, 10Thumbor: Per-format monitoring for Thumbor - https://phabricator.wikimedia.org/T376538#10204718 (10Don-vip) Would per-format canaries allow to allocate different hardware resources per file format? TIFF files would benefit to get more memory, for example.
[17:44:03] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204721 (10hnowlan) Just to explain the issue - a while ago a rate-limiting feature that was known to be problemat...
[18:27:15] <wikibugs>	 06serviceops, 06All-and-every-Wikisource, 10Thumbor, 13Patch-For-Review: Elevated 429 responses from Thumbor on codfw starting 2024-08-14 00:00 UTC - https://phabricator.wikimedia.org/T372470#10204737 (10ShakespeareFan00) Can confirm recovery occuring at English Wikisoruce on files mentioned previously, es...