[00:00:01] * AntiComposite blames Flow [00:01:23] [telegram] well it worked now.. [00:01:24] [telegram] https://phabricator.wikimedia.org/T295743 [00:01:25] [telegram] This has been closed.. [00:01:27] [telegram] So this behavior then has nothing to do with the code of mediawiki.. [00:01:28] [telegram] But then what's the problem on both en.wikipedia.org and wiki.openstreetmap.org that large collections of images fail to load properly .. [00:01:30] [telegram] Also I noticed it's not just gallery feature, also if you add just a lot of links on a page after a few dozen images the issue start to come up.. for some sooner and others later.. [00:01:31] [telegram] For me it doesn't matter what is causing this - I just want it to work - and it doesnt 😢 [00:02:39] fwiw I don't see what you're looking at on the enwiki page [00:03:04] [telegram] That's what I get on https://en.wikipedia.org/wiki/Comparison_of_European_road_signs#Speed_limitAnd a lot of people can reproduce it.. although some claim for them it loads perfectly instantly : https://tools-static.wmflabs.org/bridgebot/138b0a3a/image_2021_12_23_08_02_38.png [00:03:13] interesting [00:05:06] does it work if you refesh (normally, not purging?) [00:07:36] [telegram] refreshing doesn't change anything. I really need to purge. [00:07:37] [telegram] I should mention - I thought to wiki.openstreetmap.org it was the high latency maybe, because my connection to Amsterdam from here in the western pacific is 240-260ms [00:07:39] [telegram] but then someone in the Netherlands said for him same issue. [00:07:40] [telegram] And later someone else said he has the same issue on en.wikipedia.org [00:07:42] [telegram] and when I ping that, it's only 35-40ms to a host obviously somewhere close here in the western pacific. [00:07:43] [telegram] So the latency (time-out) can't be the problem (re @wmtelegram_bot: [irc] does it work if you refesh (normally, not purging?)) [00:07:51] [telegram] Likely a global filter that does not have that message locally on mww (re @HikeAndMap: ⧼abusefilter-warning-linkspam⧽ [00:07:51] [telegram] If I want to write on the service desk of mediawiki.org I get this.. [00:07:52] [telegram] any hints what's going wro...) [00:08:10] Thecladis, nope, log says they're hitting local filter 59 [00:08:24] [telegram] Ah [00:08:53] which is why I'm blaming Flow instead, because it's not any of the normal stuff :) [00:09:41] [telegram] This is what I see if I include a https link : https://tools-static.wmflabs.org/bridgebot/775ed073/image_2021_12_23_08_09_31.png [00:10:07] HikeAndMap: on enwiki? [00:10:18] [telegram] My other version was that perhaps @HikeAndMap uses a different interface language, but it seems that it is not the case [00:10:35] [telegram] https://www.mediawiki.org/wiki/Topic:Wj7dl62zs6r7nh31 [00:10:36] [telegram] no posting a message here gives me that spam warning [00:10:37] [telegram] if I include a link [00:11:07] [telegram] here everything is English.. (re @Thecladis: My other version was that perhaps @HikeAndMap uses a different interface language, but it seems that it is not the case) [00:11:25] [telegram] Maybe the spam warning is a bit sensible [00:26:44] [telegram] Should be fine once you hit 2 edits on the wiki (re @HikeAndMap: https://www.mediawiki.org/wiki/Topic:Wj7dl62zs6r7nh31 [00:26:45] [telegram] no posting a message here gives me that spam warning [00:26:46] [telegram] if I include a link) [00:35:22] [telegram] Maybe, but shouldn't everyone who discovers a bug be able to post a bug report? (re @Thecladis: Should be fine once you hit 2 edits on the wiki) [00:36:31] [telegram] Hmm, for this, I guess at [[Project:Village Pump]] perhaps [00:36:45] [telegram] Well for the filter itself I mean [00:36:56] [telegram] Altbough it probably works as intended [00:37:09] [telegram] As to the message not rendering [[phab:]] [00:37:24] [telegram] [[phabricator:]] [00:37:33] [telegram] Hm, why neither works [00:38:06] [telegram] Anyway, rather [[How to report a bug]] [00:38:20] [telegram] Although I won't be surprised if it is already reported [00:50:45] [telegram] writing bug reports is rather discouraging when you report a problem which happens on wiki.openstreetmap.org (it's not like they have a tiny impotent server) as well as en.wikipedia.org (it's not like their servers are underpowered either) - and then the answer is:"don't see a problem so closing it" [00:50:46] [telegram] I don't know what's causing this behavior, but it doesn't change the fact there is a problem evidently. [00:50:48] [telegram] So whether it's the code or a server setting - either way - I would expect an answer to resolve the problem rather than:"Don't see it so going to ignore and close it" [00:50:49] [telegram] Even if it's an error on the server settings on both en.wikipedia.org and wiki.openstreetmap.org [00:50:51] [telegram] Shouldn't the code somehow detect some users aren't getting what they're supposed to be getting and then in some error-dump or "error notification" file or admin panel this being exposed to the administrator using mediawiki so he/she is informed some of their users have issues retrieving data and this is because of "setting" or "performance" [00:50:52] [telegram] Anyway - the maintenance guy of wiki.openstreetmap.org said their performance levels are everywhere <10% utilization when this happens. [00:50:54] [telegram] So I feel not being fairly heard - when I report there is this issue, I'm not the only one, of course I first talked in chat groups if I'm the only one or if it's more common - which it is.. [00:50:55] [telegram] and then it's just being closed and ignored .. [00:50:57] [telegram] I don't think that's the right way to address a user taking the time to point out there's somewhere a problem.. [00:50:58] [telegram] Users aren't supposed to be the technical gurus knowing how to program PHP and the underlying system characteristics and memory management within the scripting language to precisely identify what/where/how goes wrong. [00:51:00] [telegram] That's just my opinion.. [00:51:01] [telegram] based on a the fact a lot of users - if something doesn't work for them, they simply don't use it instead of reporting it - so I wouldn't chose to ignore the person who does report it... just my 5 cents on the topic. Fact is, the problem is there and real and it's commented in channels here on telegram that it does exist this problem. That's the fact. What is causing it - is still unknown.. [00:51:03] [telegram] And the person who closed it - also closed it without actually saying what's causing it or how to resolve it. [00:51:04] [telegram] As long as one doesn't know how to resolve the issue and exactly what's causing it - how can one then claim:"let's just close and ignore it" [00:51:06] [telegram] And if the person knows what's causing it, how to resolve it - first then he can be sure it's not the code itself - then before you close it - be so kind and let en.wikipedia.org and wiki.openstreetmap.org administrators know how to resolve the issue for the users experiencing this. [00:51:07] [telegram] again, just my 5 cents on this matter [00:57:53] [telegram] You seem to be describing in part what would be considered "error logs" [01:12:36] [telegram] I suppose .. unless this behavior is in fact an error in the code and not on the server side.. [01:12:37] [telegram] I don't know what's causing it - therefore I've asked help [01:12:39] [telegram] but no one knows so it seems (re @tehreedy: You seem to be describing in part what would be considered "error logs") [01:24:03] [telegram] The HTTP 429... "Too many requests"... The requests are coming from the clients/users web browser. This isn't so much of a MW error. It's the browsers fault (even if there's a load of images) - https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/429 [01:24:50] [telegram] would lazy loading on MW's end remedy this? (re @tehreedy: The HTTP 429... "Too many requests"... The requests are coming from the clients/users web browser. This isn't so much of a MW er...) [01:25:07] [telegram] That sort of thing could help, indeed [02:41:53] [telegram] That's an error he reported which I or anyone else never experienced and thus unrelated to the reported issue. (re @tehreedy: The HTTP 429... "Too many requests"... The requests are coming from the clients/users web browser. This isn't so much of a MW er...) [02:42:25] [telegram] No it’s not unrelated [02:47:53] [telegram] If you know how to debug it yourself, do it. If you just keep telling everyone else they're wrong, but not able to justify why, or prove otherwise, people are not going to want to try and help you [02:57:26] [telegram] You may be right that it might not be the whole issue, and that's fine. But it is perfectly reasonable on a page that is requesting a lot of images (which potentially means many requests) that it results in a HTTP 429 error [03:33:19] [telegram] it seems weird to say it's not a mediawiki error and blame browsers when mediawiki generated the html of the page and browsers loading all the images you put on the page is hardly new or uncommon behaviour (in fact I don't remember ever using a browser that did it any other way) [03:34:40] [telegram] it's not even using the loading=lazy attribute to tell the browser not to load it immediately [03:35:56] [telegram] https://caniuse.com/loading-lazy-attr [03:36:03] [telegram] It's not universally supported [03:36:18] lazy loading has mostly fixed the issue on Special:NewFiles, if this problem is indeed the Thumbor ratelimit [03:36:30] [telegram] is that a sensible reason to not use it? [03:36:59] [telegram] I'm just saying it's not a guaranteed fix [03:37:24] [telegram] https://phabricator.wikimedia.org/T148047 [03:37:33] [telegram] Is a bug from 2016... [03:38:57] [telegram] Of course, MediaWiki is open source software. If a bug affects you, you are more than welcome to post a patch to try and resolve it [03:39:24] [telegram] it's not a guaranteed fix anyway, because the problem is actually that it needs some sort of rate limiting. the page with the speed signs will probably always have problems otherwise because there are a lot of small images being displayed together, so a lot of them will appear simultaneously [03:40:06] [telegram] With a well setup wiki, with caching setup for images etc... Serving those images should be cheap after they've been rendered/thumbnailed once [03:40:44] [telegram] so wikimedia's wikis are badly set up? [03:41:04] the screenshots I've seen don't look like hitting the Thumbor concurrent request ratelimits [03:41:35] I'd expect white boxes (in Firefox) or white boxes with a broken image glyph (in Chrome), not a plain link to the file page [03:41:41] that comes from MediaWiki [03:43:40] if it was the Thumbor ratelimit, a simple page refresh would fix the issue. No purging of the MediaWiki/HTTP caches should be required [03:44:19] [telegram] Well, there's many known issues with Thumbor... Including but not limited to it being not maintained upstream AIUI (re @Nikki: so wikimedia's wikis are badly set up?) [03:44:54] [telegram] it shows the filename because that's the alt text (re @wmtelegram_bot: [irc] I'd expect white boxes (in Firefox) or white boxes with a broken image glyph (in Chrome), not a plain link...) [03:49:31] it would be easy to confirm if this is 404-thumbnailing-related with the network tab in browser dev tools [03:50:47] as well as looking at the generated HTML to see if it's an image or the MediaWiki broken image placeholder [03:58:58] [telegram] the errors are 429 [04:01:59] on wiki.osm.o or on enwiki? [04:02:07] [telegram] enwiki [04:02:54] [telegram] I'm wondering - assume it is because the PHP code that is being generated makes too many requests at once for a server.. [04:02:54] [telegram] Assume on 500 images it opens 500 connections at a time [04:02:55] [telegram] PHP can't be tweaked that this never happens and instead max 50 or so at a time being downloaded? [04:02:57] [telegram] Although even if it's 1000 images connections, I can hardly image a server like en.wikipedia.org couldn't handle that [04:03:01] Nikki: right, so that one's definitely the Thumbor concurrent requests ratelimit [04:03:15] [telegram] PHP isn't doing the downloads, your browser is [04:03:28] lazy loading or preloading are the only options for that one [04:03:31] [telegram] but PH generates the information the browser using to download? (re @tehreedy: PHP isn't doing the downloads, your browser is) [04:03:44] [telegram] It generates the HTML, sure [04:03:56] [telegram] is there then in the browser a way to manipulate to not download more than xx requests at once to the server? [04:03:58] on wiki.osm.o it's a bit different [04:04:09] the broken images have no element [04:05:08] [telegram] and the is being generated by the server right? (re @wmtelegram_bot: [irc] the broken images have no element) [04:05:15] yes [04:05:36] [telegram] so that's then a result of the PHP html generation on the server? [04:08:14] my thought the last few times you've asked is that the InstantCommons mw <-> mw API requests are the issue [04:08:41] [telegram] maybe the PHP on the server side verifies if these links are actually existing real images.. [04:08:42] [telegram] That's how I would program it, but I don't know PHP only c# and c++ , have a verification if these links are actually real alive images [04:08:43] [telegram] and that might be the issue then.. PHP checks the images, if they're real existing and alive.. [04:08:45] [telegram] then that times-out partially [04:08:46] [telegram] thus no generation [04:08:48] [telegram] Then it's not a client-side browser issue, but the PHP should be given more time to verify if the images are alive [04:11:44] [telegram] I really wish someone with advanced PHP skills looking into the code to see why it doesn't generate the tags for these images.. [04:11:45] [telegram] Because my hunch tells me that's where the problem lies.. [04:11:46] [telegram] even if a server times-out on images.. [04:11:48] [telegram] then have the user wait and show a rotating clock or so.. [04:11:49] [telegram] and behind the scenes PHP continues to poll the server to see if the images are real.. [04:11:51] [telegram] or add a functionality to the gallery that a user can "force" to generate the on large collections to avoid this - and then it's up to the user to make sure the links to the instantcommons are real, correct and alive [04:12:19] the problem is that InstantCommons makes 2-3 API requests per image [04:12:28] they're synchronous and not batched [04:12:42] [telegram] ahhh (re @wmtelegram_bot: [irc] the problem is that InstantCommons makes 2-3 API requests per image) [04:13:46] [telegram] so that will cause a lot of "at the same time" parallel requests coming in at once.. [04:13:46] [telegram] A firewall might even consider it a DDoS 🤪 if it would come from many different IP's.. [04:14:22] bawolff is working on some improvements, https://www.mediawiki.org/wiki/Extension:QuickInstantCommons but it'll take some time before it filters up to core for everyone [04:14:57] [telegram] so either the server performance should be upped to handle more requests at once.. [04:14:58] [telegram] or [04:15:00] [telegram] the synchronous requests all at once be rewritten into batches with a max size the server actually could handle [04:15:07] and then back down to everyone running the LTS [04:15:10] [telegram] I'll read up on that, thanks for sharing (re @wmtelegram_bot: [irc] bawolff is working on some improvements, https://www.mediawiki.org/wiki/Extension:QuickInstantCommons but it'...) [04:16:33] [telegram] It took 1038.18 seconds with MediaWiki core's instant commons, 18.50 seconds with this extension with prefetching disabled, and 1.10 seconds with this extension but with prefetching enabled [04:16:34] [telegram] ohhhh, that's some serious improvement [04:25:58] if you're curious, this is what the requests look like currently (from my local dev wiki) for those galleries (with no timeouts) https://phabricator.wikimedia.org/P18260 [04:29:11] the most stable workaround would be to split the gallery sections into seperate pages, instead of trying to display them all on the same page [04:31:57] [telegram] yes I guess that's what I'm going to do - although it invalidates the concept of "quick lookup" [04:31:57] [telegram] The idea for this page was that a human with the eye - can almost instantly recognize a specific sign and then read the official code below it.. [04:31:58] [telegram] That's something computers are still outperformed by humans, instantly recognize a specific image out of tons of images.. [04:32:00] [telegram] having it split up into several pages - the novice user would then have to know which page to be on or browse through them one at a time (re @wmtelegram_bot: [irc] the most stable workaround would be to split the gallery sections into seperate pages, instead of trying t...) [04:32:24] that still seems better than having 3/4 of the images not load [04:33:11] [telegram] I agree - if a solution isn't in sight - then no other option left (re @wmtelegram_bot: [irc] that still seems better than having 3/4 of the images not load) [04:35:23] [telegram] https://github.com/openstreetmap/operations/issues/466 [04:35:24] [telegram] Could this be related? [04:37:04] yes, it's the same issue with InstantCommons API requests [04:42:03] the solution in Extension:QuickInstantCommons is likely the best future solution. I don't know what the OSM ops folks attitude toward running a beta extension would be, and I don't know what Bawolff's plans are for that extension [04:47:50] [telegram] Yes, let's see if the OSM server maintenance guy is willing to give it a go.. (re @wmtelegram_bot: [irc] the solution in Extension:QuickInstantCommons is likely the best future solution. I don't know what the OS...) [05:26:23] fwiw, #mediawiki on Libera or #mediawiki:matrix.org are generally better places for MediaWiki support questions/help [05:28:26] in other news, I just published a new blog post about a new library to parse MediaWiki titles in Rust: https://blog.legoktm.com/2021/12/23/what-it-takes-to-parse-mediawiki-page-titlesin-rust.html [06:34:50] @lucaswerkmeister: I just finished checking out m3api, the combining requests part looks really neat. I've never seen that before and now I'm wondering how to implement it as well! [07:02:25] [telegram] Is now the best time to invest? [07:34:59] [telegram] Hi Jan, Thank you so much for your question! By editors we mean contributors who only edit, and content moderators is a more broad definition for contributors who plus to editing also write reviews etc. But definitely, this would have been 1 single category! Thank you for your contribution to our survey! (re @Jan_ainali: I got stuck on the first question. What is the difference between a "editor" and [07:42:23] [telegram] Hi Lucas, yes thank you! Is not only for video tools but for all essential tools. [07:42:24] [telegram] We are making demo videos to explain tools that are helpful to smaller wiki communities to help people to get started. Thank you for contributing to our survey! (re @lucaswerkmeister: this isn't limited to video-related tools, right? (because I think I misunderstood that the first time I read the message)) [10:59:10] [telegram] "parsing MediaWiki" 😬 "page titles" 😌 "in Rust" 🥰 (re @wmtelegram_bot: [irc] in other news, I just published a new blog post about a new library to parse MediaWiki titles in Rust: https:...) [10:59:28] [telegram] thanks, I'm really happy with that part :) (re @wmtelegram_bot: [irc] @lucaswerkmeister: I just finished checking out m3api, the combining requests part looks really neat. I've ne...) [18:27:53] Wow, on that [[w:en:Comparison_of_European_road_signs]] page `document.getElementsByTagName("img").length` returns 3762... not sure how it is MediaWiki or Wikipedia's fault that this page has such a large number of images to load. [18:30:44] Content like that really needs image sprites to be remotely useful -- https://css-tricks.com/css-sprites/ [18:34:59] bd808: Or to be split up, frankly. [18:35:10] well, yes [18:36:54] I'm reminded a bit of the window of time where the wikidata item for Germany could only be edited if you happened to hit a HHVM backend server. That page of image horror is kind of the same thing, only its about where your web browser is in relation to download.wikimedia.org and how much ram you have locally. :) [18:37:08] Indeed. :-( [20:44:00] [telegram] [[w:en:Comparison_of_European_road_signs]] [20:44:43] [telegram] Nice, looks like a handy thing for playing Geoguessr 😊 [21:21:48] [telegram] That's true, 3000 times 50kB per icon makes roughly 150MB that's definitely something that's going to break every single computer and server in the 21st century. (re @wmtelegram_bot: [irc] Wow, on that [[w:en:Comparison_of_European_road_signs]] page `document.getElementsByTagName("img").length` returns...) [21:53:41] [telegram] did not load all the images on the first attempt, but loaded fine on next ones, I guess just a question of caching. [21:56:18] [telegram] a random image I've opened, https://upload.wikimedia.org/wikipedia/commons/thumb/4/41/Leaving_built_up_area_Ireland.png/50px-Leaving_built_up_area_Ireland.png , is just 5kB [21:58:44] [telegram] Well I'm sure even 15MB is too much for modern servers and computers to handle. We can't expect a site hosting 15MB of data to work properly of course. (re @Thecladis: a random image I've opened, https://upload.wikimedia.org/wikipedia/commons/thumb/4/41/Leaving_built_up_area_Ireland.png/50px-Lea...) [21:59:10] [telegram] 5 710 items, totalling 25,5 MB (37,0 MB on disk) [21:59:10] [telegram] downloaded [21:59:43] [telegram] I am not sure if you are being sarcastic or not though [22:22:38] [telegram] If what I'm saying is ridiculous, I usually am 😂 (re @Thecladis: I am not sure if you are being sarcastic or not though) [22:29:31] [telegram] Hosting so many small icons shouldn't be a problem in the 21st century. Even systems 20 years ago can do it. I host for my own family a private apache media server html and dlna in one on a server 50 times less potent, internal over vpn. And there's the option to list a bunch of image previews very tiny.. Over 100.000 photos by now. Just takes a really really long time to load it, but all 100.000 images are [22:43:12] @HikeAndMap: Are you just trolling us now? [22:45:36] [telegram] If addressing ignorance means trolling then I'm sorry. Obviously we chose to ignore a problem and pretend it's not here. So then, okay I'll accept then we're going to ignore problems and pretend they don't exist. Sorry for pointing out there's "not a problem " [22:49:00] You're continuing to be a troll [22:49:14] People have been helping you. Potential solutions have been suggested etc [22:49:23] It's not our fault you're creating galleries with thousands of images [22:54:10] I can come up with no charitable interpretation of "Even systems 20 years ago can do it." or implying that your private apache media server can handle the traffic level and availability of Wikimedia Commons