[15:31:10] hi [16:39:59] Amir1: nice! [23:32:25] Krinkle: Hey, I've been AFK for a few weeks. Regarding the server side latency issue, yes, I mean page loads. [23:35:05] Krinkle: One obvious thing to consider is that the wiki is using a lot of semantic mediawiki queries, and it's actually still pulling data from on-prem. [23:35:48] Krinkle: But, we might be able to rule that out since it's happening with literally every page (2x load times), regardless of the complexity of the SMW query, or if there's a query at all. [23:37:26] Krinkle: I've gone over all the performance tuning stuff I can find. I didn't know if there were any more tools I could use for analyzing where the bottlenecks are, apart from just slamming the site with Apache Bench. [23:38:19] rootd00d: did you enable debug logs to look for possible warnings, errors, timeouts, config issues etc? [23:38:59] Krinkle: I've tried to do some profiling, but I'm not even really sure how to interpret the results. The wiki/php/mysql versions are all ancient also. I've had to port everything over using Docker to get compatible versions. They'll have a ton of plugins that need to be upgraded/reworked/replaced as we step up the wiki version up to LTS. [23:39:12] Krinkle: MW-specific debug logs? [23:39:24] Yeah [23:39:46] Krinkle: I just have PHP error logging enabled, methinks. [23:40:08] rootd00d: which MW version and which PHP version before/after? [23:40:58] Krinkle: ah I should get on my other pooter... Pretty sure it's MW 1.23, and PHP 5.5. [23:41:57] rootd00d: so the MW and PHP versions are the same on both metal and VM. [23:42:10] and on the new VM you're running additionally within a container? [23:42:50] I'm not sure there's much we can do for those versions. I suggest taking the perf hit, upgrading from LTS to LTS until latest, and then seeing what the perf is like and improving it via profiling. [23:43:10] I was going to suggest profiling with excimer on old metal and new VM to see where the difference is primarily, but that's not really feasible on such old versions [23:43:48] https://www.mediawiki.org/wiki/Version_lifecycle#Versions_and_their_end-of-life [23:43:49] https://www.mediawiki.org/wiki/Compatibility#PHP [23:44:01] That should help you find appropiate intermediary versions [23:44:32] we generally keep quite long compat so you don't actually have to do each intermediary version [23:44:39] https://github.com/wikimedia/mediawiki/blob/master/UPGRADE#L20 [23:44:50] the UPGRADE file will tell you how far back you can jump from [23:46:52] We use to keep indefinite upgrade paths from any 1.x to current. We stopped that in 1.35 afaik [23:47:02] 1.36 T259771 [23:47:02] T259771: RFC: Drop support for older database upgrades - https://phabricator.wikimedia.org/T259771 [23:47:31] So you should be able to jump to 1.35 LTS directly, and from there... to 1.39 [23:48:17] Feel free to ask more about that in #mediawiki (myself and others are there)