[00:14:38] Can someone remind me what the error in the last test at Z24329 means: "Call tuples failed in returnOnFirstError. Error: Error: wikilambda_fetch threw a fetching error: invalid json response body at http://localhost:6501/w/api.php?action=wikilambda_fetch&format=json&uselang=content&zids=Z17410 reason: Unexpected token 'u', \"upstream c\"... is not valid JSON." [00:18:00] The infinite running bug now also seems to affect the built-in function Z811 (re @genocation: Thank you for filing this! We've observed the same behavior in https://www.wikifunctions.org/wiki/Z19509: the errors caused retu...) [00:33:27] I’m not seeing that error. I’m seeing Actual Monday, Expected 1. (for Z24158) (re @u99of9: Can someone remind me what the error in the last test at Z24329 means: "Call tuples failed in returnOnFirstError. Error: Error: ...) [00:38:46] Yes, it seems to have resolved. It was appearing on Z24143 (re @Al: I’m not seeing that error. I’m seeing Actual Monday, Expected 1. (for Z24158)) [00:40:15] I'm not sure about that test. It's not really our place to say how Wikidata should use it's "mul" labels. (re @wikilinksbot: Z24158 – Monday in mul is 1 (?)) [01:11:50] Indeed. If the “mul” label is “Monday”, the “mul” label is “Monday”. By the way, I think it’s worth using Z22839 rather than Z811 (which it wraps, with Z802), in case there is no label in any specified language. I’m thinking of putting some default into Z24116, presumably the first found. (re @u99of9: I'm not sure about that test. It's not really our place to message> [01:11:51] say how Wikidata should use it's "mul" labels.) [01:18:23] I considered Z22839 but in these particular compositions we can guarantee that at least "en" (always appended) will have a label (outside of Wikidata-blanking vandalism). So for efficiency of rendering I think it's okay to go with Z811 here. (re @Al: Indeed. If the “mul” label is “Monday”, the “mul” label is “Monday”. By the way, I think it’s worth using Z22839 [01:18:23] [01:18:24] rather than Z81...) [01:22:35] I think having a first-found desperation default would be a good boolean option-flag in that function, but I think we also probably want the option to go without (and perhaps switch to lexemes in an even more elaborate fallback scheme). (re @Al: Indeed. If the “mul” label is “Monday”, the “mul” label is “Monday”. By the way, I think it’s worth using Z22839 rather than [01:22:35] Z81...) [01:32:39] That’s another good reason to prefer Z22839. I’d rather switch to alternative strategies at the top of the tree, so long as it doesn’t involve re-visiting the original item. But that needs thinking through… 🤔 (re @u99of9: I think having a first-found desperation default would be a good boolean option-flag in that function, but I think we also proba...) [07:11:30] Maybe I'm imagining it, but the system seems lethargic. I last edited Z12767 29 minutes ago, and when I refresh I still see a wall of "unable to run tests" : https://tools-static.wmflabs.org/bridgebot/0db1c9ea/file_70224.jpg [07:20:15] I’m pretty sure it’s a side-effect of *T391435*. The Python implementations have a lot of timing out to do. Maybe this tilts it towards P1 rather than P2 but it still seems to be stuck awaiting triage 🤷‍♂️ (re @u99of9: Maybe I'm imagining it, but the system seems lethargic. I last edited one of the components of Z12767 29 minutes ago, and when I...) [07:24:03] It could be. My normal heuristic is that timeouts could take just over 10 seconds, so multiplying that by 6 tests and 3 implementations, I'd still expect it done in about 3 minutes. (re @Al: I’m pretty sure it’s a side-effect of T391435. The Python implementations have a lot of timing out to do. Maybe this tilts it to...) [07:31:36] Yes, I do also want this fixed soon though, it's a real problem to rely on only one efficient code language. (re @wikilinksbot: T391435 – Investigate performance of Python implementations where there is an argument containing an explicitly typed list [open...) [07:33:16] Yes. I’m talking about general sluggishness. (re @u99of9: It could be. My normal heuristic is that timeouts could take just over 10 seconds, so multiplying that by 6 tests and 3 implemen...) [07:33:46] For example, some of the rational number functions never got written in JS because returning keys is much harder than returning a single variable. (re @u99of9: Yes, I do also want this fixed soon though, it's a real problem to rely on only one efficient code language.) [07:35:01] Maybe the Z12767 specific issue is a new infinite timeout glitch. (re @Al: Yes. I’m talking about general sluggishness.) [07:40:11] I think so. It looks like each of the individual tests completed. (re @u99of9: Maybe the Z12767 specific issue is a new infinite timeout glitch.) [07:48:02] And now I’m getting service unavailable… Cause and effect? There seems to have been some thrashing about going on. (re @u99of9: Maybe the Z12767 specific issue is a new infinite timeout glitch.) [07:54:26] So now you have a lot of failures on the function page, but at least it’s not writing perpetually infinite logs… or whatever. (re @u99of9: Maybe the Z12767 specific issue is a new infinite timeout glitch.) [08:01:11] Sorry, @Sannita function evaluations seem generally unavailable now. [08:05:23] …temporarily, it seems, but there is a risk of a repeat… @u99of9 let’s not even look at that page for a while! (re @Al: Sorry, @Sannita function evaluations seem generally unavailable now.) [08:10:00] It's all yours. I've been out walking since my last post! 🚶 (re @Al: …temporarily, it seems, but there is a risk of a repeat… @u99of9 let’s not even look at that page for a while!) [08:52:26] Thanks… visiting the page is fine, of course. Each of the implementations has orderly results for all connected tests but Z13233 seems problematic, although it has results for each implementation. Despite this, the function page is now (eventually) displaying “unable to run tests” for all combinations. Oh… and now we’re back to “service unavailable”… stepping away! 😱 [09:02:11] There’s a similar problem on Z889. [09:11:46] We'll look into it (re @Al: Sorry, @Sannita function evaluations seem generally unavailable now.) [12:37:00] I came back a few hours later. I can't see any improvement at Z12767. (re @Al: …temporarily, it seems, but there is a risk of a repeat… @u99of9 let’s not even look at that page for a while!) [12:44:05] the problem is that half of staff is currently arriving or already at the Hackathon in Istanbul... [12:49:38] The first two implementations have results from “1 hour ago” and the third from nearly half an hour ago. It’s perhaps a cache access issue because looking at the third implementation does require a bit of patience. This is from looking at the implementations themselves. (re @u99of9: I came back a few hours later. I can't see any improvement at Z12767.) [12:53:55] Thanks for the update. I think it’s quite an isolated problem, but with the potential to become more widespread… (or it might be a coincidence). (re @Sannita: the problem is that half of staff is currently arriving or already at the Hackathon in Istanbul...) [13:02:30] (…but be careful about trying to use lists with numbers in, if demonstrating Wikifunctions.org…) 😎 (re @Sannita: the problem is that half of staff is currently arriving or already at the Hackathon in Istanbul...) [13:09:58] I nudged a few list functions, and found another infinite runner: Z12864 [13:15:38] Yeah… My hypothesis is that Python times out cleanly with short lists whereas JavaScript doesn’t fail until the lists are longer, then fails rather gracelessly, with optional logging issues. (re @u99of9: I nudged a few list functions, and found another infinite runner: Z12864) [13:19:26] (See Z13237, for example… gateway timeout with Z989.) (re @u99of9: I nudged a few list functions, and found another infinite runner: Z12864) [13:22:08] Here's another lot that are "unable to run". I also guess that these are two sides of the same coin. Z13397 [13:31:57] Z22971 looks fixable. I think the reference in the list is mismatched with the item, then there’s a sprinkling of excessive logging… I didn’t scroll down too far. (re @u99of9: Here's another lot that are "unable to run". I also guess that these are two sides of the same coin. Z13397) [13:43:16] (🤔 actually, if you’re lucky, they’re both function calls. But the one pulled from the list probably won’t get evaluated…) (re @Al: Z22971 looks fixable. I think the reference in the list is mismatched with the item, then there’s a sprinkling of excessive logg...) [14:08:50] (Now complete, following a WikiLambda system update) (re @u99of9: Here's another lot that are "unable to run". I also guess that these are two sides of the same coin. Z13397) [16:39:56] we're looking into it now, can you give me some more info about it? (re @Al: Sorry, @Sannita function evaluations seem generally unavailable now.) [16:40:15] like, how long they were out? was it generalised or localised? [16:49:58] I think the localised problem persists at Z12767, Z889 etc. It only became generalised when looking at Z18614, I think. Once we got service unavailable, even Z10000 failed, but maybe for just a few minutes. [16:50:53] Around 08:00 UTC. (re @Sannita: like, how long they were out? was it generalised or localised?) [16:57:34] @u99of9 updated T392905 (re @Sannita: like, how long they were out? was it generalised or localised?) [16:57:56] we're following the ticket [17:05:32] I haven’t linked it to *T391435 *yet, but I believe that’s the underlying problem. (re @Sannita: we're following the ticket) [17:50:05] While we work on resolving today's problem, we have some good news to share: the MacArthur Foundation has chosen Abstract Wikipedia as one of the five finalists for its 100&Change competition (https://www.100andchange.org/about)! [17:50:06] The contest, which received 869 submissions, is aimed at funding a project that can make real, measurable progress on a critical problem of our time. [17:50:07] You can read more and comment about this at the Project Chat (https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#c-Sannita_(WMF)-20250430174700-Abstract_Wikipedia_is_a_MacArthur_Foundation%E2%80%99s_100&Change_finalist).