[00:50:53] <wm-bot>	 !log tools.mjolnir <legoktm> Upgraded mjolnir to 1.2.1
[00:50:56] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mjolnir/SAL
[00:51:11] <wm-bot>	 !log tools.mjolnir <legoktm> Upgraded uatu to 0.1.7
[00:51:12] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mjolnir/SAL
[09:52:54] <zache-tool>	 Hi, i am running some php scripts on toolforge  (example https://fiwiki-tools.toolforge.org/api/testalbum.php ) and in last week or so fiwiki-tools has been timeouting / stuck. Static files seems to work and restarting webservice helps. However, is there any idea what is going on? Also is this something which is related to my scripts. (it seems
[09:52:55] <zache-tool>	 that all php scripts are stuck when it stucks)
[09:54:20] <zache-tool>	 it would be also helpful if somebody would have any hint how to debug this
[10:54:03] <dcaro>	 zache-tool: when the tool is stuck, you can check the pod events with (kubectl --namespace=tool-fiwiki-tools describe pod), that might give a hint if it's something on the kubernetes system. Another option is also looking at the pod logs (kubectl --namespace=tool-fiwiki-tools logs <pod_name>, where <pod_name> is from kubectl --namespace=tool-fiwiki-tools get pods)
[11:04:58] <mdipietro>	 !log paws added julia to paws with 12bfdad0b4d7c60737b79914b6098029235ce9ef
[11:05:00] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL
[12:02:00] <zache-tool>	 dcaro: thanks!
[12:57:26] <mdipietro>	 !log paws added julia to paws with 7b58fb09292a27f939f93ffba6ff00f8028302f2
[12:57:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Paws/SAL
[17:41:57] <legoktm>	 Cyberpower678: you there?
[17:42:12] <Cyberpower678>	 legoktm: kind of
[17:42:49] <legoktm>	 InternetArchiveBot is making 157 req/second against the API cluster
[17:42:58] <legoktm>	 that's way too fast
[17:43:10] <Cyberpower678>	 On one wiki?
[17:43:39] <legoktm>	 across a bunch
[17:44:15] <Cyberpower678>	 But I IABot is actively working on dozens of wikis though.
[17:44:25] <legoktm>	 it's going too fast
[17:44:27] <Cyberpower678>	 Each wiki is it's own process.
[17:44:47] <legoktm>	 please stop it or we're going to have to block it or institute some rate limit
[17:46:02] <Cyberpower678>	 How is IABot supposed to effectively work on multiple wikis if it's confined a rate limit?
[17:46:09] <Cyberpower678>	 I'm confused.
[17:46:28] <Cyberpower678>	 So every wiki is on the same cluster?
[17:47:04] <Cyberpower678>	 And also, what is it supposed to be keeping in line of?'
[17:47:29] <Cyberpower678>	 *global rate limit
[17:47:49] <Cyberpower678>	 legoktm: ^
[17:47:56] <legoktm>	 yes, wikis share resources
[17:48:17] <legoktm>	 I don't have an answer on an appropriate rate limit, but 150req/sec for action=parse is just too fast
[17:48:58] <Cyberpower678>	 legoktm: is there another way to get page text that doesn't hit the API?
[17:49:16] <legoktm>	 first, please turn off your bot or slow it down dramatically
[17:50:58] <Cyberpower678>	 legoktm: give me a bit.
[17:52:57] <Cyberpower678>	 legoktm: a bunch of workers should be shutting down now.
[17:54:41] <legoktm>	 thanks, looks like its down to 30req/sec
[17:55:25] <legoktm>	 why are you using action=parse&prop=wikitext to get page text?
[17:55:42] <legoktm>	 and not prop=revisions&rvprop=content, which allows for getting it in bulk?
[17:56:03] <Cyberpower678>	 IABot processes one article at a time.
[17:56:32] <Cyberpower678>	 Is parse much more expensive the prop=revision?
[17:56:36] <Reedy>	 Yes
[17:56:43] <Cyberpower678>	 Oh.  Sorry about that.
[17:56:59] <Reedy>	 prop=revisions will basically give you the raw wikitext
[17:57:01] <Cyberpower678>	 I'll change over to use prop=revisions then.
[17:57:07] <Cyberpower678>	 That's what I want.
[17:58:10] <legoktm>	 Cyberpower678: I'm sure you can figure out how to batch requests to get wikitext
[17:58:25] <legoktm>	 Pywikibot has a generator that lets you preload page content for example
[17:59:13] <legoktm>	 getting Parsoid HTML (not wikitext) from https://en.wikipedia.org/api/rest_v1/#/Page%20content also has a much higher rate limit
[18:00:08] <Cyberpower678>	 legoktm: limitation of IABot, no that's not a possibility.  BUT IABot 3 is being drafted and this will go on that list.
[18:00:50] <legoktm>	 please implement some concurrency limiting then
[18:02:03] <Cyberpower678>	 legoktm: Mark is sending you an email.
[18:02:12] <Cyberpower678>	 Just a heads up
[18:03:03] <Cyberpower678>	 I may actually have some idea on what I can do to batch the requests.
[18:03:23] <Cyberpower678>	 But today is a day of hell for me, so it's not happening today.
[18:03:51] <legoktm>	 ok
[18:04:22] <Cyberpower678>	 legoktm: any chance I can monitor this for myself.  Not my intention to blow up production.  I've done that enough already. :p
[18:04:56] <Cyberpower678>	 But eventually the end-game for IABot is to run on all wikis
[18:05:22] <legoktm>	 do you log your outbound requests? I'm just tailing all API requests and grepping for IAB's username
[18:05:36] <Cyberpower678>	 No.