[07:23:42] <snitch>	 [[Tech]]; AlPaD; /* Special:ChangePassword doesn't work but throws no errors */Reply; https://meta.wikimedia.org/w/index.php?diff=23185884&oldid=23180642&rcid=23853120
[17:02:48] <Krinkle>	 Reedy: regarding TMH job failures - I suspect we've lost some there given exhausting of 3 retries, right?
[17:03:12] <Krinkle>	 ref v
[17:03:13] <Krinkle>	 ref T306697
[17:03:13] <stashbot>	 T306697: Videos scalers cannot create jobs: Failed creating job from description - https://phabricator.wikimedia.org/T306697
[17:03:59] <Reedy>	 Maybe
[17:04:07] <Reedy>	 I requeued all the errored stuff on commons
[17:04:20] <Reedy>	 https://commons.wikimedia.org/wiki/Special:Transcode_statistics
[17:04:58] * Krinkle gets a timeout
[17:05:12] <Reedy>	 it's a bit slow
[17:05:13] <Reedy>	 >11,493 queued transcodes
[17:05:59] <Krinkle>	 ack, that's a lot. do we have a maintenance script to automate that?
[17:06:09] <Krinkle>	 that is.. how did you requeue them?
[17:06:11] <Reedy>	 yeah
[17:06:18] <Krinkle>	 neat
[17:06:20] <Krinkle>	 I didn't know.
[17:06:20] <Reedy>	 it's in the TMH maintenance folder
[17:06:30] <Reedy>	 Noting some of the ones requeued ones could be long failed too
[17:06:37] <Reedy>	 That failed due to other reasons, and may fail again
[17:06:51] <Krinkle>	 ah because we have the db table dedicated for this
[17:06:58] <Krinkle>	 cool
[17:07:00] <Krinkle>	 right
[19:43:27] <RhinosF1>	 Reedy: the jobrunner & videoscaler keeps firing probe down. Could that be related given they'll be suddenly more jobs?
[19:43:46] <Reedy>	 They're probably a lot busier :)
[19:44:03] <Reedy>	 So might explain not responding to more background stuff
[19:44:29] <RhinosF1>	 Reedy: quite possibly, I just noticed the noise in -operations
[19:44:42] <RhinosF1>	 And blaming you for breaking stuff is fun :)