[18:09:35] <JJMC89>	 What would make k8s pods take a long time to start after the job starts? I'm seeing mine take 12 or so minutes (was ~8 yesterday) when it used to be seconds.
[18:15:20] <taavi>	 JJMC89: that indeed sounds wrong. do you have an example I can have a look at?
[18:15:44] <taavi>	 the only slow thing happening before a job starts would be a potential docker image pull, but even that should not take that long
[18:16:27] <JJMC89>	 taavi: You can look at the jobs on https://k8s-status.toolforge.org/namespaces/tool-jjmc89-bot/. No pods are active now but there are two jobs running.
[18:23:24] <taavi>	 !log tools.mismatch-finder-staging deleted cronjob "mismatch-finder-staging.schedule" creating too many job objects
[18:23:28] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mismatch-finder-staging/SAL
[18:23:44] <taavi>	 hopefully that should make things work a bit better
[18:26:48] <JJMC89>	 looking better - most recent job had the pod start within 15 seconds.
[18:26:52] <JJMC89>	 thanks
[18:36:53] <taavi>	 !log tools.mismatch-finder-staging manually add T301081 fixes to prevent further k8s cluster issues
[18:36:56] <taavi>	 !log tools.mismatch-finder manually add T301081 fixes to prevent further k8s cluster issues
[18:36:58] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mismatch-finder-staging/SAL
[18:36:59] <stashbot>	 T301081: toolforge: Fix job/cronjob quotas - https://phabricator.wikimedia.org/T301081
[18:37:02] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.mismatch-finder/SAL
[18:55:24] <wm-bot>	 !log tools.notwikilambda <lucaswerkmeister> Checked out WSOAuth change Ie0a828e368 PS2 (commit c85a5a1c42), hopefully fixes T298351
[18:55:27] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.notwikilambda/SAL