10^-3 would be the highest learning rate I’d recommend. It’s just a starting point. Ideally if you want to do search for model performance start at either 10^-3 or 10^-4 and go from there.

For fine tuning like that you’ll just have to try it out and move lower when necessary.

It depends on the model and the task, so there is no general answer. But you dont have to search randomly. Plot your loss over time. If the lr is too high, the loss will behave almost randomly, and its almost constant if lr is too low