& often don't even need to fine tune because of instruction pre training and few shot prompting

gradient descent takes the direction of the minimum at the step size according to the taylor series of degree n at that point. in neural nets we do first degree, as if it was a plane. in a lot of other optimization settings they do second order approx to find the optimal direction

first degree optimal direction

you definitely should contact people higher