In practice, this is done by numerical optimization, using ready-made optimization software.

A theory which was applied for designing the trajectory for the first "civilian" US space rocket "Vanguard", was presented in an article "Satellite launching vehicle trajectories" by Joseph W. Siry, published in 1959. The article was published in "Orbit Theory, Proceedings of the Ninth Symposium in Applied Mathematics", which unfortunately is slightly difficult to get a hold of.

More recent publications are easy to find, from lecture slides, to NASA technical reports, to various academic publications.