"They hardly build solution that scales as well. So it comes down to the MLOps platform to be able to meet with their weird requirements."

This.

MLOps is not the bottleneck, go with something as simple as possible, as you will be spending most of your time pretending you are not rewriting their entire code.