Thank you. Figure 1 is exactly the models I am considering for this problem.

Can you elaborate?

The initial error is (y - y_hat)^2 where y_hat is mean(y1, ... yn). So the error is divided up among the y1...yn sequence based on how bad they contribute to the y. If models are trained separately, then thefull error of y is backproped. If the models are trained together, one model might have a lot of error which will influence the proportion assigned to the rest which I believe effectively lowers the learning rate. Is this what you mean by "loss values will be smoother."

Is there a mistake here?

This isn't for a production model

perceptual manifold priors using deep networks

Your very last thought seems the most reasonable. I can't imagine shrinking the model. I would surely think this would bias the results.

zero dollars USD