>If it makes it correctly, it will update its parameters to reinforce its confidence

Nonsense. If it makes it correctly, the loss will be zero and the parameters are allowed to remain as they are. Only if it makes a mistake, the loss will be non-zero and change the parameters as it propagates backward through the network.