I will tell you in terms of India.

So just because it is legal now, the prisons don't open their gates to the prisoners. However, if the prisoner makes an application to the court, saying that the crime for which he has been imprisoned is no longer a crime, then he'll be released immediately. Because once the decriminalization takes place, it essentially means that the act is no longer a crime. The society has changed its view that the act is alright. So there is no sense in keeping somebody inside and punishing them.