Common sense is often quite narrow and short-sighted though. It doesn't seem particularly hard for governments to give amnesties when they decriminalise things, so why isn't this the norm?

It's not hard to think of potential problems.

What if it looks like a law is going to be changed and people start breaking it anticipating that change? Even worse, what if it then isn't changed?

Does it undermine people's respect for laws and fear of punishment?

Does the law punish people based on the impact of their actions or their character? Eg. we might now believe there's little harm in selling weed, but if the consensus when someone sold some weed was it was dangerous, shouldn't they have paid attention to that? (This seems similar to how we don't apply new crimes retrospectively.)

What if the law changes because the situation changes? For example, the benefits and harms of drug prohibition differ in different circumstances.

With examples like this there are usually a bunch of reasons why things are done the way they are that aren't obvious or have been forgotten. They may or may not be persuasive reasons, but you don't know until you know what they are.