Someone's going to inevitably mention silphium and it going extinct because of people using it as contraceptive, which is a misunderstood concept, but it does have some basis in reality in that a lot of herbs and such were used in a variety of way to help with contraceptive. Some of these things were taken orally, with the belief that it might help (silphium was mainly used as an herb for food, so if it did have these properties there would be a lot of missing babies throughout the Mediterranean). Others would be made into a paste and put inside the woman to essentially collect the sperm within that paste, or it could be dried and placed inside to have the same effect but a bit less messy -- well at least I'd imagine so.

You also had methods we wouldn't recommend in the modern day, when far safer options exist, but in antiquity are not bad, like the pull-out method. Using this method greatly reduces the risk (though still not completely, and you're stuck with hoping the person does this properly).

Ancient Romans would discard unwanted children after birth (if they had deformities) on refuse piles.