If you feel that strongly about staying in the apartment then do it. A 3-4k difference won’t mean anything to you in 6 years (if everything goes well)

You really don’t get those college years back. Do everything that you want to do.