One thing to do, that can test this without any cost? Hang up a comforter or a heavy blanket across the door. If that improves things, you know you need those heavy curtains.

:)