You don't need to turn off a hard drive to unplug them. The act of unplugging the drive is what turns it off (one of the plugs is for power). Drives can otherwise really only be turned off by the OS's power management plan, but that isn't a pre-requisite for disconnecting it.

That said: any storage media you wish to disconnect while the system is on, you should ensure it not have any pending writes or is not actively being written to. If you disconnect a storage medium while it's in use, you could corrupt its data. Windows has an "eject storage device" feature for external storage devices for exactly this purpose. For internal drives, generally they shouldn't be disconnected while the system is on, even if it's not a problem electrically^([1]); unless the system is specifically designed for it^([2]).

[1] Eg. SATA is hot-pluggable - it can be safely (dis)connected while the system is powered

[2] Eg. servers which can remain operational while dead drives are replaced