Stellar mass black holes are similar to large stars and these are what are roaming around. SMBH are basically only found at the centre of galaxies and due to that are exceedingly rare. Black holes pose was less of a threat to us than stars do as stars outnumber black holes a million to one or something else ridiculous.