To my knowledge, it's the sort of rare thing a very small percentage of people with preexisting neurological conditions would have to worry about (e.g. heritable schizophrenia risks). Generally speaking, that "neverending trip" stuff is fear-mongering anti-drug propaganda from the days when the War On Drugs first started and programs like DARE were used to try and scare kids away from experimentation. But don't take my word for it; Spend a bit of time learning about psychedelic compounds and how they interact with the brain. It's pretty interesting stuff, actually.