Depends on what your needs and budget are, but Irvington has the least amenities of the three you listed (small town, big pockets of underdeveloped, no train station).

Irvington has some nice quiet parts, but if you’re going to deal with urban issues, you might as well live in a larger city if the home prices are comparable.