During the day you should be ok but it's a bit sketch a few blocks north and down around Lexington Market. I'd walk down Park and across Fayette or Baltimore, much less sketchy than the other routes.