Hm, so I don't know of any exclusively Jewish delis left but I've had pretty good bagels here. (lived in NYC for 6 years) Coffee & Bagel Connection in Narragansett had some tasty bagels with a good exterior crunch. Providence Bagel in URI was decent too. Second the recommendation for Bunz on Aquidneck for babka.

EDIT: I forgot to mention Jamos bagels who distributed out of Nitro bar but I heard he's closing shop to go to college. The Cheddar Jalapeño was SO GOOD.