I've been to The Lafayette in Hay-Adams Hotel which is the roof terrace restaurant across the street from the white house.

We went during cherry blossom season and they had a Cherry Blossom special afternoon tea so desserts were cherry blossom themed. It was pricey, $85/pp with champagne ($65 wo) but it actually was a lot of food and they packed the leftovers us lol. We sat outside on terrace which was fantastic. I think everyone who wanted a terrace view got an outdoor table.

I've done afternoon tea at Mandarin Oriental (now salamander) and would definitely pick The Lafayette for ambience and quality of food.

Great special occasion spot.

I was celebrating my mom and we walked to check out blossom after.