Take the CT Rail from Hartford to New Haven, and switch to a Metro North train for NYC.

If you only want one train I would drive down to Milford (no offense, but it's nicer than New Haven and West Haven). I take the Metro North out of Bridgeport. It's very nice in that specific area, but kind of a drive for you guys.

This is random, but here's a list of public bathrooms in NYC.

Between the train station and Times Square

Grand Central. Downstairs. Multiple options. There's usually a line, but it moves fast. Cleaner than expected, but not great.

Bryant Park. It's this random building at the edge. It's actually really nice.

The New York Public Library. Somewhere upstairs. They have signs to follow.

Disney Store in Times Square.

M&M store in Times Square. I have seen it be closed before though. It's usually dirty.

Over in Rockefeller Center

FAO Swartz. Top floor. Clean, but small.

The American Girl Doll Store. Bottom floor. Very clean.

Bottom of Central Park

If you walk into the park at the bottom, toward the middle, and just kind of keep going there's this random public bathroom. It's a big grey building near some play systems. It's hard for me to explain. I kind of just know where it is when I get there.

There's another public bathroom across from The Central Park Zoo. I remember the food court attached to the zoo had a bathroom, but after COVID sometimes it kept being closed. Either way you can go to that one across from the zoo.

Below 42nd Street

The Harry Potter store. If you go down 5th Ave to around 22nd street the epic Harry Potter store is there. The bathroom is on the bottom floor. The bathroom is single person, so there's always a line. You see the Flat Iron building first, walk slightly past it, and there's the Harry Potter store.

Macy's on 34th over on 7th Ave I believe it is, has a bathroom. You have to go up a certain amount of floors. There are signs to follow.