New Haven. Take Metro North. We just did it Friday. Super easy. You can buy a ticket that is also a loaded Metrocard for the subway. You buy tickets from a machine at the station.

I live in the Providence area now, but have lived and worked in NYC, but you can be a complete NY novice and do this with confidence.