There are lots of parks and spots along the water / rivers in New Haven. Fair Haven, Fair Haven Heights, and Long Wharf have some nice little spaces on the water. If you search for boat or canoe launches, you'll find areas that you can access in a car that are on the water.