Not sure what qualifies as oddball, but these are some of the places I have gone when bored:

• Abenaki Tower, Melvin Village: Old rich guy thought the view was nice so he built a tower. The view is nice.
• The Libby Museum, Wolfeboro: Old rich guy collected a bunch of random crap and decided the world needed to see it so he bequeathed all his money to ensuring everyone could view his crap in perpetuity. There's lots of stuffed animals. No, the weird and slightly creepy kind.
• Tilton Arch, Northfield: Old rich guy thought arches were cool so he built a big-ass triumphal arch in basically the middle of nowhere for basically no reason. He also installed a bunch of other random, pointless sculptures throughout the town that bears his name. It's a fun scavenger hunt.
• Hannah Dustin Monument, Boscawen: Old rich guy decided to build a memorial in honor of a white lady who killed and scalped a bunch of people, so he apparently had the sculptor model it after his mom...holding a hatchet and bunch of bloody scalps. It was erected about 200 years ago and about 200 years after the heavily whitewashed and somewhat apocryphal events took place. Or possibly didn't. If you go the wrong way from the parking lot trail there is a lovely little beach on the river. It's actually quite a nice spot to mull over the unnecessarily violent and shockingly racist inscriptions on the monument itself while enjoying a picnic lunch. The graffiti art on the nearby trestle is also well above par and worth a gander.
• Endicott Rock, Laconia: Old rich guy sends a bunch of white guys to find out just how much of the natives' territory he controls. They get lost and inscribe a marker to memorialize their failed orienteering. Two hundred years later, some other white guys dredge it up and build a little house for it. And, right up the hill...
• The American Classic Arcade Museum, Laconia: Old rich guy is too cheap to update the games in the giant arcade he owns so he decides to call it a "museum." I kid, but only just. (For a small fee I will show up and stand behind you saying "flap, flap, flap" repeatedly while you play the Joust table game. I've been told this is helpful. I'll also do it if you want to play Joust II; because this is obviously a game that needed a sequel.)

Let me know if you need more.