If the duct seems like an oval, than I would suggest you pick up a round-to-oval adapter, and see if that works. If it does, great; problem solved. If the oval part of the adapter doesn't seem to match up well with the duct in your wall, than it may be that someone just squished a piece of round duct.

Not sure why someone attached the piece of rigid duct to the semi-rigid duct. I don't see any reason why you can't simply remove it.