>but with current materials you'd struggle to get anything more than a tiny amount of bouyancy from the second example

This is the principle used to propel Autonomous Underwater Vehicles (gliders)