>Good question. If I go on the internet and find a copyrighted image and alter it to create a new image, under US law, that would still be copyright infringement.

It actually depends on the extent of the alteration and whether the result then falls into a protected class like parody

>until the generated image is as close as it can get based on what it has been trained on

This isn't the case; if it was you wouldn't be able to create novel images by combining different ideas, but of course you can