Other galaxies would not have made the sky brighter, at least not 4 billion years ago. Andromeda is the closest galaxy and it is barely visible to the naked eye as is. Cosmic expansion means other galaxies would have been closer to us in the past, but not as close as Andromeda. Maybe if you go back like 13 billion years to when galaxies were being first formed, the cosmic neighborhood would have been 8 times denser back then, and maybe multiple galaxies would have been visible in the sky, including some that Milky Way has since collided with.

What would have been visible 4 billion years ago is more brighter stars. The brightest stars are the heaviest but also the most short-lived. They are only visible for a few million years after they are formed, but make up most of the starlight reaching us. The number of heavy bright stars is proportional to number of new stars formed, but star formation in galaxies is always slowing down... There were more bright stars in the past, and there will never be as many bright stars as there are in the sky today, except for a short period of time in 5 billion years when Andromeda crashes into the Milky Way, causing a temporary burst in new star formation.