Actually, for intelligent life to occur (and not be destroyed) there may be even more conditions to be fulfilled than the amount of galaxies.

Besides, you are looking only at one dimension here - size of the universe. There is one, even more hard to comprehend - time. And I find it entirely possible that there may have been civilizations before but what can be almost impossible - is to have the right matching for time and place.

Anyways, not taking aliens as a matter of fact is the right scientific approach for now. And probability may go to hell with such a sample size, as someone pointed above.