The naval strength was already there; the Vandals just came in and ousted the Roman governor and said "hey north African navy, you work for us now!"

By this point in history the empire did not have very good cohesion, so there wasn't really a unified and effective "imperial navy" that could coordinate a retaliation; any force which might have done so was busy elsewhere. The empire was being attacked from all sides by different groups of barbarians and the Muslims. For those sailors and marines who were in North Africa when the Vandals came, the easiest and safest thing to do was to continue going about their business as if nothing had changed. "I, for one, welcome our new Germanic overlords!" If you don't piss off the new boss, you get to keep your job.