Capacitors are devices that store energy by moving charge carriers (usually electrons) in places it really doesn't want to be in--which means they have to be forced in that position--which takes energy that you can get back later, quite easily. Charge carriers like electrons can move around efficiently and quickly in conductors like metal, so capacitors generally can be charged and discharged rapidly.

Batteries are devices that store energy by making ions stay in energetically excited conditions that it doesn't really want to be in. Ions are generally floating charged particles in a liquid solution, so they take much longer to float around and do their chemistry to get your energy back. The upside is that you get to store a lot more energy into the structure, but the downside is that the rate at which you can charge and discharge is beholden to the chemistry and the speed at which ions move, so it's quite slow. They are also inefficient since ions aren't efficiently transported, as much as electrons are in metal.