The detector itself is technically part of the brain, as is all of the retina. It's true that the response of the cells that detect light is the opposite of normal neuron activation - hyperpolarization instead of depolarization. That gets swapped around before the signal leaves the retina though, along with some early processing taking place.