Viewing a single comment thread. View all comments

CommunismDoesntWork t1_jdcloqz wrote

Reply to comment by FrereKhan in [P] New toolchain to train robust spiking NNs for mixed-signal Neuromorphic chips by FrereKhan

Is there specialized hardware for SNNs yet?

1

FrereKhan OP t1_jdcn3b6 wrote

Yes, a few options. Rockpool is designed to work with SNN chips from SynSense (https://synsense.ai ). Intel has Loihi, there is also Akida from BrainChip…

1

CommunismDoesntWork t1_jdcpcv9 wrote

Are those chips general purpose SNN accelerators in the same way GPUs are general purpose NN accelerators? If so, what's stopping someone from creating a 100B parameter SNN similar to LLMs?

1

FrereKhan OP t1_jdcvobu wrote

Sort of yes; Xylo is a general-purpose SNN accelerator, but the scale is for smaller problems, in the order of 1000 neurons.

But in principle there's nothing standing in the way of building a 100B parameter SNN.

1

Art10001 t1_jdd0ag1 wrote

Brainchip has 1 million neurons already. Loihi and Loihi2 similar.

1

CommunismDoesntWork t1_jdcxx5u wrote

>But in principle there's nothing standing in the way of building a 100B parameter SNN.

That's awesome. In that case, I'd pivot my research if I were you. These constrained optimization problems on limited hardware are fun and I'm sure they have some legitimate uses, but LLMs have proven that scale is king. Going in the opposite direction and trying to get SNNs to scale to billion of parameters might be world changing.

Because NNs are only going to get bigger and more costly to train. If SNNs and their accelerators can speed up training and ultimately reduce costs, that would be massive. You could be the first person in the world to create a billion parameter SNN. Once you show the world that it's possible, the flood gates will open.

0