Viewing a single comment thread. View all comments

CommunismDoesntWork t1_jdcpcv9 wrote

Reply to comment by FrereKhan in [P] New toolchain to train robust spiking NNs for mixed-signal Neuromorphic chips by FrereKhan

Are those chips general purpose SNN accelerators in the same way GPUs are general purpose NN accelerators? If so, what's stopping someone from creating a 100B parameter SNN similar to LLMs?

1

FrereKhan OP t1_jdcvobu wrote

Sort of yes; Xylo is a general-purpose SNN accelerator, but the scale is for smaller problems, in the order of 1000 neurons.

But in principle there's nothing standing in the way of building a 100B parameter SNN.

1

Art10001 t1_jdd0ag1 wrote

Brainchip has 1 million neurons already. Loihi and Loihi2 similar.

1

CommunismDoesntWork t1_jdcxx5u wrote

>But in principle there's nothing standing in the way of building a 100B parameter SNN.

That's awesome. In that case, I'd pivot my research if I were you. These constrained optimization problems on limited hardware are fun and I'm sure they have some legitimate uses, but LLMs have proven that scale is king. Going in the opposite direction and trying to get SNNs to scale to billion of parameters might be world changing.

Because NNs are only going to get bigger and more costly to train. If SNNs and their accelerators can speed up training and ultimately reduce costs, that would be massive. You could be the first person in the world to create a billion parameter SNN. Once you show the world that it's possible, the flood gates will open.

0