Viewing a single comment thread. View all comments

ianitic t1_ja5fsnj wrote

Reply to comment by Cryptizard in So what should we do? by googoobah

So says you too. Transformers are marginal in the grand scheme of technological progress. If transformers were even 10x more efficient than CNNs or LSTMs, transformers would still be an improvement that came orders of magnitude slower than Moores law. CNNs/LSTMs being decades old.

There's a reason why all articles regarding a singularity uses Moore law as it's base, it's been the largest contributor to our increase in technological advancement over the years. That contributor is ending.

1

Cryptizard t1_ja5ipf2 wrote

>That contributor is ending.

Now its my turn to point out that they have been saying that since the 80s.

3

ianitic t1_ja5jy61 wrote

That's true, but it was always known to not a be forever thing and it has slowed down. I think I remember the last big milestone where they said that was die size of 45nm or so because of quantum tunneling. Thing is, there is a physical limit to how small we can make transistors.

Once we're dealing with transistors that are as thin as atoms, where do we go from there? Yes quantum computing, optical transistors, graphene, etc, exist, but do they provide a higher performance per dollar than silicon transistors? Probably not and it's all about price per performance.

0

Cryptizard t1_ja5mqse wrote

Nvidia seems to disagree with you. They think it is speeding up.

0

ianitic t1_ja5pfbj wrote

A CEO trying to sell their products says that their products are going to be even better in their future? They're trying to make Nvidia seem relevant and ease investor concerns with all the other big tech companies taking a hit recently.

0