Comments

You must log in or register to comment.

Ortus12 t1_is9vvva wrote

Ai designing better algorithms for Ai. Ai designing better computer chips for Ai.

It has begun.

176

TheSingulatarian t1_isbi8de wrote

Recursive self improvement does seem like the beginning of the end.

28

whenhaveiever t1_isbomv2 wrote

Or is it... (insert dramatic music) ...just the end of the beginning?

32

[deleted] t1_isbrn6q wrote

[deleted]

28

iSpatha t1_isdhtjs wrote

For real. I've been following this stuff for 8 years and seeing AI finally begin to truly advance is pretty cool.

4

maniloona t1_iseo3vv wrote

People watch one "AI takes over and tries to kill everyone" stuff and they suddenly think they're experts.

3

ZoomedAndDoomed t1_isafze0 wrote

GPT-3 summary and question answers:

What is the breakthrough discussed in this article?

The breakthrough discussed in this article is a more efficient way to perform matrix multiplication, discovered by DeepMind.

How is this breakthrough significant?

This breakthrough is significant because it can translate into large efficiency gains for AI applications, allowing them to run more quickly on existing hardware.

How does this breakthrough work?

This breakthrough works by training a neural network called AlphaTensor to play a game in which the goal is to zero out a matrix. AlphaTensor gradually improves over time and eventually discovers more efficient algorithms for matrix multiplication.

Give me a short description of the article/breakthrough.

The article discusses a breakthrough by DeepMind in which a more efficient algorithm for matrix multiplication was discovered. This algorithm can translate into large efficiency gains for AI applications, allowing them to run more quickly on existing hardware.

164

garden_frog t1_isatx32 wrote

This should become a bot that automatically reply to every article.

113

ZoomedAndDoomed t1_iscjkkh wrote

If you can make the bot with OpenAI playground, or find someone who can, it will be truly game changer for people who have low reading comprehension

8

cy13erpunk t1_isdo5t6 wrote

this is going to be valuable for anyone who values time in and of itself

AI employed in this way can save millions/billions/trillions of manhours

3

darkblitzrc t1_isawzy7 wrote

How do you enable article summary inside gpt3?? Thats awesome!

19

was_der_Fall_ist t1_isb3fez wrote

Just put the article in the prompt and ask it to summarize it.

19

ZoomedAndDoomed t1_iscjg5w wrote

I prompted it this

"Can you please summarize this article and answer these four questions.

  1. What is the breakthrough discussed in this article
  2. How is this breakthrough significant?
  3. How does this breakthrough work?
  4. Give me a short description of the article/breakthrough. Here's the article:"

And then I literally just copy the whole article (you can press select all text on phone, and It'll sort through all the other details) and it'll output it to you. I have been using this method for days as I have low reading comprehension but GPT-3 had greatly helped me understand more in less time.

20

Chop1n t1_isdci89 wrote

Do you get some kind of kick out of being polite to the AI?

2

Sethicus99 t1_isdddyy wrote

I don’t know much, but it never hurts to be polite to something that may someday be much, much more intelligent than me.

11

Chop1n t1_isddw5d wrote

Good point. I suppose one may as well make a habit of it before it actually matters.

6

ebolathrowawayy t1_isfalg2 wrote

Also keep in mind you might be recorded at all times by all nearby devices, you can never be sure! My policy is to try to be nice at all times so AI can't hold a past transgression against me in 2030.

3

flyblackbox t1_isbtgx5 wrote

What is your preferred method to install, or create an online account, to access and use GPT-3?

3

PoliticsRealityTV t1_isc4yh3 wrote

If you want to mess around with gpt-3, you can use OpenAI’s playground. I think a new account gives you a couple months of free access and then it costs to continue. For anything automated you’d probably want to look into their API

3

treebeard280 t1_isaduh1 wrote

Am I understanding correctly that this means computers will now use less power to perform the same task?

34

comments247 t1_isanrth wrote

Or more power, depending how you look at it.

0

whenhaveiever t1_isbouao wrote

If it's fewer steps per calculation, why would it take more power per task?

11

KantusFury t1_iscg2h1 wrote

Because they are improving so fast that they will reach a point in which they will need more power to do even bigger calculations. Rinse and repeat

3

2D_VR t1_isanph7 wrote

That is a weird algorithm. I don't think I would have come up with it given a million years

12

korben2600 t1_isbcn5t wrote

Right? Lol. Just looking at the promoted comment with the comparison between "textbook" 2x2 matrix multiplication versus Strassen's algorithm, I'm amazed at how he was able to achieve that efficiency improvement. I can see why nobody's made progress in 50 years. The 5x5 matrix algorithm which knocked off 3 steps from the older method must be insane.

19

DakPara t1_isawlu4 wrote

I find this very impressive.

10

ByThisKeyboardIRule t1_isba5yx wrote

Too bad that in real world things like proper cache utilization are more important than operations count. Show us the benchmarks or it doesn't matter.

9

FeezusChrist t1_isdjo3n wrote

If you spent more than a minute digging into the research:

“Algorithms in this rich space have different mathematical and practical properties. Leveraging this diversity, we adapted AlphaTensor to specifically find algorithms that are fast on a given hardware, such as Nvidia V100 GPU, and Google TPU v2. These algorithms multiply large matrices 10-20% faster than the commonly used algorithms on the same hardware, which showcases AlphaTensor’s flexibility in optimising arbitrary objectives.”

https://www.deepmind.com/blog/discovering-novel-algorithms-with-alphatensor

11

GoGayWhyNot t1_isgzyky wrote

You comment a criticism but you didn't read the entire thing, classic. You must think you are much smarter than the people who decided it was worthy of being the cover of the most respected scientific journal in existence with that 30 second big brain problem finding that nobody else must have thought about

5

ByThisKeyboardIRule t1_itgt5e4 wrote

Yeah? Still no benchmarks in the article, just generalizations like 10% (which is how much they cut the number of operations). It is good that another guy here referenced the original report, where there are actual benchmarks. The so called Strassen algorithm is actually slower on CPU than the standard algorithm for reasonably sized matrices, no matter that it performs significantly smaller number of operations. Mind blowing, huh? Seems computers are no so simple after all.

So, my question was pretty reasonable. Stop being lazy and learn something instead of insulting people who do.

1

HarryCHK t1_isbj66h wrote

Nah , it really has two version of the game, One is machine-independent one And one is machine-dependent

2

ghostfuckbuddy t1_isbv1vj wrote

That's cool and all, but when are they going to make Alphazero open-source so I can apply it to my math problem?

6

mcilrain t1_isbru8i wrote

Maybe I don't understand matrix multiplication but isn't that just iterating over two arrays of numbers and multiplying each pair together?

I don't understand how that could be optimized, it shouldn't be possible to make it simpler than LOAD -> MULTIPLY -> STORE, right?

2

Nostr0m t1_isbt2ks wrote

You can save multiplication operations by clever additions and subtractions. See Strassen's algorithm for an example.

8

mcilrain t1_isbtvx2 wrote

That wouldn't work in all cases though, right? Wouldn't the logic needed to determine when it's safe make it slow? Or are errors worth the increased performance?

1

Nostr0m t1_iscv4x4 wrote

No, these are deterministic algorithms guaranteed to produce the right answer, so there are no errors involved. If you would like to learn more look into an intro to algorithms course, pretty interesting stuff

5

mcilrain t1_iscw9kh wrote

Floating point number calculations are always slightly inaccurate to a certain degree as a performance trade-off, increasing the inaccuracy in the result in exchange for even greater efficiency is plausible.

I'd expect an algorithms course to go right over my head, I'm good with logic but terrible with numbers.

3

Chop1n t1_isdcwxh wrote

Don't tell me: your name is a portmanteau of Nick Bostrom.

1

Nostr0m t1_isggbvd wrote

Haha not intentionally, but that's interesting now you mentioned it.

3

pie3636 t1_iscem6o wrote

> isn't that just iterating over two arrays of numbers and multiplying each pair together?

It isn't. That would be elementwise multiplication, which is sometimes used but not nearly as useful/ubiquitous.

2

PolymorphismPrince t1_ismf6h6 wrote

Matrix multiplication is taking the dot product of every row with every column.

1

mcilrain t1_ismffnr wrote

I don't know what a dot product is, I tried googling it and it spat greek at me.

1

PolymorphismPrince t1_ismncj6 wrote

For example, to find the entry in row 4 and column 3 of the matrix you get out of the product, you take all the entries in the fourth row of the first matrix, all the entries in the 3rd column of the second matrix, multiply those lists together in the way you were talking about in your comment, and then add up all those multiplications.

1

Chop1n t1_isdc0ri wrote

This really stretches the limits of what the word "record" means. When you "break a record", it's in terms of performance during a specific kind of procedure, game, whatever.

This is a matter of changing the procedure itself. It's a new convention, or maybe even what you could call a "paradigm shift" within the domain of matrix multiplication, but calling it a "math record" is utterly weird. Breaking a "math record" would be like, I don't know, greatest number of problems solved in your local high school mathletes competition or something.

2

Lawjarp2 t1_isii3vy wrote

47 calculations instead of 49 is pretty small tbh. But then it is impressive that it's possible at all

0

Spoffort t1_isxn8i7 wrote

The text mentions about a 10-20% increase in speed, instead of training the neural network for 10 months, we will only spend 8-9 on it, one month is relatively long, in my opinion :)

1