Comments
MackelBLewlis t1_isa5sav wrote
Good Work!
treebeard280 t1_isaduh1 wrote
Am I understanding correctly that this means computers will now use less power to perform the same task?
ZoomedAndDoomed t1_isafze0 wrote
GPT-3 summary and question answers:
What is the breakthrough discussed in this article?
The breakthrough discussed in this article is a more efficient way to perform matrix multiplication, discovered by DeepMind.
How is this breakthrough significant?
This breakthrough is significant because it can translate into large efficiency gains for AI applications, allowing them to run more quickly on existing hardware.
How does this breakthrough work?
This breakthrough works by training a neural network called AlphaTensor to play a game in which the goal is to zero out a matrix. AlphaTensor gradually improves over time and eventually discovers more efficient algorithms for matrix multiplication.
Give me a short description of the article/breakthrough.
The article discusses a breakthrough by DeepMind in which a more efficient algorithm for matrix multiplication was discovered. This algorithm can translate into large efficiency gains for AI applications, allowing them to run more quickly on existing hardware.
[deleted] t1_isalddd wrote
[removed]
comments247 t1_isanrth wrote
Or more power, depending how you look at it.
garden_frog t1_isatx32 wrote
This should become a bot that automatically reply to every article.
darkblitzrc t1_isawzy7 wrote
How do you enable article summary inside gpt3?? Thats awesome!
Prize_Huckleberry_79 t1_isb1a09 wrote
Plot twist: Deepmind composed this reply…
was_der_Fall_ist t1_isb3fez wrote
Just put the article in the prompt and ask it to summarize it.
ByThisKeyboardIRule t1_isba5yx wrote
Too bad that in real world things like proper cache utilization are more important than operations count. Show us the benchmarks or it doesn't matter.
korben2600 t1_isbcn5t wrote
Right? Lol. Just looking at the promoted comment with the comparison between "textbook" 2x2 matrix multiplication versus Strassen's algorithm, I'm amazed at how he was able to achieve that efficiency improvement. I can see why nobody's made progress in 50 years. The 5x5 matrix algorithm which knocked off 3 steps from the older method must be insane.
Blu4stone t1_isbe75x wrote
Excellent idea
TheSingulatarian t1_isbi45v wrote
The explain it to me like I'm 5 years old bot.
TheSingulatarian t1_isbi8de wrote
Recursive self improvement does seem like the beginning of the end.
HarryCHK t1_isbj66h wrote
Nah , it really has two version of the game, One is machine-independent one And one is machine-dependent
whenhaveiever t1_isbomv2 wrote
Or is it... (insert dramatic music) ...just the end of the beginning?
whenhaveiever t1_isbouao wrote
If it's fewer steps per calculation, why would it take more power per task?
TheSingulatarian t1_isbozff wrote
Fair enough.
[deleted] t1_isbrn6q wrote
[deleted]
mcilrain t1_isbru8i wrote
Maybe I don't understand matrix multiplication but isn't that just iterating over two arrays of numbers and multiplying each pair together?
I don't understand how that could be optimized, it shouldn't be possible to make it simpler than LOAD -> MULTIPLY -> STORE, right?
Nostr0m t1_isbt2ks wrote
You can save multiplication operations by clever additions and subtractions. See Strassen's algorithm for an example.
flyblackbox t1_isbtgx5 wrote
What is your preferred method to install, or create an online account, to access and use GPT-3?
mcilrain t1_isbtvx2 wrote
That wouldn't work in all cases though, right? Wouldn't the logic needed to determine when it's safe make it slow? Or are errors worth the increased performance?
ghostfuckbuddy t1_isbv1vj wrote
That's cool and all, but when are they going to make Alphazero open-source so I can apply it to my math problem?
powerscunner t1_isbwcpw wrote
(slightly less dramatic music)
NeutrinosFTW t1_isbxpk1 wrote
The plot thins
modestLife1 t1_isc32l3 wrote
ooh-ooh! ah-ah-ah!! 🐒
PoliticsRealityTV t1_isc4yh3 wrote
If you want to mess around with gpt-3, you can use OpenAI’s playground. I think a new account gives you a couple months of free access and then it costs to continue. For anything automated you’d probably want to look into their API
pie3636 t1_iscem6o wrote
> isn't that just iterating over two arrays of numbers and multiplying each pair together?
It isn't. That would be elementwise multiplication, which is sometimes used but not nearly as useful/ubiquitous.
KantusFury t1_iscg2h1 wrote
Because they are improving so fast that they will reach a point in which they will need more power to do even bigger calculations. Rinse and repeat
ZoomedAndDoomed t1_iscjg5w wrote
I prompted it this
"Can you please summarize this article and answer these four questions.
- What is the breakthrough discussed in this article
- How is this breakthrough significant?
- How does this breakthrough work?
- Give me a short description of the article/breakthrough. Here's the article:"
And then I literally just copy the whole article (you can press select all text on phone, and It'll sort through all the other details) and it'll output it to you. I have been using this method for days as I have low reading comprehension but GPT-3 had greatly helped me understand more in less time.
ZoomedAndDoomed t1_iscjkkh wrote
If you can make the bot with OpenAI playground, or find someone who can, it will be truly game changer for people who have low reading comprehension
Nostr0m t1_iscv4x4 wrote
No, these are deterministic algorithms guaranteed to produce the right answer, so there are no errors involved. If you would like to learn more look into an intro to algorithms course, pretty interesting stuff
mcilrain t1_iscw9kh wrote
Floating point number calculations are always slightly inaccurate to a certain degree as a performance trade-off, increasing the inaccuracy in the result in exchange for even greater efficiency is plausible.
I'd expect an algorithms course to go right over my head, I'm good with logic but terrible with numbers.
Chop1n t1_isdc0ri wrote
This really stretches the limits of what the word "record" means. When you "break a record", it's in terms of performance during a specific kind of procedure, game, whatever.
This is a matter of changing the procedure itself. It's a new convention, or maybe even what you could call a "paradigm shift" within the domain of matrix multiplication, but calling it a "math record" is utterly weird. Breaking a "math record" would be like, I don't know, greatest number of problems solved in your local high school mathletes competition or something.
Chop1n t1_isdci89 wrote
Do you get some kind of kick out of being polite to the AI?
Chop1n t1_isdcwxh wrote
Don't tell me: your name is a portmanteau of Nick Bostrom.
Sethicus99 t1_isdddyy wrote
I don’t know much, but it never hurts to be polite to something that may someday be much, much more intelligent than me.
Chop1n t1_isddw5d wrote
Good point. I suppose one may as well make a habit of it before it actually matters.
iSpatha t1_isdhtjs wrote
For real. I've been following this stuff for 8 years and seeing AI finally begin to truly advance is pretty cool.
FeezusChrist t1_isdjo3n wrote
If you spent more than a minute digging into the research:
“Algorithms in this rich space have different mathematical and practical properties. Leveraging this diversity, we adapted AlphaTensor to specifically find algorithms that are fast on a given hardware, such as Nvidia V100 GPU, and Google TPU v2. These algorithms multiply large matrices 10-20% faster than the commonly used algorithms on the same hardware, which showcases AlphaTensor’s flexibility in optimising arbitrary objectives.”
https://www.deepmind.com/blog/discovering-novel-algorithms-with-alphatensor
cy13erpunk t1_isdo5t6 wrote
this is going to be valuable for anyone who values time in and of itself
AI employed in this way can save millions/billions/trillions of manhours
maniloona t1_iseo3vv wrote
People watch one "AI takes over and tries to kill everyone" stuff and they suddenly think they're experts.
ebolathrowawayy t1_isfalg2 wrote
Also keep in mind you might be recorded at all times by all nearby devices, you can never be sure! My policy is to try to be nice at all times so AI can't hold a past transgression against me in 2030.
HeinrichTheWolf_17 t1_isfsddd wrote
Take my upvote, you made me a bit happier today.
Nostr0m t1_isggbvd wrote
Haha not intentionally, but that's interesting now you mentioned it.
GoGayWhyNot t1_isgzyky wrote
You comment a criticism but you didn't read the entire thing, classic. You must think you are much smarter than the people who decided it was worthy of being the cover of the most respected scientific journal in existence with that 30 second big brain problem finding that nobody else must have thought about
PolymorphismPrince t1_ismf6h6 wrote
Matrix multiplication is taking the dot product of every row with every column.
mcilrain t1_ismffnr wrote
I don't know what a dot product is, I tried googling it and it spat greek at me.
PolymorphismPrince t1_ismncj6 wrote
For example, to find the entry in row 4 and column 3 of the matrix you get out of the product, you take all the entries in the fourth row of the first matrix, all the entries in the 3rd column of the second matrix, multiply those lists together in the way you were talking about in your comment, and then add up all those multiplications.
Spoffort t1_isxn8i7 wrote
The text mentions about a 10-20% increase in speed, instead of training the neural network for 10 months, we will only spend 8-9 on it, one month is relatively long, in my opinion :)
ByThisKeyboardIRule t1_itgt5e4 wrote
Yeah? Still no benchmarks in the article, just generalizations like 10% (which is how much they cut the number of operations). It is good that another guy here referenced the original report, where there are actual benchmarks. The so called Strassen algorithm is actually slower on CPU than the standard algorithm for reasonably sized matrices, no matter that it performs significantly smaller number of operations. Mind blowing, huh? Seems computers are no so simple after all.
So, my question was pretty reasonable. Stop being lazy and learn something instead of insulting people who do.
[deleted] t1_is9k3uw wrote
[removed]