Submitted by donnygel t3_11rjm6h in technology
Rohit901 t1_jc9cfm5 wrote
Imagine if google decided to not share their paper on transformers with the world, citing the same reasons as cited by open AI.
ninjasaid13 t1_jc9m41f wrote
They said competitive landscape reasons or safety reasons but we all know it's the former.
SidewaysFancyPrance t1_jcbeccw wrote
AI arms race, which is 100% about money and greed. Morality and ethics slow things down. Regulations slow things down. Wall Street and investors are watching for a prospective winner to bet big on. Tech is stale, old news. They need the next big win.
AI domination is going to be "first past the post" and all these companies know it. We're going to hear all kinds of BS about how they're trying to be ethical/etc but they really want to be first and snap up lucrative deals with their protected IP. What was open will stop being open very quickly once a public tech demo like this gets attention, and be strictly protected for that reason.
chuntus t1_jcbmk62 wrote
Why will it be ‘first past the post’? The market tolerates multiple versions of other technologies why not in this field?
CactusSmackedus t1_jcdmi15 wrote
It's not, the commenter doesn't know what they're talking about. There's a paper out in the last few days (I think) showing that weaker systems can be fine tuned on input/output from stronger model and approximate the better models' results. This implies any model with paid or unpaid API access could be subject to a sort of cloning. It suggests that competitive moats will not be able to hold.
Plus (I have yet to reproduce since I've been away from my machine) APPARENTLY a Facebook model weights got leaked in the last week and apparently someone managed to run the full 60B weights model on a raspberry pi (very very slowly) but two implications:
-
"Stealing" weights continues to be a problem, this isn't the first set of model weights to get leaked iirc, and once you have a solid set of model weights out, experience with stable diffusion suggests there might could be an explosion of use and fine tuning.
-
Very very very surprisingly (I am going to reproduce it if I can because if true this is amazingly cool) consumer grade GPUs can run these LLMs in some fashion. Previous open sourced LLMs that fit in under 16Gb of vram are super disappointing because to get the model size small enough to fit on the card you have to limit the number of input tokens, which means the model "sees" very few words of input with which to produce output, pretty useless.
Now I don't think this year we'll have competitive LLMs running on GPUs at home, but, even if openAI continues to be super lame and political about their progress, eventually the moat will fall.
Also all the money to be made (aside from bing eating google) or maybe I should say most of the value is going to be captured by skilled consumers/users of LLMs not by glorified compute providers.
Smeagollu t1_jcczori wrote
It's likely that the first broadly usable general AI will be ahead of all others by long enough that it can grab up most of the market. It would make Bing the new go-to search engine or Google could manifest it's place.
CactusSmackedus t1_jcdk9kb wrote
MONEY
AND GREEEEEEEEED
🙄🙄🙄 Garbage populist memerey
Grateful_Dude- t1_jcau4jc wrote
Meta seems doing strong on the open source front
[deleted] t1_jca4smw wrote
[removed]
neuronexmachina t1_jcax06f wrote
2017 transformers paper for reference: "Attention is all you need" (cited 68K times)
>The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data
7thKingdom t1_jcbx5yj wrote
Frankly it should be illegal to name your software based company anything "open" if you refuse to be open source. It is intentionally misleading and harmful and creates a false sense of security and trust.
Rohit901 t1_jcbxbxv wrote
Elon who invested 100m USD in it initially just tweeted the same haha that he’s confused about these turn of events at openAI
DidQ t1_jcf2pzl wrote
OpenAI is as open as North Korea is democratic. North Korea full and official name is "Democratic People's Republic of Korea", and there is exactly 0 democracy there. The same with OpenAI, there's zero openness there.
Transmatrix t1_jcae2rb wrote
Doesn’t Google kind of do this with refusing to disclose how most of their algorithms work?
Rohit901 t1_jcarred wrote
Google has mostly been transparent most of the time and has published lot of groundbreaking AI research to the public thus advancing the field. OpenAI on the other hand seems to be closed source and trying to compete directly with Google. Maybe in future, Google might not be willing to make its research public if things go like this and we don’t want power to be concentrated in a single company or person. Thus I hope we are able to get better open source models
Transmatrix t1_jcayko9 wrote
Well, if anything, a company with the word “Open” in their name should probably be a bit more open source…
mailslot t1_jcb7td4 wrote
Google publishes a lot of papers. PageRank, one of the the original key algorithms for Google search, was released to the public. Unfortunately, this empowered spammers to game the search engine and create link farms & other “gems.” That’s a big reason why they don’t share in that space any longer. Too many people looking to exploit search results by any means necessary.
On other topics, Google has been the sole inspiration for some substantial projects. AI aside, their paper on BigTable (their own product) caused a ton of interest and sparked independent projects like Hadoop, HBase, & Cassandra). The modern way we work with big data is thanks to Google.
In any case, Google has no obligation to give away algorithms or the designs for entire products, yet they often do.
Viewing a single comment thread. View all comments