Comments

You must log in or register to comment.

gullydowny t1_j9vc9y1 wrote

Here’s what it’s trained on - from their paper which I cant link to because of arbitrary Reddit mod rules but this looks like an extremely cool project

16

KarmaStrikesThrice t1_j9vgzt4 wrote

AI is not computationally demanding to run, learning is the part that needs a supercomputer level resources for months and months, but once the neural network is complete, using it is quite simple. How else would chatGPT be able to service 100+ million users at once if each user required a whole gpu resource-wise?

2

malevolent_keyboard t1_j9xg8ia wrote

Replying to Divided_World (#1,986,180)

Probably missing some details, but back in the early-mid 2000’s, most tech companies formed a not-so-secret-anymore pact to keep pay low for developers. Facebook was the only company who said “not interested” and paid SWE’s what all the companies knew they were worth. Then the other companies lost workers to Facebook for much higher pay and benefits, forcing those companies to follow suit. This was mostly Zuck’s doing.

11

Cloudly-so t1_j9xsahx wrote

Will be very interesting to see if the development will be to run the models locally (on mobile, PC etc) or the need for the cloud.

Will vary by use-case. Image generation is for example fitted in to much smaller models them languages. The rout it will take will effect the tech ecosystem in many ways with someone like Apple benefitting much more on local models, and AWS, Azure etc benefiting from larger models.

1

Vegan_Honk t1_j9xuuyo wrote

Going a little fast there guys. Almost like you're trying not to drown in this current market.

0

KarmaStrikesThrice t1_j9y13vs wrote

Replying to nicuramar (#1,990,336)

But is it the size that is limiting or the performance? ChatGPT is definitely too huge for 1gpu (even the A100 server gpus with 80GB of memory), but once you connect enough gpus to have the space available, i bet you the performance is quite fast. It is similar tu human brain, it takes us days, weeks, years to learn something, but we can then access it in a split of a second. The fastest supercomputers today have tens of thousands of gpus, so if chatgpt can have millions of users running it at the same time, one gpu can have hundreds and thousands of users using it.

1

freediverx01 t1_j9yb1ro wrote

Meta unveils _____ , which no one with any sense should consider using, given the company’s leadership, culture, and track record.

6

RuairiSpain t1_j9yf7g4 wrote

Replying to CerebralBypass01 (#1,975,554)

The model is huge though and needs to be in GPU memory for performance calculations (sparse matrix dot product).

Probably one thing teams are working in is reducing the dimensions of the sparse matrix so it can fit on fewer GPUs. Also looking at reduced precision of floating point multiplication, 8 bit floats is probably enough for AI matrix maths. Maybe combining matrix multiplication AND the activation functions (typically ReLU or Sigmoid) so two maths operations can be done in one pass through GPU. That involves refactoring their math library.

Or the build custom TPUs with all this build into the hardware.

The future is bright 🌞 for AI. Until we hit the next brick wall

2

Total_loss_2b_boss t1_j9yhs2w wrote

Replying to HunterofSnowmen (#1,987,782)

I know that Facebook's BERT model is hugely useful in various AI tasks but I didn't know that they were behind pytorch.

Damn. Pytorch kind of matters a lot in AI. Like, a lot. All of the open source AI stuff I've been tinkering with uses pytorch.

4

KarmaStrikesThrice t1_j9zvqll wrote

Replying to ActuatorMaterial2846 (#1,993,474)

No I meant it more generally. Neural networks dont contain any super complicated math and equations that are difficult to solve, it is a network of simple cells whose inputs are outputs of previous layer of cells and the output is fed to the next layer. Popular example of a cell is Perceptron, which computes a simple linear equation y=Ax+b. The main problem is the size of a network, which can be billions or even trillions of cells in case of chatgpt. But not all cells are always used, based on the input only some cells are active (the same way our brain does not activate cells that learned math when we are asked what is the capital of New York state for example).

So the most computationally difficult part is learning, and then having enough memory to store the whole network into fast memory, the AI doesnt know what you are about to ask it, so the whole network needs to be ready. But once we ask a specific question, like "are cats carnivores?", 99.99...% of cells remain inactive and only those storing information about biology, mammals, cats, food, meat, diets, carnivores, etc. are engaged and produce answer. So extracting the output based on given inputs is much simpler and can be done by personal computers (if our computers had many terabytes/petabytes of RAM and storage, which they dont)

The advanced compression alhorithms reduce the memory required to store the network, but it doesnt really improve performance aside from some minor cache optimizations.

2