KarmaStrikesThrice
KarmaStrikesThrice t1_j9y13vs wrote
Reply to comment by nicuramar in ChatGPT on your PC? Meta unveils new AI model that can run on a single GPU by 10MinsForUsername
But is it the size that is limiting or the performance? ChatGPT is definitely too huge for 1gpu (even the A100 server gpus with 80GB of memory), but once you connect enough gpus to have the space available, i bet you the performance is quite fast. It is similar tu human brain, it takes us days, weeks, years to learn something, but we can then access it in a split of a second. The fastest supercomputers today have tens of thousands of gpus, so if chatgpt can have millions of users running it at the same time, one gpu can have hundreds and thousands of users using it.
KarmaStrikesThrice t1_j9vgzt4 wrote
Reply to ChatGPT on your PC? Meta unveils new AI model that can run on a single GPU by 10MinsForUsername
AI is not computationally demanding to run, learning is the part that needs a supercomputer level resources for months and months, but once the neural network is complete, using it is quite simple. How else would chatGPT be able to service 100+ million users at once if each user required a whole gpu resource-wise?
KarmaStrikesThrice t1_iujosqu wrote
I have been tought that physics allow 1.5nm to be the smallest possible size of a silicone transistor, because it uses the smallest possible amount of atoms to create a working transistor. The technology process size of chip companies is often something different than size of the transistor, TSMC usually says what is the resolution at which they can make changes to the atomic structure, so they can go well below 1, we might even see 0.9nm or 0.8nm technologies in a decade.
But thw truth is that silicone technology is coming to its limit. Up until now smaller manufacturing processes were responsible for the most improvement by far, lowering the nanometers to a half meant 2x the amount of transistors, 2x the performance and half the power consumption and manufacturing price. It is the main reason AMD was able to overtake Intel back when intel was stuck at 14nm and AMD was at 11nm and then 9-10nm. Intel had better more advanced chips, but the manufacturing process was just too good for AMD and worse 10nm chip was 5-10% better than 14nm Intel, not to mention AMD started making 16 core processors whereas Intel had 10 max, which was just enough in most games (except for Total War strategies where big maps with 16 AI players could use 16 cores at 100%).
KarmaStrikesThrice t1_itwiw64 wrote
Reply to comment by InspectorGadget76 in Why do my monitors look so completely different? by DotBitGaming
true, but if you buy from the same shop at the same time, it should be from the same batch. The seller can help you match the same batch monitors as he knows when each batch of monitors was ordered and arrived by serial number. And in the end you can test the monitors next to each other before buying. However I dont think there wont be much difference if any, as long as the producing company didnt change the internal hardware between batches.
KarmaStrikesThrice t1_itu46iw wrote
Unfortunately this is the reason you should always have the exact same monitors for dual monitor setup, without calibration the image may be completely different, especially in cheaper monitor where the color delta may be up to 10-15 (up to 1.5 is considered professional grade accuracy and up to 3 is considered very good for non-professional use, over 5 is considered off). Some monitors are warmer and have vanilla white instead of fresh chalk white, some monitors have different gammuts, contrasts, brightness, colors, input lags... so many things can be different that the overall experience is "annoying", because the bad attributes are amplyfied if have comparison right next to it.
So you can try to play with different setting but I am afraid that you wont be able to fix more than 50% of the overall difference, choose which monitor you like better, get another one and sell the worse one.
KarmaStrikesThrice t1_itrn1sg wrote
Reply to comment by 685327594 in Nvidia investigating reports of RTX 4090 power cables burning or melting by JumpinKing
the board of rtx4090 is actually small, the second back fan blows air from bottom up without any board in the way, the card is literally 40% longer just because of the heatsink + it is fat as hell, hard to imagine a bigger gpu. air cooling abilities are also maxed out, I dont think future cards can go over 500W in reference designs. rtx4090 is a true beast and I wish I could have it, but $1600 for a card just to play games is ridiculous now that gpu mining is dead. A year ago rtx4090 could mine over $5-6 per day, so you could theoretically mine its value back in 12-14 months depending on electricity prices,but today there is no way to recuperate the purchase price back. But if I ever have like $3000-5000 spare extra money, I might might finally build a water cooled rtx4090 maxine with 16-core ryzen, 64GB ram and 2x2tb gen4 nvme in raid0.
KarmaStrikesThrice t1_j9zvqll wrote
Reply to comment by ActuatorMaterial2846 in ChatGPT on your PC? Meta unveils new AI model that can run on a single GPU by 10MinsForUsername
No I meant it more generally. Neural networks dont contain any super complicated math and equations that are difficult to solve, it is a network of simple cells whose inputs are outputs of previous layer of cells and the output is fed to the next layer. Popular example of a cell is Perceptron, which computes a simple linear equation y=Ax+b. The main problem is the size of a network, which can be billions or even trillions of cells in case of chatgpt. But not all cells are always used, based on the input only some cells are active (the same way our brain does not activate cells that learned math when we are asked what is the capital of New York state for example).
So the most computationally difficult part is learning, and then having enough memory to store the whole network into fast memory, the AI doesnt know what you are about to ask it, so the whole network needs to be ready. But once we ask a specific question, like "are cats carnivores?", 99.99...% of cells remain inactive and only those storing information about biology, mammals, cats, food, meat, diets, carnivores, etc. are engaged and produce answer. So extracting the output based on given inputs is much simpler and can be done by personal computers (if our computers had many terabytes/petabytes of RAM and storage, which they dont)
The advanced compression alhorithms reduce the memory required to store the network, but it doesnt really improve performance aside from some minor cache optimizations.