Viewing a single comment thread. View all comments

pkuba208 t1_jcx3d9i wrote

Reply to comment by Art10001 in [Research] Alpaca 7B language model running on my Pixel 7 by simpleuserhere

Well... I run this model on a raspberry pi 4B, but you will need AT LEAST 8gb ram

1

Art10001 t1_jcy2sb5 wrote

Raspberry Pi 4 is far slower than modern phones.

Also there was somebody else saying it probably actually uses 4/6 GB.

1

pkuba208 t1_jcy717u wrote

I know, but android uses 3-4gb ram itself. I run it myself, so I know that it uses from 6-7 gb of ram on the smallest model currently with 4bit quantization

1

Art10001 t1_jcy7rqs wrote

Yes, that's why it was tried in a Pixel 7 which has 8 GB of RAM and maybe even swap.

1

pkuba208 t1_jcy83gf wrote

I use swap too. For now, it can only run on flagships tho. You have to have at least 8gb of ram, because running it directly on let's say 3gb(3gb used by system) ram and 3-5gb SWAP may not even be possible and if it is, then it will be very slow and prone to crashing

1