Viewing a single comment thread. View all comments

blueSGL t1_jcjga2i wrote on March 17, 2023 at 7:40 AM

Is it possible to split the model and do inference across multiple lower VRAM GPUs or does a single card have to have the minimum 16gig VRAM?

bo_peng OP t1_jcjuhix wrote on March 17, 2023 at 10:58 AM

Yes ChatRWKV v2 supports that :)

Take a look at the "strategy" guide: https://pypi.org/project/rwkv/