bo_peng OP t1_jcjuvg9 wrote
Reply to comment by londons_explorer in [R] RWKV 14B ctx8192 is a zero-shot instruction-follower without finetuning, 23 token/s on 3090 after latest optimization (16G VRAM is enough, and you can stream layers to save more VRAM) by bo_peng
Yeah that will be cool. You are welcome to try it and I can help.
The rwkv pip package: https://pypi.org/project/rwkv/
Viewing a single comment thread. View all comments