bo_peng OP t1_jcmajpx wrote
Reply to comment by mikljohansson in [R] RWKV 14B ctx8192 is a zero-shot instruction-follower without finetuning, 23 token/s on 3090 after latest optimization (16G VRAM is enough, and you can stream layers to save more VRAM) by bo_peng
- RWKV-LM is now mainly for training, while ChatRWKV is for optimal inference.
- Someone in RWKV Discord tried it using LoRA (https://github.com/Blealtan/RWKV-LM-LoRA) and the result is quite nice. Join RWKV Discord for latest updates :)
Viewing a single comment thread. View all comments