Viewing a single comment thread. View all comments

bo_peng OP t1_jb1qws0 wrote

Reply to comment by Spare_Side_5907 in [R] RWKV (100% RNN) can genuinely model ctx4k+ documents in Pile, and RWKV model+inference+generation in 150 lines of Python by bo_peng

TNN is like convolution, while RWKV can be written as a CNN too (RWKV v1 is a CNN). So there's some similarity, though not much :)

2

estrafire t1_jc2umln wrote

Any particular reason for moving from CNN to RNN?

1