Viewing a single comment thread. View all comments

gwern t1_j8pg3g7 wrote

Reply to comment by maizeq in [R] RWKV-4 14B release (and ChatRWKV) - a surprisingly strong RNN Language Model by bo_peng

https://arxiv.org/abs/1805.04623 https://arxiv.org/abs/1702.04521

1

farmingvillein t1_j8pni5v wrote

Neither of these offer a comparative look against transformers, although they are certainly a useful look against the limitations of your basic RNN/LSTM.

1