Submitted by Educational_Grab_473 t3_10dsyd0 in singularity
red75prime t1_j4pogqn wrote
Reply to comment by SoylentRox in How long until an AI is able to write a book? by Educational_Grab_473
> <salient context> + <current symbol buffer> + neural network
That's RNN (recurrent neural network). As far as I know LSTM is still state of the art for them. And it struggles with long-term dependencies.
[He checks papers]
It looks like combination of transformer and LSTM does provide some benefits, but nothing groundbreaking yet.
Viewing a single comment thread. View all comments