Viewing a single comment thread. View all comments

keepthepace t1_j2k0pvh wrote

I think you may be interested in neural Turing machines and it's successor: diffferentiable neural computers. They basically force a network to accomplish a task through a Turing machine.

https://en.m.wikipedia.org/wiki/Differentiable_neural_computer

2

currentscurrents OP t1_j2k1t74 wrote

Interesting! I have heard about these, but it doesn't look like there's been much work on them in the last few years - it's mostly 2014-2018 papers.

1

keepthepace t1_j2m2xrp wrote

Yes, they never reached the level of very complex algorithms but also no one ever tried to throw a lot of compute at them before we created gigantic language models able to mostly do these tasks with orders of magnitude more parameters.

I do suspect that if we threw a bit more of compute at a DNC we would get very interesting results, but that's little more than a hunch.

1