Viewing a single comment thread. View all comments

smegdawg t1_iw02s28 wrote

Reply to comment by empty_string_ in Scientists Taught an AI to ‘Sleep’ So That It Doesn't Forget What It Learned, Like a Person. Researchers say counting sleep may be the best way for AIs to exhibit life-long learning. by mossadnik

Isn't this essentially the difference between RAM and ROM?

Or are they adding a third level RUM, Really-Unfungible-Memory?

72

chazzmoney t1_iw12gmj wrote

ML “memory” is not the same as hardware “memory”.

There are multiple methods that can be utilized, but catastrophic forgetting refers to the modification of weights within a neural network such that it loses capabilities it previously had.

Lets say you train a system to identify cats and dogs. It gets to a specific accuracy, say 80% correct, 20% incorrect choosing between the two.

Then, you stop training it to do that task, and instead start training it to identify pizza and hot dogs. It trains faster, and gets to an accuracy of 95% on pizzas and hot dogs.

Now, you go back and see how accurate it is on identifying cats and dogs - but it turns out that it is only 52% accurate- almost the same as a coin toss.

It has essentially “forgotten” how to perform the original task. This is catastrophic forgetting (no pun intended).

Edit: Cat-astrophic

53

me_team t1_iw1kxcx wrote

Deep down I know that pun was absolutely intended. Sly!

3

StevenTM t1_iw3mng5 wrote

Yes but why does it happen?

3

chazzmoney t1_iw4edfl wrote

Great question. First, some concepts that the explanation depends on…

A neural network is made up of “weights”. These weights are floating point values which are multiplied by whatever they receive from the prior neuron.

While each neuron separately doesn’t necessarily do something easily understood / clear, each neuron (and weight) is part of an overall algorithm that solves the trained problem.

When you take any network and train it to perform a specific task, the weights of the network are updated towards solving that specific task.

So, with that background, lets go back to your question- why does catastrophic forgetting happen?

If you were to compare the weights from a network trained to detect pizza/hot dog and the weights from a network trained to detect cat/dog: the two networks would have some weights that were similar and some that were different.

This is because some parts of the algorithm to detect food and the algorithm to detect animals are the same, and some parts of the algorithms are different.

Thus, when you start with one network and train for a different task, the “correct” (useful) parts remain and the “incorrect” (not useful) parts are replaced with useful parts. The parts that are not useful, the weights get changed sufficiently to prevent that part of the algorithm from working.

This is why catastrophic forgetting happens.

The cause is basically:

  1. Having a network of specific size / bias to be able to successfully train for one task.

  2. The algorithm for two different tasks being sufficiently different for some portion of the network.

  3. Training on each task separately.

Thus there are some “solutions” which have been and continued to be explored:

  1. Having a much larger network, with a bias towards generality rather than specificity.

  2. Adding / removing / training specific subsections of the network for each task - and leaving the earlier part as “general”

  3. Training for multiple purposes at the same time

4

StevenTM t1_iw4gxco wrote

Why isn't it possible to just create a copy of the weights for task 1/task 2 at various points, or even continuously?

Storage space is ridiculously cheap, and even high powered debugging traces (like Microsoft's iDNA (TTT Debugging), which basically captures full process dumps in millisecond increments (albeit for a single running process) aren't THAT huge.

Then when you re-run task 1, it just uses the weights from the latest snapshot for task 1. I don't see why it wouldn't or what the benefit of using the (obviously mismatched) weights from task 2 would be (while running task 1).

I mean.. i know it sounds like a stupidly obvious suggestion, and I'm fairly certain it isn't used as a solution, but can't figure out why

3

chazzmoney t1_iw4l6c8 wrote

If you only need to detect hotdog/pizza and dog/cat, its a fine solution. I was using those as examples, but usually its much more drastic - “transcribe speech to text” and “identify speech from background noise” and “identify speaker”. Or “answer trivia question”, “fill in the blank with the correct choice”, “determine if the text has a positive or negative sentiment”, “determine the main topic of the text”,. Etc…. quite complicated tasks.

Thus, there are a few reason it doesn’t work in practice:

  • Generality

  • Efficiency (hardware memory)

  • Efficiency (training computation)

  • Efficiency (inference latency)

Having a general network is more interesting - a single network that can solve multiple problems is more useful and more applicable to problems that may be similar or you don’t know you have yet. It can also be easier to “fine-tune” an existing network because you don’t have enough data on a given problem to train it from scratch.

Efficiency is (in my opinion), the bigger one:

  1. To run a network, the entire network and parameters for it must be stored in memory. These days, they are on the order of gigabytes for “interesting” networks. Putting a multiplier (multiple networks) on this makes scaling quite challenging.

  2. Training one general network may be harder, but it is much faster than training a new network from scratch for each problem. If you have thousands of problems, you don’t want to be training thousands of networks.

  3. The size of “interesting” models makes inference challenging as well. The bigger (more interesting)the model, the more computation it must perform on each input; some modeling techniques are loops and require thousands of runs for each input. This seems fine at first but if a single inputs takes more than 10ms, a thousand loops will take longer than one second. Usually this means that the most interesting models have to be used on high end cloud equipment, which brings about further scaling challenges.

So your answer isn’t wrong; the practicality of the situation just makes it infeasible when you are working with large models (vision, video, language, etc) - the number of parameters is often in the billions.

4

StevenTM t1_iw4nmz3 wrote

Thank you for the in-depth answer, it was very interesting!

3

chazzmoney t1_iw4s9ic wrote

Of course - thanks for the great questions!

3