Viewing a single comment thread. View all comments

fedetask OP t1_iud6830 wrote

Reply to comment by eigenham in [R] Deep model with inputs of unbalanced sizes by fedetask

  1. the number of elements in an actual vector input
3

eigenham t1_iud7sg6 wrote

Thanks and just to make sure I understand you: are these inputs of different sizes available all the time simultaneously (e.g. could theoretically be concatenated into a single vector)?

Or are only some of them available at a time (and you've found that the smaller vectors are more predictive of the more important class)?

1

fedetask OP t1_iud863u wrote

They are available at the same time. Imagine that the input is a 251-dimensional vector where the first 200 values are related to some feature A, the next 5 to feature B, and the last value to feature C. But features B and C are very important for the prediction

3

eigenham t1_iudbxis wrote

Ok so you really have one input vector but you're concerned that some important elements of it are going to get ignored or underutilized. Normally that's the whole point of the optimization process in the fitting problem: if those features result in the most gain during training, the information from them should be prioritized (up to getting stuck in local minima). Why do you think this wouldn't be the case for your problem? Is this small set of inputs only relevant for a minority class or something like that (unless addressed, this would make them underrepresented in your optimization problem)?

1

fedetask OP t1_iuduy9k wrote

My concern is that since the training process is noisy (RL) the optimization could take more time to "isolate" those features, and maybe some smarter model architecture could bias the algorithm in giving more importance to them from the beginning

2

eigenham t1_iue47f3 wrote

If you know for sure certain inputs should have a greater role in the final decision, you can help the model not lose that information layer over layer by giving it skip connections to later layers.

3