YamEnvironmental4720 t1_ituam06 wrote on October 26, 2022 at 11:11 AM

Reply to comment by ash-050 in [D] Simple Questions Thread by AutoModerator

It depends on how you define importance. Entropy could be one such definition but even in forest classifiers there are alternatives to entropy.

ash-050 t1_iu3awlr wrote on October 28, 2022 at 6:22 AM

Thank you so much. My case the alternatives are on regression

YamEnvironmental4720 t1_iu3frfr wrote on October 28, 2022 at 7:29 AM

Ok, in that case there is the cost function, defined on the model's parameters, that measures the average distance from the sample points to your hypothesis. This is the average error the model has for the fixed parameters. In the case of linear regression, the importance of a certain variable is given by the weight parameter attached to that variable.

If you are familiar with multidimensional calculus, the dependence of a fixed such parameter is given by the partial derivative of the cost function in this direction.

This is quite well explained in Andrew Ng's video lecture on linear regression: https://www.youtube.com/watch?v=pkJjoro-b5c&list=PLLssT5z_DsK-h9vYZkQkYNWcItqhlRJLN&index=19.