Viewing a single comment thread. View all comments

ash-050 t1_ittne1q wrote

Reply to comment by YamEnvironmental4720 in [D] Simple Questions Thread by AutoModerator

Thank you so much u/YamEnvironmental4720 for your reply. Would I be having the same results if I used the trained model's feature importance ?

1

YamEnvironmental4720 t1_ituam06 wrote

It depends on how you define importance. Entropy could be one such definition but even in forest classifiers there are alternatives to entropy.

1

ash-050 t1_iu3awlr wrote

Thank you so much. My case the alternatives are on regression

1

YamEnvironmental4720 t1_iu3frfr wrote

Ok, in that case there is the cost function, defined on the model's parameters, that measures the average distance from the sample points to your hypothesis. This is the average error the model has for the fixed parameters. In the case of linear regression, the importance of a certain variable is given by the weight parameter attached to that variable.

If you are familiar with multidimensional calculus, the dependence of a fixed such parameter is given by the partial derivative of the cost function in this direction.

This is quite well explained in Andrew Ng's video lecture on linear regression: https://www.youtube.com/watch?v=pkJjoro-b5c&list=PLLssT5z_DsK-h9vYZkQkYNWcItqhlRJLN&index=19.

1