Problem about extracting and ranking characteristic features from models #19314
Replies: 2 comments 3 replies
-
We did not merge the PR where the example is written but you can access the rendering: https://130375-843222-gh.circle-artifacts.com/0/doc/auto_examples/inspection/plot_model_inspection_cross_validation.html Regarding the inspection of the training score (this is not added in the example), you can pass I think that in your case, what is really important is to check if the feature selection is stable, meaning that you are picking most of the time the same feature for the 50 models cross-validated. It is a similar study to checking the variance of |
Beta Was this translation helpful? Give feedback.
-
If you use linear SVMs, you can directly look at the model coefficients but then you have to be aware of some common pitfalls. If you use a non-linear SVM (e.g. with RBF kernels), then you cannot look at the coef directly and instead you might want to have a look at permutation importances and SHAP. Be careful of possibly misleading results with permutation importance on model trained with correlated features. See the scikit-learn example linked in the above doc to learn more about this problem and some potential workaround. Another alternative is to iteratively include or exclude a feature one at a time (greedily). |
Beta Was this translation helpful? Give feedback.
-
Hi guys,
I've started learning Scikit-learn and Python, and I tried to use support vector machines for classification, but I got a problem about feature selection. More specifically, I use Receiver Operating Characteristic (ROC) with cross validation yielding 50 models to evaluate the classification performance. It is possible to calculate the mean area under the ROC curve, and see the variance of the curve when the training set is split into different subsets (discovery sets and validation sets) to build the models. However, I am really struggling with extracting and ranking characteristic features from these 50 models. I don’t know how to do that. Would you be able to give me any suggestion?
Many thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions