How to present features selected by learners of ”surv.coxph” or ”surv.coxboost”

[difficultli]

Dear all, I am a super new user of mlr3.
I read some scientific papers that used mlr package to analyze survival (time to death) data.
Paper link: https://pubmed.ncbi.nlm.nih.gov/33230128/
In this paper, the authors showed average number of features selected by each survival model (e.g., "surv.coxboost", "surv.coxph"...) extracted by internal feature selection.

When I run code below, it only gives me performance evaluation in each iteration but not features selected.
And, does makeFilter() method (only see examples from mlr) still work in mlr3 for selecting features for survival data?

Thank you very much!

library(mlr3)
library(simsurv)
library(mlr3proba)
library(mlr3extralearners)

t= tgen("simsurv")$generate(50)
l=lrn("surv.coxboost")
l$train(t)
l$model 　
# Console shows 
# "100 boosting steps resulting in 3 non-zero coefficients  partial log-likelihood: -114.4135 " 

l=lrn("surv.coxph")
l$train(t)
l$model 　
# Console shows
# Call: survival::coxph(formula = task$formula(), data = task$data(), 
#  x = TRUE)
#                   coef          exp(coef)   se(coef)      z         p
# height        0.001418  1.001419  0.010006  0.142   0.8873
# treatment -0.020864  0.979352  0.336671 -0.062  0.9506
# weight      -0.034417  0.966169  0.018095 -1.902  0.0572
# Likelihood ratio test=4.27  on 3 df, p=0.2337
# n= 50, number of events= 38 

Is there any way to see what features selected and their coefficients values?
Thank you!

[pat-s]

Thanks for posting your question here. Could you please wrap your code in backticks so its better readable?

cc @RaphaelS1 as the topic is survival :)

[RaphaelS1]

Hi @difficultli , without knowing what paper you read it's hard to help too much. But a few points:

1. You are confusing validation and feature importance. Using $score and $aggregate just validates your model across folds, it does not tell you anything about features
2. We have mlr3fselect and mlr3filters to replace makeFilter
3. For learners that do support feature importance, such as mboost, there is a method called importance():

library(mlr3)
library(mlr3proba)
library(mlr3extralearners)
t = tgen("simsurv")$generate(50)
l = lrn("surv.mboost", baselearner = "bols")
l$train(t)
l$importance()
#>       height    treatment       weight 
#> 0.0123077879 0.0010131154 0.0002999083

^{Created on 2021-10-12 by the reprex package (v2.0.1)}

[difficultli]

Hello @RaphaelS1 , Thank you for your time and reply.

I modified my question above, for the previous task (tsk("rats")) is not appropriate example for surv.coxboost.
It gave error message like this.
(Error: TaskSurv:rats has the following unsupported feature types: factor)

I also added the paper I referred to and made my question more specific.

[RaphaelS1]

Hi again I'm still not entirely sure what you're trying to achieve. However one thing to point out is that Cox PH does not include feature selection, the authors in that paper use a regularization method, so you could instead use surv.glmnet:

library(mlr3)
library(simsurv)
library(mlr3proba)
#> Warning: package 'mlr3proba' was built under R version 4.1.1
library(mlr3learners)
#> Warning: package 'mlr3learners' was built under R version 4.1.1

t = tgen("simsurv")$generate(50)
l = lrn("surv.glmnet")
l$train(t)
l$model
#> 
#> Call:  (if (cv) glmnet::cv.glmnet else glmnet::glmnet)(x = data, y = target,      family = "cox") 
#> 
#>    Df %Dev   Lambda
#> 1   0 0.00 0.147900
#> 2   1 0.12 0.134800
#> 3   1 0.22 0.122800
#> 4   1 0.31 0.111900
#> 5   1 0.37 0.102000
#> 6   1 0.43 0.092900
#> 7   1 0.48 0.084650
#> 8   1 0.52 0.077130
#> 9   1 0.55 0.070280
#> 10  1 0.57 0.064030
#> 11  1 0.60 0.058350
#> 12  1 0.61 0.053160
#> 13  1 0.63 0.048440
#> 14  1 0.64 0.044140
#> 15  1 0.65 0.040220
#> 16  1 0.66 0.036640
#> 17  1 0.67 0.033390
#> 18  1 0.67 0.030420
#> 19  1 0.68 0.027720
#> 20  1 0.68 0.025260
#> 21  2 0.69 0.023010
#> 22  2 0.69 0.020970
#> 23  2 0.70 0.019110
#> 24  2 0.70 0.017410
#> 25  3 0.70 0.015860
#> 26  3 0.71 0.014450
#> 27  3 0.71 0.013170
#> 28  3 0.72 0.012000
#> 29  3 0.72 0.010930
#> 30  3 0.73 0.009962
#> 31  3 0.73 0.009077
#> 32  3 0.73 0.008270
#> 33  3 0.73 0.007536
#> 34  3 0.73 0.006866
#> 35  3 0.73 0.006256
#> 36  3 0.73 0.005700
#> 37  3 0.74 0.005194
#> 38  3 0.74 0.004733
#> 39  3 0.74 0.004312
#> 40  3 0.74 0.003929
#> 41  3 0.74 0.003580
#> 42  3 0.74 0.003262
#> 43  3 0.74 0.002972
#> 44  3 0.74 0.002708
#> 45  3 0.74 0.002468
#> 46  3 0.74 0.002248
#> 47  3 0.74 0.002049
l$selected_features()
#> Warning: Multiple lambdas have been fit. Lambda will be set to 0.01 (see
#> parameter 's').
#> [1] "height"    "treatment" "weight"

^{Created on 2021-10-18 by the reprex package (v2.0.1)}

[difficultli]

Hi @RaphaelS1, thank you again.
It is helpful to know that some survival learners do not have PROPERTIES of "importance" or "selected_feature".
However, these learners (e.g. surv.coxboost) do have internally decided some features in the trained model as log showed below.

"100 boosting steps resulting in 3 non-zero coefficients partial log-likelihood: -114.4135 "
I think it is important to know which non-zero coefficients and features were selected for better interpretation of the result.
Thank you.

[RaphaelS1]

I agree, so my suggestion is that for any learners you think should have an importance/selected_features method you should open a corresponding issue. So can you please open an issue in mlr3extralearners listing any learners that you think should have importance/selected but do not?

[difficultli]

Thank you for your suggestion. I would open an issue in mlr3extralearners.