Reproduction of the results in JMLR19 paper

[minxueric]

I find the results in the examples notebook are quite different from the results shown in your JMLR2019 paper.

For example, the Cox-time method on the metabric dataset:
in paper: Concordance=0.662, Integrated Brier score=0.172(best), Integrated binomial log-likelihood=-0.511
in notebook: Concordance=0.675, Integrated Brier score=0.159, Integrated binomial log-likelihood=-0.470

Could you please explain the difference and give the experiment code for reproduction of the results in the paper?

[havakv]

The examples here are just examples with rather arbitrary hyperparameters. In the paper we have a common hyperparameter tuning strategy all methods we compare, to make the comparison fair (it's very easy to overtfit to such small datasets by just trying some configurations by hand). This is listed in section 6.1.1 of the paper. If you want to reproduce the results, you would need to follow the experiments as explained there.