DLTK's implementation of DeepMedic underperforms original.

[Kamnitsask]

Hello,

Let us note here to the users that DLTK's implementation of DeepMedic is significantly underperforming the original implementation (publicly available at link).

If users seek to evaluate performance of DeepMedic, eg to find the capabilities of the model, compare with other methods, etc, they must use the original version.

Please leave this issue open until the implementation of this version is brought on par with the original.

Thank you,
Konstantinos Kamnitsas

[mrajchl]

Considering this implementation comes from our group, people might expect the same performance as the original code base. I've pushed a WARNING/Note comment at the top of dltk/models/segmentation/deepmedic with 61afd38 stating:

# WARNING/NOTE
# This implementation is work in progress and an attempt to implement a 
# scalable version of the original DeepMedic [1] source. It will NOT 
# yield the same accuracy performance as described in the paper.
# If you are running comparative experiments, please refer to the 
# original code base in [1].
#
# [1] https://github.com/Kamnitsask/deepmedic

If you want this highlighted better, let me know. Let's keep this thread open until this is resolved.

[Kamnitsask]

Looks good Martin, thanks. Yes, with the other model implementations looking good on the DLTK paper, users could take for granted that this is polished as well. So lets leave it visible for now that this model has not been mirrored/transferred from the original.

I would leave the issue open for now, to avoid confusion of users. At least that's the approach I take with issues on my project (as "strong warnings" to users), not sure it's standard engineering practice though.

I'll try to make time to look into this myself in the future.

Cheers,
K