Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Which metrics to use for model evaluation ? #17

Open
George3d6 opened this issue Feb 16, 2020 · 0 comments
Open

Which metrics to use for model evaluation ? #17

George3d6 opened this issue Feb 16, 2020 · 0 comments

Comments

@George3d6
Copy link

Hello, thank you for maintaining this library, I find it takes away one of the most daunting tasks in regards to tertiary structure predictions for those that aren't that familiar with the field.

However, there's one thing that I can't figure out and reading through CASP related material (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4394854/) doesn't really help, how do you actually validate the models ?

As in, once you have a model that seems to predicted scaled values for the 3x3 matrices well under a given loss function that's practical for optimizing the model, how do you determine if the predicted structures would be correct for domain specific purposes ?

Assuming said evaluation is extremely hard is there some simpler metric that gets you 9x% of the way there so that you can at least tell if your model is heading in the right direction ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant