Learning to predict defect structures

[tomasfbouvier]

For my project we need to predict properly defects e.g self interstitials, vacancies, dumbells etc. Those structures are defined as defects within a non defected structure. For example in Silicon a self interstitial represents a single atom within a pristine diamond lattice. In the training sets it is usual to see structures of more than 200 atoms among which only one is relevant. This makes the formation energy of the defect difficult to learn and currently NEP performs worse than GAP in prediciting them (for Si). Would it be possible to implement a special computation routine of the energy loss function for those cases taking into account not only the defected structure but also the reference lattice?
L = ( E(N+n)- Et(N+n) - (1+n/N)(E(N) - Et(N))/n
where N is the number of atoms in the reference cell, n the number of atoms in the defect, L the corresponding loss to the structure and E and Et the predicted and ground training energies respectively

[brucefan1983]

In principle that can be implemented but I do not think it could be a general or very good solution. Have you tried to give some higher-than-usual weights to the relevant structures in train.xyz? In this file, one can specify the relative weight for a structure by adding weight=XXX to the second line, where XXX can be a positive number. For example, using XXX=5 for the relevant structures might help.

[brucefan1983]

This is similar to the "regularization" parameter $\sigma$ in GAP.

[tomasfbouvier]

Hi, thank you for answering. We tried to increase the weight given to point defect structures by a lot (aka 10) but yet NEP underestimates the formation energy by one eV. Within a 64 atoms box (which is the typical size of my box) 1 eV error for 1 atom corresponds to less than 20 meV in the contribution to energy RMSE. Moreover by increasing this much the weight on a single structure I believe that the weight on other structures is penalised and the ability of the model to extrapolate. I think this issue is not only relevant for point defect structures but also for structures for which a subset of the atoms is important than the others. For example cracks, voids, clusters amorphous pockets, loops etc.

Another solution that came to my mind is to have the possibility of add weights per atom in the structure so learning on those that matter is prioritised. For the current implementation of NEP I suppose this would only be relevant for forces but that's already a step.

[brucefan1983]

Yes, atom-wise weight is only relevant for force and it will not be hard to implement, as there is already a type-wise weight for force. So I can consider this as a possible extension. Could you also share the parameters in nep.in so that I can have better understanding of your training? Perhaps there is space for tunning?

[tomasfbouvier]

Sure, this is the nep.in that we are using right now.

We tried many other models, some with more complexity. I'm currently attempting to increase the number of neurons in the hidden layer but i'm concerned of overfitting.

[brucefan1983]

Thanks. If you are worrying about overfitting, lambda_1 and lambda_2 can be increased to say 0.1. If you are dealing with a complex or "general-purpose" potential, it might need even larger basis_size such as 15 15. If all these do not help, I will try to implement the atom-wise weight approach.

[brucefan1983]

Another note is about the number of generations. Did you only trian for 165000 generations? It usually requires more generations to converge for complex training data set.

[brucefan1983]

If you need high accuracy for energy, a feasible method might be to do a further training (restarting from one of your previous), setting lambda_e to a high value such as 10 for say, 50000 steps. From our experience, this can significanlty improve the energy accuracy while only has small effect on force/virial.

[tomasfbouvier]

Hi , thank you. We'll try that. In the meantime though we didn't fix the issue we are able to predict properly the number of defects in radiation cascades which is what we wanted initially.