Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consistent Non-deterministic Behavior in Thermo (polycylic) and Kinetics (Peroxyl_Disproportionation) #2490

Open
rwest opened this issue Jun 22, 2023 · 5 comments
Assignees
Labels
bug bug which will never be closed by the actions bot

Comments

@rwest
Copy link
Member

rwest commented Jun 22, 2023

Bug Description

The regression tests are showing up differences between models, when no relevant changes to RMG have been made. The differences seem to be random

Non-identical kinetics!
original:
rxn: CCCC(C)O[O](20) + CCCCCO[O](103) <=> oxygen(1) + CCCC(C)[O](64) + CCCCC[O](128)		origin: Peroxyl_Disproportionation
tested:
rxn: CCCC(C)O[O](20) + CCCCCO[O](103) <=> oxygen(1) + CCCC(C)[O](64) + CCCCC[O](128)		origin: Peroxyl_Disproportionation
k(1bar)|300K   |400K   |500K   |600K   |800K   |1000K  |1500K  |2000K  

k(T):  |   3.77|   4.45|   4.86|   5.14|   5.48|   5.68|   5.96|   6.09
k(T):  |   7.83|   7.49|   7.23|   7.02|   6.68|   6.42|   5.95|   5.61

Kinetics: Arrhenius(A=(3.2e+12,'cm^3/(mol*s)'), n=0, Ea=(3.756,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R""")
Kinetics: Arrhenius(A=(3.18266e+20,'cm^3/(mol*s)'), n=-2.694, Ea=(0,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing""")
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing

and in a different model

Non-identical kinetics!
original:
rxn: CCCC(C)O[O](20) + CCCCCO[O](103) <=> oxygen(1) + CCCC(C)[O](64) + CCCCC[O](128)		origin: Peroxyl_Disproportionation
tested:
rxn: CCCC(C)O[O](20) + CCCCCO[O](103) <=> oxygen(1) + CCCC(C)[O](64) + CCCCC[O](128)		origin: Peroxyl_Disproportionation
k(1bar)|300K   |400K   |500K   |600K   |800K   |1000K  |1500K  |2000K  

k(T):  |   3.77|   4.45|   4.86|   5.14|   5.48|   5.68|   5.96|   6.09
k(T):  |   7.83|   7.49|   7.23|   7.02|   6.68|   6.42|   5.95|   5.61

Kinetics: Arrhenius(A=(3.2e+12,'cm^3/(mol*s)'), n=0, Ea=(3.756,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R""")
Kinetics: Arrhenius(A=(3.18266e+20,'cm^3/(mol*s)'), n=-2.694, Ea=(0,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing""")
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing

See my comment #2452 (comment)

Comparing the model edge for the liquid_oxidation

Screenshot 2023-06-09 at 2 13 34 PM ``` Non-identical kinetics! original: rxn: CCCCCO[O](104) + CC(CC(C)OO)O[O](103) <=> oxygen(1) + CCCCC[O](128) + CC([O])CC(C)OO(127) origin: Peroxyl_Disproportionation tested: rxn: CCCCCO[O](104) + CC(CC(C)OO)O[O](103) <=> oxygen(1) + CCCCC[O](127) + CC([O])CC(C)OO(129) origin: Peroxyl_Disproportionation k(1bar)|300K |400K |500K |600K |800K |1000K |1500K |2000K

k(T): | 3.52| 4.27| 4.71| 5.01| 5.39| 5.61| 5.91| 6.06
k(T): | 7.79| 7.46| 7.21| 7.00| 6.67| 6.41| 5.94| 5.60

Kinetics: Arrhenius(A=(3.2e+12,'cm^3/(mols)'), n=0, Ea=(4.096,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R""")
Kinetics: Arrhenius(A=(3.18266e+20,'cm^3/(mol
s)'), n=-2.694, Ea=(0.053,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing Ea raised from 0.0 to 0.2 kJ/mol to match endothermicity of reaction.""")
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing_Ext-5R-R
kinetics: Estimated from node Root_Ext-5R-R_7R!H->C_N-7C-inRing Ea raised from 0.0 to 0.2 kJ/mol to match endothermicity of reaction.


Perhaps an instance of something like #2010 ?

This may be related #1027 but is 6 years old and quite likely outdated.

We need to decide what counts as a regression worth failing the tests for!

and Matt's reply
#2452 (comment)

Those two nodes are parent and child and look like this:
image
image
I'm wondering a little bit if this is something related to how the atoms are labeled in that family...in R_Recombination we handle the fact that the radicals aren't unique by using the same label for both radicals...but here the -O[O]'s are labeled differently so I could imagine the prediction is dependent on where the isomorphism (that will match both ways) puts the labels. (I.E. if it puts *1 and *2 on the branched species and *3 and *4 on the other it will match the less specific group while if it puts *1 and *2 on the non-branched species it will match the more specific group.

How To Reproduce

Run (or look at) the regression tests for a pull request that doesn't change RMG code (eg. it makes some other change). Two models built with identical code will have different results.

Expected Behavior

RMG results wouldn't be random.

Installation Information

Describe your installation method and system information.

  • OS: latest ubuntu github actions runner
  • Installation method: from source in conda environment
  • RMG version information:
    • RMG-Py: current main
    • RMG-database: current main
@rwest
Copy link
Member Author

rwest commented Jun 22, 2023

Probably a different bug but it's similar symptom:
The "failed regression comparison" artifact on this regression test run

Non-identical kinetics!
original:
rxn: NCO(66) <=> O1[C]=N1(126)		origin: Intra_R_Add_Endocyclic
tested:
rxn: NCO(66) <=> O1[C]=N1(126)		origin: Intra_R_Add_Endocyclic
k(1bar)|300K   |400K   |500K   |600K   |800K   |1000K  |1500K  |2000K  

k(T):  | -66.25| -46.19| -34.19| -26.21| -16.28| -10.36|  -2.54|   1.31
k(T):  | -49.54| -33.65| -24.16| -17.85| -10.01|  -5.35|   0.80|   3.82

Kinetics: Arrhenius(A=(6.95187e+18,'s^-1'), n=-1.628, Ea=(111.271,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Backbone0_N-2R!H-inRing_N-1R!H-inRing_Sp-2R!H-1R!H""")
Kinetics: Arrhenius(A=(6.95187e+18,'s^-1'), n=-1.628, Ea=(88.327,'kcal/mol'), T0=(1,'K'), comment="""Estimated from node Backbone0_N-2R!H-inRing_N-1R!H-inRing_Sp-2R!H-1R!H""")
Identical kinetics comments

at first I was surprised by the 10 orders of magnitude change in kinetics given that they used identical nodes, but then noticed it's because the thermo is different

Non-identical thermo!
original:	O1[C]=N1
tested:	O1[C]=N1
Hf(300K)  |S(300K)   |Cp(300K)  |Cp(400K)  |Cp(500K)  |Cp(600K)  |Cp(800K)  |Cp(1000K) |Cp(1500K) 
    141.64|     58.66|     12.26|     12.27|     12.09|     11.96|     12.26|     12.72|     12.15
    116.46|     53.90|     11.62|     12.71|     13.49|     13.96|     14.14|     13.85|     13.58
thermo: Thermo group additivity estimation: group(O2s-CdN3d) + group(N3d-OCd) + group(Cd-HN3dO) + ring(oxirene) + radical(CdJ-NdO)
thermo: Thermo group additivity estimation: group(O2s-CdN3d) + group(N3d-OCd) + group(Cd-HN3dO) + ring(Cyclopropene) + radical(CdJ-NdO)

it seems one matches oxirene and the other cyclopropene.
...which is exactly this issue: #2010

@github-actions
Copy link

This issue is being automatically marked as stale because it has not received any interaction in the last 90 days. Please leave a comment if this is still a relevant issue, otherwise it will automatically be closed in 30 days.

@github-actions github-actions bot added the stale stale issue/PR as determined by actions bot label Sep 21, 2023
@JacksonBurns JacksonBurns added bug bug which will never be closed by the actions bot and removed stale stale issue/PR as determined by actions bot labels Sep 28, 2023
@JacksonBurns JacksonBurns changed the title Non-deterministic (random) kinetics in Peroxyl_Disproportionation Consistent Non-deterministic Behavior in Thermo (polycylic) and Kinetics (Peroxyl_Disproportionation) Oct 24, 2023
@JacksonBurns
Copy link
Contributor

I am broadening the scope of this issue since similar non-deterministic behavior has been detected in thermo estimations as well.

The 'good' news is that the examples are reproducible (consistent).

@xiaoruiDong has offered to investigate further (thanks!).

@mjohnson541
Copy link
Contributor

I think the kinetics issue would resolve if we restructured Peroxyl_Disproportionation to operate like R_Recombination, where the labels on each reactant are the same, doing that would be a matter of doing a find-replace on labels in the training dictionary file, adapting the top group, regenerating the tree and then tweaking some logic in the database (should be able to search for mentions of "R_Recombination" to find the locations).

@JacksonBurns
Copy link
Contributor

@mjohnson541 let's catch up offline at some point about how to do this - we should document it for future devs as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bug which will never be closed by the actions bot
Projects
None yet
Development

No branches or pull requests

4 participants