Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Questions about unexpected situations where the stop method stops a thread #115

Open
MrLi2018 opened this issue Aug 11, 2021 · 7 comments
Open

Comments

@MrLi2018
Copy link

MrLi2018 commented Aug 11, 2021

When executing scripts, we need to invoke other third-party remote services. However, an exception occurs, and our services are unavailable.
The following code is involved:
NashornSandboxImpl.LOG.error(this.getClass().getSimpleName() + ": Thread hard shutdown!");
this.threadToMonitor.stop();
The logs are as follows:
[ERROR] ThreadMonitorExt: Thread hard shutdown!
`java.lang.reflect.InvocationTargetException: null
at sun.reflect.GeneratedMethodAccessor130.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_272]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_272]
at com.huawei.ies.cache.redis.RedisProxy.invoke(RedisProxy.java:64)
at com.sun.proxy.$Proxy77.smembers(Unknown Source) ~[?:?]
at com.huawei.ies.cache.redis.SimpleOper.smembers(SimpleOper.java:53)
at com.huawei.ies.cache.manage.CacheDataService.getRelationObjectOnOther
at com.huawei.ies.cache.manage.CacheDataService.lambda$null$7(CacheDataService.java:308) ~
at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_272]
at com.huawei.ies.cache.manage.CacheDataService.lambda$getCacheRelationObject$8
at java.util.HashMap.forEach(HashMap.java:1289) [?:1.8.0_272]
at com.huawei.ies.cache.manage.CacheDataService.getCacheRelationObject
at com.huawei.ies.cache.manage.CacheDataService.getRelationFromRedis
at com.huawei.ies.cache.service.OpenDataServiceImpl.getRelationData
at com.huawei.ows.ceae.infrastructure.script.call.dim.DIMQueryUtil.getRelations
at jdk.nashorn.internal.scripts.Script$Recompilation$1166$2255A$^eval_._consumerFun#getCloudLineIdByCircuitNo(:81) [?:?]
at jdk.nashorn.internal.scripts.Script$Recompilation$964$1932$^eval_._consumerFun#main(:49) [?:?]
at jdk.nashorn.internal.scripts.Script$Recompilation$962$1815A$^eval_._consumerFun(:39) [?:?]
at jdk.nashorn.internal.scripts.Script$Recompilation$961$1716A$^eval_.run(:31) [?:?]
at com.huawei.ows.ceae.infrastructure.script.api.RunScript$$NashornJavaAdapter.run(Unknown Source) [?:?]
at com.huawei.ows.ceae.infrastructure.script.sandox.SecureJSExecutor.operate(SecureJSExecutor.java:26)
at com.huawei.ows.ceae.infrastructure.script.sandox.BaseSecureJsOperator.run(BaseSecureJsOperator.java:71)
at com.huawei.ows.ceae.infrastructure.script.sandox.SecureJSExecutor.run(SecureJSExecutor.java:15)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_272]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_272]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_272]
Caused by: java.lang.ThreadDeath
at java.lang.Thread.stop(Thread.java:853) ~[?:1.8.0_272]
at delight.nashornsandbox.internal.ThreadMonitor.run(ThreadMonitor.java:124)

@tangrammer @turbanoff @obourgain @sinaa `Please answer.

@mxro
Copy link
Collaborator

mxro commented Aug 19, 2021

Hi @MrLi2018 thank you for raising this issue!

I assume this error should be raised when the script runs for longer than it is allowed to run or the memory limit is exhausted. Could this be the case here?

Would it be possible to refactor the script to use less CPU or to increase the limits?

Also if memory limits are used, these are often not very accurate due to JVM limitations, especially if threads are long running.

Hope this helps!

@MrLi2018
Copy link
Author

Thank you for answering. However, the RESTful interface of another service is invoked in my script. The execution time of the CPU cannot be controlled, @mxro

@mxro
Copy link
Collaborator

mxro commented Aug 20, 2021

If that is the case, could you then maybe simply not set the CPU limit?

Otherwise it may also be an option to do the REST call before running the script and then just inject the results?

@MrLi2018
Copy link
Author

MrLi2018 commented Dec 13, 2021 via email

@MrLi2018
Copy link
Author

MrLi2018 commented Dec 13, 2021 via email

@tellmewhattodo
Copy link

tellmewhattodo commented Dec 13, 2021 via email

@mxro
Copy link
Collaborator

mxro commented Dec 14, 2021

Sorry can't think of much more what to do than increase CPU and memory limits. Otherwise you could maybe add a timeout to the REST call? So when that takes longer than a second or so, you could terminate the script then and know for sure what was the issue?

Yeah maybe worth a try to explore how the same works in GraalVM, there is a sandbox for that too: https://github.com/javadelight/delight-graaljs-sandbox

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants