# A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680088, tid=0x00007f77c605c700

[godjuhyuk]

안녕하세요.. for loop로 konlpy 의 Okt 클래스를 이용해 텍스트를 전처리하고 전처리 과정을 거쳐 딥러닝 모델 학습까지 진행하는 코드를 짰습니다. 앞에서 말한 코드가 21번 돌아야하는데 자꾸 4번째 쯤에서 Java runtime error가 나더라구요..
늘 딥러닝 모델 학습이 끝나고 나서 다시 for loop의 처음으로 돌아가 Okt로 텍스트 처리를 할 때 아래와 같은 오류가 났습니다.
Python 코드를 돌리는데 Java Runtime Error가 발생하는데, konlpy가 Java 기반이라고 알고 있어서 혹시 konlpy와 관련이 있나? 하는 마음에 질문하게 되었습니다..
코드가 너무 복잡해서 (초보라서 죄송합니다 ㅠ..) 코드를 올리지는 못하겠고 혹시 메모리문제인가 해서 오류를 해결하기 위해 Okt 클래스에서 max_heap_size 파라미터를 128부터 4096까지 조정해봤지만 결과가 늘 똑같았습니다..
아래와 같은 오류를 혹시 해결한 사례가 있을까요?

97/116 [========================>.....] - ETA: 0s - loss: 0.1731 - mse: 0.0679�������������������������������������������������������������������������������
99/116 [========================>.....] - ETA: 0s - loss: 0.1727 - mse: 0.0671�������������������������������������������������������������������������������
101/116 [=========================>....] - ETA: 0s - loss: 0.1735 - mse: 0.0673�������������������������������������������������������������������������������
103/116 [=========================>....] - ETA: 0s - loss: 0.1736 - mse: 0.0671�������������������������������������������������������������������������������
105/116 [==========================>...] - ETA: 0s - loss: 0.1726 - mse: 0.0663�������������������������������������������������������������������������������
107/116 [==========================>...] - ETA: 0s - loss: 0.1725 - mse: 0.0661�������������������������������������������������������������������������������
109/116 [===========================>..] - ETA: 0s - loss: 0.1731 - mse: 0.0666�������������������������������������������������������������������������������
111/116 [===========================>..] - ETA: 0s - loss: 0.1735 - mse: 0.0670�������������������������������������������������������������������������������
113/116 [============================>.] - ETA: 0s - loss: 0.1735 - mse: 0.0667�������������������������������������������������������������������������������
115/116 [============================>.] - ETA: 0s - loss: 0.1734 - mse: 0.0662�������������������������������������������������������������������������������
116/116 [==============================] - 5s 44ms/step - loss: 0.1731 - mse: 0.0658 - val_loss: 0.2079 - val_mse: 0.0766
[0.20414724946022034, 0.07085125893354416]
/home/line-dev/git_fnb/fnb_py/module/learn_ai.py:189: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
 df['date_change'] = df.news_date.apply(lambda x : True if format(x,'%H:%M:%S')>'15:30:00' else False)
/home/line-dev/git_fnb/fnb_py/module/learn_ai.py:190: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
 df['news_date'] = df.news_date.apply(lambda x : format(x+timedelta(days=1),'%Y-%m-%d') if format(x,'%H:%M:%S')>'15:30:00' else format(x,'%Y-%m-%d'))
@
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680088, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680088.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680090, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680090.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680092, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680092.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680202, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680202.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680251, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680251.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680288, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680288.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680334, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680334.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680377, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680377.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680423, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680423.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f78440234f0, pid=2680468, tid=0x00007f77c605c700
#
# JRE version: OpenJDK Runtime Environment (8.0_312-b07) (build 1.8.0_312-b07)
# Java VM: OpenJDK 64-Bit Server VM (25.312-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libpthread.so.0+0x94f0]  __GI___pthread_timedjoin_ex+0x10
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/line-dev/git_fnb/fnb_py/hs_err_pid2680468.log
#
# If you would like to submit a bug report, please visit:
#   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

[hexists]

@godjuhyuk
안녕하세요.
오류 해결하셨을까요?
위에 있는 메세지만 보고는 어떤 오류인지, konlpy의 오류인지 파악이 어렵습니다.

먼저는 for loop이 돌 때 메모리가 증가하는지 확인해보시고, 그 원인이 konlpy와 관련 있는 것인지 확인해보는 것이 좋을 것 같습니다.

혹시 혼자 해결하기 힘드시다면, 코드를 올려주시면 같이 볼 수 있을 것 같아요.

감사합니다.

[godjuhyuk]

@hexists
안녕하세요! 한동안 내버려두다가 이제야 다시 고치기 시작했습니다.
아직 해결 못한 상태고 조언해주신대로 메모리 체크 한 번 더 해보고
그래도 해결되지 않으면 코드 올리겠습니다.. 너무 지저분해서요 ㅠㅠ..
감사합니다!

[hexists]

@godjuhyuk
네~ 확인해보시고 결과 공유 가능하면 공유 부탁드립니다.
감사합니다.