Segmentation fault for SIMULINK version in linux #730

abhineet-gupta · 2021-04-26T01:40:27Z

Bug description
Cannot run the simulink version of OpenFAST due to segmentation fault. The s-function compiles without any issues.
This behavior exists in both main(718d46f) and dev(216bccb) branches.
I am showing the results from the dev brach as the main branch has an bug where simulink errors are not reported.

System Details
Linux Mint 20.1, MATLAB 2020b, openblas, gcc and gfortran

To Reproduce
Following options were used ()

cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DBUILD_SHARED_LIBS=ON
make
make install

Screenshots, if applicable
The crash report is attached

OpenFAST Version
dev(216bccb)

matlab_crash_dump.txt

The text was updated successfully, but these errors were encountered:

andrew-platt · 2021-04-26T14:13:15Z

Which model are you testing with?

abhineet-gupta · 2021-04-26T14:19:42Z

I modified the "Run_OpenLoop.m" to run the 5MW_Land_BD_Linear turbine.

andrew-platt · 2021-04-26T14:28:43Z

Did the original "Run_OpenLoop.m" also experience a segmentation fault?

abhineet-gupta · 2021-04-26T14:36:18Z

With the AOC_Wst turbine, I get a weird error shown below.

%%%%%%%%%%%%%%%%%%%
Error reported by S-function 'FAST_SFunc' in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function':
FAST_InitializeAll: IfW_Init:IfW_UniformWind_Init: Could not read upflow column in uniform wind files. Assuming
upflow is 0.
FAST_InitializeAll:SrvD_Init:SrvD_ReadInput:ReadPrimaryFile:Invalid logical input for file
"/home/abhineet/Documents/MyOpenFAST/openfast/reg_tests/r-test/glue-codes/openfast/AOC_WSt/AOC_WSt_ServoDyn.dat"
occurred while trying to read CompNTMD.
%%%%%%%%%%%%%%%%%%%%%

andrew-platt · 2021-04-26T14:41:55Z

the format of the ServoDyn input file has changed between v2.5.0 and what is currently in the dev branch. The CompNTMD keyword was used v2.5.0 release (main branch), but not in the current dev branch. So it looks like you are running the input files from the dev branch with the v2.5.0 compiled version of OpenFAST.

abhineet-gupta · 2021-04-26T14:49:47Z

I checked out the 'dev' (e0a73e19) branch of the r-test submodule.

I get the following error for AOC_Wst
%%%%%%%%%%%%%%%%%
Error using Run_OpenLoop (line 15)
Error reported by S-function 'FAST_SFunc' in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function':
FAST_InitializeAll: IfW_Init:IfW_UniformWind_Init: Could not read upflow column in uniform wind files. Assuming
upflow is 0.
FAST_InitializeAll:SrvD_Init:SrvD_ReadInput:ReadPrimaryFile:Invalid logical input for file
"/home/abhineet/Documents/MyOpenFAST/openfast/reg_tests/r-test/glue-codes/openfast/AOC_WSt/AOC_WSt_ServoDyn.dat"
occurred while trying to read CompNTMD.
%%%%%%%%%%%%%%%%%%%

but I get an error instead of segmentation fault for 5MW_Land_BD_Linear
%%%%%%%%%%%%%%%%%%%%%%%
Error using Run_OpenLoop (line 15)
Error reported by S-function 'FAST_SFunc' in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function':
FAST_InitializeAll:FAST_Init:ValidateInputData:LinOutJac can be used only when LinInputs=LinOutputs=2.
FAST_InitializeAll:ED_Init:ED_CalcContStateDeriv:LAPACK_DGETRF: illegal value in argument -4.
%%%%%%%%%%%%%%%%%%%%%%%

abhineet-gupta · 2021-04-26T14:53:37Z

I had also tried running the simulations for 'main' branches and got errors as well.
I can rerun those and share the errors if that helps in debugging.
(The main branch had a bug that simulink errors were not displayed (fixed in the dev branch) so I thought that might be less useful for bug reporting. )

andrew-platt · 2021-04-26T14:54:42Z

Since your error message states occurred while trying to read CompNTMD, the OpenFAST version that simulink is running is not the dev version. What version does OpenFAST report in the first few output lines?

abhineet-gupta · 2021-04-26T14:58:06Z

The OpenFAST output says:

%%%%%%%%%%%%%%%%%%%%%%%
OpenFAST-v2.4.0-133-gb913c945-dirty
Compile Info:

Compiler: GCC version 9.3.0
Architecture: 64 bit
Precision: double
Date: Jan 5 2021
Time: 21:12:45
Execution Info:
Date: 04/26/2021
Time: 09:48:55-0500
%%%%%%%%%%%%%%%%%%%%%%%

Also, heres a screenshot of my git log

andrew-platt · 2021-04-26T15:06:50Z

commit b913c94 is on the main branch from 4 months ago. Can you recompile your currently checked out version and rerun the create_FAST_SFunc.m script? I would expect to see the git hash ending in -g216bccbf in the OpenFAST output if you actually are using the dev version that is checked out.

abhineet-gupta · 2021-04-26T15:10:41Z

Just to make sure, when you say recompile, you mean I should run the following commands (with the current checked out version)

cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DBUILD_SHARED_LIBS=ON
make
make install

andrew-platt · 2021-04-26T15:14:00Z

Yes. First though, delete everything in the build directory (rm -rf * inside the build directory). This will make absolutely certain that there are no leftover modules from a prior build.

You may also want to get the recent tags (git doesn't grab them by default for some reason): get fetch --all --tags

abhineet-gupta · 2021-04-26T15:14:38Z

Alright, will do.

bjonkman · 2021-04-26T15:14:54Z

The segmentation fault could be due to a conflict in LAPACK libraries between Matlab and OpenFAST. I'd recommend looking at #482 (particularly the last comments on that issue) to see if that helps.

andrew-platt · 2021-04-26T15:43:19Z

One other thought that @rafmudaf brought up is to check the install path.

In the CMakeCache.txt file within the build directory, what does the entry for the CMAKE_INSTALL_PREFIX variable show? Is that the same as the location specified in the create_FAST_SFunc.m? If not, it is possible that an older version at the location specified in the create_FAST_SFunc.m script is getting linked in the mex-function.

abhineet-gupta · 2021-04-26T16:06:08Z

One other thought that @rafmudaf brought up is to check the install path.

In the CMakeCache.txt file within the build directory, what does the entry for the CMAKE_INSTALL_PREFIX variable show? Is that the same as the location specified in the create_FAST_SFunc.m? If not, it is possible that an older version at the location specified in the create_FAST_SFunc.m script is getting linked in the mex-function.

The CMakeCache.txt has the following like
CMAKE_INSTALL_PREFIX:PATH=/home/abhineet/Documents/MyOpenFAST/openfast/install

which is same as what I am using in create_FAST_SFUNC.m

if ( ispc ) % Windows PC
        installDir = '../../../install';
        outDir = fullfile(installDir, 'lib');
        % If there are shared libraries does it work for outDir to be the local directory?
    else
        installDir = '../../../install';  %%%%%%% <<<<<<<<< **Here**
        outDir = '.';
    end

abhineet-gupta · 2021-04-26T16:20:32Z

Yes. First though, delete everything in the build directory (rm -rf * inside the build directory). This will make absolutely certain that there are no leftover modules from a prior build.

You may also want to get the recent tags (git doesn't grab them by default for some reason): get fetch --all --tags

@andrew-platt
I tried the following based on your comments

cd openfast
rm -rf ./build
rm -rf ./install
mkdir ./build
cd ./build
cmake .. -DBUILD_OPENFAST_SIMULINK_API=ON -DBUILD_SHARED_LIBS=ON
make -j12
make install
cd glue-codes/simulink/src/
rm ./FAST_SFunc.mexa64
rm ../examples/FAST_SFunc.mexa64
edit and run create_FAST_SFunc.m (to ensure correct options for linux built_with_visualStudio = false and installDir = '../../../install; )
cp ./FAST_SFunc.mexa64 ../examples/
run Run_OpenLoop.m with AOC_WSt turbine

I get the following output in matlab

OpenFAST-v2.4.0-133-gb913c945-dirty
 Compile Info:
  - Compiler: GCC version 9.3.0
  - Architecture: 64 bit
  - Precision: double
  - Date: Jan  5 2021
  - Time: 21:12:45
 Execution Info:
  - Date: 04/26/2021
  - Time: 11:16:55-0500

and the same error
%%%%%%%%%%%%%%%%%%%
Error using Run_OpenLoop (line 12)
Error reported by S-function 'FAST_SFunc' in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function':
FAST_InitializeAll: IfW_Init:IfW_UniformWind_Init: Could not read upflow column in uniform wind files. Assuming
upflow is 0.
FAST_InitializeAll:SrvD_Init:SrvD_ReadInput:ReadPrimaryFile:Invalid logical input for file
"../../../reg_tests/r-test/glue-codes/openfast/AOC_WSt/AOC_WSt_ServoDyn.dat" occurred while trying to read
CompNTMD.
%%%%%%%%%%%%%%%%%%%

andrew-platt · 2021-04-26T16:23:22Z

I'm surprised the version info didn't change. What does git describe --tags return? I'm expecting to see something like v2.5.0-1017-g216bccbf8

rafmudaf · 2021-04-26T16:26:06Z

Also, can you do find /usr -name *openfast*?

bjonkman · 2021-04-26T16:31:50Z

Also, note that the compile date says Jan 5 2021. It is definitely not using the openfast library file that you just built.

abhineet-gupta · 2021-04-26T16:39:42Z

Also, can you do find /usr -name *openfast*?

Ah, Nice catch. The output is

/usr/local/include/openfast
/usr/local/lib/libopenfast_prelib.so
/usr/local/lib/libopenfast_postlib.so
/usr/local/lib/libopenfastlib.so
/usr/local/bin/openfast

I guess it is prioritizing older version of OpenFAST which have on my system with higher priority path.
My bad, and sorry for the oversight.

Is there an easy way to remove the OpenFAST in /usr/local? or should I just remove all the OpenFAST files from /usr folder?

I will try to run the correct version of OpenFAST meanwhile to ensure that no segmentation fault occurs.

rafmudaf · 2021-04-26T16:49:41Z

There's no catch-all, but you can do these things:

find /usr/local -name *openfast* -delete
find /usr/local -name *dyn*
# If there's something related to OpenFAST (aerodynlib, beamdynlib, etc), delete those
ls /usr/local/include
# Remove anything here related to OpenFAST

abhineet-gupta · 2021-04-26T19:23:41Z

Thank you all for the help. I got the simulation working although I had some more trouble with LAPACK libraries.

My understanding after some reading on the issue is that OpenBLAS includes 'LAPACK' library but not 'LAPACKE' which is needed for OpenFAST. This can be fixed with

sudo apt install liblapacke liblapacke-dev

and then running the following (in a new terminal to avoid any mixup)

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to openfast/install/lib>
export BLAS_VERSION=libopenblas.so
export LAPACK_VERSION=liblapacke.so
matlab

Hope this helps someone.

rafmudaf · 2021-04-26T19:46:34Z

For our reference, what was the LAPACK error you found and fixed by installing LAPACKE?

abhineet-gupta · 2021-04-26T21:43:50Z

If I use the following options

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path to openfast/install/lib>
export BLAS_VERSION=libopenblas.so
export LAPACK_VERSION=libopenblas.so
matlab

I get the following error (which was fixed by using LAPACKE)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Error using Run_OpenLoop (line 12)
Error in 'OpenLoop/FAST Nonlinear Wind Turbine/S-Function' while executing C MEX S-function 'FAST_SFunc', (mdlStart),
at time 0.0.
Caused by:
Error using Run_OpenLoop (line 12)
LAPACK loading error:
/usr/lib/x86_64-linux-gnu/libopenblas.so.0: undefined symbol: LAPACKE_sbdsdc
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

rafmudaf · 2021-04-26T21:58:24Z

Got it, thanks.

abhineet-gupta added the Type: Bug label Apr 26, 2021

abhineet-gupta closed this as completed Apr 26, 2021

isablanca mentioned this issue Nov 19, 2021

Conflicts of blas/lapack setting when using openfast with simulink in Ubuntu18.04 #924

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Segmentation fault for SIMULINK version in linux #730

Segmentation fault for SIMULINK version in linux #730

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021 •

edited

abhineet-gupta commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

bjonkman commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021 •

edited

andrew-platt commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

bjonkman commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

Segmentation fault for SIMULINK version in linux #730

Segmentation fault for SIMULINK version in linux #730

Comments

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021 • edited

abhineet-gupta commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

bjonkman commented Apr 26, 2021

andrew-platt commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021 • edited

andrew-platt commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

bjonkman commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

abhineet-gupta commented Apr 26, 2021

rafmudaf commented Apr 26, 2021

andrew-platt commented Apr 26, 2021 •

edited

abhineet-gupta commented Apr 26, 2021 •

edited