Need some serious help to link Dataframe with my C++ application

[achzhou]

Build a very simple C++ application in order to get practicing with dataframe in C++. Spend the whole day but can't even have the application compiled. I am using VS code in windows 11 (64-bits) and I am using cl.exe as C++ compiler. I used vcpkg to download and install dataframe as instructed in dataframe web site. The code itself is so simple:
`
#include
#include
#include
#include <DataFrame/DataFrame.h>
#include <DataFrame/DataFrameStatsVisitors.h>
#include <DataFrame/RandGen.h>

using namespace hmdf;
constexpr std::size_t ALIGNMENT = 256;
typedef StdDataFrame MyDataFrame;

int main (int argv, char* argc[])
{
MyDataFrame df_read;
std::cout << "Testing CPP dataframe: " << std::endl;
try
{
df_read.read("invema_zc_test.csv");
}
catch (const DataFrameError &ex)
{
std::cout << ex.what() << std::endl;
}
std::cout << "Successfully read the csv file" << std::endl;
return 0;
}
`

The following is the compiling command VS code generated for me:

cl.exe /Zi /Zc:__cplusplus /std:c++20 /EHsc /nologo D:\DellProjs\CPPProjs\EMA\ema_tst.cpp /I"c:\dev\vcpkg\vcpkg\installed\x64-windows\include" /FeD:\DellProjs\CPPProjs\EMA\ema_tst.exe /link"c:\dev\vcpkg\vcpkg\installed\x64-windows\lib\DataFrame.lib"

But the compiling kept failing. The error msg as followings:
ema_tst.obj : error LNK2019: unresolved external symbol "public: __thiscall hmdf::DateTime::DateTime(enum hmdf::DT_TIME_ZONE)" (??0DateTime@hmdf@@QAE@W4DT_TIME_ZONE@1@@z)，function "class hmdf::DateTime __cdecl hmdf::get_nan(void)" (??$get_nan@VDateTime@hmdf@@@hmdf@@ya?AVDateTime@0@XZ) quote the symbol
ema_tst.obj : error LNK2019: can't resolved external symbol "public: __thiscall hmdf::DateTime::DateTime(char const *,enum hmdf::DT_DATE_STYLE,enum hmdf::DT_TIME_ZONE)" (??0DateTime@hmdf@@QAE@PBDW4DT_DATE_STYLE@1@W4DT_TIME_ZONE@1@@z)，function "class hmdf::DateTime * __cdecl std::construct_at<class hmdf::DateTime,char (&)[8192],enum hmdf::DT_DATE_STYLE,0>(class hmdf::DateTime * const,char (&)[8192],enum hmdf::DT_DATE_STYLE &&)" (??$construct_at@VDateTime@hmdf@@AAY0CAAA@DW4DT_DATE_STYLE@2@$0A@@std@@YAPAVDateTime@hmdf@@QAV12@AAY0CAAA@D$$QAW4DT_DATE_STYLE@2@@z) quote the symbol
ema_tst.obj : error LNK2019: unresolved external symbol "public: __thiscall hmdf::DateTime::DateTime(class hmdf::DateTime const &)" (??0DateTime@hmdf@@QAE@ABV01@@z)，function "class hmdf::DateTime * __cdecl std::construct_at<class hmdf::DateTime,class hmdf::DateTime &,0>(class hmdf::DateTime * const,class hmdf::DateTime &)" (??$construct_at@VDateTime@hmdf@@aav12@$0A@@std@@YAPAVDateTime@hmdf@@QAV12@AAV12@@z) quote the symbol
ema_tst.obj : error LNK2019: unresolved external symbol "public: __thiscall hmdf::DateTime::DateTime(class hmdf::DateTime &&)" (??0DateTime@hmdf@@QAE@$$QAV01@@z)，function "class hmdf::DateTime * __cdecl std::construct_at<class hmdf::DateTime,class hmdf::DateTime,0>(class hmdf::DateTime * const,class hmdf::DateTime &&)" (??$construct_at@VDateTime@hmdf@@v12@$0A@@std@@YAPAVDateTime@hmdf@@QAV12@$$QAV12@@z) quote the symbol
ema_tst.obj : error LNK2019: unresolved external symbol "public: __thiscall hmdf::DateTime::~DateTime(void)" (??1DateTime@hmdf@@QAE@XZ)，function "public: unsigned int __thiscall hmdf::DataFrame<unsigned int,struct hmdf::HeteroVector<0> >::load_column(char const *,class std::vector<class hmdf::DateTime,class std::allocator > &&,enum hmdf::nan_policy,bool)" (??$load_column@VDateTime@hmdf@@@?$DataFrame@IU?$HeteroVector@$0A@@hmdf@@@hmdf@@QAEIPBD$$QAV?$vector@VDateTime@hmdf@@v?$allocator@VDateTime@hmdf@@@std@@@std@@W4nan_policy@1@_N@Z) quote this symbol
ema_tst.obj : error LNK2019: unresolved external symbol "public: class hmdf::DateTime & __thiscall hmdf::DateTime::operator=(class hmdf::DateTime const &)" (??4DateTime@hmdf@@QAEAAV01@ABV01@@z)，function "private: void __thiscall std::vector<class hmdf::DateTime,class std::allocator >::Assign_counted_range<class hmdf::DateTime *>(class hmdf::DateTime ,unsigned int)" (??$_Assign_counted_range@PAVDateTime@hmdf@@@?$vector@VDateTime@hmdf@@v?$allocator@VDateTime@hmdf@@@std@@@std@@AAEXPAVDateTime@hmdf@@i@Z) quote the symbol
ema_tst.obj : error LNK2019: unresolved external symbol "public: void __thiscall hmdf::DateTime::set_time(__int64,int)" (?set_time@DateTime@hmdf@@QAEX_JH@Z)，function "public: void _thiscall hmdf::DataFrame<unsigned int,struct hmdf::HeteroVector<0> >::ColVectorPushBack<class hmdf::DateTime,class std::vector<class hmdf::DateTime,class std::allocator >,void>::operator()(class std::vector<class hmdf::DateTime,class std::allocator > &,class std::basic_istream<char,struct std::char_traits > &,class hmdf::DateTime (__cdecl)(char const *,char * *),enum hmdf::io_format)const " (??R?$ColVectorPushBack@VDateTime@hmdf@@v?$vector@VDateTime@hmdf@@v?$allocator@VDateTime@hmdf@@@std@@@std@@x@?$DataFrame@IU?$HeteroVector@$0A@@hmdf@@@hmdf@@QBEXAAV?$vector@VDateTime@hmdf@@v?$allocator@VDateTime@hmdf@@@std@@@std@@aav?$basic_istream@DU?$char_traits@D@std@@@4@P6A?AVDateTime@2@PBDPAPAD@ZW4io_format@2@@z) quote the symbol
c:\dev\vcpkg\vcpkg\installed\x64-windows\lib\DataFrame.lib : warning LNK4272:lib computer type“x64” conflicts with the target computer type “x86”
D:\DellProjs\CPPProjs\EMA\ema_tst.exe : fatal error LNK1120: 7 external commands unresolved

It looks like that something is related to 64-bits or 32-bits. But my OS is windows 11 (64-bits), my compiler, my vs code are all 64-bits. vcpkg installed dataframe as x64-windows automatically. I don't where this x64 and x86 issues come from. Need help here.

Thanks a lot! I really want to involve datafram lib into my new project.

[hosseinmoein]

I am not familiar with Windows development environment, but from the error messages the problem is kind of obvious.
DataFrame is not a header only library. You have to first build (ar) the DataFrame library and then link your executable (your test program) with the DataFrame library.
If you have cmake, you can follow the build instructions in README and should just work.

I hope that's helpful.

[achzhou]

Thanks for your response, especially the link. It provided many info I was looking for. It helped me a lot. Even though I didn't fully replicate your compiling process and compiling result, I finally have my DataFrame.lib compiled in my windows using cl.exe and cmake. Most of tester executbles are created. However, hello_world.cc failed to be compiled by using cmake. I have been trying to compile hello_world.cc from windows command prompt by using cl.exe command line. But I still didn't quite get it done successfully. I even tried to simplify hellow_world.cc. I only kept small part of the beginning of source code, pretty much until reading ibm.csv file. But I still could not get it compiled. Here is the compiling command I used:

cl /std:c++20 /c /I"C:\Dev\DataFrame\include" hello_world_cz.cpp
link /NODEFAULTLIB:library /out:hello_world.exe /libpath:C:\Dev\DataFrame\build\Debug DataFrame.lib hello_world_cz.obj

But unfortunely I got the following error msg:
Microsoft (R) Incremental Linker Version 14.37.32824.0
Copyright (C) Microsoft Corporation. All rights reserved.

DataFrame.lib(DateTime.obj) : error LNK2038: detect “_ITERATOR_DEBUG_LEVEL” unmatched: vale “2” unmatched “0”(hello_world_cz.obj 中)
DataFrame.lib(DateTime.obj) : error LNK2038: detect “RuntimeLibrary” unmatched: “MDd_DynamicDebug”不 unmatched “MT_StaticRelease”(hello_world_cz.obj)
msvcprtd.lib(MSVCP140D.dll) : error LNK2005: "public: __cdecl std::_Lockit::_Lockit(int)" (??0_Lockit@std@@qeaa@H@Z) already defined in libcpmt.lib(xlock.obj)
msvcprtd.lib(MSVCP140D.dll) : error LNK2005: "public: __cdecl std::_Lockit::~_Lockit(void)" (??1_Lockit@std@@qeaa@XZ) already defined libcpmt.lib(xlock.obj)
LINK : warning LNK4098: default lib “MSVCRTD”conflicts with others；Please use /NODEFAULTLIB:library
LINK : warning LNK4217: symbol “_invalid_parameter”(already defined in “ libucrt.lib(invalid_parameter.obj)”) already impoted in “DataFrame.lib(DateTime.obj)”(function“"void * __cdecl std::_Allocate_manually_vector_aligned(unsigned __int64)" (??$_Allocate_manually_vector_aligned@U_Default_allocate_traits@std@@@std@@YAPEAX_K@Z)”)
LINK : warning LNK4217: symbol “__stdio_common_vsprintf” (already defined in “ libucrt.lib(output.obj)”) already imported from “DataFrame.lib(DateTime.obj)”(“vsnprintf”)
LINK : warning LNK4217: symbol “__stdio_common_vsscanf”(already defined in “ libucrt.lib(input.obj)”) already imported by “DataFrame.lib(DateTime.obj)”(function“_vsscanf_l”)
LINK : warning LNK4217: symbol “_tzset”(already defined in “ libucrt.lib(tzset.obj)”) already imported by “DataFrame.lib(DateTime.obj)”(function “"public: __cdecl hmdf::DateTime::DT_initializer::DT_initializer(void)" (??0DT_initializer@DateTime@hmdf@@qeaa@XZ)”)
LINK : warning LNK4217: symbol “isspace”(already defined in “ libucrt.lib(_ctype.obj)”) already imported by “DataFrame.lib(DateTime.obj)”(function“"public: __cdecl hmdf::DateTime::DateTime(char const *,enum hmdf::DT_DATE_STYLE,enum hmdf::DT_TIME_ZONE)" (??0DateTime@hmdf@@qeaa@PEBDW4DT_DATE_STYLE@1@W4DT_TIME_ZONE@1@@z)”中)导入
DataFrame.lib(DateTime.obj) : error LNK2019: unresolved symbol __imp__CrtDbgReport，function "void * __cdecl std::_Allocate_manually_vector_aligned(unsigned __int64)" (??$_Allocate_manually_vector_aligned@U_Default_allocate_traits@std@@@std@@YAPEAX_K@Z) quote the symbol
DataFrame.lib(DateTime.obj) : error LNK2019: unresolved external symbol __imp__putenv，function "private: static void cdecl hmdf::DateTime::change_env_timezone(enum hmdf::DT_TIME_ZONE)" (?change_env_timezone@datetime@hmdf@@CAXW4DT_TIME_ZONE@2@@z) quote the symbol
DataFrame.lib(DateTime.obj) : error LNK2019: unresolved external symbol __imp__localtime64_s，the function localtime_s quote the symbol
DataFrame.lib(DateTime.obj) : error LNK2019: unresolved external symbol __imp__mktime64，the function mktime quote the symbol
hello_world.exe : fatal error LNK1120: 4 unresolved external symbol

Any ideas???

[hosseinmoein]

I have no idea how you are compiling this. But it seems you have two problems:

1. You have conflicting libraries. So, do what this suggests LINK : warning LNK4098: default lib “MSVCRTD”conflicts with others；Please use /NODEFAULTLIB:library
2. You still have undefined symbols. So, it means you are missing some libraries on the link line.

[achzhou]

Thanks for your quick response. Here is my compiling command:

cl /std:c++20 /c /I"C:\Dev\DataFrame\include" hello_world_cz.cpp
link /NODEFAULTLIB:library /out:hello_world.exe /libpath:C:\Dev\DataFrame\build\Debug DataFrame.lib hello_world_cz.obj

I did put /NODEFAULTLIB:library in my link command. I am not sure what library I missed or why I missed. I actually copy your hello_world.cc code and even simpler. I cut it in shorter version. hello_world_cz.cpp is really really simple code. So I guess my DataFrame.lib could be not compile correctly?

[hosseinmoein]

I have no idea, sorry

[achzhou]

Sorry to bother you again and again. Regarding the link you gave to me,
https://ci.appveyor.com/project/hosseinmoein/dataframe

It looks like that your compiling process is really clean. But my compiling process was quite messy and had many warning all the way until the end. From the above link, I can see you were using windows powershell. So, is it possible for me to do me a favor?
1, make some change in cmakelists.txt so that every compiling and linking command can be printed out. Now the output only showed it was compiling but I don't know how the compiling and linking commands were constructed;
2, I saw you ran "$env:arch" to show one env variable. is it possible to show me you whole environment variables so that I can check what's wrong with me development environment?

Thanks a lot! Any of your help would be highly appreciated.

[achzhou]

I have no idea, sorry

No problem. You certianly helped me a lot. It looks like that it is quite far away to make it work in windows. I switched to Linux. I have a ubuntu linux machine ready. I have the latest version of g++ compiler (including g++, c++, gcc), cmake ready. I did exactly you instructed in your github readme.md:
cd DataFrame
mkdir Debug
cd Debug
cmake -DCMAKE_BUILD_TYPE=Debug -DHMDF_BENCHMARKS=1 -DHMDF_EXAMPLES=1 -DHMDF_TESTING=1 ..

But I got the following error msg:

You have called ADD_LIBRARY for library DataFrame without any source files. This typically indicates a problem with your CMakeLists.txt file
CMake Error at CMakeLists.txt:37 (target_compile_features):
target_compile_features specified unknown feature "cxx_std_20" for target
"DataFrame".

Why are there such errors? I downloaded the whole repository from github. The source code is definitely there. CMakeLists.txt is exactly the same as downloaded. It looks like that there are some errors in CMakeLists.txt. Can you help me with that?

-- Configuring incomplete, errors occurred!

[hosseinmoein]

I have no idea. A lot of people use this on Windows, Linux and macOS with different processors and compilers. I never heard of these problems.
When I run it on Linux I get this

~/pcode/DataFrame> mkdir Debug                                                                                                                                                                       master
~/pcode/DataFrame> cd Debug                                                                                                                                                                          master
~/pcode/DataFrame/Debug> cmake -DCMAKE_BUILD_TYPE=Debug -DHMDF_BENCHMARKS=1 -DHMDF_EXAMPLES=1 -DHMDF_TESTING=1 ..                                                                                    master
-- The CXX compiler identification is AppleClang 15.0.0.15000040
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/hosseinmoein/pcode/DataFrame/Debug
~/pcode/DataFrame/Debug> make                                                                                                                                                                        master
[  2%] Building CXX object CMakeFiles/DataFrame.dir/src/Utils/DateTime.cc.o
[  4%] Linking CXX static library libDataFrame.a
[  4%] Built target DataFrame
[  6%] Generating ../bin/data/sample_data.json
[  6%] Built target hmdf_copy_sample_data.json
[  8%] Generating ../bin/data/sample_data_dt_index.csv
[  8%] Built target hmdf_copy_sample_data_dt_index.csv
[ 10%] Generating ../bin/data/sample_data_string_index.csv
[ 10%] Built target hmdf_copy_sample_data_string_index.csv
[ 12%] Generating ../bin/data/sample_data.csv
[ 12%] Built target hmdf_copy_sample_data.csv
[ 14%] Building CXX object test/CMakeFiles/dataframe_tester.dir/dataframe_tester.cc.o
[ 16%] Linking CXX executable ../bin/dataframe_tester
[ 16%] Built target dataframe_tester
[ 18%] Generating ../bin/data/sample_data_no_index.json
[ 18%] Built target hmdf_copy_sample_data_no_index.json
[ 20%] Generating ../bin/data/csv2_format_data_2.csv
[ 20%] Built target hmdf_copy_csv2_format_data_2.csv
[ 22%] Generating ../bin/data/csv2_format_data_no_index.csv
[ 22%] Built target hmdf_copy_csv2_format_data_no_index.csv
[ 24%] Generating ../bin/data/csv2_format_data.csv
[ 24%] Built target hmdf_copy_csv2_format_data.csv
[ 26%] Generating ../bin/data/DT_IBM.csv
[ 26%] Built target hmdf_copy_DT_IBM.csv
[ 28%] Generating ../bin/data/FORD.csv
[ 28%] Built target hmdf_copy_FORD.csv
[ 30%] Generating ../bin/data/IBM.csv
[ 30%] Built target hmdf_copy_IBM.csv
[ 32%] Generating ../bin/data/SHORT_IBM.csv
[ 32%] Built target hmdf_copy_SHORT_IBM.csv
[ 34%] Generating ../bin/data/sample_data_2.csv
[ 34%] Built target hmdf_copy_sample_data_2.csv
[ 36%] Generating ../bin/data/sample_data_2.json
[ 36%] Built target hmdf_copy_sample_data_2.json
[ 38%] Generating ../bin/data/sample_data_no_index.csv
[ 38%] Built target hmdf_copy_sample_data_no_index.csv
[ 40%] Building CXX object test/CMakeFiles/dataframe_tester_2.dir/dataframe_tester_2.cc.o
[ 42%] Linking CXX executable ../bin/dataframe_tester_2
[ 42%] Built target dataframe_tester_2
[ 44%] Generating ../bin/data/AAPL_10dBucketWithVector.csv
[ 44%] Built target hmdf_copy_AAPL_10dBucketWithVector.csv
[ 46%] Building CXX object test/CMakeFiles/dataframe_tester_3.dir/dataframe_tester_3.cc.o
[ 48%] Linking CXX executable ../bin/dataframe_tester_3
[ 48%] Built target dataframe_tester_3
[ 51%] Building CXX object test/CMakeFiles/dataframe_tester_schema.dir/dataframe_tester_schema.cc.o
[ 53%] Linking CXX executable ../bin/dataframe_tester_schema
[ 53%] Built target dataframe_tester_schema
[ 55%] Building CXX object test/CMakeFiles/vectors_tester.dir/vectors_tester.cc.o
[ 57%] Linking CXX executable ../bin/vectors_tester
[ 57%] Built target vectors_tester
[ 59%] Building CXX object test/CMakeFiles/allocator_tester.dir/allocator_tester.cc.o
[ 61%] Linking CXX executable ../bin/allocator_tester
[ 61%] Built target allocator_tester
[ 63%] Building CXX object test/CMakeFiles/vector_ptr_view_tester.dir/vector_ptr_view_tester.cc.o
[ 65%] Linking CXX executable ../bin/vector_ptr_view_tester
[ 65%] Built target vector_ptr_view_tester
[ 67%] Building CXX object test/CMakeFiles/meta_prog_tester.dir/meta_prog_tester.cc.o
[ 69%] Linking CXX executable ../bin/meta_prog_tester
[ 69%] Built target meta_prog_tester
[ 71%] Building CXX object test/CMakeFiles/date_time_tester.dir/date_time_tester.cc.o
[ 73%] Linking CXX executable ../bin/date_time_tester
[ 73%] Built target date_time_tester
[ 75%] Building CXX object test/CMakeFiles/gen_rand_tester.dir/gen_rand_tester.cc.o
[ 77%] Linking CXX executable ../bin/gen_rand_tester
[ 77%] Built target gen_rand_tester
[ 79%] Building CXX object test/CMakeFiles/dataframe_thread_safety.dir/dataframe_thread_safety.cc.o
[ 81%] Linking CXX executable ../bin/dataframe_thread_safety
[ 81%] Built target dataframe_thread_safety
[ 83%] Generating ../bin/data/DT_AAPL.csv
[ 83%] Built target hmdf_copy_DT_AAPL.csv
[ 85%] Building CXX object examples/CMakeFiles/hello_world.dir/hello_world.cc.o
[ 87%] Linking CXX executable ../bin/hello_world
[ 87%] Built target hello_world
[ 89%] Building CXX object benchmarks/CMakeFiles/dataframe_performance.dir/dataframe_performance.cc.o
[ 91%] Linking CXX executable ../bin/dataframe_performance
[ 91%] Built target dataframe_performance
[ 93%] Building CXX object benchmarks/CMakeFiles/dataframe_performance_2.dir/dataframe_performance_2.cc.o
[ 95%] Linking CXX executable ../bin/dataframe_performance_2
[ 95%] Built target dataframe_performance_2
[ 97%] Building CXX object benchmarks/CMakeFiles/linkedin_benchmark.dir/linkedin_benchmark.cc.o
[100%] Linking CXX executable ../bin/linkedin_benchmark
[100%] Built target linkedin_benchmark
~/pcode/DataFrame/Debug>

[achzhou]

Thanks for showing me the whole process. In your github readme.md, you also have one more step "make install" but it looked like that you didn't do it as shown in the above. Is "make install" still necessary after exec "make"?

[hosseinmoein]

make install is only necessary if you have a standard installation site or you want to install your DataFrame in a system installed place. In other words, it is only necessary of you have a production system

[achzhou]

Thank you so much! Your help and advices are highly appreicated! :-)