The aws sdk and libtorch build failed

[cl1218]

Describe the bug

The bug manifests as undefined references to specific functions in the AWS SDK during the linking stage of your project build.

Expected Behavior

I expect the project to be built successfully.

Current Behavior

[ 50%] Building CXX object CMakeFiles/http_predict.dir/downLoad.cpp.o
[100%] Linking CXX executable http_predict
CMakeFiles/http_predict.dir/downLoad.cpp.o:(.data.rel.ro._ZTVN3Aws2S39S3RequestE[_ZTVN3Aws2S39S3RequestE]+0x30): undefined reference to `Aws::AmazonWebServiceRequest::GetAdditionalCustomHeaders() const'
CMakeFiles/http_predict.dir/downLoad.cpp.o:(.data.rel.ro._ZTVN3Aws2S39S3RequestE[_ZTVN3Aws2S39S3RequestE]+0x38): undefined reference to `Aws::AmazonWebServiceRequest::SetAdditionalCustomHeaderValue(std::string const&, std::string const&)'
CMakeFiles/http_predict.dir/downLoad.cpp.o:(.data.rel.ro._ZTVN3Aws2S38Endpoint18S3EndpointProviderE[_ZTVN3Aws2S38Endpoint18S3EndpointProviderE]+0x28): undefined reference to `Aws::Endpoint::DefaultEndpointProvider<Aws::S3::S3ClientConfiguration, Aws::S3::Endpoint::S3BuiltInParameters, Aws::S3::Endpoint::S3ClientContextParameters>::OverrideEndpoint(std::string const&)'
collect2: error: ld returned 1 exit status
make[2]: *** [http_predict] Error 1
make[1]: *** [CMakeFiles/http_predict.dir/all] Error 2
make: *** [all] Error 2

Reproduction Steps

CMakeLists.txt :

cmake_minimum_required(VERSION 3.13)
project(http_predict)
find_package(AWSSDK REQUIRED COMPONENTS s3)
message(STATUS "AWS SDK version: ${AWSSDK_LIBRARIES}")
# Set the C++ compiler based on the build target
if(BUILD_FOR_ARM)
    set(CMAKE_CXX_COMPILER aarch64-linux-gnu-gcc)
else()
    set(CMAKE_CXX_COMPILER gcc)
endif()
# Set additional library and include directories
set(EXTRA_LIB_DIR -L../lib)
set(EXTRA_INCLUDE_DIR -I../include)
file(GLOB AWSDOC_S3_HEADERS
    "include/s3download.h"
)
set(VTK_DIR "/usr/local/lib64/cmake/vtk-8.2")
set(Torch_DIR "/home/ec2-user/libtorch/share/cmake/Torch")

find_package(VTK REQUIRED)
find_package(Torch REQUIRED)
find_package(X11 REQUIRED)
# Set source files
set(SRC_DEMO downLoad.cpp)

# Set library dependencies
set(LIB -lsecurec -lstdc++)

# Add executable target
add_executable(http_predict ${SRC_DEMO})

# Set additional compile options
#target_compile_options(http_predict PRIVATE -g -fstack-protector --param ssp-buffer-size=4 -Wstack-protector)
target_link_libraries(http_predict PRIVATE ${VTK_LIBRARIES} ${TORCH_LIBRARIES}   ${X11_LIBRARIES} ${EXTRA_LIB_DIR} ${LIB} pthread aws-cpp-sdk-s3 aws-cpp-sdk-core)

# Set linking options
target_link_options(http_predict PRIVATE -Wl,--disable-new-dtags,--rpath ../lib -Wl,-z,relro,-z,now -Wl,-z,noexecstack -fPIC)

Possible Solution

No response

Additional Information/Context

Only aws getobject is used in the actual source code.There is no problem when aws and libtorch build projects separately,This problem arises when the combination comes together.libtorch is available in version 1.7.0

AWS CPP SDK version used

AWS CPP SDK-1.11.237

Compiler and Version used

gcc-7.3.1

Operating System and version

centos

[sbiscigl]

find_package(AWSSDK REQUIRED COMPONENTS s3)

how are you installing the SDK? this error to me indicates that you have drift in your installation where you are including a header that expects these functions to be present but are using a binary where they are not present. Those specific functions were added in a 1.10 version a while ago. My guess if you clean installed the SDK install to a specific directory, you would not see this problem

during the linking stage of your project build.

also just for clarity this is not the linking stage of our project, this is during the linking of your build when http_predict/downLoad.cpp is linked against the SDK. is this file part of your source?

[github-actions]

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

[cl1218]

Build S3 part of aws-sdk-cpp:
git clone --recurse-submodules https://github.com/aws/aws-sdk-cpp
cd aws-ask-cpp
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=/usr/local/ -DCMAKE_INSTALL_PREFIX=/usr/local/ -DBUILD_ONLY="s3"
make -j4
make install

http_predict/downLoad.cpp example code:

// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0

#include "torch/torch.h"
#include "torch/script.h"
#include <iostream>
#include <aws/core/Aws.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/GetObjectRequest.h>
#include <fstream>
#include "s3download.h"


bool AwsDoc::S3::GetObject(const Aws::String &objectKey,
                           const Aws::String &fromBucket,
                           const Aws::Client::ClientConfiguration &clientConfig) {
    Aws::S3::S3Client client(clientConfig);

    Aws::S3::Model::GetObjectRequest request;
    request.SetBucket(fromBucket);
    request.SetKey(objectKey);

    Aws::S3::Model::GetObjectOutcome outcome =
            client.GetObject(request);

    if (!outcome.IsSuccess()) {
        const Aws::S3::S3Error &err = outcome.GetError();
        std::cerr << "Error: GetObject: " <<
                  err.GetExceptionName() << ": " << err.GetMessage() << std::endl;
    }
    else {
        std::cout << "Successfully retrieved '" << objectKey << "' from '"
                  << fromBucket << "'." << std::endl;
    }

    return outcome.IsSuccess();
}


#ifndef TESTING_BUILD

int main() {
    Aws::SDKOptions options;
    Aws::InitAPI(options);
    {
        //TODO(user): Change bucketName to the name of a bucket in your account.
        const Aws::String bucketName = "<Enter bucket name>";

        //TODO(user): Change objectName to the name of an object in the bucket.
        //See create_bucket.cpp and put_object.cpp to create a bucket and load an object into that bucket.
        const Aws::String objectName = "<Enter object name>";

        Aws::Client::ClientConfiguration clientConfig;
        // Optional: Set to the AWS Region in which the bucket was created (overrides config file).
        // clientConfig.region = "us-east-1";

        AwsDoc::S3::GetObject(objectName, bucketName, clientConfig);
    }
    Aws::ShutdownAPI(options);

    return 0;
}

#endif // TESTING_BUILD

s3download.h:

#include <aws/core/Aws.h>
#include <aws/s3/S3Client.h>
#include <aws/s3/model/BucketLocationConstraint.h>

namespace AwsDoc {
    namespace S3 {

        bool GetObject(const Aws::String &objectKey,
                       const Aws::String &fromBucket,
                       const Aws::Client::ClientConfiguration &clientConfig);


        extern std::mutex upload_mutex;

        extern std::condition_variable upload_variable;
    } // namespace S3
} // namespace AwsDoc

[jmklix]

Can you please clean your build and install folders and then try rebuilding this sdk?