Multi thread io_service::run() SET GET DEL #48

redboltz · 2017-10-25T01:48:56Z

I'd like to use redisclient on multi thread program. In the program, io_service::run() is called from each threads.

I got some unexpected behavior.

Here is the list of behavior I ovserved:

Got stucked. Log output is stopped.
Core dumped.

*** Error in `./a.out': double free or corruption (fasttop): 0x00007f4dd0002720 ***
[1]    17510 abort (core dumped)  ./a.out

Segmentation falult.

SET: OK[1]    17549 segmentation fault (core dumped)  ./a.out

Redis returns error.

SET: unique-redis-value
Invalid value from redis: unique-redis-value

I wrote the code that reproduces the behavior based on
https://github.com/nekipelov/redisclient/blob/master/examples/async_set_get.cpp

The code continues SET, GET, DEL, SET, GET, DEL, ...
When I run the executable file again and again, the behavior above is observed.

#include <string>
#include <iostream>
#include <functional>
#include <boost/asio/ip/address.hpp>
#include <thread>

#include <redisclient/redisasyncclient.h>

static const std::string redisKey = "unique-redis-key-example";
static const std::string redisValue = "unique-redis-value";

class Worker
{
public:
    Worker(boost::asio::io_service &ioService, redisclient::RedisAsyncClient &redisClient)
        : ioService(ioService), redisClient(redisClient)
    {}

    void onConnect(bool connected, const std::string &errorMessage);
    void onSet(const redisclient::RedisValue &value);
    void onGet(const redisclient::RedisValue &value);
    void onDel(const redisclient::RedisValue &value);
    void stop();

private:
    boost::asio::io_service &ioService;
    redisclient::RedisAsyncClient &redisClient;
};

void Worker::onConnect(bool connected, const std::string &errorMessage)
{
    if( !connected )
    {
        std::cerr << "Can't connect to redis: " << errorMessage;
    }
    else
    {
        // two async requests
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));
    }
}

void Worker::onSet(const redisclient::RedisValue &value)
{
    std::cerr << "SET: " << value.toString() << std::endl;
    if( value.toString() == "OK" )
    {
        redisClient.command("GET",  {redisKey},
                            std::bind(&Worker::onGet, this, std::placeholders::_1));
    }
    else
    {
        std::cerr << "Invalid value from redis: " << value.toString() << std::endl;
    }
}

void Worker::onGet(const redisclient::RedisValue &value)
{
    std::cerr << "GET " << value.toString() << std::endl;
    if( value.toString() != redisValue )
    {
        std::cerr << "Invalid value from redis: " << value.toString() << std::endl;
    }

    redisClient.command("DEL", {redisKey},
                        std::bind(&Worker::onDel, this, std::placeholders::_1));
}

void Worker::onDel(const redisclient::RedisValue &value)
{
    std::cerr << "DEL " << value.toString() << std::endl;
    redisClient.command("SET",  {redisKey, redisValue},
                        std::bind(&Worker::onSet, this, std::placeholders::_1));
}


int main(int, char **)
{
    const char *address = "127.0.0.1";
    const int port = 6379;

    boost::asio::io_service ioService;
    redisclient::RedisAsyncClient client(ioService);
    Worker worker(ioService, client);
    boost::asio::ip::tcp::endpoint endpoint(boost::asio::ip::address::from_string(address), port);

    client.asyncConnect(endpoint, std::bind(&Worker::onConnect, &worker,
                std::placeholders::_1, std::placeholders::_2));

    std::vector<std::thread> threads;
    const int num = 8;
    threads.reserve(num);
    for (int i = 0; i != num; ++i) {
        threads.emplace_back([&]{ ioService.run(); });
    }
    for (auto& e : threads) {
        e.join();
    }

    return 0;
}

Compile command:

clang++ -std=c++14 test.cpp -lboost_system -lpthread -O3

Environment:

clang version 5.0.0 (tags/RELEASE_500/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

boost 1.65.1

Linux 4.13.3-1-ARCH #1 SMP PREEMPT Thu Sep 21 20:33:16 CEST 2017 x86_64 GNU/Linux

Can I send async command like redisClient.command("SET", ...) via the same connection from multiple threads?

The text was updated successfully, but these errors were encountered:

redboltz · 2017-10-25T01:50:15Z

It might be related to #21.

redboltz · 2017-10-25T01:55:55Z

I got more useful log for analyze using clang address sanitizer.

Compile command:

clang++ -std=c++14 async_set_get.cpp -I../src/ -lboost_system -lpthread -O0  -g -fsanitize=address -fno-omit-frame-pointer

Log:

SET: OK
SET: OK
GET unique-redis-value
GET unique-redis-value
DEL 
DEL 
SET: =================================================================
OK
==21775==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700070b690 at pc 0x5584fd3ad384 bp 0x7f7427af9d40 sp 0x7f7427af94f0
READ of size 69 at 0x60700070b690 thread T3
    #0 0x5584fd3ad383 in read_iovec(void*, __sanitizer::__sanitizer_iovec*, unsigned long, unsigned long) (/home/kondo/work/redisclient/examples/a.out+0x93383)
    #1 0x5584fd3c4081 in read_msghdr(void*, __sanitizer::__sanitizer_msghdr*, long) (/home/kondo/work/redisclient/examples/a.out+0xaa081)
    #2 0x5584fd3c6a33 in __interceptor_sendmsg (/home/kondo/work/redisclient/examples/a.out+0xaca33)
    #3 0x5584fd45ea67 in boost::asio::detail::socket_ops::send(int, iovec const*, unsigned long, int, boost::system::error_code&) /usr/include/boost/asio/detail/impl/socket_ops.ipp:1170:43
    #4 0x5584fd45d574 in boost::asio::detail::socket_ops::non_blocking_send(int, iovec const*, unsigned long, int, boost::system::error_code&, unsigned long&) /usr/include/boost/asio/detail/impl/socket_ops.ipp:1244:30
    #5 0x5584fd45cdbe in boost::asio::detail::reactive_socket_send_op_base<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > > >::do_perform(boost::asio::detail::reactor_op*) /usr/include/boost/asio/detail/reactive_socket_send_op.hpp:54:12
    #6 0x5584fd461175 in boost::asio::detail::reactor_op::perform() /usr/include/boost/asio/detail/reactor_op.hpp:40:12
    #7 0x5584fd45fceb in boost::asio::detail::epoll_reactor::start_op(int, int, boost::asio::detail::epoll_reactor::descriptor_state*&, boost::asio::detail::reactor_op*, bool, bool) /usr/include/boost/asio/detail/impl/epoll_reactor.ipp:242:15
    #8 0x5584fd45ba95 in boost::asio::detail::reactive_socket_service_base::start_op(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, int, boost::asio::detail::reactor_op*, bool, bool, bool) /usr/include/boost/asio/detail/impl/reactive_socket_service_base.ipp:221:16
    #9 0x5584fd45a721 in void boost::asio::detail::reactive_socket_service_base::async_send<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > >, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > > const&, int, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >&) /usr/include/boost/asio/detail/reactive_socket_service_base.hpp:216:5
    #10 0x5584fd45a0c2 in boost::asio::async_result<boost::asio::handler_type<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::stream_socket_service<boost::asio::generic::stream_protocol>::async_send<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > >, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >(boost::asio::detail::reactive_socket_service<boost::asio::generic::stream_protocol>::implementation_type&, boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > > const&, int, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >&&) /usr/include/boost/asio/stream_socket_service.hpp:334:19
    #11 0x5584fd4591bc in boost::asio::async_result<boost::asio::handler_type<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >::async_write_some<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > >, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >(boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > > const&, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >&&) /usr/include/boost/asio/basic_stream_socket.hpp:731:32
    #12 0x5584fd456967 in boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >::operator()(boost::system::error_code const&, unsigned long, int) /usr/include/boost/asio/impl/write.hpp:181:19
    #13 0x5584fd453adc in boost::asio::async_result<boost::asio::handler_type<std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::async_write<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >(boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >&, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > const&, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>&&) /usr/include/boost/asio/impl/write.hpp:621:3
    #14 0x5584fd44f905 in redisclient::RedisClientImpl::asyncWrite(boost::system::error_code const&, unsigned long) /home/kondo/work/redisclient/examples/../src/redisclient/impl/redisclientimpl.cpp:193:9
    #15 0x5584fd462274 in void std::__invoke_impl<void, void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>(std::__invoke_memfun_deref, void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:73:14
    #16 0x5584fd46202b in std::__invoke_result<void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>::type std::__invoke<void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>(void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:95:14
    #17 0x5584fd461ec4 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::__call<void, boost::system::error_code const&, unsigned long const&, 0ul, 1ul, 2ul>(std::tuple<boost::system::error_code const&, unsigned long const&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:467:11
    #18 0x5584fd459e19 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::operator()<boost::system::error_code const&, unsigned long const&, void>(boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:549:17
    #19 0x5584fd456b5b in boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >::operator()(boost::system::error_code const&, unsigned long, int) /usr/include/boost/asio/impl/write.hpp:192:9
    #20 0x5584fd45cb4f in boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>::operator()() /usr/include/boost/asio/detail/bind_handler.hpp:127:5
    #21 0x5584fd45cae4 in void boost::asio::asio_handler_invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, ...) /usr/include/boost/asio/handler_invoke_hook.hpp:69:3
    #22 0x5584fd45cac1 in void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>&) /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37:3
    #23 0x5584fd45ca80 in void boost::asio::detail::asio_handler_invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >*) /usr/include/boost/asio/impl/write.hpp:565:5
    #24 0x5584fd45c91f in void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >&) /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37:3
    #25 0x5584fd45c31d in boost::asio::detail::reactive_socket_send_op<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > >, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/reactive_socket_send_op.hpp:107:7
    #26 0x5584fd47fa07 in boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operation.hpp:38:5
    #27 0x5584fd4cb9f9 in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:372:12
    #28 0x5584fd4cac01 in boost::asio::detail::task_io_service::run(boost::system::error_code&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:149:10
    #29 0x5584fd4ca8d3 in boost::asio::io_service::run() /usr/include/boost/asio/impl/io_service.ipp:59:25
    #30 0x5584fd43e80c in main::$_0::operator()() const /home/kondo/work/redisclient/examples/async_set_get.cpp:96:45
    #31 0x5584fd43e7bc in void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:60:14
    #32 0x5584fd43e74c in std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:95:14
    #33 0x5584fd43e724 in _ZNSt6thread8_InvokerISt5tupleIJZ4mainE3$_0EEE9_M_invokeIJLm0EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:234:13
    #34 0x5584fd43e6f4 in std::thread::_Invoker<std::tuple<main::$_0> >::operator()() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:243:11
    #35 0x5584fd43e5c8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:186:13
    #36 0x7f742c359a6e in execute_native_thread_routine /build/gcc-multilib/src/gcc/libstdc++-v3/src/c++11/thread.cc:83
    #37 0x7f742c62e089 in start_thread (/usr/lib/libpthread.so.0+0x7089)
    #38 0x7f742b67024e in __GI___clone (/usr/lib/libc.so.6+0xf624e)

0x60700070b690 is located 0 bytes inside of 69-byte region [0x60700070b690,0x60700070b6d5)
freed by thread T7 here:
    #0 0x5584fd435449 in operator delete(void*) (/home/kondo/work/redisclient/examples/a.out+0x11b449)
    #1 0x5584fd44aa7c in __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/ext/new_allocator.h:125:2
    #2 0x5584fd44aa4f in std::allocator_traits<std::allocator<char> >::deallocate(std::allocator<char>&, char*, unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/alloc_traits.h:462:13
    #3 0x5584fd44aa0a in std::_Vector_base<char, std::allocator<char> >::_M_deallocate(char*, unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:180:4
    #4 0x5584fd449503 in std::_Vector_base<char, std::allocator<char> >::~_Vector_base() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:162:9
    #5 0x5584fd445640 in std::vector<char, std::allocator<char> >::~vector() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:435:33
    #6 0x5584fd4549e4 in void std::_Destroy<std::vector<char, std::allocator<char> > >(std::vector<char, std::allocator<char> >*) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_construct.h:98:19
    #7 0x5584fd4549ae in void std::_Destroy_aux<false>::__destroy<std::vector<char, std::allocator<char> >*>(std::vector<char, std::allocator<char> >*, std::vector<char, std::allocator<char> >*) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_construct.h:108:6
    #8 0x5584fd45496c in void std::_Destroy<std::vector<char, std::allocator<char> >*>(std::vector<char, std::allocator<char> >*, std::vector<char, std::allocator<char> >*) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_construct.h:136:7
    #9 0x5584fd454900 in void std::_Destroy<std::vector<char, std::allocator<char> >*, std::vector<char, std::allocator<char> > >(std::vector<char, std::allocator<char> >*, std::vector<char, std::allocator<char> >*, std::allocator<std::vector<char, std::allocator<char> > >&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_construct.h:206:7
    #10 0x5584fd4548b8 in std::deque<std::vector<char, std::allocator<char> >, std::allocator<std::vector<char, std::allocator<char> > > >::_M_destroy_data_aux(std::_Deque_iterator<std::vector<char, std::allocator<char> >, std::vector<char, std::allocator<char> >&, std::vector<char, std::allocator<char> >*>, std::_Deque_iterator<std::vector<char, std::allocator<char> >, std::vector<char, std::allocator<char> >&, std::vector<char, std::allocator<char> >*>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/deque.tcc:861:3
    #11 0x5584fd45421c in std::deque<std::vector<char, std::allocator<char> >, std::allocator<std::vector<char, std::allocator<char> > > >::_M_destroy_data(std::_Deque_iterator<std::vector<char, std::allocator<char> >, std::vector<char, std::allocator<char> >&, std::vector<char, std::allocator<char> >*>, std::_Deque_iterator<std::vector<char, std::allocator<char> >, std::vector<char, std::allocator<char> >&, std::vector<char, std::allocator<char> >*>, std::allocator<std::vector<char, std::allocator<char> > > const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_deque.h:2072:4
    #12 0x5584fd453f79 in std::deque<std::vector<char, std::allocator<char> >, std::allocator<std::vector<char, std::allocator<char> > > >::_M_erase_at_end(std::_Deque_iterator<std::vector<char, std::allocator<char> >, std::vector<char, std::allocator<char> >&, std::vector<char, std::allocator<char> >*>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_deque.h:2089:2
    #13 0x5584fd45338b in std::deque<std::vector<char, std::allocator<char> >, std::allocator<std::vector<char, std::allocator<char> > > >::clear() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_deque.h:1841:9
    #14 0x5584fd44f559 in redisclient::RedisClientImpl::asyncWrite(boost::system::error_code const&, unsigned long) /home/kondo/work/redisclient/examples/../src/redisclient/impl/redisclientimpl.cpp:174:16
    #15 0x5584fd462274 in void std::__invoke_impl<void, void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>(std::__invoke_memfun_deref, void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:73:14
    #16 0x5584fd46202b in std::__invoke_result<void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>::type std::__invoke<void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&>(void (redisclient::RedisClientImpl::*&)(boost::system::error_code const&, unsigned long), std::shared_ptr<redisclient::RedisClientImpl>&, boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:95:14
    #17 0x5584fd461ec4 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::__call<void, boost::system::error_code const&, unsigned long const&, 0ul, 1ul, 2ul>(std::tuple<boost::system::error_code const&, unsigned long const&>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:467:11
    #18 0x5584fd459e19 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>::operator()<boost::system::error_code const&, unsigned long const&, void>(boost::system::error_code const&, unsigned long const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:549:17
    #19 0x5584fd456b5b in boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >::operator()(boost::system::error_code const&, unsigned long, int) /usr/include/boost/asio/impl/write.hpp:192:9
    #20 0x5584fd45cb4f in boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>::operator()() /usr/include/boost/asio/detail/bind_handler.hpp:127:5
    #21 0x5584fd45cae4 in void boost::asio::asio_handler_invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, ...) /usr/include/boost/asio/handler_invoke_hook.hpp:69:3
    #22 0x5584fd45cac1 in void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)>&) /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37:3
    #23 0x5584fd45ca80 in void boost::asio::detail::asio_handler_invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >*) /usr/include/boost/asio/impl/write.hpp:565:5
    #24 0x5584fd45c91f in void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >(boost::asio::detail::binder2<boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >, boost::system::error_code, unsigned long>&, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> >&) /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37:3
    #25 0x5584fd45c31d in boost::asio::detail::reactive_socket_send_op<boost::asio::detail::consuming_buffers<boost::asio::const_buffer, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> > >, boost::asio::detail::write_op<boost::asio::basic_stream_socket<boost::asio::generic::stream_protocol, boost::asio::stream_socket_service<boost::asio::generic::stream_protocol> >, std::vector<boost::asio::const_buffer, std::allocator<boost::asio::const_buffer> >, boost::asio::detail::transfer_all_t, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::_Placeholder<1>, std::_Placeholder<2>))(boost::system::error_code const&, unsigned long)> > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/reactive_socket_send_op.hpp:107:7
    #26 0x5584fd47fa07 in boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operation.hpp:38:5
    #27 0x5584fd4cb9f9 in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:372:12
    #28 0x5584fd4cac01 in boost::asio::detail::task_io_service::run(boost::system::error_code&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:149:10
    #29 0x5584fd4ca8d3 in boost::asio::io_service::run() /usr/include/boost/asio/impl/io_service.ipp:59:25

previously allocated by thread T8 here:
    #0 0x5584fd4346b1 in operator new(unsigned long) (/home/kondo/work/redisclient/examples/a.out+0x11a6b1)
    #1 0x5584fd44985b in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/ext/new_allocator.h:111:27
    #2 0x5584fd4497fb in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/alloc_traits.h:436:20
    #3 0x5584fd4497a2 in std::_Vector_base<char, std::allocator<char> >::_M_allocate(unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:172:20
    #4 0x5584fd44965e in std::_Vector_base<char, std::allocator<char> >::_M_create_storage(unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:187:33
    #5 0x5584fd448d44 in std::_Vector_base<char, std::allocator<char> >::_Vector_base(unsigned long, std::allocator<char> const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:138:9
    #6 0x5584fd44897b in std::vector<char, std::allocator<char> >::vector(std::vector<char, std::allocator<char> > const&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/stl_vector.h:327:9
    #7 0x5584fd44c7aa in void std::__invoke_impl<void, void (redisclient::RedisClientImpl::*&)(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>), std::shared_ptr<redisclient::RedisClientImpl>&, std::vector<char, std::allocator<char> >&, std::function<void (redisclient::RedisValue)>&>(std::__invoke_memfun_deref, void (redisclient::RedisClientImpl::*&)(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>), std::shared_ptr<redisclient::RedisClientImpl>&, std::vector<char, std::allocator<char> >&, std::function<void (redisclient::RedisValue)>&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:73:47
    #8 0x5584fd44c45b in std::__invoke_result<void (redisclient::RedisClientImpl::*&)(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>), std::shared_ptr<redisclient::RedisClientImpl>&, std::vector<char, std::allocator<char> >&, std::function<void (redisclient::RedisValue)>&>::type std::__invoke<void (redisclient::RedisClientImpl::*&)(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>), std::shared_ptr<redisclient::RedisClientImpl>&, std::vector<char, std::allocator<char> >&, std::function<void (redisclient::RedisValue)>&>(void (redisclient::RedisClientImpl::*&)(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>), std::shared_ptr<redisclient::RedisClientImpl>&, std::vector<char, std::allocator<char> >&, std::function<void (redisclient::RedisValue)>&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:95:14
    #9 0x5584fd44c374 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>::__call<void, , 0ul, 1ul, 2ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:467:11
    #10 0x5584fd44c177 in void std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>::operator()<, void>() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/functional:549:17
    #11 0x5584fd44c074 in void boost::asio::asio_handler_invoke<std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)> >(std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>&, ...) /usr/include/boost/asio/handler_invoke_hook.hpp:69:3
    #12 0x5584fd44bd91 in void boost_asio_handler_invoke_helpers::invoke<std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)> >(std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>&, std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)>&) /usr/include/boost/asio/detail/handler_invoke_helpers.hpp:37:3
    #13 0x5584fd44b7eb in boost::asio::detail::completion_handler<std::_Bind<void (redisclient::RedisClientImpl::*(std::shared_ptr<redisclient::RedisClientImpl>, std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>))(std::vector<char, std::allocator<char> >, std::function<void (redisclient::RedisValue)>)> >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/completion_handler.hpp:68:7
    #14 0x5584fd47fa07 in boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operation.hpp:38:5
    #15 0x5584fd47f738 in boost::asio::detail::strand_service::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/impl/strand_service.ipp:167:10
    #16 0x5584fd47fa07 in boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) /usr/include/boost/asio/detail/task_io_service_operation.hpp:38:5
    #17 0x5584fd4cb9f9 in boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:372:12
    #18 0x5584fd4cac01 in boost::asio::detail::task_io_service::run(boost::system::error_code&) /usr/include/boost/asio/detail/impl/task_io_service.ipp:149:10
    #19 0x5584fd4ca8d3 in boost::asio::io_service::run() /usr/include/boost/asio/impl/io_service.ipp:59:25
    #20 0x5584fd43e80c in main::$_0::operator()() const /home/kondo/work/redisclient/examples/async_set_get.cpp:96:45
    #21 0x5584fd43e7bc in void std::__invoke_impl<void, main::$_0>(std::__invoke_other, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:60:14
    #22 0x5584fd43e74c in std::__invoke_result<main::$_0>::type std::__invoke<main::$_0>(main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/invoke.h:95:14
    #23 0x5584fd43e724 in _ZNSt6thread8_InvokerISt5tupleIJZ4mainE3$_0EEE9_M_invokeIJLm0EEEEDTclsr3stdE8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:234:13
    #24 0x5584fd43e6f4 in std::thread::_Invoker<std::tuple<main::$_0> >::operator()() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:243:11
    #25 0x5584fd43e5c8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<main::$_0> > >::_M_run() /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/thread:186:13
    #26 0x7f742c359a6e in execute_native_thread_routine /build/gcc-multilib/src/gcc/libstdc++-v3/src/c++11/thread.cc:83

Thread T3 created by T0 here:
    #0 0x5584fd355d62 in pthread_create (/home/kondo/work/redisclient/examples/a.out+0x3bd62)
    #1 0x7f742c359d65 in __gthread_create /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:662
    #2 0x7f742c359d65 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /build/gcc-multilib/src/gcc/libstdc++-v3/src/c++11/thread.cc:163
    #3 0x5584fd43e140 in void __gnu_cxx::new_allocator<std::thread>::construct<std::thread, main::$_0>(std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/ext/new_allocator.h:136:23
    #4 0x5584fd43d87c in void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0>(std::allocator<std::thread>&, std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/alloc_traits.h:475:8
    #5 0x5584fd43ca06 in void std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0>(main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/vector.tcc:100:6
    #6 0x5584fd43c471 in main /home/kondo/work/redisclient/examples/async_set_get.cpp:96:17
    #7 0x7f742b59af69 in __libc_start_main (/usr/lib/libc.so.6+0x20f69)

Thread T7 created by T0 here:
    #0 0x5584fd355d62 in pthread_create (/home/kondo/work/redisclient/examples/a.out+0x3bd62)
    #1 0x7f742c359d65 in __gthread_create /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:662
    #2 0x7f742c359d65 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /build/gcc-multilib/src/gcc/libstdc++-v3/src/c++11/thread.cc:163
    #3 0x5584fd43e140 in void __gnu_cxx::new_allocator<std::thread>::construct<std::thread, main::$_0>(std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/ext/new_allocator.h:136:23
    #4 0x5584fd43d87c in void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0>(std::allocator<std::thread>&, std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/alloc_traits.h:475:8
    #5 0x5584fd43ca06 in void std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0>(main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/vector.tcc:100:6
    #6 0x5584fd43c471 in main /home/kondo/work/redisclient/examples/async_set_get.cpp:96:17
    #7 0x7f742b59af69 in __libc_start_main (/usr/lib/libc.so.6+0x20f69)

Thread T8 created by T0 here:
    #0 0x5584fd355d62 in pthread_create (/home/kondo/work/redisclient/examples/a.out+0x3bd62)
    #1 0x7f742c359d65 in __gthread_create /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/x86_64-pc-linux-gnu/bits/gthr-default.h:662
    #2 0x7f742c359d65 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) /build/gcc-multilib/src/gcc/libstdc++-v3/src/c++11/thread.cc:163
    #3 0x5584fd43e140 in void __gnu_cxx::new_allocator<std::thread>::construct<std::thread, main::$_0>(std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/ext/new_allocator.h:136:23
    #4 0x5584fd43d87c in void std::allocator_traits<std::allocator<std::thread> >::construct<std::thread, main::$_0>(std::allocator<std::thread>&, std::thread*, main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/alloc_traits.h:475:8
    #5 0x5584fd43ca06 in void std::vector<std::thread, std::allocator<std::thread> >::emplace_back<main::$_0>(main::$_0&&) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../include/c++/7.2.0/bits/vector.tcc:100:6
    #6 0x5584fd43c471 in main /home/kondo/work/redisclient/examples/async_set_get.cpp:96:17
    #7 0x7f742b59af69 in __libc_start_main (/usr/lib/libc.so.6+0x20f69)

SUMMARY: AddressSanitizer: heap-use-after-free (/home/kondo/work/redisclient/examples/a.out+0x93383) in read_iovec(void*, __sanitizer::__sanitizer_iovec*, unsigned long, unsigned long)
Shadow bytes around the buggy address:
  0x0c0e800d9680: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
  0x0c0e800d9690: fd fd fd fd fd fa fa fa fa fa fd fd fd fd fd fd
  0x0c0e800d96a0: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c0e800d96b0: fd fa fa fa fa fa fd fd fd fd fd fd fd fd fd fa
  0x0c0e800d96c0: fa fa fa fa fd fd fd fd fd fd fd fd fd fa fa fa
=>0x0c0e800d96d0: fa fa[fd]fd fd fd fd fd fd fd fd fa fa fa fa fa
  0x0c0e800d96e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e800d96f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e800d9700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e800d9710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e800d9720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==21775==ABORTING

redboltz · 2017-10-25T02:08:03Z

I guess that it is the following issue.
https://stackoverflow.com/questions/12794107/why-do-i-need-strand-per-connection-when-using-boostasio/12801042?noredirect=1#comment79777780_12801042

Should I serialize command sending to the redis connection?

And can I call multiple async command concurrently? Or I can call async command only from its callback function?

Boost.Asio async_write has the limitation:

This operation is implemented in terms of zero or more calls to the stream's
async_write_some function, and is known as a composed operation. 
The program must ensure that the stream performs no other write operations
(such as async_write, the stream's async_write_some function, or any other
composed operations that perform writes) until this operation completes.

http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/async_write/overload1.html

And one solution:
https://stackoverflow.com/questions/7754695/boost-asio-async-write-how-to-not-interleaving-async-write-calls

redboltz · 2017-10-25T02:15:46Z

I found the answer for thread safety:

#34 (comment)

Is it thread safe? Can I use the RedisSyncClient with one global connection? Or should I go for RedisAsyncClient?

Is not thread-safe. Use locks, connection per thread or special thread.

I will use lock or one special thread for the redis connection.

So rest of my question is "can I call multiple async command concurrently? Or I can call async command only from its callback function?"

like this:

        // two async requests
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));
        redisClient.command("SET",  {redisKey, redisValue},
                            std::bind(&Worker::onSet, this, std::placeholders::_1));

nekipelov · 2017-10-25T10:55:52Z

So rest of my question is "can I call multiple async command concurrently? Or I can call async command only from its callback function?"

Yes you can multiple async call. RedisClient serializes write operations in internal queue.

redboltz · 2017-10-25T11:18:47Z

Thank you! It's easy to use :)

redboltz closed this as completed Oct 25, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi thread io_service::run() SET GET DEL #48

Multi thread io_service::run() SET GET DEL #48

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017 •

edited

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017

nekipelov commented Oct 25, 2017

redboltz commented Oct 25, 2017

Multi thread io_service::run() SET GET DEL #48

Multi thread io_service::run() SET GET DEL #48

Comments

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017 • edited

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017

nekipelov commented Oct 25, 2017

redboltz commented Oct 25, 2017

redboltz commented Oct 25, 2017 •

edited