correct use CapnProto

[angelskieglazki]

Hi all. i want to use capnproto as serialization library. Am I using the library correctly? or it can be done more optimally and correctly.
(I have to build into the my current message packaging system. so I only need the size in bytes and the bytes themselves. This Blob - my class)

size_t EncodeTo( Blob& dest, const BaseCommand& arg) {
  ::capnp::MallocMessageBuilder message;
  Base::Builder base = message.initRoot<Base>();

  base.setCapBaseField0(arg.BaseField0);
  base.setCapBaseField1(enums::BASE_ENUM(arg.BaseField1));

  kj::Array<capnp::word> words = messageToFlatArray(message);
  kj::ArrayPtr<kj::byte> bytes = words.asBytes();
  dest.appendBytes((uint32_t)bytes.size());
  std::vector<uint8_t> v(bytes.begin(), bytes.end());
  dest.appendBytes(v);

  return bytes.size() + sizeof(uint32_t);
}

size_t DecodeTo(const Blob& src, Blob::Pos& pos, BaseCommand& arg) {
  Blob::Pos innerPos = pos;
  uint32_t payloadSize = src.fetchBytes<uint32_t>(innerPos);
  std::vector<uint8_t> payload(payloadSize);
  payload = src.fetchBytes<std::vector<uint8_t>>(innerPos, payloadSize);

  kj::ArrayBuilder<uint8_t> builder = kj::heapArrayBuilder<uint8_t>(payloadSize);
  builder.addAll(payload);
  auto array = builder.finish();
  capnp::FlatArrayMessageReader arr_reader(kj::arrayPtr(
          reinterpret_cast<const capnp::word*>(array.begin()),
          reinterpret_cast<const capnp::word*>(array.end())));
  auto reader = arr_reader.getRoot<worldshards::Base>();
  arg.BaseField0 = reader.getCapBaseField0();
  arg.BaseField1 = (enums::BASE_ENUM)reader.getCapBaseField1();

  pos = innerPos;
  return payloadSize + sizeof(payloadSize);
}

regards, max

[kentonv]

  kj::Array<capnp::word> words = messageToFlatArray(message);
  kj::ArrayPtr<kj::byte> bytes = words.asBytes();
  dest.appendBytes((uint32_t)bytes.size());
  std::vector<uint8_t> v(bytes.begin(), bytes.end());
  dest.appendBytes(v);

This looks like it's making at least three copies of the message content: messageToFlatArray makes one, constructing a vector is a second, and appendBytes() is a third. This could be more efficient if you implement a subclass of kj::OutputStream which writes directly to your Blob, then use capnp::writeMessage(message, stream).

Similarly on the read side you are reading into a std::vector first, and then copying that into a kj::Array. The best would be if you could get a pointer directly to the bytes inside the Blob to use directly, or failing that, only make one copy into kj::Array<capnp::word>.

[angelskieglazki]

for reading - I figured out with my colleague how to do it without unnecessary allocations. just like you described.

for writing - I will try implement a subclass of kj::OutputStream. Thank you so much 🤗