Cap'n Proto unaligned data error while trying to send&receive serialized object over ZeroMQ pub-sub socket (C++)

Question

I am using Cap'n Proto with ZeroMQ for interprocess communication in my C++ project. I have a sender and a receiver, and I am sending a Cap'n Proto message from the sender to the receiver via ZeroMQ.

I am getting the following error when I try to receive and deserialize the message at the receiver:

*Bprepheading
terminate called after throwing an instance of 'kj::ExceptionImpl'
  what():  capnp/arena.c++:76: failed: expected reinterpret_cast<uintptr_t>(segment.begin()) % sizeof(void*) == 0 [2 == 0]; Detected unaligned data in Cap'n Proto message. Messages must be aligned to the architecture's word size. Yes, even on x86: Unaligned access is undefined behavior under the C/C++ language standard, and compilers can and do assume alignment for the purpose of optimizations. Unaligned access may lead to crashes or subtle corruption. For example, GCC will use SIMD instructions in optimizations, and those instrsuctions require alignment. If you really insist on taking your changes with unaligned data, compile the Cap'n Proto library with -DCAPNP_ALLOW_UNALIGNED to remove this check.
stack: 7fc92d717865 7fc92d7178b2 7fc92d734def 55d19f2b114c 55d19f2b0528 7fc92d347d09 55d19f2b0299
Aborted

I assume i need to properly align the data on the receiving end, but i do not know how to do it.

My code for Sender.cpp

#include "../schema/message.capnp.h"
  
  #include <capnp/message.h>
  #include <capnp/serialize.h>
  #include <kj/std/iostream.h>
  #include <zmq.hpp>

int main() {
  
  // create a message builder
  capnp::MallocMessageBuilder message;
  Message::Builder messageBuilder = message.initRoot<Message>();

  // set the preparation and heading fields
  messageBuilder.setPreparation("prep");
  messageBuilder.setHeading("heading");

  zmq::context_t context(1);
  zmq::socket_t socket(context, ZMQ_PUB);

  socket.bind("tcp://*:5006");

  // serialize the message to a memory buffer
  kj::Array<capnp::word> serialized_message = capnp::messageToFlatArray(message);  
  
  while(true)
  {
    // create a ZeroMQ message from the serialized buffer
    zmq::message_t zmq_message(serialized_message.size() * sizeof(capnp::word));
    memcpy((void*)zmq_message.data(), serialized_message.begin(), serialized_message.size() * sizeof(capnp::word));
    socket.send(zmq_message);
  }

  return 0;
}

My code for Receiver.cpp

#include "../schema/message.capnp.h"

#include <kj/array.h>
#include <capnp/message.h>
#include <capnp/serialize.h>
#include <kj/std/iostream.h>
#include <zmq.hpp>

int main() {
  
zmq::context_t context(1);
zmq::socket_t socket(context, ZMQ_SUB);

socket.connect("tcp://127.0.0.1:5006"); //or *
// socket.set(zmq::sockopt::subscribe, "");
socket.setsockopt(ZMQ_SUBSCRIBE, "", 0);

zmq::message_t zmq_message;
socket.recv(zmq_message, zmq::recv_flags::none);
std::cout << zmq_message.to_string() << std::endl;

// create a memory buffer from the received message
kj::ArrayPtr<capnp::word> buffer(reinterpret_cast <capnp::word*>(zmq_message.data()), zmq_message.size() / sizeof(capnp::word));

// create an input stream from the memory buffer
capnp::FlatArrayMessageReader message_reader(buffer); 
Message::Reader message = message_reader.getRoot<Message>();

// print the preparation and heading fields
std::cout << "Preparation: " << message.getPreparation().cStr() << std::endl;
std::cout << "Heading: " << message.getHeading().cStr() << std::endl;

return 0;
}

My cap'n proto schema language Message.capnp

@0xbf5147cbbecf40c1;

struct Message {
  preparation @0 :Text;
  heading @1 :Text;
}

Answer 1

Unfortunately, it appears the buffer coming from ZeroMQ is not aligned on a word boundary. Therefore, you will have to make a copy. You can do that with code like:

auto buffer = kj::heapArray<capnp::word>(zmq_message.size() / sizeof(capnp::word));
memcpy(buffer.asBytes().begin(), zmq_message.data(), buffer.asBytes().size());

It's annoying that you lose the zero-copy benefit of Cap'n Proto this way. However, a memcpy is still a lot faster than parsing would normally be. Unfortunately there is no way around this -- even though many CPUs are OK with unaligned reads, the C++ language standard insists that they are undefined behavior, and as a result compilers may perform optimizations that assume reads are aligned. For example, they might assume the lower bits of a pointer are zero. So it turns out you really do need alignment.