i am using flatbuffers and nng. using FB i am creating seralized buffers and adding it to nng msg. I wanted to eliminate this copying of FB to NNG.
To achieve this i wrote a custom allocator as below.
class CustomAllocator : public flatbuffers::Allocator
{
private:
nng_msg *m_nng_msg = nullptr;
public:
~CustomAllocator()
{
if (m_nng_msg) {
nng_msg_free(m_nng_msg);
m_nng_msg = nullptr;
}
}
nng_msg *get() { return m_nng_msg; }
nng_msg *release()
{
nng_msg *t_nng_msg = m_nng_msg;
m_nng_msg = nullptr;
return t_nng_msg;
}
uint8_t *allocate(size_t size) override
{
if (nng_msg_alloc(&m_nng_msg, size) != 0) {
return nullptr;
}
return static_cast<uint8_t *>(nng_msg_body(m_nng_msg));
}
void deallocate(uint8_t *p, size_t size) override
{
if (m_nng_msg) {
nng_msg_free(m_nng_msg);
m_nng_msg = nullptr;
}
}
};
and then attaching it to FB as below
CustomAllocator allocator;
flatbuffers::FlatBufferBuilder builder(1024, &allocator);
... fill using builder
nng_msg *msg = allocator.get();
nng_msg_trim(msg, nng_msg_len(msg) - builder.GetSize()); // this is required as FB adds data to the end of the buffer, i checked the *_trim and it does not do realloc
nng_msg_set_pipe(msg, pipe);
nng_sendmsg(s, msg, 0);
for the case of double copy i use it like this
flatbuffers::FlatBufferBuilder builder(1024);
... fill using builder
nng_msg *msg = nullptr;
nng_msg_alloc(&msg, 0);
nng_msg_append(msg, builder.GetBufferPointer(), builder.GetSize());
nng_msg_set_pipe(msg, pipe);
nng_sendmsg(s, msg, 0);
My expectation was that no-copy would be be slightly better than double copy. But when i ran tests i am getting different results.
So i am wondering what could be the cause of this behavior?
each scenario was run 20 times to get the avg times.
Message Size 180 bytes
Message Count 20000 80000 100000
no-copy copy no-copy copy no-copy copy
time in ms 198 185 809 784 997 950
std-dev 47 44 85 66 72 83