why is flatbuffers and nng no copy is slower that double copy

514 Views Asked by At

i am using flatbuffers and nng. using FB i am creating seralized buffers and adding it to nng msg. I wanted to eliminate this copying of FB to NNG.

To achieve this i wrote a custom allocator as below.

class CustomAllocator : public flatbuffers::Allocator
{
private:
    nng_msg *m_nng_msg = nullptr;

public:

    ~CustomAllocator()
    {
        if (m_nng_msg) {
            nng_msg_free(m_nng_msg);
            m_nng_msg = nullptr;
        }
    }

    nng_msg *get() { return m_nng_msg; }

    nng_msg *release()
    {
        nng_msg *t_nng_msg = m_nng_msg;
        m_nng_msg = nullptr;
        return t_nng_msg;
    }

    uint8_t *allocate(size_t size) override
    {
        if (nng_msg_alloc(&m_nng_msg, size) != 0) {
            return nullptr;
        }
        return static_cast<uint8_t *>(nng_msg_body(m_nng_msg));
    }

    void deallocate(uint8_t *p, size_t size) override
    {
        if (m_nng_msg) {
            nng_msg_free(m_nng_msg);
            m_nng_msg = nullptr;
        }
    }
};

and then attaching it to FB as below

CustomAllocator allocator;
flatbuffers::FlatBufferBuilder builder(1024, &allocator);
... fill using builder

nng_msg *msg = allocator.get();
nng_msg_trim(msg, nng_msg_len(msg) - builder.GetSize()); // this is required as FB adds data to the end of the buffer, i checked the *_trim and it does not do realloc
nng_msg_set_pipe(msg, pipe);
nng_sendmsg(s, msg, 0);

for the case of double copy i use it like this

flatbuffers::FlatBufferBuilder builder(1024);
... fill using builder
nng_msg *msg = nullptr;
nng_msg_alloc(&msg, 0);
nng_msg_append(msg, builder.GetBufferPointer(), builder.GetSize());
nng_msg_set_pipe(msg, pipe);
nng_sendmsg(s, msg, 0);

My expectation was that no-copy would be be slightly better than double copy. But when i ran tests i am getting different results.

So i am wondering what could be the cause of this behavior?

each scenario was run 20 times to get the avg times.

Message Size      180 bytes
Message Count     20000             80000             100000
                  no-copy  copy     no-copy  copy     no-copy  copy
time in ms        198      185      809      784      997      950
std-dev            47       44       85       66       72       83
0

There are 0 best solutions below