I'm working on a serialization library for an embedded target. The library is based on a classic visitor pattern with a templated pack() method implemented in each struct that I want to serialize. Example:
struct softwareInfo
{
uint32_t version;
std::array<uint8_t, 40> commitHash;
uint32_t buildDate;
template <class T>
void pack(T& archive)
{
archive.process(HVP(version));
archive.process(HVP(commitHash));
archive.process(HVP(buildDate));
}
};
HVP macro packs the field with it's hashed name into a hashValuePair class that is used later on.
constexpr uint32_t compileTimeHash(const char* str, size_t index = 0, uint32_t hash = 0)
{
return (str[index] == '\0') ? hash : compileTimeHash(str, index + 1, hash * 31 + str[index]);
}
template <typename T>
class hashValuePair
{
public:
hashValuePair(uint32_t hash, T* value) : hash(hash), value(value) {}
uint32_t getHash() const { return hash; }
T& getValueRef() { return *value; }
private:
uint32_t hash = 0;
T* const value = nullptr;
};
#define HVP(var) hashValuePair(compileTimeHash(#var), &var)
There are two more modules:
- The PersistentStorage class that is used to register objects that can be serialized. It holds the following buffer:
std::array<StorageEntry, maxEntries> entries
where:
struct StorageEntry
{
std::aligned_storage_t<sizeof(Entry<void>), alignof(Entry<void>)> buffer;
EntryBase* object;
struct Header
{
uint32_t hashName;
uint32_t size;
} header;
uint32_t datacrc32;
Status packStatus;
Status unpackStatus;
};
class EntryBase
{
public:
virtual ~EntryBase() {}
virtual void pack(Packer::Packer& packer) = 0;
virtual void pack(Packer::Unpacker& unpacker) = 0;
};
template <typename T>
class Entry : public EntryBase
{
public:
explicit Entry(T* item) : item(item) {}
void pack(Packer::Packer& packer) override { item->pack(std::forward<Packer::Packer&>(packer)); }
void pack(Packer::Unpacker& unpacker) override { item->pack(std::forward<Packer::Unpacker&>(unpacker)); }
private:
T* const item;
};
It has two main methods - doRead() and doWrite() that take all the registered structs and packs/unpacks them one by one using the pack and unpack methods shown below.
- The second module is a pair of Packer/Unpacker classes that is used to serialize and deserialize structs by visiting every member listed using the process method.
namespace Packer
{
class Packer
{
public:
template <class T>
void process(hashValuePair<T>&& hashValuePair)
{
packType(hashValuePair.getValueRef());
}
private:
template <class T>
void packType(T& value)
{
(...)
copyToBuf(value);
}
template <class T>
void copyToBuf(T value)
{
it += serialize<T>(value, it);
}
};
class Unpacker
{
public:
template <class T>
void process(hashValuePair<T>&& hashValuePair)
{
//search for value hash
unpackType(hashValuePair.getValueRef());
}
private:
template <class T>
void unpackType(T& value)
{
(...)
tryUnpack(*it++, value);
}
template <typename T>
void tryUnpack(uint8_t maybeType, T& value)
{
if (maybeType != getType(value))
//error
copyToValue(value);
}
template <typename T>
void copyToValue(T& value)
{
value = deserialize<T>(it);
it += sizeof(value);
}
};
template <class PackableObject>
Status pack(PackableObject& obj, std::span<uint8_t>& buf, size_t& packedSize)
{
Status status{};
auto packer = Packer(buf, packedSize, &status);
obj.pack(packer);
return status;
}
template <class PackableObject>
Status unpack(PackableObject& obj, std::span<uint8_t>& buf)
{
Status status{};
auto unpacker = Unpacker(buf, &status);
obj.pack(unpacker);
return status;
}
}
I intentionally omitted the rest of the implementation (error handling, searching for field hashes) to leave only the most basic flow.
Now, what I can see is that there is a relatively large memory footprint associated with the implementation provided above. I can imagine the templated methods have to be instantiated with types that are used, however i'm not sure I'm doing everything in the most optimal way. For example serializing a softwareInfo struct uses 464B of FLASH memory Packer FLASH footprint divided to functions. When I add another uint32_t field I get a 44B increase in size which feels quite large, especially considering I had already specialized this type with previous uint32_t field Packer FLASH footprint divided to functions with additional uint32_t field. The build type is release with -Os.
Can you see any obvious flaws in the current design? Could something like std::variant help in this situation, since its runtime polymorphism?