intermediate log services working directly on flexbuffer map

32 Views Asked by At

We have a custom log pipeline and currently, each step/service parses incoming json and modifies/adds tags. I was exploring options to reduce the cost of this serialization/de-serialization since some of the services don't modify data and just need to read/route the logs.

I was trying out flexbuffers (since the logs are from different services and don't share same schema).

I was thinking convert to flexbuffers::to_vec() and send this vector to next service. The downstream services can deserialize into other formats using flexbuffers::from_slice(). But I'm looking for ways where this vector can be directly used as flexbuffer::MapReader instead of going through intermediate steps.

fn test_flexbuffer_map() {
        let log = serde_json::json!({"time": 123, "level": "INFO", "extra" : {"message" : "Hello! World"}});
        let vec_fb = flexbuffers::to_vec(&log).unwrap();

        let map_fb = ??;
        let level = map_fb.idx("level"); // want to access fields directly
    }

-- Update --

Not sure if this is correct way but this works.

        let u8_fb: &[u8] = &vec_fb;
        let map_fb = flexbuffers::Reader::get_root(u8_fb).unwrap().as_map();
        let level = map_fb.idx("level").as_str();
        println!("{level}");
0

There are 0 best solutions below