Js binding for large rust object using wasm-bindgen

Question

I want to write a vscode extension that displays the content of a large binary file, written with bincode:

#[macro_use]
extern crate serde_derive;

use std::collections::HashMap;
use std::fs::File;
use std::io::{BufReader, BufWriter};

#[derive(Serialize, Deserialize)]
pub struct MyValue {
    pub name: String,
}

#[derive(Serialize, Deserialize)]
pub struct MyStruct {
    pub data: HashMap<String, MyValue>,
}

impl MyStruct {
    pub fn dump(&self, filename: &str) -> Result<(), String> {
        let file = File::create(filename).map_err(|msg| msg.to_string())?;
        let writer = BufWriter::new(file);
        bincode::serialize_into(writer, self).map_err(|msg| msg.to_string())
    }

    pub fn load(filename: &str) -> Result<Self, String> {
        let file = File::open(filename).map_err(|msg| msg.to_string())?;
        let reader = BufReader::new(file);
        bincode::deserialize_from::<BufReader<_>, Self>(reader).map_err(|msg| msg.to_string())
    }
}

Therefore there is a wasm binding:

#[wasm_bindgen]
#[derive(Clone)]
pub struct PyMyStruct {
    inner: Arc<MyStruct>,
}

#[wasm_bindgen]
impl PyMyStruct {
    pub fn new(filename: &str) -> Self {
        Self {
            inner: Arc::new(MyStruct::load(filename).unwrap()),
        }
    }

    pub fn header(self) -> Array {
        let keys = Array::new();
        for key in self.inner.data.keys() {
            keys.push(&JsValue::from_str(key));
        }
        keys
    }

    pub fn value(&self, name: &str) -> JsValue {
        if let Some(value) = self.inner.data.get(name) {
            JsValue::from_serde(value).unwrap_or(JsValue::NULL)
        } else {
            JsValue::NULL
        }
    }
}

which provides a simple interface to the JavaScript world in order to access the content of that file. Using Arc in order to prevent expensive unintended memory copy when handling on the JavaScript side. (It might look strange that keys is not marked as mutable but the rust compiler recomended that way)

When running the test code:

const {PyMyStruct} = require("./corejs.js");

let obj = new PyMyStruct("../../dump.spb")
console.log(obj.header())

you get the error message:

Error: null pointer passed to rust

Does someone know how to handle this use case?

Thank you!

Answer 1

I solved that problem by using https://neon-bindings.com instead of compiling the API to web-assembly.

The binding here looks as follow:

use core;
use std::rc::Rc;
use neon::prelude::*;

#[derive(Clone)]
pub struct MyStruct {
    inner: Rc<core::MyStruct>,
}

declare_types! {
    pub class JsMyStruct for MyStruct {
        init(mut cx) {
            let filename = cx.argument::<JsString>(0)?.value();

            match core::MyStruct::load(&filename) {
                Ok(inner) => return Ok(MyStruct{
                    inner: Rc::new(inner)
                }),
                Err(msg) => {
                    panic!("{}", msg)
                }
            }
        }

        method header(mut cx) {
            let this = cx.this();
            let container = {
                let guard = cx.lock();
                let this = this.borrow(&guard);
                (*this).clone()
            };
            let keys = container.inner.data.keys().collect::<Vec<_>>();
            let js_array = JsArray::new(&mut cx, keys.len() as u32);
            for (i, obj) in keys.into_iter().enumerate() {
                let js_string = cx.string(obj);
                js_array.set(&mut cx, i as u32, js_string).unwrap();
            }
            Ok(js_array.upcast())
        }

        method value(mut cx) {
            let key = cx.argument::<JsString>(0)?.value();
            let this = cx.this();
            let container = {
                let guard = cx.lock();
                let this = this.borrow(&guard);
                (*this).clone()
            };
            if let Some(data) = container.inner.data.get(&key) {
                return Ok(neon_serde::to_value(&mut cx, data)?);
            } else {
                panic!("No value for key \"{}\" available", key);
            }
        }
    }
}

register_module!(mut m, {
    m.export_class::<JsMyStruct>("MyStruct")?;
    Ok(())
});

Answer 2

The issue here is that you are using new PyMyStruct() instead of PyMyStruct.new(). In wasm-bindgen's debug mode you will get an error about this at runtime. Using .new() will fix your issue:

let obj = PyMyStruct.new("../../dump.spb")

If you add the #[wasm_bindgen(constructor)] annotation to the new method, then new PyMyStruct() will work as well:

#[wasm_bindgen]
impl PyMyStruct {
    #[wasm_bindgen(constructor)]
    pub fn new(filename: &str) -> Self {
        Self {
            inner: 1,
        }
    }
}

Now this is fine:

let obj = new PyMyStruct("../../dump.spb")