Save ARCore video for further processing?

Question

Is there a video standard that is able to save output from ARCore for further processing? Like encoding the world information beyond the actual video content.

Answer 1

Depends on how you're using it. If you're using Unity3D or Unreal, its possible.

Unity for example has RenderTexture. Check the following tutorial to create a rendertexture:

https://www.youtube.com/watch?v=pA7ZC8owaeo

Run a script and in its update loop, use the following code:

int i = 0; // in Start()

int width = Screen.width;
int height = Screen.height;
Texture2D tex = new Texture2D(width, height, TextureFormat.RGB24, false);

// Read screen contents into the texture
tex.ReadPixels(new Rect(0, 0, width, height), 0, 0);
tex.Apply();

// Encode texture into PNG
byte[] bytes = tex.EncodeToPNG();
File.WriteAllBytes(Application.dataPath + "/../pic" + i + ".png", bytes);
i++;

The above code has been borrowed from here

Now you can use ffmpeg to convert those PNGs into a MP4 file using the following command:

ffmpeg -r 60 -f image2 -s 1920x1080 -i pic%04d.png -vcodec libx264 -crf 25  -pix_fmt yuv420p test.mp4

Hope that helps.