Using: https://github.com/ash3D/VideoRecorder Wrapped with: https://github.com/ash3D/UEVideoRecorder
I added the VP9 codec to the "FindEncoder" function of the CVideoRecorder (part of the VideoRecorder repository). Using VP9 codec works generally, however the memory consumption is enormous and within a good 30 seconds, the memory usage goes up from 3GB to 6GB, upon which (round about) FFMPEG throws out of memory errors and aborts the encoding process. I started up the Diagnostics Tool of Visual Studio and it told me that avcodec.dll is holding all the memory, which is not released until the encoding is terminated (peacefully). The function within which the allocating call is executed is the CVideoRecorder::Encode() function, which inside it calls avcodec_send_frame().
H264 and HVENC work fine with a constant memory consumption of around 300MB.
I tried setting various settings on the VP9 codec like the bitrate, cfr and quality, none of which changed the behavior in any to me recognizable way.