Forwarding and duplicating RTP packets with VP8 encoded video

Question

For a project, I am using a small SBC that captures video frames, encodes this video to a VP8 stream (using gstreamer) and sends this video to a forwarding server. The forwarding server in turn forwards the incoming stream to connected web client. All forms of communication utilize webRTC. A schema is shown below:

┌───────────┐             ┌───────────────┐            ┌──────────┐
│           │             │               │            │          │
│  Video    │             │  Forwarding   │            │   Web    │
│  capture  ├────────────►│    server     ├───────────►│  client  │
│           │  WebRTC     │               │  WebRTC    │          │
└───────────┘  VP8 video  └───────────────┘  VP8 video └──────────┘
               track                         track

When I remove the forwarding server "hop" in the middle (so SBC -> web client directly) the video stream works as it should, but when I use the forwarding server to just forward the raw bytes (no modification or inspection) I cannot get it to work in any way. The Chrome webRTC debugger shows that the video track is being created correctly, but the NACK count and PLI count increase at the same rate as the bytes received (i.e. all frames get NACKed and PLIed).

Video capture

To create the the video stream and webRTC connection, I am using Golang with gstreamer and pion/webrtc. This is the most important code (simplified):

cmd := exec.Command( 
    "gst-launch-1.0",
    // Use test footage
    "videotestsrc",
    // config video width, height and framerate
    "!", "video/x-raw,width=320,height=240,framerate=15/1",
    // encode to vp8
    "!", "vp8enc",
    // output to cmd StdoutPipe
    "!", "filesink", "location=/dev/stdout",
)

out, cmdErr := cmd.StdoutPipe()
if cmdErr != nil {
    panic(cmdErr)
}

buf := make([]byte, 1024*512)
for {
    // log the start time to calc samples
    start := time.Now()

    // read stdout pipe data
    n, readErr := out.Read(buf)
    if readErr != nil {
        panic(readErr)
    }

    // get this time duration
    duration := time.Since(start)

    // output to videoTrack (an instance of webrtc.TrackLocalStaticSample)
    if err = videoTrack.WriteSample(media.Sample{Data: buf[:n], Duration: duration}); err != nil {
        panic(err)
    }
}

Forwarding server

The forwarding server (also in Go) reads the incoming bytes from the video capture and forwards them to the web client over webRTC. The forwarding server also uses a PLI factory to request a new base frame every three seconds (since the RTCP packets of the web client cannot traverse back to the video capture.

PLI factory

intervalPliFactory, err := intervalpli.NewReceiverInterceptor()
if err != nil {
    panic(err)
}
i.Add(intervalPliFactory)

// Use the default set of Interceptors
if err = webrtc.RegisterDefaultInterceptors(&mediaEngine, i); err != nil {
    panic(err)
}

Byte forwarding

for {
    start := time.Now()

    buf := make([]byte, 1024*512*5000)
    outBuf := make([]byte, 1024*512*5000)

    // Read from incoming video capture
    i, _, err := remoteTrack.Read(buf)
    if err != nil {
        panic(err)
    }
    log.Printf("Received %d bytes", i)

    // get this time duration
    duration := time.Since(start)

    // output to web client video track
    if err = webClient.Track.WriteSample(media.Sample{Data: buf[:n], Duration: duration}); err != nil {
        panic(err)
    }
    log.Printf("Forwarded %d bytes", n)
}

Web client

The web client is a svelte application. The Chrome internal webRTC debug tools show that the NACKs and PLIs grow at the same rate as the bytes received, so I am assuming that the video stream is not being decoded correctly. I don't think that the error is in the web client, because when I remove the forwarding server hop, the video stream is being shown correctly.

Question

So the question is: how can VP8 RTP packets be forwarded correctly? And also, can I just duplicate the bytes? Because I want to be able to forward the incoming video stream to multiple connected web clients.

It might not seem sensical to use webRTC between the video capture and forwarding server, but this eases the setup of a duplex connection between the two (because I also want to send other data back to the video capture, but that is not important right now).