Using cache in ExoPlayer for persisting streamed video

Question

UPDATE:

I found this post, which details exactly the same problem I am seeing. It turns out that the fact I am using a Pipe approach in my DocumentsProvider to stream content from DropBox means that ExoPlayer doesn't know the size of the file ahead of time, and so by default was not saving it to the cache.

So I ended up doing what I presume the author did - I created a custom CacheDataSource for these situations that alters the DataSpec.flags variable in the open() method of that class:

public long open(DataSpec dataSpec) throws IOException {
    try {
        key = cacheKeyFactory.buildCacheKey(dataSpec);
        uri = dataSpec.uri;
        actualUri = getRedirectedUriOrDefault(cache, key, /* defaultUri= */ uri);
        httpMethod = dataSpec.httpMethod;
        if ( !dataSpec.isFlagSet(DataSpec.FLAG_ALLOW_CACHING_UNKNOWN_LENGTH) ) { // <-- update here
            flags = (dataSpec.flags | DataSpec.FLAG_ALLOW_CACHING_UNKNOWN_LENGTH);
        } else {
            flags = dataSpec.flags;
        }
        readPosition = dataSpec.position;

Not the optimum solution, and I also chimed in on the other post with a request for a more supported way to indicate this flag should be set.

But at least now my streamed files are being saved in the cache.

---

I am implementing a customer CacheDataSourceFactory for ExoPlayer2, in order to implement a cache to store videos streamed to ExoPlayer.

I have reviewed several posts here, this one was helpful in getting the general approach right to have a video cached into the directory of my choice.

I noticed that when handling a URI that resolves to my custom DocumentsProvider, the Cache defined by the CacheDataSourceFactory is only used to store what looks like a "pointer" or "index" file ("cached_content_index.exi"). Looking in that file I see the URI of the video streamed by my custom DocumentsProvider. However the actual video is not in the cache.

Here is the relevant portion of my Provider, it's quite straight forward:

        // Return a descriptor that will stream the file
        Timber.d("In openDocument of DropboxProvider for Id: %s, streaming from source", documentId);
        ParcelFileDescriptor[] pipe;
        try {
            pipe = ParcelFileDescriptor.createPipe();

            // Get input stream for the pipe
            DbxDownloader downloader = mDbxClient.files().download(fileMetadata.getPathLower(), fileMetadata.getRev());
            new TransferThread(downloader.getInputStream(), new ParcelFileDescriptor.AutoCloseOutputStream(pipe[1]), signal, fileMetadata.getSize()).start();
            return pipe[0];
        } catch (DbxException dbe) {
            Timber.d("Got IDbxException when streaming content: %s", dbe.getMessage());
        } catch (IOException ioe) {
            Timber.d("Got IOException when streaming content: %s", ioe.getMessage());
        } catch (Exception e) {
            Timber.d("Got Exception when streaming content: %s", e.getMessage());
        }
        return null;

And the TransferThread:

private static class TransferThread extends Thread {
    final InputStream in;
    final OutputStream out;
    final CancellationSignal signal;
    final long size;

    TransferThread(InputStream in, OutputStream out, CancellationSignal signal, long size) {
        this.in = in;
        this.out = out;
        this.signal = signal;
        this.size = size;
    }
    @Override
    public void run() {
        int biteSize = (8*1024);
        if ( size <= (biteSize * 8) ) {
            biteSize = Math.max( ((int)(size / (biteSize*2))) * (biteSize * 2), biteSize);
        }
        Timber.d("TransferThread: File size is: %s, buffer biteSize set to: %d", InTouchUtils.getFormattedFileSize(size), biteSize);
        byte[] buf = new byte[biteSize];
        int len;
        try {
            while ( ((len=in.read(buf)) >= 0) && (signal == null || !signal.isCanceled()) ) {
                out.write(buf, 0, len);
            }
        } catch (IOException e) {
            // When Glide is used to request a URI where this provider resolves the query,
            // it will close the stream out from under us once it has fetched enough bytes
            // to render a single frame as an image if the if it is to a video, so
            // we swallow that exception here, only logging the error if it isn't that EPIPE
            // (broken pipe due to one end being closed) exception.
            if ( !(e.getMessage().contains("EPIPE"))) {
                Timber.d("TransferThread: Got IOException transferring file: %s", e.getMessage());
            }
        } finally {
            try {
                if (in != null) {
                    in.close();
                }
                if ( out != null ) {
                    out.flush();
                    out.close();
                }
                Timber.d("TransferThread: Finished streaming file.");
            } catch (IOException ioe) {
                Timber.d("TransferThread: Got IOException closing file: %s", ioe.getMessage());

            }
        }
    }
}

Again - ExoPlayer seems quite happy with the ParcelFileDescriptor it receives from the DocumentsProvider in this case - it takes the bytes streamed to it and plays the video. I am just not seeing the video file end up in the cache.

I also tried an example streaming a video from my Google Drive (which uses the out-of-the-box documents provider from the SAF), and this time the video did wind up in the cache.

Since they both use the same MediaSource instance - there must be an approach that the Google Docs provider takes so that ExoPlayer knows to place the resulting streamed video in the cache that my custom Dropbox DocumentsProvider is not doing.

Does anyone know how to get to the source code of the DocumentsProvider that ships with the SAF that manages access to Google Docs? I'd like to see what it is doing in its openDocument() method.

Is the fact that the Dropbox provider is utilizing a Pipe in its ParcelFileDescriptor something that ExoPlayer doesn't handle?

Other Ideas?