Downloading a file with transfer-encoding header on android

Question

I have a strange problem that I would really some insight on. I have some code to download various files. Some files return the "transfer-encoding" header with a value of "chunked" and I'm pretty sure this is what's causing me problems.

Here's where it gets really strange. When downloading on my Motorolla Droid on 3g the files with this encoding do not download correctly, as in it finishes downloading before all the bytes have been read, therefore I have a corrupt file. However, on the exact same phone using WiFi the same file download just fine.

Does this sound like it would this be a server issue? android issue? 3g issue? my code issue?

Here's some code I'm using to read the response headers:

private void readResponseHeaders(HttpResponse response) throws StopRequest {
        Header header = response.getFirstHeader("Content-Disposition");
        if (header != null) {
            mInfo.mInnerState.mHeaderContentDisposition = header.getValue();
        }
        header = response.getFirstHeader("Content-Location");
        if (header != null) {
            mInfo.mInnerState.mHeaderContentLocation = header.getValue();
        }
        if (mInfo.mMimeType == null) {
            header = response.getFirstHeader("Content-Type");
            if (header != null) {
                mInfo.mMimeType = sanitizeMimeType(header.getValue());
            }
        }
        header = response.getFirstHeader("ETag");
        if (header != null) {
            mInfo.mInnerState.mHeaderETag = header.getValue();
        }
        String headerTransferEncoding = null;
        header = response.getFirstHeader("Transfer-Encoding");
        if (header != null) {
            headerTransferEncoding = header.getValue();
        }
        if (headerTransferEncoding == null) {
            header = response.getFirstHeader("Content-Length");
            if (header != null) {
                mInfo.mInnerState.mHeaderContentLength = header.getValue();
                mInfo.mTotalBytes = Long.parseLong(mInfo.mInnerState.mHeaderContentLength);

                if (!mFileManager.hasSufficientSpace(mInfo.mTotalBytes, mBasePath)){
                    throw new StopRequest(ServiceDownloadConstants.STATUS_INSUFFICIENT_SPACE_ERROR, "Not Enough Space");
                }
            }
        } else {
            // Ignore content-length with transfer-encoding - 2616 4.4 3
            if (ServiceDownloadConstants.LOGVV) {
                Log.v(ServiceDownloadConstants.TAG,
                        "ignoring content-length because of xfer-encoding");
            }
        }
        if (ServiceDownloadConstants.LOGVV) {
            Log.v(ServiceDownloadConstants.TAG, "Content-Disposition: " + mInfo.mInnerState.mHeaderContentDisposition);
            Log.v(ServiceDownloadConstants.TAG, "Content-Length: " + mInfo.mInnerState.mHeaderContentLength);
            Log.v(ServiceDownloadConstants.TAG, "Content-Location: " + mInfo.mInnerState.mHeaderContentLocation);
            Log.v(ServiceDownloadConstants.TAG, "Content-Type: " + mInfo.mMimeType);
            Log.v(ServiceDownloadConstants.TAG, "ETag: " + mInfo.mInnerState.mHeaderETag);
            Log.v(ServiceDownloadConstants.TAG, "Transfer-Encoding: " + headerTransferEncoding);
        }

        boolean noSizeInfo = mInfo.mInnerState.mHeaderContentLength == null
                && (headerTransferEncoding == null
                    || !headerTransferEncoding.equalsIgnoreCase("chunked"));
        if (!mInfo.mNoIntegrity && noSizeInfo) {
            throw new StopRequest(ServiceDownloadConstants.STATUS_HTTP_DATA_ERROR,
            "can't know size of download, giving up");
        }
    }

Here's some code I'm using to transfer the data:

private void transferData(byte[] data, InputStream entityStream) throws StopRequest {
        for (;;) {
            int bytesRead = readFromResponse( data, entityStream);
            if (bytesRead == -1) {
                // success, end of stream already reached
                handleEndOfStream();
                return;
            }

            writeDataToDestination(data, bytesRead);
            mInfo.mInnerState.mBytesSoFar += bytesRead;

            /** Update SHA1 if needed **/
            if( mInfo.mEnclosure.getHash() != null && mSha1 != null ){
                mSha1.update( data, 0, bytesRead );
            }

            if (ServiceDownloadConstants.LOGVV) {
                Log.v(ServiceDownloadConstants.TAG, "downloaded " + mInfo.mInnerState.mBytesSoFar + " for "
                     + mInfo.mEnclosure.getUrl());
            }

            checkPausedOrCanceled();
        }
    }

Edit Upon further debugging, I've received the exact same results even when the "transfer-encoding" header isn't being sent, so it would seem my problem is as generic as it works on Wifi and it doesn't on 3g.

Answer 1

3G providers cache files, and caching is sometimes broken. Try to set no-cache headers on your request.