Uploading a SQLite database file to an S3 bucket

Question

I'm currently working on a project in Android Studio that implements AWS S3. The use of S3 is to upload/store my local SQLite database file on it so that I can download and use it on command. I realize that this is by no means the most optimal way to be using databases with S3.

I'm able to upload and download files to and from my S3 bucket. But for some reason, the database file is corrupted according to logcat. The SQLite database file is stored in the device's database folder when it is downloaded.

Here's the implemented code:

public void uploadFile(String fileName) {
    File exampleFile = new File(getApplicationContext().getDatabasePath("Login.db").getPath());

    try {
        BufferedWriter writer = new BufferedWriter(new FileWriter(exampleFile));
        writer.append("Example file contents");
        writer.close();
    } catch (Exception exception) {
        Log.e("MyAmplifyApp", "Upload failed", exception);
    }

    Amplify.Storage.uploadFile(
        fileName,
        exampleFile,
        result -> Log.i("MyAmplifyApp", "Successfully uploaded: " + result.getKey()),
        storageFailure -> Log.e("MyAmplifyApp", "Upload failed", storageFailure)
    );
}

public void downloadFile() {
    Amplify.Storage.downloadFile(
        "Login.db",
        new File(getApplicationContext().getDatabasePath("Login.db") + ""),
        result -> Log.i("MyAmplifyApp", "Successfully downloaded: " + result.getFile().getName()),
        error -> Log.e("MyAmplifyApp",  "Download Failure", error)
    );
}

I'm looking for some insight on this matter. I'm just not sure what is causing the file corruption. I was thinking it could be the file path but I believe that is being navigated correctly.

Answer 1

I suspect that your issue is that the file is in fact corrupted and probably due to you using writer.append("Example file contents"); and saving a file that contains just that.

You could perhaps add code to do some checks before uploading and after downloading a file.

The following is a snippet from code that does as such (in this case checking that an asset is a valid sqlite file, it's perhaps a little over the top) :-

/**
 * DBHEADER
 * The the header string, the first 16 bytes, of the SQLite file
 *  This should never be changed.
 */
private static final String DBHEADER = "SQLite format 3\u0000"; // SQLite File header first 16 bytes

/**
 * Constants that represent the various stages of the copy
 */
private static final int
        STAGEOPENINGASSETFILE = 0,
        STAGEOPENINGDATABASEFILE = 1,
        STAGECOPYING = 3,
        STAGEFLUSH = 4,
        STAGECLOSEDATABSE = 5,
        STAGECLOSEASSET = 6,
        STAGEALLDONE = 100
                ;
/**
 * Constants for message codes
 */
private static final int
        MSGCODE_EXTENDASSETFILE_ADDEDSUBDIRECTORY = 20,
        MSGCODE_EXTEANDASSETFILE_EXTENDEDFILENAME = 21,
        MSGCODE_CHECKASSETFILEVALIDTY_OPENEDASSET = 30,
        MSGCODE_CHECKASSETFILEVALIDITY_OPENFILED = 31,
        MSGCODE_CHECKASSETFILEVALIDITY_NOTSQLITEFILE = 32,
        MSGCODE_CHECKASSETFILEVALIDITY_VALIDSQLITEFILE = 33,
        MSGCODE_COPY_FAILED = 40,
        MSGCODE_COPY_OK = 41
                ;

/**
 * The default buffer size, can be changed
 */
private static final int DEFAULTBUFFERSIZE = 1024 * 32; // 32k buffer

....


/**
 * Check that the asset file to be copied exists and optionally is a valid
 * SQLite file
 * @param cntxt                     The Context
 * @param extendedAssetFilename     The asset file name including subdirectories
 * @param showstacktrace            true id to show the stack-trace if an exception as trapped
 * @param checkheader               true if the SQLite file header should be checked
 * @return                          true if the checks are ok
 */
public static boolean checkAssetFileValidity(Context cntxt, String extendedAssetFilename, boolean showstacktrace, boolean checkheader) {
    boolean rv = true;
    InputStream is;
    try {
        is = cntxt.getAssets().open(extendedAssetFilename);
        messages.add(
                new Msg(
                        MSGCODE_CHECKASSETFILEVALIDTY_OPENEDASSET,
                        Msg.MESSAGETYPE_INFORMATION,
                        "Successfully Opened asset file " + extendedAssetFilename
                )
        );
        if (checkheader) {
            byte[] fileheader = new byte[DBHEADER.length()];
            is.read(fileheader,0,fileheader.length);
            if (!(new String(fileheader)).equals(DBHEADER)) {
                messages.add(
                        new Msg(
                                MSGCODE_CHECKASSETFILEVALIDITY_NOTSQLITEFILE,
                                Msg.MESSAGETYPE_ERROR,
                                "Asset file " +
                                        extendedAssetFilename +
                                        " is NOT an SQlite Database, instead found " + (new String(fileheader))
                        )
                );
                is.close();
                return false;
            } else {
                messages.add(
                        new Msg(
                        MSGCODE_CHECKASSETFILEVALIDITY_VALIDSQLITEFILE,
                                Msg.MESSAGETYPE_INFORMATION,
                                "Successfully validated asset file " + extendedAssetFilename +
                                        " . It has a valid SQLite Header."
                        )
                );
            }
        }
        is.close();
    } catch (IOException e) {
        messages.add(
                new Msg(
                        MSGCODE_CHECKASSETFILEVALIDITY_OPENFILED,
                        Msg.MESSAGETYPE_ERROR,
                        "Unable to open asset " + extendedAssetFilename + "."
                )
        );
        if (showstacktrace) {
            e.printStackTrace();
        }
        return false;
    }
    return rv;
}

• the Msg class and associated methods haven't been included, they are just for logging messages. Shout if you want them added.

The main thing is checking the header.