Android Out of Memory Error: How to solve this?

Question

I have an application, and I am trying to set up a fairly large SQLite database (one table with roughly 5000 rows) into it. I have built the DB classes and everything, and my app works when I tested on a smaller scale (400 rows), but now when I want to import my database, I get the out of memory error which I can't seem to find a way to get around.

The database is initially on MySQL on my web server, and I couldn't convert it for some odd reason but I managed to generate a text file with the queries to add all 5000 rows, which is 11.5mb in size. I have this file in my assets folder, and I am trying this to put it into my DB:

public void onHandleIntent(Intent intent) {

        DBAdapter db = new DBAdapter(getApplicationContext());
        db.open();

        try {
            InputStream is = getAssets().open("verbs_sql.txt");
            db.executeSQL(convertStreamToString(is));
        } catch (IOException e) {}

        db.close();

        // Run main activity
        Intent i = new Intent(DatabaseReceiver.this, BaseActivity.class);
        i.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_EXCLUDE_FROM_RECENTS);
        DatabaseReceiver.this.startActivity(i);
    }

    public static String convertStreamToString(InputStream is) throws IOException {
        Writer writer = new StringWriter();

        char[] buffer = new char[2048];
        try {
            Reader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
            int n;
            while ((n = reader.read(buffer)) != -1) {
                writer.write(buffer, 0, n);
            }
        } finally {
            is.close();
        }
        String text = writer.toString();
        return text;
    }
}

The out of memory error occurs on the StringWriter(), so it looks likes it putting that big file on to the memory. How can I solve this? I have also tried looping through all 5000 rows but after maybe 30 seconds I got the out of memory again.

Answer 1

Sqlite databases are just files, when you're trying to run thousands of inserts on the phone you're hitting the SD card over and over for file access.

What you'll want to do is create the sqlite database on your desktop and include the already created database in the app. If you need to regularly update the information in the database you could post it on a website and have the app download it, just make sure to only do large downloads over Wifi.

Check out this Tech Talk for more information.

Edit: See this for more information on creating an sqlite database in windows and including it in your app.

Answer 2

I believe you could do this in the way you want. The problem with the code you posted is that you are trying to convert the entire file to a string. I am fairly certain that this would fail even on a desktop machine.

I believe that you would have better luck if you tried to read in one line at a time and execute the SQL. Then read the next line. You could also reduce the size of the file by passing it through zip.

If I can find a couple of minutes, I will attach some code.

Answer 3

I had the same problem. I tried so many ways to solve it but failed. At last i found the reason and i wondered. In my case the reason for the error was i was printing the entire response string in log cat. That data was very huge and it took heap memory. Take care of the following

1. Remove Log cat printing of bulk data.
2. Try to use only one J-SON Array for all Operation under one resonse(Reuse it for all).
3. Try to avoid array-list usage.
4. Insert item when each item iterate from J-Son Array. That means don't follow the method in which we are taking the item as object and put it in to an array-list and passing array-list to DB-helper and from there iterate the object and insert.