Parsing text file line by line to array using BufferedInputStream with RTL characters in Java

Question

Guys I need to understand something: the \n comes at the begining of a new line currect? If so, I am trying to parse a file that has RTL characters in it and they are at the begining of a line, so:

1. xxx xxxx, ABC DEFG, 1, 11, 111, 786
2. xxx xxxx, ABC DEFG, 1, 11, 111, 786
3. etc...

when parsing the txt file (android from assets) I keep getting the first word from the next line concatenated to the Integer from the previous line. I have tried everything but with no luck.

Here is a code snippet:

            InputStream is;
            try {
                is = new BufferedInputStream(getAssets().open(fileName));
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }

            try {
                InputStream is = new BufferedInputStream(getAssets().open(fileName));
                byte[] c = new byte[128];
                byte[] d = null;
                int readChars = 0;
                int lineNumber = 0;
            String line;
                String[] paramLineArray = null;
                int k;
                while ((readChars = is.read(c)) != -1) {
                    for (int i = 0; i < readChars; i++) {
                        if (c[i] == '\n') {
                            lineNumber++;
                            line = new String(c);
                            k = 0;
                            StringTokenizer st = new StringTokenizer(line,",");
                            paramLineArray = new String[st.countTokens()];
                            while (st.hasMoreTokens()) {
                              // get next token and store it in the Line
                              paramLineArray[k] = st.nextToken();
                              k++;
                            }
                        }                       
                    }
                    publishProgress(((int) (1 / (float) lineNumber) * 100));
                    populateTables(paramLineArray, tblName, tblElements);
                }

What I am trying to achieve is this:

Parse the text file really fast Line by line into an array that is the inserted into a DB...

Any ideas???

Help is much appreciated as I've been at it for days now (loosing my hair :-()...

Currently I have code working with InputStreamReader but it is very slow!!!!!

Thank you.

JadeYe.

Answer 1

Use:

BufferedReader in = new BufferedReader(
        new InputStreamReader(getAssets().open(fileName), "UTF-8"));
try {
    for (;;) {
        String line = in.readLine();
        if (line == null)
            break;
        ...
     }
} finally {
    in.close();
}