Dont want to add header rows when i have looped through first file

Question

What am i trying to do

Basically im looping through all my folders to catch if there is a .txt document. When i have loop through this i want to add the headers and rows from this document to a DataTable. My columns are always the same from each file.

The code i used so far

DataTable csvData = new DataTable();

string[] fullpath = Directory.GetFiles(@"D:\Stack", "*.txt", System.IO.SearchOption.AllDirectories);

            foreach(string s in fullpath)
            {

                DataTable eachfile = new DataTable();

            using (TextFieldParser csvReader = new TextFieldParser(s))
            {


                        csvReader.SetDelimiters(new string[] { ";" });
                        csvReader.HasFieldsEnclosedInQuotes = false;
                        string[] colFields = csvReader.ReadFields();
                        foreach (string column in colFields)
                        {
                            DataColumn datecolumn = new DataColumn(column);
                            datecolumn.AllowDBNull = true;
                        eachfile.Columns.Add(datecolumn);
                        }

                    eachfile.Columns.Add("SalesNo");
                    eachfile.Columns.Add("Date");


                    while (!csvReader.EndOfData)
                    {

                        csvReader.SetDelimiters(new string[] { "," });

                        string[] fieldData = csvReader.ReadFields();
                        //Making empty value as null
                        for (int i = 0; i < fieldData.Length; i++)
                        {
                            if (fieldData[i] == "")
                            {
                                fieldData[i] = null;
                            }
                        }

                        Match conum = Regex.Match(s, @"SalesNo\d+");
                        Match cycle = Regex.Match(s, @"Date\d{8}");


                        DataRow row = eachfile.NewRow();

                        //Alternativ
                        //row["FirstName"] = fieldData[0];
                        row[0] = fieldData[0];
                        row[1] = fieldData[1];
                        row[2] = conum;
                        row[3] = cycle;

                        eachfile.Rows.Add(row);

                        csvData = eachfile.Copy();
                    }



                }
            }
        }

What is my issue

Well in my code i try to add data to a DataTable called eachfile. However i cannot add this to my main DataTable csvData, since it will be overwritten eachtime. What I really wanted to is to collect all my data into one DataTable csvData but only add the HeaderRows once (from first file i loop through)

What do i need

I need to re-write the code i think so when i loop through first file, it uses the code here which adds my header to the csvData DataTable

 csvReader.SetDelimiters(new string[] { ";" });
                        csvReader.HasFieldsEnclosedInQuotes = false;
                        string[] colFields = csvReader.ReadFields();
                        foreach (string column in colFields)
                        {
                            DataColumn datecolumn = new DataColumn(column);
                            datecolumn.AllowDBNull = true;
                        eachfile.Columns.Add(datecolumn);
                        }

                    eachfile.Columns.Add("CoNum");
                    eachfile.Columns.Add("CycleDate");

When this is done and i have inserted data from the first file, i do not want to add the header rows again for the next file, since it will fail due to duplicate names. But only add the data to the already existing csvData (and not overwriting the previous added data)

What do i think i can do?

I think i need to count if it is the first file, and do something like this where i count the number of fullpath and if that is 1 i go through the headers else i go to my else where i readline first so i wont get first row?

 if (fullpath.Count() == 1)
                    {

                        csvReader.SetDelimiters(new string[] { ";" });
                        csvReader.HasFieldsEnclosedInQuotes = false;
                        string[] colFields = csvReader.ReadFields();
                        foreach (string column in colFields)
                        {
                            DataColumn datecolumn = new DataColumn(column);
                            datecolumn.AllowDBNull = true;
                            csvData.Columns.Add(datecolumn);
                        }

                        csvData.Columns.Add("CoNum");
                        csvData.Columns.Add("CycleDate");


                        while (!csvReader.EndOfData)
                        {

                            csvReader.SetDelimiters(new string[] { "," });

                            string[] fieldData = csvReader.ReadFields();
                            //Making empty value as null
                            for (int i = 0; i < fieldData.Length; i++)
                            {
                                if (fieldData[i] == "")
                                {
                                    fieldData[i] = null;
                                }
                            }

                            Match conum = Regex.Match(s, @"CoNum\d+");
                            Match cycle = Regex.Match(s, @"CycleDate\d{8}");


                            DataRow row = csvData.NewRow();

                            //Alternativ
                            //row["FirstName"] = fieldData[0];
                            row[0] = fieldData[0];
                            row[1] = fieldData[1];
                            row[2] = conum;
                            row[3] = cycle;

                            csvData.Rows.Add(row);
                        }
                    }
                    else
                    {
                        csvReader.ReadLine();
                        while (!csvReader.EndOfData)
                        {

                            csvReader.SetDelimiters(new string[] { "," });

                            string[] fieldData = csvReader.ReadFields();
                            //Making empty value as null
                            for (int i = 0; i < fieldData.Length; i++)
                            {
                                if (fieldData[i] == "")
                                {
                                    fieldData[i] = null;
                                }
                            }

                            Match conum = Regex.Match(s, @"CoNum\d+");
                            Match cycle = Regex.Match(s, @"CycleDate\d{8}");


                            DataRow row = csvData.NewRow();

                            //Alternativ
                            //row["FirstName"] = fieldData[0];
                            row[0] = fieldData[0];
                            row[1] = fieldData[1];
                            row[2] = conum;
                            row[3] = cycle;

                            csvData.Rows.Add(row);
                        }


                    }

Thanks in advance !

Answer 1

Your problem seems to be related to the detection of the first iteration of a foreach loop. All the other building blocks, as adding the columns and adding the data rows, look fine and will work.

There are many options. One is to use a for loop instead, and add the columns only when the loop counter has its start value. One is to add a flag that ensures that the columns are added only in the first iteration:

var isfirstfile = true;
foreach(var file in files)
{
    if(isfirstfile)
    {
        /* add the columns */


        isfirstfile = false;
    }
    else 
    {
        /* skip the first row of the current input file */
    }
    /* add the data rows */

}

Of course this assumes that all files have the same number and types of columns, in the same order.