Adding a new Field to FDMemTable when loading an existing data

Question

I'm using TFDMemTable, i have created a dataset and populated my table with data and then used FDMemTable.saveToFile to save my data.
Now here is the question, how can i add a new Field to this already saved data and fill all records with a default value ?
I tried adding the new Field to the FDMemTable and then loading the information hoping it would automatically fill each Field using it's FieldName and the new field with an empty space, but i got an error saying : `---------------------------

Debugger Exception Notification

Project Project1.exe raised exception class EDatabaseError with message 'Field 'Episode' not found'.

Break Continue Help

`
How can i fix this ? is there a work around for adding a new field with default value to an already existing data ?
Here is a testcase:

// here is a simple example, i have a few fields
FDMemTable1.FieldDefs.Add('ID', ftInteger, 0, false);
FDMemTable1.FieldDefs.Add('name', ftString, 30, false);
FDMemTable1.FieldDefs.Add('QualityID', ftInteger, 0, false);

FDMemTable1.CreateDataSet;

// i fill the the table with some value
FDMemTable1.Open;
FDMemTable1.AppendRecord([1, 'Movie1', 1]);
FDMemTable1.AppendRecord([2, 'Movie2', 2]);
FDMemTable1.AppendRecord([3, 'Movie3', 1]);

// after seeing the value on the grid, i push a button and save the table as XML
FDMemTable1.saveToFile('Data.xml');

// now after closing the program and running it again, i want to load the data with a new FieldDef Called Episode with a default value 0
// the table is connected to cxGrid, and the moment i try to load, i get the error i mentioned.

FDMemTable1.FieldDefs.Add('ID', ftInteger, 0, false);
FDMemTable1.FieldDefs.Add('name', ftString, 30, false);
// this line is NEW
FDMemTable1.FieldDefs.Add('Episode', ftInteger, 0, false);
FDMemTable1.FieldDefs.Add('QualityID', ftInteger, 0, false);

FDMemTable1.CreateDataSet;

// i try to load but i get an error
FDMemTable1.loadFromFile('Data.xml');

Update 1 (based on Victoria's answer) :

//All the codes below are executed after the code at the top, after the loadFromFile.
  FDMemTable1.ResourceOptions.StoreItems := FDMemTable1.ResourceOptions.StoreItems + [siMeta];
  FDMemTable1.Close;

  //i tried putting FDMemTable1 as the owner but when i try the line "FDMemTable1.Fields.add" i get an Duplicate error!
  // and when i try putting nil, i get access violation ! so i tried putting the owner someother random table and it fixed the problem
  fieldDefs := TFieldDefs.Create(someRandomFDMemTable);
  fieldDefs.Add('Episodes', ftString, 30, false);

  FDMemTable1.Fields.Add(fieldDefs.Items[0].CreateField(FDMemTable1));
  FDMemTable1.Open;

As you can see i have two problem,
1- I have a problem with adding a new field! where i get an error, i first tried using TFieldDef instead of TFieldDefs but i couldn't get it to work.
2- Is the fact that all the columns are empty and there is no data on the grid.
Problem 2 occurs when i try solving the problem 1 forcefully.

Answer 1

The code below works fine for me in Delphi Seattle. It's closely based on yours but with a few minor changes (see comments) and one important addition.

The thing is, if you closely observe FDMemTable1.loadFromFile, it actually clears the FieldDefs/Fields, so there's not a lot of point setting them up again in the first place, so your added Episode field is just going to get omitted from the loaded dataset.

To avoid that, the addition I've made is to use a temporary TFDMemTable, loaded the XML file into that, then copied its contents to FDMemTable1 using CopyDataSet. FDMemTable1 retains the added Episode field in the process, and you can of course add code to set the Episode field data once CopyDataSet has been called.

procedure TForm1.AddFieldTest;
var
  DataFN : String;
  TempMemTable : TFDMemTable;
begin
  // Added, to avoid relative path for data file name
  DataFN := ExtractFilePath(Application.ExeName) + 'Data.Xml';

  FDMemTable1.FieldDefs.Add('ID', ftInteger, 0, false);
  FDMemTable1.FieldDefs.Add('name', ftString, 30, false);
  FDMemTable1.FieldDefs.Add('QualityID', ftInteger, 0, false);

  FDMemTable1.CreateDataSet;

  // i fill the the table with some value
  FDMemTable1.Open;
  FDMemTable1.AppendRecord([1, 'Movie1', 1]);
  FDMemTable1.AppendRecord([2, 'Movie2', 2]);
  FDMemTable1.AppendRecord([3, 'Movie3', 1]);

  // after seeing the value on the grid, i push a button and save the table as XML
  FDMemTable1.saveToFile(DataFN);

  // Added, close the dataset and clear the FieldDefs
  // Without the FieldDefs.Clear call, your code produces a "Duplicate field ID" error 
  FDMemTable1.Close;
  FDMemTable1.FieldDefs.Clear;

  // now after closing the program and running it again, i want to load the data with a new FieldDef Called Episode with a default value 0
  // the table is connected to cxGrid, and the moment i try to load, i get the error i mentioned.

  FDMemTable1.FieldDefs.Add('ID', ftInteger, 0, false);
  FDMemTable1.FieldDefs.Add('name', ftString, 30, false);
  // this line is NEW
  FDMemTable1.FieldDefs.Add('Episode', ftInteger, 0, false);
  FDMemTable1.FieldDefs.Add('QualityID', ftInteger, 0, false);

  FDMemTable1.CreateDataSet;

  // check the FieldCount and existence of the Episode field
  Caption := IntToStr(FDMemTable1.FieldCount);
  Assert(FDMemTable1.FindField('Episode') <> Nil);

  //  Create a temporary TFDMemTable
  TempMemTable := TFDMemTable.Create(Nil);
  try
    //  load the data into the temporary TFDMemTable
    TempMemTable.loadFromFile(DataFN);
    //  copy the data from the temporary TFDMemTable into FDMemTable1
    FDMemTable1.CopyDataSet(TempMemTable, [coAppend]);

  // check the FieldCount and existence of the Episode field
    Caption := IntToStr(FDMemTable1.FieldCount);
    Assert(FDMemTable1.FindField('Episode') <> Nil);
  finally
    TempMemTable.Free;
  end;
end;

Answer 2

Add that field (not field definition) after you load dataset from the file (assuming siMeta is included in the StoreItems property of the ResourceOptions of your TFDMemTable). Be aware, dataset must be closed at the time you add the field to the field collection.

So, close the dataset right after loading from file, add the field and open it again.