Problem adding lots of strings to a TStringList

Question

I have a problem adding strings to a TStringList. I've searched other posts but couldn't find an answer to this.

What I'm trying to do is to add a big amount of strings to a TStringList (more than 14000) but somewhere in the process I get an EAccessViolation. Here's the code I'm using:

procedure TForm1.FormCreate(Sender: TObject);
begin
    List := TStringList.Create;
    List.Duplicates := dupAccept;
end;

procedure TForm1.ButtonStartClick(Sender: TObject);
begin
    List.Clear;
    List.Add('125-AMPLE');
    List.Add('TCUMSON');
    List.Add('ATLV 4300');
    List.Add('150T-15');
    List.Add('TDL-08ZE');
    List.Add('RT20L');
    List.Add('SIN LINEA');
    List.Add('TIARA');
    List.Add('FL200ZK1');
    List.Add('FL250ZK1');
    List.Add('SIN LINEA');
    List.Add('CENTAURO-70 S.P.');
    List.Add('CORSADO');

{ This list continues to about 14000 strings...}

    List.Add('VOSJOD 2');
    List.Add('Z 125');
    List.Add('ZUMY');
    List.Add('NEW AGE 125');
    List.Add('SIN LINEA');
end;

procedure TForm1.FormClose(Sender: TObject; var Action: TCloseAction);
begin
    FreeAndNil(List);
end;

¿What's wrong with this code? The list contains duplicate strings so I set the Duplicates property to dupAccept. I was able to load the list using LoadFromFile, but I don't want to have a text file outside my application.

I hope you can help me!!! Please tell me if you need any further information.

Thank you very much. I really appreciate your help.

Answer 1

The suggestions for using an external file are on the mark here. However, your post indicates your desire for not having an external file. I would then suggest you link the file to the executable as a resource. You can easily do this following these steps:

Place all the strings into a text file called stringdata.txt (or whatever name you choose). Then create a .rc file of whatever name you choose and put the following into it (STRING_DATA can be any identifier you choose):

STRING_DATA RCDATA "stringdata.txt"

Create a .res file from the .rc:

BRCC32 <name of rc>.rc

Now reference this file from the source code. Place the following someplace in the unit:

{$R <name of res>.res}

Instead of loading from a file stream, load from a resource stream:

StringData := TResourceStream.Create(HInstance, 'STRING_DATA', RT_RCDATA);
try
  List.LoadFromStream(StringData);
finally
  StringData.Free;
end;

If you do command-line automated builds, I would suggest you keep the .rc file under source control and build the .res during the build process. This way you can also keep the stringdata.txt file under source control and any edits are automatically caught on the next build without having to explicitly build the .res file each time the .txt file changes.

Answer 2

What Delphi version are you using? Some older versions had a bug in the memory manager that can cause an access violation when trying to reallocate an array to a size that's too large.

Try adding FastMM4 to your project to replace the old memory manager and see if that helps.

Also, you're probably better off keeping the list in an external file. Yes, it's another file, but it also means that you can change the list without having to recompile the entire program. This also makes creating (and distributing!) updates easier.

Answer 3

Mason is probably right for the cause of the AV; this is quite a large array to grow.
On a side note, when doing such a long processing on a StringList, it's recommended to surround it by BeginUpdate/EndUpdate to avoid firing any update event.
Even if you don't have any now, they might be added later and you'll get problems.

Answer 4

Set list.capacity to the number of items you plan to add, immediately after you create the list. Alternatively, place the list in an RC file (named other than with the name of your project) and add it to your project. This gets compiled into your application, but does not involve executable code to create the list.

Answer 5

You may also want to try THashedStringList, could see a speed boost (although not in this function), although I'm not sure if the add method is a whole lot different.

Answer 6

I would also worry about compiler integrity with a 14,000 line procedure. People have found other cases where going beyond anything reasonable breaks the compiler in various ways.

Answer 7

try using the following instead of your code to add the strings to the StringList

var
Str: string;
begin
Str := '125-AMPLE' + #13#10;
Str := Str + 'TCUMSON' + #13#10;
Str := Str + 'ATLV 4300' + #13#10;
Str := Str + '150T-15' + #13#10;
................

List.Text := Str;

end;