Within a small DDD project my repositories are returning arrays of entities. However it seems that some data sets will grow quite large and I want to optimize the architecture to limit the amount of objects that go through the internal data mapper of the repos.
The first obvious thing I realized is that my repository queries do not have to return entities, in some cases, they could very well return a much more convenient list of ids.
One very common case is a table filtered dynamically by a user input. At each keystroke the list must be reloaded, but each of the first characters will generate thousand of query results whenever only the 10 first rows are being displayed.
I am trying to learn how to deal with this without marrying myself to some specific framework with lazy loading. Also I am starting to have seconds thoughts about how much of the arrangement of objects displayed in a table goes into a repository, and how much goes somewhere else. Ideally I would like to avoid exposing the need of query result limits outside my repositories.
Which approach should I evaluate to deal with large data sets without using any specific framework outside my repos?
I do have a custom Query class that just wraps the query result object of different databases and frameworks. I considered hiding inside a limit for the initial fetch and automatically expand if more data is needed, however fetching by parts can be a pain if the results are updated externally.
All this made me think that I need some type of smart array with some custom lazy loading. However I am not sure which one is the right place for it within the DDD patterns.
There is nothing wrong with limiting the number of rows fetched. You should fetch only 10 if you need 10 entities. Your view layer could ask for X number of entities.
Regarding fetching only IDs, that's not really OOP/DDD way but since you are limited with your platform you could fetch an array of ID=>name.
E.g.
return: array of ID=>name