Get a specific arraylist element in yaml for retriving data from datastore

Question

I am trying to pull all entities from google cloud datastore in csv format.

I found a great link which helped me a lot (https://vikashazrati.wordpress.com/2010/09/20/downloading-datastore-data-from-a-java-application-on-google-app-engine/)

My application is made in java.

I have 2 concerns:

a. Only indexed fields of subclasses are being retrieved and not all. Should I index all the fields/properties of the SubClass (my understanding is that indexing all the fields is not a good idea, please correct me if I am wrong)

b. I have an arrayList as part of this entity and I need only the element on max index, something like arrayList[arrayList.size() -1] and I am not able to retrieve it. (just FYI, arrayList is made up of objects of a specific class, ie this is a composite property and after retrieving the required object, I plan to print retrieve the members of this composite property 1 by 1 something like arrayList[arrayList.size() -1].member1 )

Any pointers would be helpful.

Thanks in advance

Answer 1

Wrt (a), indexing everything is feasible -- the issue is it can get rather costly if you write a lot of entities (each entity-put you do will need to update many indices). The Remote API, https://cloud.google.com/appengine/docs/java/tools/remoteapi , is likely to be a better approach.

Wrt (b), the usual solution to such issues is to "denormalize" -- introduce some redundancy for ease of querying and fetching. So, for example, if you have a foo member that's a list, add a redundant last_foo member that's maintained (a small amount of code each time you modify foo) to equal the last item of foo. A little extra cost on writing, but, for many applications, disproportionate saving on querying and fetching (and for most apps queries and fetches are more frequent).