bwu-datagrid - Implementing Lazy Loading of Large Data Set

191 Views Asked by At

I am working on implementing lazy loading utilizing bwu-grid's dataview implementation in Dart. My server is implemented with Google App Engine (Java) over cloudendpoint API. When returning over 1,000+ rows, if no fetch option is set, I get an exception thrown.

I currently have JQuery's pagination in the bottom of dataview. I want to implement lazy loading utilizing jdo's query cursor in such way that upon changing the page, it loads the particular segment of the dataset.

I looked over dataview class implementation and found the following listener I can potentially utilize:

  async.Stream<core.PagingInfoChanged> get onBwuPagingInfoChanged =>
      _eventBus.onEvent(core.Events.PAGING_INFO_CHANGED);

The implementation plan is the following:

  1. Query for 1st page of data and total row count, update the datagrid with total row count
  2. Listen for the page changing event
  3. Make cloud endpoint using query cursor that returns the data segment
  4. Insert the data segment into the grid and refresh the data-grid

How is my implementation plan? What else do I have to consider when implementing this way? Is there any examples showing lazy loading support in bwu-datagrid? Any suggestions would be appreciated.

    W 12:06:21.550 com.google.appengine.api.datastore.BaseQueryResultsSource logChunkSizeWarning: This query does not have a chunk size set in FetchOptions and has returned over 1000 results.  If result sets of this size are common for this query, consider setting a chunk size to improve performance.
        To disable this warning set the following system property in appengine-web.xml (the value of the property doesn't matter): 'appengine.datastore.disableChunkSizeWarning'
    E 12:07:19.760 com.google.api.server.spi.SystemService invokeServiceMethod: exception occurred while calling backed method
      javax.jdo.JDODataStoreException: Datastore Failure
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:421)
        at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:367)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
        at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)
        at com.sports_tla.MLBMatchEndpoint.listMLBMatch(MLBMatchEndpoint.java:81)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:45)
        at com.google.api.server.spi.SystemService.invokeServiceMethod(SystemService.java:359)
        at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:160)
        at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:118)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
        at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
        at com.google.tracing.CurrentContext.runInContext(CurrentContext.java:220)
        at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
        at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
        at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
        at java.lang.Thread.run(Thread.java:724)
      NestedThrowablesStackTrace:
      com.google.appengine.api.datastore.DatastoreFailureException: Unexpected failure
        at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:78)
        at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:62)
        at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:41)
        at com.google.appengine.datanucleus.WrappedDatastoreService.get(WrappedDatastoreService.java:60)
        at com.google.appengine.datanucleus.FetchFieldManager.lookupOneToOneChild(FetchFieldManager.java:581)
        at com.google.appengine.datanucleus.FetchFieldManager.fetchRelationField(FetchFieldManager.java:541)
        at com.google.appengine.datanucleus.FetchFieldManager.fetchObjectField(FetchFieldManager.java:405)
        at org.datanucleus.state.AbstractStateManager.replacingObjectField(AbstractStateManager.java:2353)
        at com.sports_tla.MLBMatch.jdoReplaceField(MLBMatch.java)
        at com.sports_tla.MLBMatch.jdoReplaceFields(MLBMatch.java)
        at org.datanucleus.state.JDOStateManager.replaceFields(JDOStateManager.java:1935)
        at org.datanucleus.state.JDOStateManager.replaceFields(JDOStateManager.java:1962)
        at com.google.appengine.datanucleus.EntityUtils$1.fetchFields(EntityUtils.java:974)
        at org.datanucleus.state.JDOStateManager.loadFieldValues(JDOStateManager.java:764)
        at org.datanucleus.state.JDOStateManager.initialiseForHollow(JDOStateManager.java:205)
        at org.datanucleus.state.StateManagerFactory.newForHollowPopulated(StateManagerFactory.java:89)
        at org.datanucleus.state.ObjectProviderFactory.newForHollowPopulated(ObjectProviderFactory.java:75)
        at org.datanucleus.ObjectManagerImpl.findObject(ObjectManagerImpl.java:2882)
        at com.google.appengine.datanucleus.EntityUtils.entityToPojo(EntityUtils.java:1014)
        at com.google.appengine.datanucleus.query.DatastoreQuery$2.apply(DatastoreQuery.java:229)
        at com.google.appengine.datanucleus.query.DatastoreQuery$2.apply(DatastoreQuery.java:226)
        at com.google.appengine.datanucleus.query.LazyResult.resolveNext(LazyResult.java:96)
        at com.google.appengine.datanucleus.query.LazyResult.resolveAll(LazyResult.java:121)
        at com.google.appengine.datanucleus.query.StreamingQueryResult.closingConnection(StreamingQueryResult.java:90)
        at org.datanucleus.store.query.AbstractQueryResult.disconnect(AbstractQueryResult.java:108)
        at com.google.appengine.datanucleus.query.StreamingQueryResult.disconnect(StreamingQueryResult.java:78)
        at com.google.appengine.datanucleus.query.JDOQLQuery$1.managedConnectionPreClose(JDOQLQuery.java:199)
        at com.google.appengine.datanucleus.DatastoreConnectionFactoryImpl$DatastoreManagedConnection.close(DatastoreConnectionFactoryImpl.java:147)
        at org.datanucleus.store.connection.ConnectionManagerImpl.closeAllConnections(ConnectionManagerImpl.java:181)
        at org.datanucleus.store.AbstractStoreManager$1.preClose(AbstractStoreManager.java:260)
        at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java:1112)
        at org.datanucleus.api.jdo.JDOPersistenceManager.internalClose(JDOPersistenceManager.java:359)
        at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.releasePersistenceManager(JDOPersistenceManagerFactory.java:1106)
        at org.datanucleus.api.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java:343)
        at com.sports_tla.MLBMatchEndpoint.listMLBMatch(MLBMatchEndpoint.java:81)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:45)
        at com.google.api.server.spi.SystemService.invokeServiceMethod(SystemService.java:359)
        at com.google.api.server.spi.SystemServiceServlet.execute(SystemServiceServlet.java:160)
        at com.google.api.server.spi.SystemServiceServlet.doPost(SystemServiceServlet.java:118)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      <continued in next message>
0

There are 0 best solutions below