Salat: How to debug stack trace to know what field in case class is causing exception

Question

I am using Salat in my Scala / Scalatra project to serialize MongoDB objects. This is a general question in that I am trying to pull a complex MongoDB object from my database that embeds different hashes and arrays of its own that I have created case classes for. I could comment out field by field and figure out what in the case class is broken, but that is just an absurd way to debug this. The stack trace I get looks like the following:

DefaultArg.safeValue$lzycompute(Grater.scala:406)
com.novus.salat.DefaultArg.safeValue(Grater.scala:402)
com.novus.salat.ConcreteGrater.safeDefault(Grater.scala:363)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$3.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$3.apply(Grater.scala:262)
scala.Option.orElse(Option.scala:257)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:247)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.immutable.List.foreach(List.scala:318)
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
scala.collection.AbstractTraversable.map(Traversable.scala:105)
com.novus.salat.ConcreteGrater.asObject(Grater.scala:247)
com.novus.salat.transformers.in.DBObjectToInContext$class.transform0(Injectors.scala:333)
com.novus.salat.transformers.in.DBObjectToInContext$class.transform(Injectors.scala:339)
com.novus.salat.transformers.in.package$$anon$1.transform(Injectors.scala:66)
com.novus.salat.transformers.Transformer$$anonfun$1.apply(Transformer.scala:85)
com.novus.salat.transformers.Transformer$$anonfun$1.apply(Transformer.scala:85)
scala.Option.flatMap(Option.scala:170)
com.novus.salat.transformers.Transformer.transform_$bang(Transformer.scala:85)
com.novus.salat.Field.in_$bang(Field.scala:64)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$2.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$2.apply(Grater.scala:262)
scala.Option.flatMap(Option.scala:170)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:247)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.immutable.List.foreach(List.scala:318)
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
scala.collection.AbstractTraversable.map(Traversable.scala:105)
com.novus.salat.ConcreteGrater.asObject(Grater.scala:247)
com.novus.salat.transformers.in.DBObjectToInContext$class.transform0(Injectors.scala:333)
com.novus.salat.transformers.in.DBObjectToInContext$class.transform(Injectors.scala:338)
com.novus.salat.transformers.in.package$$anon$2.com$novus$salat$transformers$in$TraversableInjector$$super$transform(Injectors.scala:123)
com.novus.salat.transformers.in.TraversableInjector$class.transformElement(Injectors.scala:370)
com.novus.salat.transformers.in.package$$anon$2.transformElement(Injectors.scala:123)
com.novus.salat.transformers.in.TraversableInjector$$anonfun$after$1.apply(Injectors.scala:385)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.immutable.List.foreach(List.scala:318)
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
scala.collection.AbstractTraversable.map(Traversable.scala:105)
com.novus.salat.transformers.in.TraversableInjector$class.after(Injectors.scala:385)
com.novus.salat.transformers.in.package$$anon$2.after(Injectors.scala:123)
com.novus.salat.transformers.Transformer$$anonfun$1.apply(Transformer.scala:85)
com.novus.salat.transformers.Transformer$$anonfun$1.apply(Transformer.scala:85)
scala.Option.flatMap(Option.scala:170)
com.novus.salat.transformers.Transformer.transform_$bang(Transformer.scala:85)
com.novus.salat.Field.in_$bang(Field.scala:64)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$2.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5$$anonfun$apply$2.apply(Grater.scala:262)
scala.Option.flatMap(Option.scala:170)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:262)
com.novus.salat.ConcreteGrater$$anonfun$5.apply(Grater.scala:247)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
scala.collection.immutable.List.foreach(List.scala:318)
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
scala.collection.AbstractTraversable.map(Traversable.scala:105)
com.novus.salat.ConcreteGrater.asObject(Grater.scala:247)
com.sportsnetwork.api.models.MLBPlayByPlay$$anonfun$findByGameId$1.apply(MLBPlayByPlay.scala:38)
com.sportsnetwork.api.models.MLBPlayByPlay$$anonfun$findByGameId$1.apply(MLBPlayByPlay.scala:38)
scala.Option.map(Option.scala:145)
com.sportsnetwork.api.models.MLBPlayByPlay$.findByGameId(MLBPlayByPlay.scala:38)
com.sportsnetwork.api.MLBController$$anonfun$3.apply(MLBController.scala:49)
org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$liftAction(ScalatraBase.scala:250)
org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:245)
org.scalatra.ScalatraBase$$anonfun$invoke$1.apply(ScalatraBase.scala:245)
org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:175)
com.sportsnetwork.api.MLBController.withRouteMultiParams(MLBController.scala:20)
org.scalatra.ScalatraBase$class.invoke(ScalatraBase.scala:244)
com.sportsnetwork.api.MLBController.org$scalatra$json$JsonSupport$$super$invoke(MLBController.scala:20)
org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:65)
org.scalatra.json.JsonSupport$$anonfun$invoke$1.apply(JsonSupport.scala:57)
org.scalatra.ApiFormats$class.withRouteMultiParams(ApiFormats.scala:175)
com.sportsnetwork.api.MLBController.withRouteMultiParams(MLBController.scala:20)
org.scalatra.json.JsonSupport$class.invoke(JsonSupport.scala:57)
com.sportsnetwork.api.MLBController.invoke(MLBController.scala:20)
org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$7.apply(ScalatraBase.scala:220)
org.scalatra.ScalatraBase$$anonfun$runRoutes$1$$anonfun$apply$7.apply(ScalatraBase.scala:218)
scala.Option.flatMap(Option.scala:170)
org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:218)
org.scalatra.ScalatraBase$$anonfun$runRoutes$1.apply(ScalatraBase.scala:217)
scala.collection.immutable.Stream.flatMap(Stream.scala:442)
org.scalatra.ScalatraBase$class.runRoutes(ScalatraBase.scala:217)
org.scalatra.ScalatraServlet.runRoutes(ScalatraServlet.scala:49)
org.scalatra.ScalatraBase$class.runActions$1(ScalatraBase.scala:163)
org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply$mcV$sp(ScalatraBase.scala:175)
org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175)
org.scalatra.ScalatraBase$$anonfun$executeRoutes$1.apply(ScalatraBase.scala:175)
org.scalatra.ScalatraBase$class.org$scalatra$ScalatraBase$$cradleHalt(ScalatraBase.scala:190)
org.scalatra.ScalatraBase$class.executeRoutes(ScalatraBase.scala:175)
org.scalatra.ScalatraServlet.executeRoutes(ScalatraServlet.scala:49)
org.scalatra.ScalatraBase$$anonfun$handle$1.apply$mcV$sp(ScalatraBase.scala:113)
org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113)
org.scalatra.ScalatraBase$$anonfun$handle$1.apply(ScalatraBase.scala:113)
scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
org.scalatra.DynamicScope$class.withResponse(DynamicScope.scala:80)
org.scalatra.ScalatraServlet.withResponse(ScalatraServlet.scala:49)
org.scalatra.DynamicScope$$anonfun$withRequestResponse$1.apply(DynamicScope.scala:60)
scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
org.scalatra.DynamicScope$class.withRequest(DynamicScope.scala:71)
org.scalatra.ScalatraServlet.withRequest(ScalatraServlet.scala:49)
org.scalatra.DynamicScope$class.withRequestResponse(DynamicScope.scala:59)
org.scalatra.ScalatraServlet.withRequestResponse(ScalatraServlet.scala:49)
org.scalatra.ScalatraBase$class.handle(ScalatraBase.scala:111)
org.scalatra.ScalatraServlet.org$scalatra$servlet$ServletBase$$super$handle(ScalatraServlet.scala:49)
org.scalatra.servlet.ServletBase$class.handle(ServletBase.scala:43)
com.sportsnetwork.api.MLBController.org$scalatra$scalate$ScalateSupport$$super$handle(MLBController.scala:20)
org.scalatra.scalate.ScalateSupport$class.handle(ScalateSupport.scala:122)
com.sportsnetwork.api.MLBController.org$scalatra$CorsSupport$$super$handle(MLBController.scala:20)
org.scalatra.CorsSupport$$anonfun$handle$1.apply$mcV$sp(CORSSupport.scala:202)
org.scalatra.CorsSupport$$anonfun$handle$1.apply(CORSSupport.scala:190)
org.scalatra.CorsSupport$$anonfun$handle$1.apply(CORSSupport.scala:190)
scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
org.scalatra.DynamicScope$class.withResponse(DynamicScope.scala:80)
org.scalatra.ScalatraServlet.withResponse(ScalatraServlet.scala:49)
org.scalatra.DynamicScope$$anonfun$withRequestResponse$1.apply(DynamicScope.scala:60)
scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
org.scalatra.DynamicScope$class.withRequest(DynamicScope.scala:71)
org.scalatra.ScalatraServlet.withRequest(ScalatraServlet.scala:49)
org.scalatra.DynamicScope$class.withRequestResponse(DynamicScope.scala:59)
org.scalatra.ScalatraServlet.withRequestResponse(ScalatraServlet.scala:49)
org.scalatra.CorsSupport$class.handle(CORSSupport.scala:188)
com.sportsnetwork.api.MLBController.handle(MLBController.scala:20)
org.scalatra.ScalatraServlet.service(ScalatraServlet.scala:54)
javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:669)
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
org.eclipse.jetty.server.Server.handle(Server.java:361)
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:926)
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:988)
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
java.lang.Thread.run(Thread.java:701)

An example of my case classes is something such as the following:

case class MLBPlayByPlay(datetime: DateTime, gamedate: DateTime, game_id: Int, home_team_id: Int, away_team_id: Int,
                          home_team_score: Int, away_team_score: Int, sport: String, season_year: Int, season_type: String, location: String, status: String,
                          home_team_abbr: String, away_team_abbr: String, hp_umpire: String, fb_umpire: String, sb_umpire: String, tb_umpire: String,
                          play_by_play_seqs: Seq[MLBPlayByPlaySeq], created_at: Option[Date] = None, updated_at: Date)

case class MLBPlayByPlaySeq(batter: Option[MLBBatter] = None, next_batter: Option[MLBBatter] = None, pitchers: Option[Seq[MLBPitcher]] = None, pitches: Option[Seq[MLBPitches]] = None, fielders: Option[Seq[MLBFielder]] = None,
                            seq_id: Int, inning: Option[Int] = None, outs: Option[Int] = None, no_play: Option[Int] = None, home_team_score: Int, away_team_score: Int)

case class MLBBatter(batter_id: Int, team_id: Int, first_name: String, last_name: String, action: Option[String] = None, out: Option[Int] = None, adv: Option[Int] = None,
                     to_base: Option[Int] = None, scored: Option[Int] = None, bats: Option[String] = None, appear: Option[Int] = None, pos: Option[String] = None,
                     pitcher_id: Option[Int] = None, pthrows: Option[String] = None, error_ids: Option[String] = None, location: Option[String] = None, traj: Option[String] = None,
                     field: Option[String] = None, coords: Option[String] = None, ab: Option[Int] = None, gndout: Option[Int] = None)

case class MLBPitcher(pitcher_id: Int, team_id: Int, first_name: String, last_name: String, inning: Int, outs: Int, throws: String, bbats: String, ab: Int, bf: Int)
case class MLBFielder(fielder_id: Int, team_id: Int, first_name: String, last_name: String, pos: String, assist: Int, put_out: Int)
case class MLBPitches(count: Int, total_pitches: Int, balls: Int, strikes: Int, value: String, batter_id: Int, pitcher_id: Int, runner_1_id: Int, runner_2_id: Int, 
                      runner_3_id: Int, final_pitch: String)

I started debugging this by comment out field by field starting at the MLBPlayByPlay case class and was getting the API to respond just fine. Eventually when I uncommented play_by_play_seqs: Seq[MLBPlayByPlaySeq] everything blew up. Now I have to go into the MLBPlayByPlaySeq case class which embeds its own case class types and start commenting out one by one there... surely there is an easier way to debug Salat? This is how I'm using the grater:

 val res = MLBPlayByPlayDAO.findByGameId(gameId)
 res.map(grater[MLBPlayByPlay].asObject(_))

The actual exception:

k.api.MLBController - exception occured: com.novus.salat.DefaultArg.safeValue$lzycompute(Grater.scala:406)

Will update thread shortly after I remove Option[Seq[T]] from fields.

** UPDATE 2 **

After removing all Option[Seq[T]] from my case classes, I narrowed down the problem to be when next_batter is an empty Hash in my database for MLBPlayByPlaySeqs case class. I changed the definition from next_batter: Option[MLBBatter] = None (this would cause it to explode on an empty hash) to next_batter: Map[String, Any] = Map.empty and it works. The problem here is that:

• I lost type safety by using a map
• I can no longer hand pick what attributes I want listed in my serialized JSON output. Things like created_at and updated_at for the embedded Hash show up in my JSON and those are things I explicitly left out in the case class that I can't do by changing next_batter to a Map vs a case class.

Answer 1

Salat author here - could you share the actual exception from the stack trace?

Salat does not support options that contain collections - I see you have a few of those in MLBPlayByPlaySeq. Try turning them into Seq with a default value of Seq.empty and see if that clears things up.

If not, then please post the actual exception and I'll look at it again.