RealityKit - Moving an Entity constantly at a detected surface infront of the center of the screen

Question

I am trying to move an Entity (a brown MeshResource box like the one placed on the table). But I want the box to move at a detected point in the center of the screen (I use raycast) constantly as I move the camera. The object should stick to the wall (and rotate 90 degrees as if the wall is the new table) or stick to the table or floor.

So far I am doing fine and the box glides to the wall and floor as i pan the phone across the room. But If I detect an uneven surface such as the desk partition pictured here, the box will get crazy (moving and shaking randomly) and gets thrown away off the screen permanently even if I pan back to an even surface it wont come back. There is an error in the console printing constantly after the object gets thrown away (like below). Googling the error RETransformComponentSetLocalSRT yields zero results and Im lost on how to resolve this.

Call to RETransformComponentSetLocalSRT contains NaN. Skipping.

Here is my code...

let model = ModelEntity(mesh: shape, materials: [material])
let anchorEntity   = AnchorEntity(world: transform)
anchorEntity.addChild( model as! RealityKit.Entity)


if let raycastresult = arView.raycast(from: CGPoint(x: arView.frame.width/2, y: arView.frame.height/2), allowing: .estimatedPlane, alignment: .any).first {
    let wTransform = raycastresult.worldTransform
    let translation = SCNVector3(
        wTransform.columns.3.x,
        wTransform.columns.3.y,
        wTransform.columns.3.z
    )
    let rotation = wTransform.orientation
    let transform = Transform(scale: model.scale, rotation: rotation, translation: SIMD3(translation))
    model.move(to: transform, relativeTo: model.parent, duration: 1.5)
}

Any tips would be appreciated! Thanks in advance!