ARKit combining ARImageAnchor position with offset values

Question

This is what I'm trying to figure out how to do. Let's say I have a SCNNode in my scene. (Eg: ARImageAnchor) Can be flat on a table or upright.

I'm trying to get the position and rotation if the image and draw a line with an offset value from the ARImage when I trigger a button. So instead of drawing line from the centre of the ARImage, It should be slightly off-centre depending on the offset values provided. But for some reason the world origin is not set accurately the first time I press the button. If i keep pressing the button it gets accurate progressively and finally gets drawn on the correct coordinates. Sometimes only takes two taps. Below is from the button tap function. ARSCNView is initiated when the ViewController is launched.

var offsetX: Float = 0.05
var offsetY: Float = 0.05
var offsetZ: Float = 0

let arMarkerPosition = SCNVector3(x, y, z)
let translationVector = SCNVector3(x: offsetX, y: offsetY, z: offsetZ)

// Create a translation matrix
let translationMatrix = SCNMatrix4MakeTranslation(translationVector.x, translationVector.y, translationVector.z)

// Apply translation to the arMarkerPosition
let translationTransform = SCNMatrix4Translate(translationMatrix, arMarkerPosition.x, arMarkerPosition.y, arMarkerPosition.z)


let pan: Float = 0
let rotationVector = SCNVector3(x: 0, y: 1, z: 0)

// Create a rotation matrix
let rotationMatrix = SCNMatrix4Rotate(SCNMatrix4Identity, pan, rotationVector.x, rotationVector.y, rotationVector.z)

// Combine translation and rotation into a single transformation matrix
let combinedTransform = SCNMatrix4Mult(translationTransform, rotationMatrix)

// Create a new SCNNode
let newNode = SCNNode()

// Apply the combined transformation to the new node's transform
newNode.transform = combinedTransform

// Add the new node to the scene
self.sceneView.scene.rootNode.addChildNode(newNode)

// Update the world origin to match the new node's position and orientation
self.sceneView.session.setWorldOrigin(relativeTransform: newNode.simdWorldTransform)

Any help would be much appreciated!