Unity C# Climbing issue

Question

i got a climbing script in unity for VR and i have some issues with it.

the first issue is that if i start climbing on a object i get teleported a little before my hand is at the position it should be again.

the second issue is that its responding too slow. its smoothing my movement out which it shouldnt it should be moving my player exactly when im moving my hand too.

this is the part that is handling the climbing which is in FixedUpdate():

Vector3 deltaPosition = currentObjectClimbingOn.position - lastHandPosition;
        rb.MovePosition(rb.position + rb.transform.rotation * deltaPosition * 10f * Time.fixedDeltaTime);
        lastHandPosition = transform.position;

i already tried removing the Time.fixedDeltaTime and also tried changing the multiplier from 10f to something like 100f or 1f but with 100f or no Time.fixedDeltaTime im getting flung around and with 1f the movement is even slower.

i also tried using the XRControllers velocity before but when moving the hand quickly it was moving away from the object i grabbed on so players were able to fly.