Integrating Pivot Controls (drei) with Physics World (rapier) using three-react/fiber three-react/drei and three-react/rapier

Question

The use-case I am building is to have in a physics world an object that can be moved (in any direction) and on release (exit of pivot controls), the physics world should apply viz. if the object is lifted, it should fall down by gravity.

I find that when the object (box) is moved, it is visible but in the physics world it remains at the originals setting. This can be confirmed using the debug mode.

Below is the code I have:

import * as THREE from 'three'
import { useRef, useState } from 'react'
import { Canvas } from '@react-three/fiber'
import { OrbitControls, PivotControls, PerspectiveCamera } from '@react-three/drei'
import { Physics, RigidBody, useFixedJoint } from '@react-three/rapier'

export default function App() {

    const Mat = () => {
        return (
            <RigidBody type={"fixed"} mass={50} colliders={"cuboid"} restitution={0.8} position={[0, 0.02, 0]}>
                <mesh>
                    <boxGeometry args={[10, 0.04, 10]} />
                    <meshBasicMaterial color="#8a8463" />
                </mesh>
            </RigidBody>
        )
    }
    const ref = useRef(null);
    return (
        <Canvas shadows>
            <PerspectiveCamera makeDefault position={[-2, 0, 2]} />
            <ambientLight intensity={Math.PI / 2} />
            <Physics debug>
                <RigidBody>
                    <PivotControls ref={ref} scale={0.3} disableRotations={true} autoTransform={true} anchor={[0, 0, 0]}
                        onDrag={(local) => {
                            const position = new THREE.Vector3()
                            const scale = new THREE.Vector3()
                            const quaternion = new THREE.Quaternion()
                            local.decompose(position, quaternion, scale)
                            // apply these to anything you want
                            ref.current.position.copy(position)
                            ref.current.scale.copy(scale)
                            ref.current.quaternion.copy(quaternion);
                            //console.log(position);
                        }}

                        onDragEnd={() => {
                            console.log('end');
                            console.log(ref.current.position);
                        }}
                    >
                        <mesh position={[0, 0.5, -3]}>
                            <boxGeometry args={[0.2, 0.2, 0.2]} />
                            <meshBasicMaterial attach="material" color="red" />
                        </mesh>
                    </PivotControls>
                </RigidBody>
                <Mat />
            </Physics>
            <OrbitControls makeDefault />
        </Canvas>
    )
}

Any suggestions on how to update the position in the physics world. Perhaps on the exit of the Pivot Controls. Thanks