I am retargeting a cat object (target) from another object's (source) bone positions in Unity/C# environment.
- What I know: Source rest bone positions, target rest bone positions, source pose bone positions
- What I don't know: Source model, source rotations
- What I want: target pose bone positions (no rotations) retargeted from the source pose bone positions
My Current Approach:
I compute the local rest and pose directions for each bone in the source (from immediate parent to each bone), and then apply the rotation from the source rest to the source pose in the target model. I'm scaling the bone lengths based on a ratio calculated from the rest poses of the source and target.
Current Output:
The spine positions and the relative positions of leg bones seem correct. However, the first leg bone's relative position from the body/spine is incorrect, causing all child leg bone positions to be incorrect as well.
Possible Issue:
I think it is incorrect to calculate and apply rotation in this way. With different rest bone orientations of the two objects, the rotation becomes incorrect. In the spine and the bones in the leg chain, the relative direction from the parent is similar in the source and the target. But the direction from the spine4 (the spine bone that connects the front legs) to the first bones of the front legs are opposite in the two models. As I apply the rotation from source rest dir -> source pose dir to the target rest dir, it yields a wrong result. I thought I compute relative directions from the immediate parent so orientation won't be an issue, but I'm struggling for days.
Please help :(
Target pose calculation code
For each frame, I fix the root position first to be identical to that of the source, and then I iterate down the hierarchy to compute next bone's position.
// 1. Initial Setup - Rest Pose Positions
List<Vector3> sourceRestPositions = new List<Vector3> { /* the bone positions at source rest pose */ };
List<Vector3> targetRestPositions = new List<Vector3> { /* the bone positions at target rest pose */ };
// Source & Target - Calculate relative lengths and unit directions
Vector3[] sourceRestDirections = new Vector3[numJoints];
Vector3[] targetRestDirections = new Vector3[numJoints];
float[] restLengthRatios = new float[numJoints];
for (int i = 0; i < numJoints; i++)
{
if (bones[i] == null) continue; // Skip if no corresponding bone in the target
// Source
var sourceParentRestPosition = sourceRestPositions[(int)GetParentBone(i)];
var sourceLength = Vector3.Distance(sourceParentRestPosition, sourceRestPositions[i]);
sourceRestDirections[i] = (sourceRestPositions[i] - sourceParentRestPosition).normalized;
// Target
var targetParentRestPosition = targetRestPositions[(int)GetParentBone(i)];
var targetLength = Vector3.Distance(targetParentRestPosition, targetRestPositions[i]);
targetRestDirections[i] = (targetRestPositions[i] - targetParentRestPosition).normalized;
// Length ratio
restLengthRatios[i] = targetLength / sourceLength;
}
// 2. Retargeting
// Calculate retargeted joint positions
_targetJointSequence = new Vector3[sourceJointSequence.Length][];
for (int f = 0; f < sourceJointSequence.Length; f++)
{
// Fix root position
_targetJointSequence[f] = new Vector3[numJoints];
_targetJointSequence[f][(int)Bone.Spine0_BLegConnect] = sourceJointSequence[f][(int)Bone.Spine0_BLegConnect];
foreach (var childIdx in retargetOrder)
{
// Compute the local directions in source pose
int parentIdx = (int)GetParentBone(childIdx); // Get immediate parent
Vector3 sourcePoseParentPosition = sourceJointSequence[f][parentIdx];
Vector3 sourcePoseChildPosition = sourceJointSequence[f][childIdx];
Vector3 sourcePoseDirection = (sourcePoseChildPosition - sourcePoseParentPosition).normalized;
// Compute the length and rotation in source pose
float sourcePoseBoneLength = Vector3.Distance(sourcePoseChildPosition, sourcePoseParentPosition);
Quaternion sourceRotation = Quaternion.FromToRotation(sourceRestDirections[childIdx], sourcePoseDirection);
// Apply the rotation to the target
Vector3 targetPoseDirection = sourceRotation * targetRestDirections[childIdx];
// Compute the global positions in target pose
Vector3 targetParentPosePosition = _targetJointSequence[f][parentIdx];
Vector3 targetChildPosePosition = targetParentPosePosition + targetPoseDirection * restLengthRatios[childIdx] * sourcePoseBoneLength;
// Set the target pose position
_targetJointSequence[f][childIdx] = targetChildPosePosition;
}
Bone positions at rest position: Source (left) and Target (right)
The joints I use for retargeting are marked with spheres (blue: head, red: spine, white: legs, green: tail)

Bone positions at a pose: Source (left) and Target (right)

At a pose similar to the rest pose but different orientation
