Recursive type iteration of object does not properly associate keys with values in Typescript

Question

I'm trying to define a TypeScript type that matches strings which work as accessors similar to JsonPath on strictly typed objects. So for a object with the type

{ root: { inner: { child: string } } } 

the string $root.inner.child should be a valid type, but $root.something.else should not be a valid type.

Here's what I've currently got (Typescript Playground):

type PropertyAccessor<Properties, Prefix extends string = ''> =
  Properties extends Primitive
    ? Prefix : (
      Properties extends { [Key in keyof Properties]: infer Value }
        ? PropertyAccessor<Value, `${Prefix}${Prefix extends '' ? '$' : '.'}${StringOrNever<keyof Properties>}`>
        : never
      );

// Helper types
type StringOrNever<T> = T extends string ? T : never;
type Primitive = string | number | boolean | bigint | symbol | null | undefined;

It almost works. When looking at the following example:

const example = {
    root1: {
        sub1: {
            subSub1: {
                subSubSub1: 'value'
            }
        },
        sub2: {
            subSub2: {
                subSubSub2: 'value'
            }
        },
    }
}

// Works as intended
const accessor1: PropertyAccessor<typeof example> = '$root1.sub1.subSub1.subSubSub1'; 

// Should throw an type error, but does not!
const accessor2: PropertyAccessor<typeof example> = '$root1.sub2.subSub1.subSubSub1';

// Correctly throws type error
const accessor3: PropertyAccessor<typeof example> = '$root1.sub1.subSub2.subSubSub1';

// Correctly throws type error
const accessor4: PropertyAccessor<typeof example> = '$root1.sub1.subSub1.subSubSub2';

it works for some strings, but not all. I think the issue is that, by deconstructing the object type into keyof Properties as Key, and Value = Properties[keyof Properties], I would get something like Key = 'sub1' | 'sub2' and Value = { subSub1: ... } | { subSub2: ... }, so individual keys are not properly associated with their respective values. However, the fact that type errors are correctly thrown for the examples accessor3 and accessor4 gives me hope that it should be doable somehow.

Does someone have an idea how I could adapt the type such that type errors are thrown for accessor2 as well, without accessor1 throwing a type error?`

The entire code is available as Typescript Playground.