React useInView triggers before Element is in view

Question

I am building a scroll triggered animation in React, however some elements are triggered before they are in the view. For this I have built a small component for triggering:

import { useInView } from 'react-intersection-observer';
import { useAnimation } from 'framer-motion';

export const useScroll = (treshhold = .2) => {
    const controls = useAnimation();
    const [element, view] = useInView({ threshold: treshhold, triggerOnce: true });
    if (view) {
        controls.start('show');
    } else {
        controls.start('hidden');
    }
    return [element, controls];
}

For a better illustration, I have sketched it out here:

The black frame is to represent the viewport. There is a hero section which is also animated when the page is loaded (the red one in the black frame).

In the next section, which uses the scrollTriger controls, the elements have already been triggered and roughly corresponds to the viewport height. (Green frames under the viewport)

The blue frames below are the same elements as those in the green frame, but are triggered as soon as they enter the viewport. (desired behaviour) The blue and green frames have the same parent and do not differ further.

In the animations, the elements are only scaled, i.e. not shifted in height.

I can't find any reason why it behaves the way it does.