Moving Three.js camera using device orientation - Strange behavior when the device is placed vertically

Question

I'm creating a small project using three.js where the camera is inside a textured sweater. Rotating the camera is to be done using the orientation of the phone in space. At first glance, my solution works fine, but as the device approaches the vertical position (angle Beta approaches 90 degrees), the image goes crazy and rotates unexpectedly. It looks like the device has rotated 360 degrees around one of the axes (corresponding to the alpha angle).

createScene() {
    const scene = new THREE.Scene();
    console.log('scene', scene);

    // create a new Three.js perspective camera
    const camera = new THREE.PerspectiveCamera(
      75, // field of view
      window.innerWidth / window.innerHeight, // aspect ratio
      0.1, // near plane
      1000 // far plane
    );

    // create a new Three.js renderer
    const renderer = new THREE.WebGLRenderer();

    // set the renderer size to the window dimensions
    renderer.setSize(window.innerWidth, window.innerHeight);

    // get the container element for the background
    const backgroundContainer = document.querySelector('.vr-sphere');

    if (backgroundContainer == null) return;

    backgroundContainer.appendChild(renderer.domElement);

    const loader = new THREE.TextureLoader();

    const texture = loader.load(
      '../../assets/images/oil_painting_van_gogh_starry_night.jfif'
    );

    const geometry = new THREE.SphereGeometry(500, 60, 40);

    // invert the geometry to render it inside out
    geometry.scale(-1, 1, 1);

    const material = new THREE.MeshBasicMaterial({
      map: texture,
    });

    const sphere = new THREE.Mesh(geometry, material);

    scene.add(sphere);

    // set the camera's position in the scene
    camera.position.set(0, 0, 0);

    // enable VR mode based on DeviceOrientation API
    if (window.DeviceOrientationEvent) {
      window.addEventListener('deviceorientation', (event) => {
        let alpha = event.alpha ? THREE.MathUtils.degToRad(event.alpha) : 0; // Z
        let beta = event.beta ? THREE.MathUtils.degToRad(event.beta) : 0; // X'

        if (alpha && beta) {
          if (beta >= 0 && beta <= 180) {
            // up/down rotation
            camera.rotation.set(-Math.PI / 2 + beta, 0, 0);
            // left/right rotation
            sphere.rotation.y = -alpha;
        }
      });
    }

    function animate() {
      // request the next frame of the animation
      requestAnimationFrame(animate);

      renderer.render(scene, camera);
    }

    animate();

  }

I've read that the solution to this problem can be to use quanterions or rotation matrices. I tried implementing them using three.js built-in functions but the effect was the same. Has anyone encountered a similar problem and knows how to solve it?

Answer 1

You found the problem: use quaternions.

I wrote this class that you're free to reuse or modify (citation is welcome), to add device orientation controls to three.js project.

Instead of DeviceOrientationEvent, I use AbsoluteOrientationSensor

The crucial part is here:

// Retrieve device absolute rotation quaternion
const q1 = new THREE.Quaternion().fromArray(this.sensor.quaternion)

// Ispired by https://gist.github.com/kopiro/86aac4eb19ac29ae62c950ad2106a10e
// Apply rotation around x axis (camera points toward the back of the phone, not up)
// and then update camera quaternion
const q2 = new THREE.Quaternion().setFromAxisAngle( new THREE.Vector3(1, 0, 0), -Math.PI/2 )
this.camera.quaternion.multiplyQuaternions(q2, q1)