OffScreenCanvas and touch events

Question

I'm trying to use the new API for the OffScreenCanvas. The idea is to move all logic regarding drawing and updating data for player, but leave some other logic in the main thread, like touch events (because worker cannot reach window object).

So I have class Player.js

export class Player {
  constructor() {
    this.width = 100;
    this.height = 100;
    this.posX = 0;
    this.posY = 0;
  }

  draw = (ctx) => {
    ctx.fillRect(this.posX, this.posY, this.width, this.height);
  }

  updatePos = (x, y) => {
    this.posX = x;
    this.posY = y;
  }
}

I generate instance of player in another module named playerObject.js

import { Player } from "./Player.js";

export const player = new Player();

OffScreenCanvas is created like this

const canvas = document.querySelector('#myGame');
const offscreen = canvas.transferControlToOffscreen();
const worker = new Worker('offscreencanvas.js', { type: "module" });
worker.postMessage({ canvas: offscreen }, [offscreen]);

Now I import playerObject to OffScreenCanvas worker

import {player} from "./playerObject.js";

addEventListener('message', (evt) => {
  const canvas = evt.data.canvas;
  const ctx = canvas.getContext("2d");

  const render = () => {
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    player.draw(ctx);

    requestAnimationFrame(render);
  }

  render();
});

and also to a class (module) that contains touch events, that are modifying player position:

import {player} from "./playerObject.js";

export class Game {
  constructor() {
    this.touch();
  }

  touch = () => {
    window.addEventListener('touchstart', (e) => {
      player.updatePos(e.touches[0].clientX, e.touches[0].clientY);

    }, {passive: true});
  }
}

The problem is that OffScreenCanvas doesn't see changes that are made by Game class. Touch itself is working (console.log shows event and also modified player object) but in OffScreenCanvas player still has the initial coorinations.

I'm still not sure whats happening there. Is worker creating a new instance of class and thats why it doesn't see changes from touch event?

Is there a way to achieve that?

Answer 1

There are things you need to know:

1. Each time you import {player} from "./playerObject.js"; you'll create a new instance of player. So it won't bridge OffscreenCanvasWorker module with Game module.
2. Instead of creating an instance of Player class in a module, why not create it globally then put it into Game's constructor parameter and Worker's message parameter.

Here's what I mean:

Player instance creation (globally)

import { Player } from "./Player.js";
const player = new Player();

Offscreen Canvas creation

const canvas = document.querySelector('#myGame');
const offscreen = canvas.transferControlToOffscreen();
const worker = new Worker('offscreencanvas.js', { type: "module" });
worker.postMessage({
    'canvas' : offscreen,
    'player' : player
}, [offscreen]);

Inside Offscreen Canvas Worker

addEventListener('message', (evt) => {
  const canvas = evt.data.canvas;
  const player = evt.data.player;
  const ctx = canvas.getContext("2d");

  const render = () => {
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    player.draw(ctx);

    requestAnimationFrame(render);
  }

  render();
});

Inside Game Module

export class Game {
  constructor(player) {
    this.player = player;
    this.touch();
  }

  touch = () => {
    const player = this.player;
    window.addEventListener('touchstart', (e) => {
      player.updatePos(e.touches[0].clientX, e.touches[0].clientY);

    }, {passive: true});
  }
}

There are actually many alternative ways if you don't like this way, but most important thing is you must transport the player instance object not its class nor new instance declaration in a module.

Additional NOTE: Just in case you don't know, in 2020 OffscreenCanvas still experimental and won't be good idea if you put this to your on-air website. Good luck!

Answer 2

You currently have two distinct Player instances, which won't communicate one with the other.

There is a proposal in rough draft for enabling passing some events from the main thread to Worker threads, but it's still really just a draft, and I'm not sure at all when it will come to day, nor in what shape exactly.

Currently, the only solution we have is to build ourselves a bridge between the main thread and the Worker thread, so that the main thread emits all the events to the Worker when they happen.
This has a lot of drawbacks, among which

• obvious latency, since we have to wait for the main thread receives the event before dispatching a new MessageEvent task,
• dependency to the main thread: it has to be free to handle the events, this means the promise that OffscreenCanvas can run smoothly even when the main thread is locked is broken here.
• hard to maintain (?) Since we don't have a clear API to access which target we are getting, we have to resort in ugly hard-coded values in both the main thread and the worker thread

But still, we can achieve something.

I just had some time to write a rough play-toy, loosely based on the current proposal I linked to, with a few changes, and no testing, so use it as a base to write your own, but don't expect it to work seamlessly in every situations.

The basic logic is

• From the main-thread we enhance the Worker interface to make it start a private communication channel with the Worker thread (through a MessageChannel object).
• We also add and addEventTarget( target, uuid ) method, to that interface.
• From the Worker thread, we initialize a receiver, when we receive the message from our main-thread's script. From there, we hold the MessageChannel and wait until it says when new delegatedTargets have been declared from the main-thread.
• When this happens, we fire a new event eventtargetadded that the user-script running in the Worker can listen to, exposing an EventDelegate instance which will create a new private communication channel between the Worker and the main threads.
• It's through this EventDelegate's channel that each event object rom the main thread will get cloned, after it has been sanitized.

But enough words and hard to grasp explanations, here is a plnkr, where it might be clearer how it works.

Here is a StackSnippet live version, probably a bit harder to read:

// StackSnippet only: build up internal path to our Worker scripts
const event_delegate_worker_script = document.getElementById( 'event-delegate-worker' ).textContent;
const event_delegate_worker_url = generateScriptURL( event_delegate_worker_script );

const user_script_worker_script = document.getElementById( 'user-script-worker' ).textContent
  .replace( "event-delegate-worker.js", event_delegate_worker_url );
const user_script_worker_url = generateScriptURL( user_script_worker_script );

function generateScriptURL( content ) {
  // Chrome refuses to importScripts blob:// URI...
  return 'data:text/javascript,' + encodeURIComponent( content );
}

// end StackSnippets only

onload = evt => {
  const worker = new EventDelegatingWorker( user_script_worker_url );
  const canvas = document.getElementById( 'canvas' );
  worker.addEventTarget( canvas, "canvas" );

  try {
    const off_canvas = canvas.transferControlToOffscreen();
    worker.postMessage( off_canvas, [ off_canvas ] );    
  }
  catch (e) {
    // no support for OffscreenCanvas, we'll just log evt
    worker.onmessage = (evt) => { console.log( "from worker", evt.data ); }
  }
};

canvas { border: 1px solid; }

<canvas id="canvas"width="500" height="500"></canvas>

<script id="user-script-worker" type="worker-script">
importScripts( "event-delegate-worker.js" );

self.addEventListener( "eventtargetadded", ({ delegatedTarget }) => {
  if( delegatedTarget.context === "canvas" ) {
    delegatedTarget.addEventListener( "mousemove", handleMouseMove );
  }
} );

let ctx;
function handleMouseMove( evt ) {
  if( ctx ) {
    draw( evt.offsetX, evt.offsetY );
  }
  else {
    // so we can log for browsers without OffscreenCanvas
    postMessage( evt );
  }
}

function draw( x, y ) {

  const rad = 30;
  ctx.clearRect( 0, 0, ctx.canvas.width, ctx.canvas.height );
  ctx.beginPath();
  ctx.arc( x, y, rad, 0, Math.PI*2 );
  ctx.fill();

}

onmessage = (evt) => {
  const canvas = evt.data;
  ctx = canvas.getContext("2d");
};
</script>

<!-- below are the two scripts required to bridge the events -->
<script id="event-delegate-main">
(()=> { "use strict";

  const default_event_options_dict = {
    capture: false,
    passive: true
  };
  const event_keys_to_remove = new Set( [
    "view",
    "target",
    "currentTarget"
  ] );
  class EventDelegatingWorker extends Worker {
    constructor( url, options ) {
      super( url, options );
      // this channel will be used to notify the Worker of added targets
      const channel = new MessageChannel();
      this._mainPort = channel.port2;
      this.postMessage( "init-event-delegation", [ channel.port1 ] );
    }
    addEventTarget( event_target, context ) {
      // this channel will be used to notify us when the Worker adds or removes listeners
      // and to notify the worker of new events fired on the target
      const channel = new MessageChannel();
      channel.port1.onmessage = (evt) => {
        const { type, action } = evt.data;
        if( action === "add" ) {
          event_target.addEventListener( type, handleDOMEvent, default_event_options_dict );        
        }
        else if( action === "remove" ) {
          event_target.removeEventListener( type, handleDOMEvent, default_event_options_dict );        
        }
      };
      // let the Worker side know they have a new target they can listen on
      this._mainPort.postMessage( context, [ channel.port2 ] );
      
      function handleDOMEvent( domEvent ) {
        channel.port1.postMessage( sanitizeEvent( domEvent ) );
      }
    }
  }
  window.EventDelegatingWorker = EventDelegatingWorker;

  // Events can not be cloned as is, so we need to stripe out all non cloneable properties
  function sanitizeEvent( evt ) {
    
    const copy = {};
    // Most events only have .isTrusted as own property, so we use a for in loop to get all
    // otherwise JSON.stringify() would just ignore them
    for( let key in evt ) {
      if( event_keys_to_remove.has( key ) ) {
        continue;
      }
      copy[ key ] = evt[ key ];      
    }
    
    const as_string = tryToStringify( copy );
    return JSON.parse( as_string );

    // over complicated recursive function to handle cross-origin access
    function tryToStringify() {
      const referenced_objects = new Set; // for cyclic
      // for cross-origin objects (e.g window.parent in a cross-origin iframe)
      // we save the previous key value so we can delete it if throwing
      let lastKey;  
      let nextVal = copy;
      let lastVal = copy;
      try {
        return JSON.stringify( copy, removeDOMRefsFunctionsAndCyclics );
      }
      catch( e ) {   
        delete lastVal[ lastKey ];
        return tryToStringify();
      }
      
      function removeDOMRefsFunctionsAndCyclics( key, value ) {
        lastVal = nextVal;
        lastKey = key;
        
        if( typeof value === "function" ) {
          return;
        }
        if( typeof value === "string" || typeof value === "number") {
          return value;
        }
        if( value && typeof value === "object" ) {
          if( value instanceof Node ) {
            return;
          }
          if( referenced_objects.has( value ) ) {
            return "[cyclic]";
          }
          referenced_objects.add( value );
          nextVal = value;
          return value;
        }
        return value;
      }
    }
  }

})();
</script>
<script id="event-delegate-worker" type="worker-script">
(()=> { "use strict";

// This script should be imported at the top of user's worker-script
function initDelegatedEventReceiver( evt ) {

  // currently the only option is "once"
  const defaultOptionsDict = {
    once: false,
  };
  // in case it's not our message (which would be quite odd...)
  if( evt.data !== "init-event-delegation" ) {
    return;
  }

  // let's not let user-script know it happend
  evt.stopImmediatePropagation();
  removeEventListener( 'message', initDelegatedEventReceiver, true );

  // this is where the main thread will let us know when a new target is available
  const main_port = evt.ports[ 0 ];

  class EventDelegate {
    constructor( port, context ) {
      this.port = port; // the port to communicate with main
      this.context = context; // can help identify our target
      this.callbacks = {}; // we'll store the added callbacks here
      // this will fire when main thread fired an event on our target
      port.onmessage = (evt) => {
        const evt_object = evt.data;
        const slot = this.callbacks[ evt_object.type ];
        if( slot ) {
          const to_remove = [];
          slot.forEach( ({ callback, options }, index) => {
            try {
              callback( evt_object );
            }
            catch( e ) {
              // we don't want to block our execution,
              // but still, we should notify the exception
              setTimeout( () => { throw e; } );
            }
            if( options.once ) {
              to_remove.push( index );
            }
          } );
          // remove 'once' events
          to_remove.reverse().forEach( index => slot.splice( index, 1 ) );
        }
      };
    }
    addEventListener( type, callback, options = defaultOptionsDict ) {

      const callbacks = this.callbacks;
      let slot = callbacks[ type ];
      if( !slot ) {
        slot = callbacks[ type ] = [];
        // make the main thread attach only a single event,
        // we'll handle the multiple callbacks
        // and since we force { passive: true, capture: false }
        // they'll all get attached the same way there
        this.port.postMessage( { type, action: "add" } );
      }
      // to store internally, and avoid duplicates (like EventTarget.addEventListener does)
      const new_item = {
          callback,
          options,
          options_as_string: stringifyOptions( options )
        };
      if( !getStoredItem( slot, new_item ) ) {
        slot.push( new_item );
      }

    }
    removeEventListener( type, callback, options = defaultOptionsDict ) {

      const callbacks = this.callbacks;
      const slot = callbacks[ type ];
      const options_as_string = stringifyOptions( options );

      const item = getStoredItem( slot, { callback, options, options_as_string } );
      const index = item && slot.indexOf( item );

      if( item ) {
        slot.splice( index, 1 );
      }
      if( slot && !slot.length ) {
        delete callbacks[ type ];
        // we tell the main thread to remove the event handler
        // only when there is no callbacks of this type anymore
        this.port.postMessage( { type, action: "remove" } );
      }

    }
  }
  // EventInitOptions need to be serialized in a deterministic way
  // so we can detect duplicates 
  function stringifyOptions( options ) {
    if( typeof options === "boolean" ) {
      options = { once: options };
    }
    try {
      return JSON.stringify(
        Object.fromEntries(
          Object.entries(
            options
          ).sort( byKeyAlpha )
        )
      );
    } 
    catch( e ) {
      return JSON.stringify( defaultOptionsDict );
    }
  }
  function byKeyAlpha( entry_a, entry_b ) {
    return entry_a[ 0 ].localeCompare( entry_b[ 0 ] );
  }
  
  // retrieves an event item in a slot based on its callback and its stringified options
  function getStoredItem( slot, { callback, options_as_string } ) {
    return Array.isArray( slot ) && slot.find( (obj) => {
      return obj.callback === callback &&
        obj.options_as_string === options_as_string;
    } );
  }

  // a new EventTarget has been declared by main thread
  main_port.onmessage = evt => {
    const target_added_evt = new Event( 'eventtargetadded' );
    target_added_evt.delegatedTarget = new EventDelegate( evt.ports[ 0 ], evt.data );
    dispatchEvent( target_added_evt );
  };
  
}
addEventListener( 'message', initDelegatedEventReceiver );

})();
</script>

---

Ps: Since this answer has been posted, I did start implementing an EventPort interface, loosely based on this PR, which may be easier to use, and closer to the final specs.
A bit long to post in a Stackoverflow answer though.
You can see live examples here.