SIgnature pad: Can not properly detect mouse coordinates after scaling

Question

I am using phrogz's solution for zooming and panning. Basically for a drawing pad.

The issue is, after zooming and panning, the mouse movement and the drawing movement have a bit of distance, i.e. I mouse the mouse somewhere, it draws elsewhere.

The blue represents my actual mouse movement Example

From his example, it may seem like he has handled it here, however, I am still facing the issue

     canvas.addEventListener('mousedown',function(evt){
      document.body.style.mozUserSelect = document.body.style.webkitUserSelect = document.body.style.userSelect = 'none';
      lastX = evt.offsetX || (evt.pageX - canvas.offsetLeft);
      lastY = evt.offsetY || (evt.pageY - canvas.offsetTop);
      dragStart = ctx.transformedPoint(lastX,lastY);
      dragged = false;
  },false);