Embed widgets with pythreejs: wrong perspective and camera look-at

Question

I am using pythreejs to visualize some 3D models.

When visualizing the models on a Jupyter notebook, everything works as expected.

But when trying to embed the widget in an HTML document, I am facing two problems:

• It seems the camera, on load, is looking at (0, 0, 0), not as expected, and once you interact with the widget, the camera will "jump" and start looking at the expected coordinate
• The projection (ortographic camera mode) is lost too

Here is the code to reproduce the error and an animation of the mentioned problem:

from ipywidgets import embed
from pythreejs import *
from IPython.display import display

base = Mesh(
    BoxBufferGeometry(20, 0.1, 20), 
    MeshLambertMaterial(color='green', opacity=0.5, transparent=True),
    position=(0, 0, 0),
)
cube = Mesh(
    BoxBufferGeometry(10, 10, 10), 
    MeshLambertMaterial(color='green', opacity=0.5, transparent=False),
    position=(0, 5, 0),
)
target = (0, 5, 0)
view_width = 600
view_height = 400
camera = CombinedCamera(position=[60, 60, 60], width=view_width, height=view_height)
camera.mode = 'orthographic'
lights = [
    PointLight(position=[100, 0, 0], color="#ffffff"),
    PointLight(position=[0, 100, 0], color="#bbbbbb"),
    PointLight(position=[0, 0, 100], color="#888888"),
    AmbientLight(intensity=0.2),
]
orbit = OrbitControls(controlling=camera, target=target)
camera.lookAt(target)
scene = Scene(children=[base, cube, camera] + lights)
renderer = Renderer(scene=scene, camera=camera, controls=[orbit],
                    width=view_width, height=view_height)
camera.zoom = 4

embed.embed_minimal_html('export.html', views=renderer, title='Renderer')
display(renderer)

The result looks good in the notebook:

But when opening the export.html file:

Note how the view of the cube "jumps" suddenly on interaction and how the projection is different: perspective instead of orthographic (parallel) projection.

Could it be an issue with ipywidgets? Since the view is okay when displayed in the notebook.

How could it be fixed?

Answer 1

This is a bug in the CombinedCamera code in pythreejs (https://github.com/jupyter-widgets/pythreejs/issues/308). The sync logic will need to call updateProjectionMatrix on the camera when any of the attributes change. The OrbitController calls this when you interact with it, which is why the view is 'fixed' when interacting.

Answer 2

It took a few days, didn't get cadquery working properly, neither the code from your first question on this topic without cadquery The code here made it possible to look at the issue...

The jumping happens because orbit.update() for target does not occurs and the function update() is not available in python; only in c++ or c#, etc. From the docs:

When animating the camera rotation above, we used the camera’s quaternion . This is the most robust method for animating free-form rotations. For example, the animation above was created by first moving the camera manually, and then reading out its position and quaternion properties at the wanted views...

The text can be found here on page 12. And also discussed here at github.

However, the jumping can be reproduced in IPython if you apply the following:

renderer = Renderer(scene=scene, camera=camera, controls=[orbit], position=target, width=view_width, height=view_height)

here position is added with target coordinates [0, 5, 0] but the update for this is only done when you mouse-click and adjust to position of the cube/camera. The jump is similar/equal to the jump as seen in the export.HTML.

Conclusion: the programmed camera position is seen as a jump after manual interference due to the absense of the .update() function of the OrbitControls python class and thus not a bug or mistake.

Update 1 - Ipython Renderer output while not ran in jupyter-notebook:

target = (0,5,0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), target=(0.0, 5.0, 0.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0,0,0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0, 5, 0) and position=target
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), target=(0.0, 5.0, 0.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

target = (0,0,0)  and position=(0, 5, 0)
Renderer(camera=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), controls=[OrbitControls(controlling=CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0))], scene=Scene(children=(Mesh(geometry=BoxBufferGeometry(depth=20.0, height=0.1, width=20.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None, transparent=True), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), Mesh(geometry=BoxBufferGeometry(depth=10.0, height=10.0, width=10.0), material=MeshLambertMaterial(alphaMap=None, aoMap=None, color='green', emissiveMap=None, envMap=None, lightMap=None, map=None, opacity=0.5, specularMap=None), position=(0.0, 5.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), CombinedCamera(height=400.0, mode='orthographic', position=(60.0, 60.0, 60.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0), width=600.0, zoom=4.0), PointLight(position=(100.0, 0.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#bbbbbb', position=(0.0, 100.0, 0.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), PointLight(color='#888888', position=(0.0, 0.0, 100.0), quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), AmbientLight(intensity=0.2, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0))), fog=None, overrideMaterial=None, quaternion=(0.0, 0.0, 0.0, 1.0), scale=(1.0, 1.0, 1.0), up=(0.0, 1.0, 0.0)), shadowMap=WebGLShadowMap())

To test the .update() for the export.html I consider a search for free SDD so I'm able to test it in a linux conda environment using my current yml-file.