Why does `IPython.display.Javascript(...)` work to run a Javascript command in a Jupyter cell but not in an IPython magic?

Question

I'm trying to write a Jupyter nbextension where the user can configure the notebook-wide extension behavior using IPython magics. I'm planning on siting the configuration data on the JS side, under an application-specific key in Jupyter.notebook.metadata, "tempvars". In order to do this, I need the magics to be able to change the appropriate values within that ....metadata["tempvars"].

If I load a Jupyter notebook and make metadata changes using IPython.display.Javascript directly from a notebook cell, they apply successfully.

JS console:

>> Jupyter.notebook.metadata["tempvars"]["universal"] = false
<- false

Jupyter cell:

[ ]: from IPython.display import Javascript
[ ]: Javascript("Jupyter.notebook.metadata['tempvars']['universal'] = true")

JS console:

>> Jupyter.notebook.metadata["tempvars"]["universal"]
<- true  // CHANGED AS EXPECTED

However, if I try to do the same thing from inside a magic, it does not work:

From my magics definition module, within the broader jupyter_tempvars package (please don't mind the very WIP implementation):

from IPython.core.magic import register_line_magic

def load_ipython_extension(ipython):
    @register_line_magic
    def tempvars(line):
        subcommand, arg = line.strip().split(" ")
        if subcommand == "universal":
            Javascript(f"Jupyter.notebook.metadata['tempvars']['universal'] = {arg.lower()};")

From a freshly loaded notebook, starting in the JS console:

>> Jupyter.notebook.metadata["tempvars"]["universal"] = false
<- false

Jupyter cells:

[ ]: %load_ext jupyter_tempvars

[ ]: %tempvars universal true

JS console:

>> Jupyter.notebook.metadata["tempvars"]["universal"]
<- false  // DID NOT CHANGE

What is different about the IPython magic context that is making this Javascript call fail?

Answer 1

According to the IPython docs for the Javascript object:

Create a Javascript display object given raw data.

When this object is returned by an expression or passed to the display function, it will result in the data being displayed in the frontend.

As it turns out, the rendering of the JS code upon simply being returned by an expression is not a universal behavior. In particular, inside an IPython magic, you must pass the Javascript instance to IPython.display.display() in order for it to be rendered.

This is what IPython does in its own built-in (and deprecation-pending) %javascript magic.