Boost.python pull object into local scope for read access

Question

I am exporting a C++ function to Python using Boost.Python. In this function, I would like to access a Python object on the local stack frame, without the need for this object to be accessed on the Python side.

Example: I have a C++ object X and a C++ function cppfun exported to Python, and then following code,

x = X()

def fun():
    cppfun()
    # no explicit access to x

Inside cppfun I can access the local stack frame using PyEval_GetLocals, but since the Python side does not contain any reference to x, this object will not be in that local stack frame (similarly, if you print(locals()) inside fun, x will not be there unless you add a reference to it, e.g. print(x)).

Is there any way in which I can access x inside cppfun, that is, can I somehow force Python to pull it into the locals without accessing it within fun on the Python side? I tried simply running boost::python::eval("x") but that runs in the wrong scope as well.

added: so I do not want to make the x from the outer frame writeable inside fun or something like that (I know that's not possible); this question is purely about how to get read access to a variable from the outer frame without accessing it on the Python side.

Answer 1

One can access the outer scope through the use of PyEval_GetGlobals(), which returns dictionary of the global variables in the current execution frame. It is fairly similar to the Python builtin globals() function, which returns the global symbol table of the module in which the function is defined. However, frame stacks are handled slightly differently between standard modules and extension modules. For extension modules, the current execution frame is the caller, and thus PyEval_GetGlobals() will return the global symbols for the caller's module, rather than the extension module's globals.

Alternative, one can obtain a handle to the current frame via PyEval_GetFrame(), then walk through the stack, examining each frame's local (f_local) and global (f_global) dictionary.

---

Here is a complete minimal example demonstrating both techniques and the subtle differences between them. In the example, get_global() will use PyEval_GetGlobals(),and search_stack_locals() will examine the local dictionary for each frame in the stack.

#include <boost/python.hpp>

namespace detail {

/// @brief Return a handle to a global variable.  If one is not found, then
///        None is returned.
boost::python::object get_global(std::string name)
{
  // Borrow a reference from the locals dictionary to create a handle.
  // If PyEval_GetGlobals() returns NULL, then Boost.Python will throw.
  namespace python = boost::python;
  python::dict globals(python::borrowed(PyEval_GetGlobals()));
  return globals.get(name);
}

/// @brief Search through the call stack for a variable.  If found, the
///        object is returned.  Otherwise, None.
boost::python::object search_stack_locals(std::string name)
{
  // Get a handle to the current frame.
  namespace python = boost::python;
  python::object frame(python::borrowed(
    reinterpret_cast<PyObject*>(PyEval_GetFrame())));

  // Iterate through the stack's frames until the variable has been found
  // or the stack has been exhausted.
  while (frame)
  {
    // If the current frame has the desired variable, then return it.
    python::dict locals(frame.attr("f_locals"));
    if (locals.has_key(name))
    {
      return locals.get(name);
    }

    // Move up the stack.
    frame = frame.attr("f_back");
  }

  // Return None
  return python::object();
}

} // namespace detail

/// @brief Mockup function to demonstrate finding non-local variables.
boost::python::object cppfun(std::string name)
{
  return boost::python::make_tuple(
    detail::get_global(name), detail::search_stack_locals(name));
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::def("cppfun", &cppfun);
}

In spam.py:

import example

x = 'spam.global.x'
y = 'spam.global.y'

def inner():
    for name in ('x', 'y', 'z'):
        print name, example.cppfun(name)

def outer():
    x = 'spam.outer.x'
    inner()

Interactive usage:

>>> x = 'main.global.x'
>>> y = 'main.global.y'
>>> z = 'main.global.z'
>>> import spam
>>> spam.outer()
x ('spam.global.x', 'spam.outer.x')
y ('spam.global.y', 'main.global.y')
z (None, 'main.global.z')

Take note that when using PyEval_GetGlobals(), the example extension module used the caller's module's global symbol table (spam). The globals declared in the interpreter's main namespace were only found when iterating through the stack.