SWIG and Python: How can I add a manually-created class to a SWIG-generated module?

Question

My project uses SWIG to automatically create wrappers for a set of C++ functions and types, in a single module called tensorflow.python.pywrap_tensorflow. I want to define a new type using the Python C API directly, and add it to that module. (In particular, I want to define a new type that implements the Python buffer protocol, so that I can expose a native buffer as a memoryview.)

I can define the requisite structs for my new type by inlining it in a .i file:

%{

typedef struct {
  PyObject_HEAD
  /* Other fields... */
} MyType;

// Define functions for the initializer, destructor, and buffer protocol:
// * MyType_init
// * MyType_dealloc
// * MyType_old_getbuffer (the readbufferproc for Python 2.7)
// * MyType_segcount (for Python 2.7)
// * MyType_getbuffer (the Python 2.7/3.x buffer protocol)

// ...

static PyBufferProcs MyType_as_buffer = {
#if PY_VERSION_HEX < 0x03000000
  (readbufferproc)MyType_old_getbuffer,
  (writebufferproc)0,
  (segcountproc)MyType_segcount,
  (charbufferproc)0,
#endif
  (getbufferproc)MyType_getbuffer,
  (releasebufferproc)0,
};

static PyTypeObject MyType_TypeObject = {
  /* PyObject header changed in Python 3 */
#if PY_VERSION_HEX>=0x03000000
  PyVarObject_HEAD_INIT(NULL, 0)
#else
  PyObject_HEAD_INIT(NULL)
  0 /* ob_size */,
#endif
  "MyType" /* tp_name */,
  sizeof(MyType) /* tp_basicsize */,
  0 /* tp_itemsize */,
  (destructor)MyType_dealloc /* tp_dealloc */,
  0 /* tp_print */,
  0 /* tp_getattr */,
  0 /* tp_setattr */,
#if PY_VERSION_HEX>=0x03000000
  0 /* tp_reserved in 3.0.1 */
#else
  0 /* tp_compare */,
#endif
  0 /* tp_repr */,
  0 /* tp_as_number */,
  0 /* tp_as_sequence */,
  0 /* tp_as_mapping */,
  0 /* tp_hash */,
  0 /* tp_call */,
  0 /* tp_str */,
  0 /* tp_getattro */,
  0 /* tp_setattro */,
  &MyType_as_buffer /* tp_as_buffer */,
  Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_NEWBUFFER /* tp_flags */,
  "Python wrapper for MyType." /* tp_doc */,
  0 /* tp_traverse */,
  0 /* tp_clear */,
  0 /* tp_richcompare */,
  0 /* tp_weaklistoffset */,
  0 /* tp_iter */,
  0 /* tp_iternext */,
  0 /* tp_methods */,
  0 /* tp_members */,
  0 /* tp_getset */,
  0 /* tp_base */,
  0 /* tp_dict */,
  0 /* tp_descr_get */,
  0 /* tp_descr_set */,
  0 /* tp_dictoffset */,
  (initproc)MyType_init /* tp_init */,
};

%}

After doing this, I want to add the type object to the SWIG-generated module, by calling PyModule_AddObject(), presumably in the module initialization block (%init %{ ... %}). However, I do not know what name to give for the value returned from Py_InitModule() (or PyModule_Create() in Python 3.x) in the generated code that creates the tensorflow.python.pywrap_tensorflow module. I have been able to create a second module in the %init %{ ... %} block, but I'd prefer to add it to the auto-generated module if possible.

An answer to either of these questions would solve my problem:

• Is there a SWIG macro for accessing the generated module object in the initialization block (e.g. by expanding to the variable name for the corresponding PyObject*)? This object seems to be stored in a local variable called m in my generated code, but I don't know if that is guaranteed to be the same in all distributions.

• Or is there a more idiomatic way to do what I am trying to achieve using SWIG? The buffer protocol typemaps seem from their documentation seemed to be designed for writing functions that accept buffers as arguments, whereas I want my wrapper type to implement the buffer protocol.

Answer 1

Just to make Jacky's answer more explicit - you probably want to do something like:

%{
static *PyBuffer mycoolbuffer_function(PyObject* self) {
    // get the C data for self
    // build your buffer
}
%}

%feature("python:tp_as_buffer") MyTypeThatWillBeWrapped "&mycoolbuffer_function";

This is sort of untested, I'm analogizing from some work I'm doing with iterators. And it certainly requires the -builtin option.

Answer 2

If you are writing you own class or structure, swig will build a python type for it, and all you have to do is add slots. Try %feature(python:slot). This feature can let you add arbitrary function/structures to generated python type slot. Also build your .i file using '-builtin' option, emit the unnecessary proxy python file.