How do I downcast a SWIG object without access to the SWIG template?

Question

Is there a way to downcast the swig proxy of a swig object on the fly?

The reason for doing this is to emulate C++ downcasting, but purely from python. For example, typical C++ usage would be

MyBase* obj = new MyBase();    // eg might come via network
if (obj.is_child())            // runtime info lets us know what we are dealing with
{
    MyChild* child = (MyChild*) obj;
    obj->some_child_function();
}

On the Python side, the proxy objects for MyBase and MyChild exist, but all objects enter python as a MyBase type. I'd like to be able to write:

obj = MyBase(); # eg might come from network
print(obj)
if obj.is_child():
    child_obj = MyChild.from_raw_ptr(obj.get_raw_ptr())  # Pseudo-code
    child_obj.some_child_function();
    print(obj)

Output like:
<MyBase; proxy of <Swig Object of type 'MyBase *' at 0x00000000069D46C0> >
<MyChild; proxy of <Swig Object of type 'MyChild *' at 0x00000000069D46C0> >

Note in the sample output, both output lines reference the same memory address.

Ideally I'd like to do this in pure python, but if it must be wrapper around some C/C++ bear in mind I don't have access to the SWIG template or original C code.

Answer 1

The tricky bit about getting this cast to work properly is not just changing the type of the proxy, which you've done in your answer, but changing the type of the this member inside the proxy also as you recognised is needed in the desired output of your question.

SWIG does store enough information to make this possible and I was able to fairly simply achieve the result you desired without needing to patch/recompile/reverse-engineer/rely upon the original module's .i file in anyway, provided you can match the SWIG version that was used to build it and compiler close enough that we can rely on the struct layouts in memory being the same. I should caveat this by saying that it feels like what I had to do to solve it ought to be more complex than necessary, but I can't see any other way of exposing the type hierarchy information from swig_type_info's internal to the module without doing it like this.

So to validate and develop this I put together the following, test.i file which is a minimal example of the problem you've got:

%module test

%inline %{

struct BaseClass {
    virtual bool is_derived() const { return false; }
    virtual ~BaseClass() {} 
};

struct DerivedClass : BaseClass {
    virtual bool is_derived() const { return true; }
    void do_something_special() {}
};

BaseClass *get_one() {
    static DerivedClass b;
    return &b;
};
%}

I compiled this with:

swig3.0 -Wall -c++ -python test.i
g++ -Wall -Wextra -o _cast.so -shared -fPIC cast_wrap.cxx -I/usr/include/python2.7

But then didn't touch it other than by using import test inside my code.

If we look inside the generated test_wrap.cxx we can see the following definition of swig_type_info, which is emitted once per type that SWIG knows of:

/* Structure to store information on one type */
typedef struct swig_type_info {
  const char             *name;                 /* mangled name of this type */
  const char             *str;                  /* human readable name of this type */
  swig_dycast_func        dcast;                /* dynamic cast function down a hierarchy */
  struct swig_cast_info  *cast;                 /* linked list of types that can cast into this type */
  void                   *clientdata;           /* language specific type data */
  int                    owndata;               /* flag if the structure owns the clientdata */
} swig_type_info;

The dcast member didn't seem to be populated (i.e. it was always null in my tests), but fortunately the cast member was. cast is a pointer to the first node in a linked-list of information about the type hierarchy, which looks like:

/* Structure to store a type and conversion function used for casting */
typedef struct swig_cast_info {
  swig_type_info         *type;                 /* pointer to type that is equivalent to this type */
  swig_converter_func     converter;            /* function to cast the void pointers */
  struct swig_cast_info  *next;                 /* pointer to next cast in linked list */
  struct swig_cast_info  *prev;                 /* pointer to the previous cast */
} swig_cast_info;

Which has both the name of the type and the converter function populated. So essentially what we need to do is walk that linked list to search for the converter function that performs the conversion we're after and then call it. You probably could at this point do exactly that using ctypes, however I chose to make use of the fact that the exact same compiler and SWIG versions would save me having to express those structs in ctypes notation and would always be correct so I simply wrote another SWIG module with some more C++.

(I should add that there are internal functions which do use this information within a module, but they have static linkage by default, so they're hard to find and use here).

Anyway my cast.i file, which exposes this neatly back to Python ended up looking like this:

%module cast

%{
#include <iostream>
%}

%inline %{
    PyObject *dyn_cast(PyObject *obj) {
        assert(SwigPyObject_Check(obj));
        SwigPyObject *s = (SwigPyObject*)obj;
        void *ptr = s->ptr;
        swig_cast_info *cast = s->ty->cast;
        while (cast) {
            std::cerr << "Cast @" << cast << ", converter: " << (void*)cast->converter << ", type: " << cast->type->str << "\n";
            if (0==strcmp(cast->type->name, "_p_DerivedClass")) break; 
            cast = cast->next;
        }
        assert(cast->converter);
        int newmem;
        s->ptr = cast->converter(ptr, &newmem);
        s->ty = cast->type;
        Py_INCREF(obj);
        return obj;
    }
%}

%pythoncode %{
import test

def base_to_derived(o):
    if not isinstance(o, test.BaseClass): raise TypeError()
    if not o.is_derived(): raise TypeError()
    c = test.DerivedClass.__new__(test.DerivedClass)
    c.this = dyn_cast(o.this)
    return c
%}

All that's doing is walking the cast linked list looking for information about the derived class. There's a little bit of extra Python to wrap that up safely and handle creating a new proxy object, but that's essentially it.

With this in place it was then possible for me to run the following Python code:

import test

o=test.get_one()
print(o)

import cast

o=cast.base_to_derived(o)
print(o)
o.do_something_special()

After having compiled my cast module:

swig3.0 -Wall -c++ -python cast.i
g++ -Wall -Wextra -o _cast.so -shared -fPIC cast_wrap.cxx -I/usr/include/python2.7
python run.py 
<test.BaseClass; proxy of <Swig Object of type 'BaseClass *' at 0xb6cd9428> >
Cast @0xb6ccf5ec, converter: 0xb6cbcdf1, type: DerivedClass *
<test.DerivedClass; proxy of <Swig Object of type 'DerivedClass *' at 0xb6cd9428> >

Personally though I'd just take the hit on maintaining a branch of the original module, pushing a patch upstream or even just use %import to write another SWIG module which extends the original module unmodified over something like this.