Typemap resources and converting a list to vector (and back)

Question

I'm using SWIG to wrap c++ in python and need to be using typemaps in order to keep my python script as simple as possible. As a first attempt, I'm just sending in 2 lists, converting them into vector's, adding the two vectors, then returning the result back into a new list.

My issue is that I've been finding the SWIG manual not very instructive, hard to follow, and not giving any solid, complete examples as to how I can write my own typemap.

My questions are:

1. How would I go about ensuring my lists are properly converted to vector's, then back again?
2. Are there any better tutorials/references out there for how to write typemaps, and what all the syntax/functions mean?

Here's my code:

add_array.h

#include <vector>
#include <functional>

std::vector<int> add_array(std::vector<int> src1, std::vector<int> src2);

add_array.i

%module add_array
%{
#include "add_array.h"
%}

%include std_vector.i 
%template(vectorInt) std::vector<int>;

%include "add_array.h"

add_array.cpp

#include "add_array.h"
#include <cassert>
#include <cstring>

std::vector<int> add_array(std::vector<int> src1, std::vector<int> src2) {  
  assert(src1.size() == src2.size());
  std::vector<int> dst;
  dst.resize(src1.size());

  for (size_t i = 0; i < src1.size(); i++) {
    dst[i] = src1[i] + src2[i];
  }
  return dst;
}

Makefile

all:
rm -f *.so *.o *_wrap.* *.pyc *.gch add_array.py
swig -c++ -python add_array.i
g++ -fpic -c add_array_wrap.cxx add_array.h add_array.cpp -I/home/tools/anaconda3/pkgs/python-3.7.3-h0371630_0/include/python3.7m/
g++ -shared add_array_wrap.o add_array.o -o _add_array.so

array.py (this is the file I'm running)

import add_array

a = [1, 2, 3, 4, 5, 6]
b = [5, 6, 7, 8, 9, 10]
c = add_array.add_array(a, b)
print(c)

Output: (6, 8, 10, 12, 14, 16)

This is coming out as a tuple (I'd like it to be a list). It looks like I'm just lucky that it can convert the input lists to vectors (while not so lucky the other direction), but I'd really like to know how that's happening and how I can change that for future code if need be.

Thanks!

Answer 1

I don't know if there is a specific reason, but the included std_vector.i converts output vectors into tuples instead of lists. If you want a list, you'll need to write a custom typemap.

Example (no error checking):

%module add_array
%{
#include "add_array.h"
%}

%include <std_vector.i>
%template(vectorInt) std::vector<int>;

// Override the template output typemap with one that returns a list.
// An "out" typemap controls how a value is returned.
// When a function returns std::vector<int> this template will convert it to
// a Python object.  In this case, a PyList.
// 
// Note: PyObject* tmp declares a local variable that will be used by this code snippet.
// Make sure to look at the generated wrapper code and find the add_array_wrap function
// and how this code is integrated into it.
// 
%typemap(out) std::vector<int> (PyObject* tmp) %{

    // Allocate a PyList object of the requested size.
    // $1 references the first type in the type list (in this case, std::vector<int>)
    // and represents the c++ return value of a function that returns
    // this type; therefore, we can call methods on that value to get the size.
    //
    // Note: The elements of the new PyList are null pointers and MUST be
    //       populated before returning it to Python.
    //
    tmp = PyList_New($1.size());

    // Populate the PyList.  PyLong_FromLong converts a C++ "long" to a
    // Python PyLong object.  PyList_SET_ITEM takes a PyList object (tmp),
    // an index (i), and a Python object to put in the list.  This particular
    // function *steals* the reference to the Python object, so you don't have to
    // Py_DECREF the object to free it later.
    //
    for(int i = 0; i < $1.size(); ++i)
        PyList_SET_ITEM(tmp,i,PyLong_FromLong($1[i]));

    // $result is where you assign the Python object that should be returned
    // after converting the C++ $1 object.  SWIG_Python_AppendOutput is not
    // well-documented, but it appends the return object to an existing
    // returned object.  It's most useful for "argout" typemaps where multiple
    // output or in/out arguments in C++ can be returned as a tuple of
    // return values in Python.  For example, a function like:
    //
    //     void func(int* pValue1, int* pValue2);
    //
    // would normally return None ($result = Py_None), but an "argout" typemap
    // could *$1 to a PyLong and use SWIG_Python_AppendOutput to add it to
    // the result.  The template would be applied twice and you'd get a tuple.
    //
    $result = SWIG_Python_AppendOutput($result,tmp);
%}

%include "add_array.h"

Output:

>>> import add_array
>>> add_array.add_array([1,2,3],[4,5,6])
[5, 7, 9]

As far as tutorials, I've only ever read the SWIG documentation and the language-specific documentation for C language extensions. It's actually pretty good as documentation goes, but you can't just pick and choose what to read. Study the first dozen or so sections for the basics and then skip to the language-specific section (such as Python). There is an Examples directory under the SWIG installation as well.

References:

• PyLong_FromLong
• PyList_New
• PyList_SET_ITEM
• std_vector.i

You'll have to look through SWIG sources to get any info about SWIG_Python_AppendOutput. Or just google for others examples.