Decorator isn't passing arguments?

Question

I've been working on serialising a n-dimensional array into a 1-dimensional "array" database:

from collections import Iterable, Mapping
import sqlite3

def pass_many(vals, some_funct, args=()):
    if not vals:
        return
    if isinstance(vals, Iterable) and not isinstance(vals, (basestring, Mapping)):
        for v in vals:
            pass_many(v, some_funct, args)
    else:
        some_funct(vals, *args)

def counter(func):
    def wrapper(v, *args, **kwargs): # added 'v' arg to no avail
        wrapper.count = wrapper.count + 1
        test_var_args(v, *args, **kwargs)
        #return func(*args, **kwargs)
    wrapper.count = 0

    return wrapper

def test_var_args(farg, *args):
    print "formal arg:", farg
    for arg in args:
        print "another arg:", arg

@counter
def insert(val, cursor, table="wordlist", logfile="queries.log"):
    print val, cursor, table, logfile
    if val:
        if isinstance(val, (basestring, Mapping)):
            val = '\"' + val + '\"'
        else: val = str(val)
        query = "insert into tablename values (?);".replace('tablename', table).replace('?', val)
        #if logfile: to_logfile(query + '\n', logfile)
        cursor.execute(query)

if __name__ == '__main__':
    connection = sqlite3.connect('andthensome.db')
    cursor = connection.cursor()
    cursor.execute("create table array (word text);")
    pass_many([["foo", "bar"], "pew"], insert, cursor)
    connection.commit()
    cursor.execute("select * from array;") # wrapped select function omitted for brevity
    print "insert() was called", insert.count, "times, and db now contains:\n", cursor.fetchall()
    cursor.close()

Output:

formal arg: foo
formal arg: bar
formal arg: pew
insert() was called 3 times, and db now contains:
[]

Output uncommenting #return func(*args, **kwargs):

formal arg: foo
Traceback (most recent call last):
    Line 42, in <module>
        pass_many([["foo", "bar"], "pew"], insert, cursor)
    Line 9, in pass_many
        pass_many(v, some_funct, args)
    Line 9, in pass_many
        pass_many(v, some_funct, args)
    Line 11, in pass_many
        some_funct(vals, *args)
    Line 17, in wrapper
        return func(*args, **kwargs)
    TypeError: insert() takes at least 2 arguments (0 given)

Expected output (omitting debug function):

insert() was called 3 times, and db now contains:
["foo","bar","pew"]

Unfortunately it doesn't seem like the insert function decorated with counter is being passed arguments correctly.

What am I doing wrong?

Answer 1

One problem seems to be that you are using *args to expand your args argument, but you pass in cursor as the value of that argument without wrapping it in a tuple. Thus your eventual call is insert("foo", *cursor), when you seem to want it to be insert("foo", cursor). Try doing pass_many([["foo", "bar"], "pew"], insert, (cursor,)).

I think what is happening is that when you do this, your test_var_args function is consuming the cursor object (which is apparently iterable) and thus leaving no more arguments to be expanded in the subsequent call to the real insert function.

Editd after your response: Don't you actually want to pass v to your func call? Your insert function as written takes two arguments, v and cursor, but you call it with only cursor. Why are you wrapping insert? What is the extra argument supposed to do? You don't use it in the wrapper, and you don't pass it to the underlying function, so what is its purpose?

Answer 2

A couple of things: first, in wrapper():

def counter(func):
    def wrapper(v, *args, **kwargs): # added 'v' arg to no avail
        wrapper.count = wrapper.count + 1
        test_var_args(v, *args, **kwargs)
        return func(*args, **kwargs)
    ...

You have mapped the first argument to v, but you aren't passing it to the real func. It'd be better to remove it completely.

Second, and most important:

def pass_many(vals, some_funct, args=()):
    if not vals:
        return
    if isinstance(vals, Iterable) and not isinstance(vals, (basestring, Mapping)):
        for v in vals:
            pass_many(v, some_funct, args)
    else:
        some_funct(vals, *args)

Notice that when you call some_funct(), you're destructuring the args parameter. The problem is, when you called pass_many(), you gave it a cursor object as the args parameter. This will fail subtly. To fix this, you can either remove the splat operator *, or, a better choice, you can wrap the cursor object in a tuple when you call pass_many, like so:

pass_many([["foo", "bar"], "pew"], insert, (cursor,))

This will still give an error of "no such table: wordlist", but that's actually because you haven't defined such a table. ('wordlist' is the default argument to insert())

Answer 3

From @BrenBarn 's suggestion I've modified it to:

def counter(func):
    def wrapper(v, *args, **kwargs):
        wrapper.count = wrapper.count + 1
        test_var_args(v, *args, **kwargs)
        return func(v, *args, **kwargs)
    wrapper.count = 0

    return wrapper

pass_many([["foo", "bar"], "pew"], insert, [cursor])