global cluster object for dispy

Question

(1) Can I have the cluster object global across machines so that once a job is submitted, that job in turn can submit other dissimilar smaller jobs?

cluster = dispy.JobCluster(compute)

(2) Can the "compute" function be different each time I invoke a submit?

Answer 1

(1) I believe you need to look into the SharedCluster object, of which I'm not familiar.

(2) You can create different "functions" inside of the compute function using if statements and passing a selection argument to compute:

def compute(param):

    if param == 'a':
        'Do something'
    if param == 'b':
        'Do something else'

cluster = dispy.JobCluster(compute)

for i in params:

    cluster.submit(i)