using dispy to distribute work to ec2 instances

414 Views Asked by At

I'm trying to use dispy to distribute work to ec2 instances.

I've followed:

Using dispy with port forwarding via ssh tunnel

http://dispy.sourceforge.net/dispy.html#cloud

but it's not going anywhere, the client script is hung and the server node doesn't receive anything .

what i have right now is:

from my machine:

ssh -i mypemfile.pem -R 51347:localhost:51347 [email protected]

then on remote machine:

sudo dispynode.py --ext_ip_addr ec2-52-30-185-175.eu-west-1.compute.amazonaws.com -d

and i get:

2016-02-03 18:38:39,410 - dispynode - dispynode version 4.6.7
2016-02-03 18:38:39,414 - asyncoro - poller: epoll
2016-02-03 18:38:39,417 - dispynode - serving 1 cpus at 172.31.26.18:51348
2016-02-03 18:38:39,417 - dispynode - tcp server at 172.31.26.18:51348
2016-02-03 18:38:39,422 - asyncoro - waiting for 2 coroutines to terminate
Enter "quit" or "exit" to terminate dispynode,
  "stop" to stop service, "start" to restart service,
  "cpus" to change CPUs used, anything else to get status:

on the client machine i run:

import dispy
def f(x):
    return 2*x
cluster = dispy.JobCluster(f,nodes=['ec2-52-30-185-175.eu-west-1.compute.amazonaws.com'],ext_ip_addr='localhost')
job = cluster.submit(2)
print('result: %s' % job())

and nothing happens, it's just stuck. thanks

0

There are 0 best solutions below