Twistd amp protocol : Producer/Consumer pattern?

165 Views Asked by At

I would like to use protocal to send data between server/client : list of unicode (string) that exceed the limit size of 65,535 bytes. A nice and clean solution to deal with the limit size problem is to Implement Producer/Consumer pattern.

Unfortunately, I read the documentation and example but I can't find how to start. For small size of transferred data, my amp implementation works very well . I show here relevant parts of my code.

command

class DataCommand(Command):
    response = [("data", ListOf(Unicode()))]
    errors = {
        KeyError: "KEY_ERROR",
        ValueError: "VALUE_ERROR",
    }    

server

class ProducerLocator(CommandLocator):
    def __init__(self,size,N):
        self.data = [id_generator(size=size) for x in range(N)]
    @DataCommand.responder
    def getData(self):
        return {'data':self.data}  ## don't work if is self.data > 64kB

client

class Consumer:     
    d = connect()
    def getData(protocol):
        return protocol.callRemote(DataCommand)
    d.addCallback(getData) 

Some code to generate data (used above in the factory constructor)

import string
import random
def id_generator(size=6, chars=string.ascii_uppercase + string.digits):
    return ''.join(random.choice(chars) for _ in range(size))
0

There are 0 best solutions below