Listening to Ollama output, using port forwarding and socket connection. Empty stream

181 Views Asked by At

My python tool want to listen to ollama output, but i have an empty stream.

Ollama listening port: localhost:11434.

iptables set up with : sudo iptables -t nat -A PREROUTING -p tcp --dport 11434 -j REDIRECT --to-port 11435

my tool:

def main():
    # Configuration
    HOST = 'localhost'
    PORT = 11435
    
    # Create a socket to listen for incoming connections
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        # Bind the socket to the host and port
        s.bind((HOST, PORT))
        
        # Listen for incoming connections
        s.listen()
        print(f"Listening on port {PORT}...")
        
        # Accept incoming connections
        conn, addr = s.accept()
        print(f"Connection established from: {addr}")
        
        try:
            # Receive and print data
            while True:
                data = conn.recv(4096)
                if not data:
                    break
                print("Received:", data.decode('utf-8'))
        
        except ConnectionResetError:
            print("Connection closed by Ollama.")
        
        print("Connection closed.")

if __name__ == "__main__":
    main()

When ollama is running and sending output, that could be the ollama run initialization output, or a response to my question, my tool keeps waiting with : Listening on port 11435.

Furthermore, when i quit ollama with Ctrl+C i don't have the message connection closed.(but this is an optional point to solve).

I don't know if mistake is from the socket python code, or something else.

Maybe there is a completely different way to parse all ollama output using it's API , i just thought writing this post.

0

There are 0 best solutions below