SSE subscribers - distributed across multiple servers?

Question

Looking at the sse-chat demo of the Clojure Pedestal Framework - which relies on the SSE features in Pedestal - I noticed the following code:

(defn publish
  [request]
  (doseq [sse-context @subscribers]
    (try
      (sse/send-event sse-context "message" (-> request :form-params (get "msg")))
      (catch java.io.IOException e
        (remove-subscriber sse-context))))
  {:status 204})

Basically this keeps a map of subscribers (a map of EventSource clients) and sends chat events to them.

My question is - suppose you wanted to scale this application across multiple servers. What is an idomatic pattern in which to do this? (hopefully in Clojure Pedestal - but could be a solution from across the Java spectrum)

Answer 1

There is presently no free lunch/ silver bullet for multi-node parallelism. Most people use an SOA approach (REST, queues, etc) to parallelize their application. Of course in the large you lose ability to coordinate access to resources, and the work-arounds can be hackish. I have heard good things about Immutant's (Jboss's) XA transactions that automatically apply to how their caching and messaging work, but have not used this approach personally. Another tool you may find useful is Storm, which allows you to set up a topology for distributed processing, adding some declarative abstraction in place of tedious manual development and provisioning of equivalent service architecture.