Distributed Actors in Clojure

2012-05-17 akka clojure erlang jvm

Here’s another post on a topic that has been discussed since the dawn-of-time: is there a nice and idiomatic way to write Erlang/Actor-style distributed programs in Clojure? There have certainly been a few attempts, but Rich’s post (above) still holds true today.

First, some clarification: I am not primarily thinking about number-crunching, map/reduce-style stuff, where Clojure has a pretty good story:

Akka and the Erlang legacy

I am trying to write programs that solve problems in the areas where Erlang typically excels, such as:

Event-driven, asynchronous, non-blocking programming model
Scalability (location transparency, etc.)
Fault tolerance (supervisors, “let it crash”)

The closest we’ve got on the JVM is Akka, which claims to have all features (and more) listed above. Akka is the “killer app” for Clojure’s sister language Scala, and is very feature-rich and performant. Leveraging its power in a safe and idiomatic way is certainly appealing.

However, interfacing with Akka from Clojure is not nice, and certainly not idiomatic. Some work is clearly needed to improve Akka/Clojure interop. The bigger question is if it’s worth pursuing. Even if the interop is made as pain-free as possible, how badly will it clash with Clojure’s underlying design and philosophy? For instance, Akka comes with a STM; how nasty will that be when used in conjunction with Clojure’s own?

Update: Two Akka/Clojure libraries have emerged since this article was written, which solve some of the problems I was facing: akka-clojure and okku. Performance compared to Scala/Akka is yet to be determined.

Wishful thinking

Ideally, Clojure should support distributed actors in it’s core, that looks, behaves and interops nicely with it’s other concurrency primitives. It’s pretty easy to create a ideal-world straw-man for how this might look from a code/syntax perspective; Termite is a good place to start. Here is a cleaned-up version of the hello-world examples in the gist above.

;; An (imaginary) actor macro takes an initial state and callback fns.
;; (actor {} (fn1 [state message]) (fn2 [state message) ...)

;; The most obvious callback fn is (receive [state message) that is called when a
;; message is consumed from the agents mailbox. receive is called with the old state
;; and the message as parameters -- it returns the new state.
;; sender is bound to the senders pid.

;; other callbacks are stuff broken link detection etc.

;; (spawn) creates a lightweight actor process, also has remote-actor semantics
;; (tell) is used to send messages to an spawned actor's pid.

(def hello-actor
  (actor {:world-actor
          (spawn
           (actor {}
                  (receive [state [message-type word :as message]]
                           (condp = message-type
                             :hello (do
                                      (tell sender (str (.toUpperCase word) "world!"))
                                      state)
                             (unhandled message)))))}
         (receive [state [message-type word]]
                  (condp = message-type
                    :start (do
                             (tell (:world-actor state) [:hello "hello"])
                             state)
                    (do
                      (println (str "Received message:" word))
                      (shutdown)))))

(let [pid (spawn hello-actor)]
  (tell pid [:start]))

Many problem arises, serialisation is a big one. Since Clojure’s data structures can contain “anything”, like Java objects, some limitations needs to be applied to strike a good usability / performance balance. Limiting the stuff you can distribute amongst actors to mimic Erlangs atoms/lists/tuples are probably a fair trade off (all you need is a hashmap right?), and maybe baking in Google Protobuf for efficiency.

For data transport / socket stuff, I’d vote for using a message queue such as 0MQ or maybe even RabbitMQ, this would simplify and empower matters greatly.

With all that in place, it would be possible to build Clojure equivalents of Erlang’s OTP, Mnesia etc, now that’s a world I want to live in! :)

Distributed Actors in Clojure

Akka and the Erlang legacy

Wishful thinking

More reading