Actors in a New “Highly Parallel” World
Fábio Corrêa
Polelo Research Lab
University of Pretoria
Pretoria, 0002
+27-12-420-3617
feamcor@gmail.com
ABSTRACT
The actor-model of computation was forged 25 years ago and,
despite its natural fitness for concurrent programming, it never got
traction on mainstream programming languages and as a
widespread programming technique, it has been restricted mainly
to academic experiments. The functional programming paradigm,
available since the inception of the first high-level programming
languages, despite the success of some of its languages, mostly on
the academic environment, and its natural ability to avoid the
inherent problems of concurrency, also never got traction on
commercial development, being overshadowed by the
proliferation of imperative languages. Recently, Moore’s Law
reached a limit and, as an alternative way of evolving processors,
multi-core ones were developed, turning parallel systems into
commodity. This paper presents the guidelines of research that is
being conducted in response to the actual software engineering
crisis, on how to effectively exploit this new highly parallel world
of multi-core processors on computers and other devices. In
search of an effective and efficient concurrent programming
model, currently available actor-model implementations will be
scrutinized in order to evaluate their strengths and weaknesses
against the omnipresent shared-state, multi-threaded model of
concurrent programming.
Categories and Subject Descriptors
D.1.3 [Programming Techniques]: Concurrent Programming –
Parallel Programming.
D.3.3 [Programming Languages]: Language Constructs and
Features – Concurrent Programming Structures.
General Terms
Performance, Experimentation, Languages.
Keywords
Multi-core, Concurrency, Parallelism, Actors, Actor-Model,
Asynchronous Programming, Functional Programming, Erlang,
Scala, Java, C#, F#, .NET, JVM, Kilim.
21
1. INTRODUCTION
1.1 A Law in Danger of Extinction
A few years ago the processor industry noticed that the continuity
of Moore’s Law was under serious threat. The traditional
approach to CPU performance improvement, through the addition
of more transistors per chip (density), and the increase of clock
speed, had hit some (physical) walls: excessive power
consumption and thermal dissipation. It was no longer possible to
keep producing single-threaded processors in the same way as
they had been for decades, i.e. executing more instructions per
cycle. A new engineering approach had to be adopted in hardware
development and, in the meantime, in the software arena, it was
proclaimed that the free (performance) lunch was over [10].
1.2 Two Heads are Better than One
Enter the multi-core processors (a.k.a. chip multiprocessors). In
simple terms, the processor industry, making use of the amazing
density capability available these days, started to put two (or
more), under-clocked processors within the same die, sharing a
single memory source, like a typical SMP configuration. These
under-clocked processors (a.k.a. cores) required less power and
consequently dissipated less heat. But, for instance, two cores
don’t mean double of performance. According to Sutter [10]:
“First, there is coordination overhead between the cores to ensure
cache coherency (a consistent view of cache, and of main
memory) and to perform handshaking. (…) Second, unless the two
cores are running different processes, or different threads of a
single process that are well-written to run independently and
almost never wait for each other, they won’t be well utilized.”
His second point gives a clear indication that the concurrent
software MUST be well written in order to have efficient
utilization of the processor. This statement takes us to where the
burden really lies on this new context: in the software.
1.3 Software for a Truly Parallel World
Sutter’s statement that [10] “concurrency is the next major
revolution in how we write software” seems pertinent. It is the
actual hot topic in software development and, definitively, not an
easy task to handle. Concurrency was “ignored” by most
developers for years, being restricted to a few application domains
and their small group of hard-core developers. But it cannot be
ignored anymore and developers will have to adapt, in order to
absorb the outcome of the effort that is being expended by both
academic and industry researchers, in order to make this transition
as smooth as possible.
Non-determinism, for instance, makes the debugging of
concurrent software much more difficult than the debugging of
essentially sequential software. Threads, synchronization, locks,
race condition, deadlocks, live locks, thread-safe components,
thread-pools, scalability, load balancing and shared-state, for
instance, are well-known issues, and sources of headache, for
those who adventure on suck tasks.
By being so difficult to master, the quest for finding a concurrent
programming model and high-level abstractions, that would be,
not only adopted by most of developers and hobbyists, but also
effective on making efficient use of the available computational
resources (in this case, the cores), in a dynamic fashion, is the
main force that drives this research.
1.4 In Search of a Programming Model
Van Roy [12] considers that the issue of programming multi-core
processors is technically solved through dataflow programming;
and that its adoption is actually a sociological issue. As you will
be able to verify in the following sections, dataflow programming
and the actor-model share many common characteristics, if they
do not actually overlap each other.
It seems that the sociological issue, in this regard, is the lack of
knowledge about the options available (which would take us to a
separate discussion about education) and, more importantly, an
imposition of a common and restricted set of development tools,
mainly on corporative environments that are usually not open to
differentiation and experimentation, but to what is common
knowledge and easy to get in the market.
Java was the first language that made thread a known term among
developers that, who up to that point, were used to the term multi-
threaded from the context of features of an operating system.
Some years ago it was very fashionable to sell the idea that one
operating system was preemptive and multi-threaded, even if most
people didn’t know the exact implications of that.
Anyway, the predominant programming model for concurrency
today is through the use of threads on a shared-state model. Lee
[7] states that “threads discard the most essential and appealing
properties of sequential computation: understandability,
predictability and determinism” and indicates that improving such
model is not the way to go as it pushes the developer to fight
against the non-determinism of such model. As a consequence of
such thoughts, deterministic concurrency is a must on this new
highly parallel world; otherwise, with the continuing increase of
complexity of new applications, it will be a burden too heavy for
developers to carry.
Sutter and Larus [11] share the same concerns and add that
shared-state is a major constraint and that synchronization
methods, like locks, by not being composable and by being global
properties of an application (and not of individual components),
are an overhead that has to be overcome. Lock-free programming
and transactional memory are suggested as alternatives to such
problem, but I consider them incipient technologies, despite, for
instance, the extensive research that is being conducted on
software transactional memory (STM).
Other concurrency abstractions like Futures [14] and Phasers [8]
are available in real and/or experimental programming languages.
Also, various other methods and tools, like specialized languages
such as Fortress and X10, are not being considered and analyzed
here by not being in the context of actors.
22
1.5 The Functional Insurgence
Let me digress a little before moving to the actor-model, in order
to contextualize the actual insurgence of functional programming
languages and its impact on the current concurrency crisis.
High-order functions, anonymous functions (closures), efficient
tail recursion, pattern matching, list comprehensions, shared-
nothing memory model (through immutable types and structures),
and a high-level programming style (through aggregation), that
besides being considered exotic for those not used to them, are
typical features of functional programming languages that are
becoming more and more widespread these days and these are
beneficial and a good fit for concurrent programming.
The increased popularity of such a paradigm is perceived from a
few scenarios:
1. Existing languages that were hidden in academic silos
and that are currently facing a rise on popularity among
development projects, e.g. Lisp and Haskell;
2. Existing languages that were hidden in niche silos and
that are currently facing a rise on popularity among
development projects, e.g. Erlang;
3. New languages that were developed to run on top of
market-consolidated VMs (like JVM and .NET CLR),
e.g. Clojure, Scala and F#; and
4. The adoption of functional features on imperative
languages; e.g. C#, Python, Ruby and Java.
2. THE ACTOR MODEL
2.1 History
Hewitt et al [6] defined the actor-model as part of their research
on artificial intelligence for the PLANNER-like family of AI
languages. Following their seminal paper [6], other works helped
to define the semantics of the actor-model [15] and culminated on
Agha [1] setting the actor-model theory in 1985. His first thoughts
in the preface of his thesis [1] (“It is generally believed that the
next generation of computers will involve massively parallel
architectures.”) took twenty years to start to become reality. Agha
is one of the most important and prolific researchers on the actor-
model and his theory is used as baseline taking in consideration
that Hewitt et al [6] are slightly obscure, and of difficult
interpretation.
2.2 Characteristics
Actors are reactive entities that have a unique name (mail address)
and, based on an incoming message received through its
individual inbox, that can [1]:
1. Send a finite set of messages to other known actors;
2. Create a finite set of new actors; and
3. Define how it will behave in relation to the next
incoming messages.
The communication between actors in an actor system is, like in
dataflow [13], asynchronous, i.e. the destination actor doesn’t
have to be ready in order to the sender to send it a message.
Actor systems are classified as message-passing systems, taking
into consideration that all communication between actors happens
exclusively through the exchange of messages.
Due to their message-passing mechanism, actors don’t share state
[4]. This means, they are not subject to memory corruption as the
one existent on shared-state systems, which require the explicit
use of synchronization devices, like locks, in order to avoid such
corruption. On one side, it eliminates the need for synchronization
of state (value). On the other side, the cost of copying such state
(value) from one actor to another requires optimized and efficient
ways of doing it, without excessively demanding intensive
memory allocation and operations, like copy.
In order to identify and model an actor system, and its dynamics,
the following, pretty much like the modeling of an object-oriented
system, must be determined [4]:
1. All type of actors of the system (like classes);
2. The contract that each actor responds to, i.e. which
messages it should receive (like interfaces); and
3. The behavior that each actor should take when receiving
a message (like methods).
Many actor languages were developed along the years, most of
them restricted to academic departments. A not so extensive list
can be found at [15], being SALSA the most prominent one.
In order to conduct the experiments of this research, I decided to
evaluate only a few real world implementations of the actor-
model.
3. IMPLEMENTATIONS
On most of the technologies below you won’t find an explicit
reference to the actor-model or to actors themselves. You will find
a mix of terms that can be confusing and deceiving most of the
time. How much of the theory of the actor-model does an
implementation implement is also not in question here. More or
less, all of them implement a minimum set of characteristics that
identify and behave like an actor system. Differences in
terminology will be described during the demonstration of the
results of the experiments. All of the implementations listed below
can make efficient use of the cores available during runtime,
avoiding any static setup and/or annoying re-compilations.
3.1 Erlang
Erlang [16] is a dynamically typed functional language that was
developed at the Ericsson Computer Science Laboratory with the
purpose of being a “better way of programming telephony
applications” [3]. It started in 1985 and was restricted to Ericsson
until 1997, when it was released as open source. It was developed
with, soft real-time, concurrency, fault-tolerance, distribution and
hot-replacement in mind, due to the very nature of the telephony
environment. Erlang is experiencing a tremendous increase in
popularity and adoption among developers worldwide.
Armstrong, one of its original developers and still the most
famous face on Erlang, released a book in 2007 [2], assisting in
spreading more inner details about the language. An ACM
conference, and a few other commercial conferences, happens
every year. Today, dozens of open source projects are being
developed using Erlang as the primary language, e.g. ejabberd,
yaws, RabbitMQ, and CouchDB. Erlang has been shown to be a
perfect match for server applications due to its OTP framework,
which provides the infrastructure for the creation of resilient
distributed applications.
The only concurrent programming model available in Erlang is
the actor-model (or Concurrency Oriented Programming, as
prefers Armstrong). Erlang has immutable types, pattern matching
and other features from a pure functional language and most of its
design inspiration came from PROLOG.
23
Erlang is today the de facto implementation of an actor-model and
it is used as reference for many of the newest actor-model
implementations. Due to its extreme scalability, it is also used as
baseline for benchmarks of other implementations. You can
consider an actor-model implementation good if it concurrently
scales at the same level, or even more, than what you would get
from Erlang.
Erlang runs on top of its own VM, which is considered the great
differentiator compared to other famous VMs like the JVM and
the .NET CLR. Its implementation of (light) processes is totally
detached from OS threads and fully implemented and controlled
by the VM.
3.2 Scala
Scala [19] is a hybrid object-oriented plus functional
programming language, inspired by Java but with characteristics
of many others, which compiles to Java bytecodes and runs on top
of the JVM. Due to its intimate relation with Java, you can freely
mix both, allowing you to take advantage of the extensive set of
libraries and frameworks available for Java. Scala allows you to
use the concurrent abstractions available in Java, e.g. threads, but
its most famous one is the Actors library that mimics the Erlang
implementation.
Haller and Odersky [5] describe the pitfalls encountered during
the development of the Actors library and the impedance between
the JVM and the requirements of an actor-model implementation.
Actually they mixed threads and events in a way that overcame
these limitations and provided a performance level and scalability
similar to those achieved by Erlang.
Scala is released as open source. Actually there are a couple of
books available about it. Some people consider Scala as the new
Java (a little bit of a hype in my opinion).
3.3 Kilim
Kilim [17] is a framework used to create robust and massively
concurrent actor systems in Java, which claims to be, at least, 3x
faster than Erlang.
Kilim makes use of a bytecode post-processor called Weaver [9]
and a run-time library. Weaver ensures that, through annotations
on the bytecode:
1. Threads will be ultra-lightweight (like fibers);
2. Messages will be treated as a special, and much simpler,
category of Java objects; and
3. Isolation is enforced on compile-time.
Besides the bytecode post-processor, the use of Kilim doesn’t
require any change to the Java language or the JVM itself.
Kilim is released as open source and is still a work-in-progress.
3.4 Microsoft CCR/DSS Toolkit
The Microsoft CCR (Concurrency and Coordination Runtime)
and DSS (Decentralized Software Services) Toolkit [18] is the
new kid in town in the list of actor-model implementations.
Originally incubated inside the Microsoft Robotics Developer
Studio (due to the inherent concurrency requirements when
dealing with robots) it became an official Microsoft product by
the end of October 2008 (during the PDC 2008).
It comprises of “a set of .NET and .NET Compact Framework
class libraries and tools that enable developers to better deal with
the inherent complexities in creating loosely-coupled concurrent
and distributed applications” [18].
There is practically no scientific research published yet about
CCR/DSS, but it promises to be a very important contender.
4. EXPERIMENTS
The experiments will consist of the definition of a list of software
engineering features, as well as benchmarking requests that will
be evaluated against the actor-model implementations listed in the
previous section and one threaded shared-state implementation.
Erlang will be considered the baseline for all the benchmarks. The
benchmarks will not only measure performance and scalability,
but also the efficiency of the underlying runtime in make effective
use of the cores available on the testing workstation.
The testing workstation is a PC with the following configuration:
• 2x Intel Xeon 2.0GHz Quad Core Processors
• 8x 2GB RAM Modules 667MHz FB-DIMM
• 4x 500GB HDs Seagate SATA 7200RPM
• 1x GeForce 9800GTX+ GPU with 512MB of VRAM
• OS Windows HPC Server 2008, XP and Vista 64bit
Distributed characteristics of the actor-model are out-of-the scope
of this research.
Object-orientation will be considered as the baseline for all the
software engineering features to be measured.
A set of testing applications will be developed on all the
implementations listed on the previous section. The programming
languages that will be used for the experiments are: Erlang, Scala,
Java, C# and F#. The CCR/DSS toolkit will be tested on two
different languages, namely C# an example of an imperative
shared-state language and F# an example of a functional language.
In case any third-party component, e.g. a database, is required on
a test application, such component has to provide a unique and
open method of interoperation for all the languages listed above.
5. CONCLUSIONS
The aim of this research is to demonstrate that the actor-model is
an effective and efficient programming model for tackling the
current concurrent crisis. It also intends to describe the differences
found between the implementations, in order to drive to a single
terminology and set of abstractions, that would be part of future
actor-model implementations, speeding-up their adoption by
developers due to a common idiom. And last, but not least, it will
describe the effort that would be required from developers to shift
to this paradigm.
6. ACKNOWLEDGMENTS
Thanks to Prof. Judith Bishop on giving me the opportunity to
pursue my masters and also for providing me with guidance and
resources for this endeavor.
24
7. REFERENCES
[1] Agha, Gul. Actors: A Model of Concurrent Computation in
Distributed Systems. MIT Artificial Intelligence Laboratory,
Ann Arbor, MI (1985)
[2] Armstrong, Joe. Programming Erlang: Software for a
Concurrent World. Pragmatic Bookshelf (2007) 1st Edition
[3] Armstrong, Joe. A History of Erlang. Proceedings of the 3rd
ACM SIGPLAN conference on History of Programming
Languages - HOPL (2007)
[4] Ayres, John. Implementing Stage: the Actor based language.
Final Report. Imperial College London (2007)
[5] Haller, Philipp and Odersky, Martin. Actors that Unify
Threads and Events. International Conference on
Coordination Models and Languages, LNCS (2007) pp. 171-
190
[6] Hewitt, Carl and Bishop, Peter and Steiger, Richard. A
Universal Modular ACTOR Formalism for Artificial
Intelligence. International Joint Conferences on Artificial
Intelligence - IJCAI (1973) pp. 235-245
[7] Lee, Edward. The Problem with Threads. COMPUTER
(2006) pp. 33-42
[8] Shirako et al. Phasers: a unified deadlock-free construct for
collective and point-to-point synchronization. ICS '08:
Proceedings of the 22nd annual international conference on
Supercomputing (2008)
[9] Srinivasan, Sriram and Mycroft, Alan. Kilim: Isolation typed
actors for Java. Proceedings of the European Conference on
Object Oriented Programming - ECOOP (2008) pp. 104-128
[10] Sutter, Herb. The free lunch is over: A fundamental turn
toward concurrency in software. Dr. Dobb’s Journal (2005)
[11] Sutter, Herb and Larus, James. Software and the concurrency
revolution. Queue (2005) vol. 3 (7) pp. 54-62
[12] Van Roy, Peter. The Challenges and Opportunities of
Multiple Processors: Why Multi-Core Processors are Easy
and Internet is Hard. (2008)
[13] Van Roy, Peter and Haridi, Seif. Concepts, Techniques and
Models of Computer Programming. MIT Press (2004)
section 4.9
[14] http://en.wikipedia.org/wiki/Future_(programming) as of 30
November 2008
[15] http://en.wikipedia.org/wiki/Actors_(concurrency) as of 30
November 2008
[16] http://www.erlang.org as of 30 November 2008
[17] http://www.malhar.net/sriram/kilim as of 30 November 2008
[18] http://www.microsoft.com/ccrdss as of 30 November 2008
[19] http://www.scala-lang.org as of 30 November 2008