Common Terminology

Concerrency vs Parallelism

Concerrency may or may not indicate that two tasks are running at the same time.

Non-blocking vs Blocking

• Blocking is when if the delay of one thread can indefinitely delay some of the other threads
• E.g., a resource can be used exclusively by one thread using mutual exclusion.
• Non-blocking means that no thread is able to indefinitely delay others

Non-blocking operations are preferred to blocking ones, as the overall progress of the system is not trivially guaranteed when it contains blocking operations.

Synchronous vs Asynchronous

• A method call is considered synchronous if the caller cannot make progress until the method returns a value or throws an exception.
• An asynchronous call allows the caller to progress after a finite number of steps, and the completion of the method may be signalled via some additional mechanism (it might be a registered callback, a Future, or a message).

A synchronous API may use blocking to implement synchrony, but this is not a necessity. A very CPU intensive task might give a similar behavior as blocking.

In general, it is preferred to use asynchronous APIs, as they guarantee that the system is able to progress.

Asynchronous vs Non-blocking

• Non-blocking means the method returns some result immediatelly. For eaxmple, in a non-blocking socket, the result is something like EAGIN/EWOULDBLOCK.
• Asynchronous indicates that not only the caller of the method does not wait for the completion but also the completion will be signaled via another mechanism.

Internal Components of Akka Actor

An Akka's Actor contains the following main components under-the-hood:

• ActorRef
• Dispatcher
• MailBox
• Actor

Actor

Definition

Actors are objects which encapsulate state and behavior, they communicate exclusively by exchanging messages which are placed into the recipient’s mailbox

• The Actor encapsulate states and actions.
• The Actors communicate by exchanging messages.
• You can think about Actors like persons.

• Decoupling: The multi-threaded programming model couples most of the concurrency logic into actual application business logic, so its very difficult to reason about and maintain it. Here, a developer has to concentrate on both, carefully. The Actor-based programming model decouples the concurrency logic from the application business logic, so it's very easy to reason about and maintain it. Here, a developer leaves concurrency issues to the programming model and concentrates on business logic only.
• API: Java threading API is a low-level concurrency API. The Akka Actor API is a high-level concurrency API.
• Scaling: Akka helps you build reliable applications which make use of multiple processor cores in one machine (“scaling up”) or distributed across a computer network (“scaling out”). Threads are good for scale-up, but not for scale-out.
• Concurrency versus parallelism: Threads are good for only concurrency, but not for parallelism. We cannot execute threads in parallel. Actors are good for both concurrency and parallelism. We can execute Actors in parallel very well.
• Lightweight:
• Threads are not lightweight; they need more space from the RAM. We can only create around 4 K threads in 1 GB RAM.
• Actors are very lightweight, and we can create around 3 million Actors in 1 GB RAM.
• Stackoverflow: Why actors are lightweight?
• Communication: Threads communicate with each other in a synchronous and blocking way, whereas Actors communicate with each other in an asynchronous and non-blocking way.
• Issues: The threading API raises many issues such as synchronization, locking, deadlock, race condition, live locks, context switching, performance issues, inefficient code, and more. Actors do not raise these kind of issues.
• Portable: Threads are not portable, whereas Actors are portable.

ActorRef

An Actor can interact with other Actors through ActorRef only; they cannot communicate with them directly. It follows the Proxy Design Pattern:

Dispatcher

Dispatcher is responsible for coordination; it takes the incoming message and hands it over to its Actor.

The Akka's dispatcher performs the following tasks:

• It picks up the incoming message from senders (Actor or non-Actor)
• It publishes that message to its Actor's mailbox message queue
• It checks whether its Actor is available or not to process that message
• If its Actor is available, it picks up the message from the message queue and hands it over to its Actor to take the necessary action
• It decouples the outside world from its Actor. In the Akka framework, the Dispatcher component follows the dispatcher design pattern
• If the required Actor is not available, it sends those messages to the DeadLetter Actor

Benefits of the dispatcher pattern

• It reduces dependencies between components, which means less coupling between components
• It increases application maintainability and testability
• It reduces boilerplate or duplicate code because we provide a common delegating logic at one place, that is, in the dispatcher

Mailbox (MessageQueue)

An Akka Mailbox holds the messages that are destined for an Actor. Normally each Actor has its own mailbox, but with for example a BalancingPool all routees will share a single mailbox instance.

By default, the MailBox follows the FIFO order to pick up and process messages from the queue.

Programming Akka Actor Model

Create ActorRef

When we execute the ActorSystem.actorOf() function, ActorSystem not only creates ActorRef, but also creates the actual Actor instance, MessageDispatcher, and Mailbox. However, the ActorSystem returns only ActorRef to the clients.

Prop

Props is a configuration class to specify options for the creation of actors, think of it as an immutable and thus freely shareable recipe for creating an actor including associated deployment information (e.g. which dispatcher to use)

ActorPath

Every Actor has a name or address to look it up from the ActorSystem. This name must be unique per level in the Akka Actor hierarchy. This name is also known as ActorPath.

A path in an actor system represents a “place” which might be occupied by a living actor. Initially (apart from system initialized actors) a path is empty. When actorOf() is called it assigns an incarnation of the actor described by the passed Props to the given path.

An actor incarnation is identified by the path and a UID.

A restart only swaps the Actor instance defined by the Props but the incarnation and hence the UID remains the same.

Top-Level Scopes for Actor Paths

At the root of the path hierarchy resides the root guardian above which all other actors are found; its name is "/". The next level consists of the following:

• "/user" is the guardian actor for all user-created top-level actors; actors created using ActorSystem.actorOf are found below this one.
• "/system" is the guardian actor for all system-created top-level actors, e.g. logging listeners or actors automatically deployed by configuration at the start of the actor system.
• "/deadLetters" is the dead letter actor, which is where all messages sent to stopped or non-existing actors are re-routed (on a best-effort basis: messages may be lost even within the local JVM).
• "/temp" is the guardian for all short-lived system-created actors, e.g. those which are used in the implementation of ActorRef.ask.
• "/remote" is an artificial path below which all actors reside whose supervisors are remote actor references

Actor lifecycle

It is important to note that Actors do not stop automatically when no longer referenced, every Actor that is created must also explicitly be destroyed. The only simplification is that stopping a parent Actor will also recursively stop all the child Actors that this parent has created.

Supervision and monitoring

In the Akka Toolkit, an Actor supervises its child Actors automatically. When a child Actor crashes or fails to do its computation, it informs its status to its supervisor or parent Actor. Then, the parent Actor will take a decision and perform one of the following actions on that Actor only, or on all of its Actors:

• Resume: It resumes only the affected Actor or all of its subordinate Actors
• Restart: It restarts only the affected Actor or all of its subordinate Actors
• Stop: It stops only the affected Actor or all of its subordinate Actors permanently
• Escalate: If it cannot decide what to do, it escalates to its supervisor Actor about the failure and fails itself

The supervisor Actor decides what to do on only the affected Actor or all of its subordinate Actors, based on the configured supervision strategy.

Message Delivery Reliability

General Rules

These are the rules for message sends (i.e. the tell or ! method, which also underlies the ask pattern):

• at-most-once delivery, i.e. no guaranteed delivery
• message ordering per sender–receiver pair

Discussion: What does “at-most-once” mean?

When it comes to describing the semantics of a delivery mechanism, there are three basic categories:

• at-most-once delivery means that for each message handed to the mechanism, that message is delivered zero or one times; in more casual terms it means that messages may be lost.
• at-least-once delivery means that for each message handed to the mechanism potentially multiple attempts are made at delivering it, such that at least one succeeds; again, in more casual terms this means that messages may be duplicated but not lost.
• exactly-once delivery means that for each message handed to the mechanism exactly one delivery is made to the recipient; the message can neither be lost nor duplicated.

The first one is the cheapesthighest performance, least implementation overhead—because it can be done in a fire-and-forget fashion without keeping state at the sending end or in the transport mechanism.

The second one requires retries to counter transport losses, which means keeping state at the sending end and having an acknowledgement mechanism at the receiving end.

The third is most expensive—and has consequently worst performance—because in addition to the second it requires state to be kept at the receiving end in order to filter out duplicate deliveries.

Discussion: Why No Guaranteed Delivery?

At the core of the problem lies the question what exactly this guarantee shall mean:

1. The message is sent out on the network?
2. The message is received by the other host?
3. The message is put into the target actor’s mailbox?
4. The message is starting to be processed by the target actor?
5. The message is processed successfully by the target actor?

Each one of these have different challenges and costs, and it is obvious that there are conditions under which any message passing library would be unable to comply; think for example about configurable mailbox types and how a bounded mailbox would interact with the third point, or even what it would mean to decide upon the “successfully” part of point five.

Another angle on this issue is that by providing only basic guarantees those use cases which do not need stronger reliability do not pay the cost of their implementation; it is always possible to add stronger reliability on top of basic ones, but it is not possible to retro-actively remove reliability in order to gain more performance.

Discussion: Message Ordering

The rule more specifically is that for a given pair of actors, messages sent directly from the first to the second will not be received out-of-order. The word directly emphasizes that this guarantee only applies when sending with the tell operator to the final destination, not when employing mediators or other message dissemination features (unless stated otherwise).

The guarantee is illustrated in the following:

Actor A1 sends messages M1, M2, M3 to A2

Actor A3 sends messages M4, M5, M6 to A2

This means that:

1. If M1 is delivered it must be delivered before M2 and M3
2. If M2 is delivered it must be delivered before M3
3. If M4 is delivered it must be delivered before M5 and M6
4. If M5 is delivered it must be delivered before M6
5. A2 can see messages from A1 interleaved with messages from A3
6. Since there is no guaranteed delivery, any of the messages may be dropped, i.e. not arrive at A2

Please note that this rule is not transitive:

Actor A sends message M1 to actor C

Actor A then sends message M2 to actor B

Actor B forwards message M2 to actor C

Actor C may receive M1 and M2 in any order

Communication of failure

Please note, that the ordering guarantees discussed above only hold for user messages between actors. Failure of a child of an actor is communicated by special system messages that are not ordered relative to ordinary user messages. In particular:

Child actor C sends message M to its parent P

Child actor fails with failure F

Parent actor P might receive the two events either in order M, F or F, M

The reason for this is that internal system messages has their own mailboxes therefore the ordering of enqueue calls of a user and system message cannot guarantee the ordering of their dequeue times.

Higher-level abstractions

Based on a small and consistent tool set in Akka’s core, Akka also provides powerful, higher-level abstractions on top it.

Messaging Patterns

As discussed above a straight-forward answer to the requirement of reliable delivery is an explicit ACK–RETRY protocol. In its simplest form this requires

• a way to identify individual messages to correlate message with acknowledgement
• a retry mechanism which will resend messages if not acknowledged in time
• a way for the receiver to detect and discard duplicates or make processing the messages idempotent on the business logic level.

The third becomes necessary by virtue of the acknowledgements not being guaranteed to arrive either. An ACK-RETRY protocol with business-level acknowledgements is supported by At-Least-Once Delivery of the Akka Persistence module. Duplicates can be detected by tracking the identifiers of messages sent via At-Least-Once Delivery.

Event Sourcing

Event sourcing (and sharding) is what makes large websites scale to billions of users, and the idea is quite simple: when a component (think actor) processes a command it will generate a list of events representing the effect of the command. These events are stored in addition to being applied to the component’s state. The nice thing about this scheme is that events only ever are appended to the storage, nothing is ever mutated; this enables perfect replication and scaling of consumers of this event stream (i.e. other components may consume the event stream as a means to replicate the component’s state on a different continent or to react to changes). If the component’s state is lost—due to a machine failure or by being pushed out of a cache—it can easily be reconstructed by replaying the event stream (usually employing snapshots to speed up the process). Event sourcing is supported by Akka Persistence.

Messages which cannot be delivered (and for which this can be ascertained) will be delivered to a synthetic actor called /deadLetters.

• This delivery happens on a best-effort basis;
• It may fail even within the local JVM (e.g. during actor termination).
• Messages sent via unreliable network transports will be lost without turning up as dead letters.

Location Transparency

Actor paths are used to enable location transparency. This special feature deserves some extra explanation, because the related term “transparent remoting” was used quite differently in the context of programming languages, platforms and technologies.

Everything in Akka is designed to work in a distributed setting: all interactions of actors use purely message passing and everything is asynchronous

The key for enabling this is to go from remote to local by way of optimization instead of trying to go from local to remote by way of generalization.

Ways in which Transparency is Broken

Designing for distributed execution poses some restrictions on what is possible:

• All messages sent over the wire must be serializable. This includes closures which are used as actor factories (i.e. within Props) if the actor is to be created on a remote node.
• Everything needs to be aware of all interactions being fully asynchronous, which in a computer network might mean that it may take several minutes for a message to reach its recipient. It also means that the probability for a message to be lost is much higher than within one JVM, where it is close to zero (still: no hard guarantee!).

How is Remoting Used?

We took the idea of transparency to the limit in that there is nearly no API for the remoting layer of Akka: it is purely driven by configuration. Just write your application according to the principles outlined in the previous sections, then specify remote deployment of actor sub-trees in the configuration file.

Peer-to-Peer vs. Client-Server

Akka Remoting is a communication module for connecting actor systems in a peer-to-peer fashion, and it is the foundation for Akka Clustering. The design of remoting is driven by two (related) design decisions:

1. Communication between involved systems is symmetric: if a system A can connect to a system B then system B must also be able to connect to system A independently.
2. The role of the communicating systems are symmetric in regards to connection patterns: there is no system that only accepts connections, and there is no system that only initiates connections.

The consequence of these decisions is that it is not possible to safely create pure client-server setups with predefined roles (violates assumption 2). For client-server setups it is better to use HTTP or Akka I/O.

Important: Using setups involving Network Address Translation, Load Balancers or Docker containers violates assumption 1, unless additional steps are taken in the network configuration to allow symmetric communication between involved systems. In such situations Akka can be configured to bind to a different network address than the one used for establishing connections between Akka nodes. See Akka behind NAT or in a Docker container.

References

https://doc.akka.io/docs/akka/2.5.5/scala/general/terminology.html

https://stackoverflow.com/questions/2625493/asynchronous-vs-non-blocking

http://blog.omega-prime.co.uk/2015/09/03/asynchronous-and-non-blocking-io/

https://www.safaribooksonline.com/library/view/learning-path-akka/9781788396271/video1_3.html

https://doc.akka.io/docs/akka/2.5.3/scala/actors.html

https://doc.akka.io/docs/akka/2.5.3/scala/general/message-delivery-reliability.html

https://www.amazon.com/Scala-Reactive-Programming-Rambabu-Posa/dp/1787288641