Introduction to Rest.li

Quick introduction

A REST framework using type-safe bindings and asynchronous, non-blocking IO.

TODO: add other features - R2, D2, Avro, health monitoring?

Rest.li Data and Control Flow Between a Rest.li Server and Client

restli-client-server-flow.png

restli-client-server-flow.png

Server

There are several server implementations:

  • Servlet — Battle tested and ready for production use. Containers supporting Servlet 3.0 API are required to benefit from asynchronous, non-blocking request processing. Jetty 8.x supports Servlet 3.0 and has been used in large production environments.
  • Netty — Experimental
  • Embedded Jetty — Primarily for integration testing as it’s trivial to spin up as part of a test suite

Client

Rest.li’s client implementation is Netty-based.

It is designed to work seamlessly with ParSeq to construct complex asynchronous request flows.

D2

Dynamic Discovery (D2) is a layer of indirection similar to DNS for the rest.li framework.

Functionally speaking, D2 translates a URI like d2:// to another address like http://myD2service.something.com:9520/someContextPath.

Dive into D2

R2

Pegasus's request/response framework, often called R2, includes abstractions for REST and RPC requests and responses, filter chains for customized processing, and transport abstraction.

R2 can be used in conjunction with the Dynamic Discovery system (also known as D2). The combined stack can be referred to as "R2D2". R2 can be used independently as well.

Dive into R2

Development flow

restli-code-gen.png

restli-code-gen.png

  • Blue boxes represent classes you will write;
  • Green boxes represent components that are created by Rest.li’s code generators;
  • Black arrows indicate a code generation process;
  • Red dashed lines indicate the use of classes that allow a server and clients to exchange data;

Dive into D2

How D2 works

  1. When a client is about to send a request, the D2 client library extract the service name from the d2 URI.
  2. Then the d2 client library queries zookeeper for the cluster that owns that service.
  3. Once d2 client know the cluster, it will then queries zookeeper for the available URIs for that cluster.
  4. The D2 client will listen for any updates related to the cluster and service it previously contacted.
  • So if there's any changes happening either because the server membership changes, or there are new services in a cluster, the d2 client can pick up the changes immediately

D2 client

D2 Load Balencer

All the load balancing happened in the client side.

D2 client keep tracks of the health of the cluster. The client may choose to "load balance" or drop requests depends on the cluster health (cluster avg latency) and client health (tracked by many things like error rate, number of calls, latency of calls, etc).

Load balancer strategy

For more details

Dive into R2