Twitter Finagle

Netty at Twitter with Finagle
Within a service-oriented architecture, services spend most of their time waiting for responses from other upstream services. Using an asynchronous library allows services to concurrently process requests and take full advantage of the hardware.

Netty isn’t tied to any particular protocol. Adding to it is as simple as creating the appropriate event handlers. 

Clients need to connect to and load balance across a cluster of servers. All services need to export metrics (request rates, latencies, etc) that provide valuable insight for debugging service behavior

Finagle relies on Netty for IO multiplexing providing a transaction-oriented framework on top of Netty’s connection-oriented model.

Finagle emphasizes modularity by stacking independent components together. Each component can be swapped in or out depending on the provided configuration.
Finagle’s Abstraction

Finagle’s core concept is a simple function (functional programming is the key here) from Request to Future of Response.

1	type Service[Req, Rep] = Req => Future[Rep]

This simplicity allows for very powerful composition. Service is a symmetric API representing both the client and the server. Servers implement the service interface. The server can be used concretely for testing or Finagle can expose it on a network interface. Clients are provided an implemented service that is either virtual or a concrete representation of a remote server.

For example, we can create a simple HTTP server by implementing a service that takes an HttpReq and returns a Future[HttpRep] representing an eventual response:

1 2 3 4 5

val s: Service[HttpReq, HttpRep] = new Service[HttpReq, HttpRep] {   def apply(req: HttpReq): Future[HttpRep] =     Future.value(HttpRep(Status.OK, req.body)) } Http.serve(":80", s)

A client is then provided with a symmetric representation of that service:

1 2 3

val client: Service[HttpReq, HttpRep] = Http.newService("twitter.com:80") val f: Future[HttpRep] = client(HttpReq("/")) f map { rep => transformResponse(rep) }

This example exposes the server on port 80 of all interfaces and consumes from twitter.com port 80. However we can also choose not to expose the server and instead use it directly:

?

1	server(HttpReq("/")) map { rep => transformResponse(rep) }

Here the client code behaves the same way but doesn’t require a network connection. This makes testing clients and servers very simple and straightforward.

there is a need for application agnostic functionality as well. Timeouts, authentication, and statics are a few examples. Filters provide an abstraction for implementing application-agnostic functionality.

Filters receive a request and a service with which it is composed.

?

1	type Filter[Req, Rep] = (Req, Service[Req, Rep]) => Future[Rep]

Filters can be chained together before being applied to a service.

?

1 2 3 4

recordHandletime andThen traceRequest andThen collectJvmStats andThen myService

This allows for clean abstractions of logic and good separation of concerns. Internally, Finagle heavily uses filters. Filters enhance modularity and reusability. They’ve proved valuable for testing as they can be unit tested in isolation with minimal mocking.

Failure Management

At scale, failure becomes common rather than exceptional;

Finagle can balance load across a cluster of hosts implicitly using latency as a heuristic. A Finagle client locally tracks load on every host it knows about. It does so by counting the number of outstanding requests being dispatched to a single host. Given that, Finagle will dispatch new requests to hosts with the lowest load and implicitly the lowest latency.

Failed requests will cause Finagle to close the connection to the failing host and remove it from the load balancer. In the background, Finagle will continuously try to reconnect. The host will be re-added to the load balancer only after Finagle can re-establish a connection. Service owners are then free to shut down individual hosts without negatively impacting downstream clients. Clients also keep per-connection health heuristics and remove the connection if it’s deemed unhealthy.

the flow of data is defined and we leave the concurrency to Finagle. We don’t have to manage thread pools or worry about race conditions. 

Recently the internal structure of Finagle has been updated to be more modular, paving the way for an upgrade to Netty 4.

Your Server as a Function by Marius Eriksen provides more insight into Finagle’s philosophy.

https://blog.twitter.com/2011/finagle-a-protocol-agnostic-rpc-system

Sophisticated network servers and clients have many moving parts: failure detectors, load-balancers, failover strategies, and so on. These parts need to work together in a delicate balance to be resilient to the varieties of failure that occur in a large production system.

Finagle provides a robust implementation of:

• connection pools, with throttling to avoid TCP connection churn;
• failure detectors, to identify slow or crashed hosts;
• failover strategies, to direct traffic away from unhealthy hosts;
• load-balancers, including “least-connections” and other strategies; and
• back-pressure techniques, to defend servers against abusive clients and dogpiling.

Additionally, Finagle makes it easier to build and deploy a service that

• publishes standard statistics, logs, and exception reports;
• supports distributed tracing (a la Dapper) across protocols;
• optionally uses ZooKeeper for cluster management; and
• supports common sharding strategies.

Finagle is flexible and easy to use because it is designed around a few simple, composable primitives: Futures, Services, and Filters.

Futures can be combined and transformed in interesting ways, leading to the kind of compositional behavior commonly seen in functional programming languages.

val service = new Service[HttpRequest, HttpResponse] {

  def apply(request: HttpRequest) =

    Future(new DefaultHttpResponse(HTTP_1_1, OK))

}

val address = new InetSocketAddress(10000)

val server: Server[HttpRequest, HttpResponse] = ServerBuilder()

  .name("MyWebServer")

  .codec(Http())

  .bindTo(address)

  .build(service)

Filters are a useful way to isolate distinct phases of your application into a pipeline. For example, you may need to handle exceptions, authorization, and so forth before your Service responds to a request.

http://www.infoq.com/cn/news/2014/05/twitter-finagle-intro

https://en.wikipedia.org/wiki/Back_pressure

Back pressure refers to pressure opposed to the desired flow of a fluid in a confined place such as a pipe. It is often caused by obstructions or tight bends in the confinement vessel along which it is moving, such as piping or air vents.