New – Application Load Balancer Support for End-to-End HTTP/2 and gRPC
- October 30, 2020
Thanks to its efficiency and support for numerous programming languages, gRPC is a popular choice for microservice integrations and client-server communications. gRPC is a high performance remote procedure call (RPC) framework using HTTP/2 for transport and Protocol Buffers to describe the interface.
To make it easier to use gRPC with your applications, Application Load Balancer (ALB) now supports HTTP/2 end-to-end, enabling you to publish gRPC services alongside non-gRPC services via a single load balancer. You can use Amazon Elastic Compute Cloud (EC2) instances or IP addresses (for example with AWS Fargate) as gRPC targets, with support for gRPC health checks for the target groups. In this way, you can use ALBs to terminate, route and load balance the gRPC traffic between your microservices or between gRPC-enabled clients and services.
ALB provides rich content based routing features to inspect the gRPC calls and route them to appropriate services. More specifically, the ALB provides health checks that can examine the gRPC status code, metrics for gRPC request count, access logs that differentiate gRPC requests, and gRPC specific response headers. Additionally, you can benefit from native features like stickiness, different load balancing algorithms, and TLS termination.
How to Use gRPC with Application Load Balancer
To test this new feature, I start by preparing the gRPC server application. I am using the Python programming language and the
route_guide demo included in the
grpc repo. I use this application because it quickly introduces some of the many ways a client and a server can interact via gRPC, such as:
First, I prepare a Dockerfile to have the
route_guide application run in a container. Technically, this isn’t necessary because I could use a plain EC2 instance, but using containers with gRPC is so common than this makes the example more relevant.
I build the container image and upload it to Amazon Elastic Container Registry.
aws ecr get-login-password --region eu-north-1 | docker login --username AWS --password-stdin 123412341234.dkr.ecr.eu-north-1.amazonaws.com docker build -t route-guide . docker tag route-guide:latest 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest docker push 123412341234.dkr.ecr.eu-north-1.amazonaws.com/route-guide:latest
In the Amazon ECS console, I create a new cluster using the Networking only template. I name the cluster
demo and create a new VPC for this cluster, leaving all other values to their default. The ECS console is using AWS CloudFormation to set up the resources for the cluster. After a few minutes, all resources have been successfully created and the cluster is ready.
To give access to the gRPC traffic, I create a security group to allow inbound TCP traffic on port
50051 from my client laptop. I name this security group
Now, I create the load balancer. In the EC2 console, I select Load Balancers on the left, and then Create Load Balancer. In the next step, I create an Application Load Balancer and name it
route-guide. I leave the default internet-facing Scheme, because I want to connect to the gRPC service over public internet. In the Listeners section, I select HTTPS as protocol and
50051 for the port. Just below, I select the newly created VPC and the two Availability Zones created by the ECS console.
Next, in the security settings, I choose a certificate I created before, managed by AWS Certificate Manager (ACM).
Then, I select the
default security group and the
gRPC security group I created above. Each VPC automatically comes with a default security group that gives network access to other resources using the same security group.
In the routing section, I create a new target group named
route-guide. Since I am planning to use AWS Fargate to run the gRPC server, I select the IP address target type. The ALB supports both secure and insecure connections for target groups using the gRPC protocol. Here, I use HTTP (because I am using an insecure connection between the load balancer and the containers running the gRPC server) with port 50051. Then, I select the VPC used by the ECS cluster. For Protocol Version, I use gRPC.
In the Advanced health check settings, I can specify which gRPC Success codes to use when checking for a correct response. I leave the default of
12 which means unimplemented.
12 is returned by a gRPC server if a method is not found. It works in this case because I’m using a path that is not implemented by the
route_guide application. More generally, checking for code
12 is a quick way to verify that your gRPC server is running correctly. To be more specific in your health checks, you can use a single code, a list, or a range, depending on what you expect from your implementation.
In the next step, I don’t register any target and complete the creation of the target group.
Back in the ECS console, I create a new task definition compatible with the Fargate launch type. I name it
route-guideand give it a minimum amount of resources: 0.5 GB of memory and 0.25 CPU units. I add a container definition using the image URI of the container image I uploaded to ECR, and the container port
50051/tcp, the network port exposed in the Dockerfile above.
route-guidetask definition selected, I select Create Service in the Actions menu. I use the Fargate launch type,
route-guide as service name, and 2 for the number of tasks. In the next step, I add the two subnets in the VPC, and select the
default security group to allow the load balancer to reach the tasks. In the Load balancing section, I select Application Load Balancer and the
route-guide load balancer.
For the Container to balance, I select
route-guide:50051:50051 and Add to load balancer.
Then, I choose the
route-guide target group.
At the next step, I select Do not adjust the service’s desired count to not use auto scaling for this demo.
I complete the creation of the ECS service. After a few minutes the two tasks are
RUNNING. Looking at the Targets tab of the
route-guide target group, I see that the two targets are healthy. The load balancer is now ready to accept traffic.
# create credentials credentials = grpc.ssl_channel_credentials() # create secure channel using credentials with grpc.secure_channel('
:50051', credentials) as channel: stub = route_guide_pb2_grpc.RouteGuideStub(channel) print("-------------- GetFeature --------------") guide_get_feature(stub) print("-------------- ListFeatures --------------") guide_list_features(stub) print("-------------- RecordRoute --------------") guide_record_route(stub) print("-------------- RouteChat --------------") guide_route_chat(stub)
Now, I start the client to test the gRPC channel with some workload. In the output, I see that the
route_guide application is using unary, server-side streaming, client-side streaming, and bi-directional streaming in the request and the response.
Well, the output is pretty long, but as I said at the beginning of this post, the
route-guide demo is showing many of the different ways a client and a server can interact using gRPC, beyond basic RPC invocations.
End-to-end HTTP/2 and gRPC Support is available today for new and existing Application Load Balancers in all regions. You can use this feature via the console, AWS Command Line Interface (CLI), AWS SDKs. We are working to add AWS CloudFormation support soon.
There is no additional cost for using the gRPC protocol with an ALB. For more information, you can see the Elastic Load Balancing pricing page.
With these new features, it’s much easier to use gRPC to integrate your applications, or to improve client/server communication. To learn more, please see the documentation.