23 Dec 2021

gRPC client-side load balancing in .NET

This blog post is my first contribution to the 2021 C# Advent Calendar many thanks to both Matthew D. Groves and Calvin A. Allen for this opportunity if you are interested to read other members blog postd so far please check them out I am sure you will find useful blog posts of this Advent Calendar.

In short, what is gRPC?

The gRPC is one of the modern open-source framework technologies that help us build applications more efficiently and with high performance. This technology is commonly used in Microservices environments to communicate between components while they depending on each other or calling each other, gRPC uses the protocol buffers over HTTP/2 for serializing structured data.Fortunately on .NET ecosystem, we can use the gRPC very easily.

Client-side load balancing

Load balancing (LB) allows us to distribute network traffics across many backend services (instances) to improve the performance and reliability of our applications, it can be categorized into two types:

Layer 4 Transport layer protocols (TCP,UDP) load balancer.
Layer 7 Application layer protocols (HTTP, TLS, DNS) load balancer.

Each of them uses different algorithms to distribute all requests between backend services (instances) and the client applications, the most common algorithm is widely used in load balancing (Round Robin, Weighted Round Robin, Least Connection, Least response time … etc) in this article we’re trying to configure layer 7 as known as the application layer load balancer. That’s the idea! a client application would send and receives requests from a list of backend services’ IP address or a list of DNS service records the client application tries to select an IP address randomly from the list to send and receives HTTP requests.

THE PROJECT CASE STUDY: In this article we’re going to create a gRPC service app that handling the feed of the .NET Blog and retrieves latest blog post that contains title,summary publishdate and links the gRPC service will parse the feed data into buffer protocol then received by gRPC Client app.

Here’s the gRPC service backend.

public class MainFeedService : FeedService.FeedServiceBase
{
private string feedUrl = "https://devblogs.microsoft.com/dotnet/feed/";
private readonly ILogger<MainFeedService> _logger;

public MainFeedService(ILogger<MainFeedService> logger)
{
  _logger = logger;
}


public override async Task<FeedResponce> GetFeed(Empty request, ServerCallContext context)
{

  var httpContext = context.GetHttpContext();

  await context.WriteResponseHeadersAsync(new Metadata { { "host", $"{httpContext.Request.Scheme}://{httpContext.Request.Host}" } });

  FeedResponce listFeedResponce = new();

  using XmlReader reader = XmlReader.Create(feedUrl, new XmlReaderSettings { Async = true });

  SyndicationFeed feed = SyndicationFeed.Load(reader);

  var query = from feedItem in feed.Items.OrderByDescending(x => x.PublishDate)

              select new FeedModel
              {

                  Title = feedItem.Title.Text,
                  Summary = feedItem.Summary.Text,
                  Link = feedItem.Links[0].Uri.ToString(),
                  PublishDate = feedItem.PublishDate.ToString()


              };

  listFeedResponce.FeedItems.AddRange(query.ToList());

  reader.Close();


  return await Task.FromResult(listFeedResponce);
}

}

In the gRPC service we added (host) to MetaData is represented as a key, value pairs it’s like an HTTP header, and the host key holds the host address value for example host = http://localhost:port.

var httpContext = context.GetHttpContext();

await context.WriteResponseHeadersAsync(new Metadata { { "host", $"{httpContext.Request.Scheme}://{httpContext.Request.Host}" } });

The gRPC service backend and client have the following .proto file

 
syntax = "proto3";

option csharp_namespace = "GrpcServiceK8s";
import "google/protobuf/empty.proto";
package mainservice;
 

 
// The feed service definition.
service FeedService {
	 rpc GetFeed (google.protobuf.Empty) returns (FeedResponce);
}
 
// The response message containing list of feed items.
message FeedResponce {
	repeated FeedModel FeedItems = 1;
}


// The response message containing the feed.
message FeedModel {

	string title = 1;
	string summary = 2;
	string link = 3;
	string publishDate = 4;

}

Configure address resolver

On the gRPC client-side to enable load balancing it needs a way to resolve addresses fortunately .NET provides many ways to configure the gRPC resolver such as :

StaticResolverFactory
Custom Resolver
DnsResolverFactory

StaticResolverFactory

In the static resolver approach the resolver doesn’t call addresses from an external source this approach is suitable if we already know all the service’s addresses.

Here’s Program.cs file which StaticResolverFactory has registered in dependency injection (DI).

var addressCollection = new StaticResolverFactory(address => new[]
    {
                new DnsEndPoint("localhost", 5244),
                new DnsEndPoint("localhost", 5135),
                new DnsEndPoint("localhost", 5155)
    });


builder.Services.AddSingleton<ResolverFactory>(addressCollection);

builder.Services.AddSingleton(channelService => {

var methodConfig = new MethodConfig
{
    Names = { MethodName.Default },
    RetryPolicy = new RetryPolicy
    {
        MaxAttempts = 5,
        InitialBackoff = TimeSpan.FromSeconds(1),
        MaxBackoff = TimeSpan.FromSeconds(5),
        BackoffMultiplier = 1.5,
        RetryableStatusCodes = { Grpc.Core.StatusCode.Unavailable }
    }
};

     
return GrpcChannel.ForAddress("static:///feed-host", new GrpcChannelOptions

{
    Credentials = ChannelCredentials.Insecure,
    ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } },
    ServiceProvider = channelService

});


});

The above code (StaticResolverFactory) returns a collection of addresses.

Also, we had configured Retry policies (MethodConfig) to provide more availability for the gRPC Client application this section is optional.

Because we used the StaticResolverFactory the schema for the addresses should be static (static:///feed-host)

gRPC in .NET provides two types of Load Balancing policies (Pick First) and (Round Robin) for our project we configured the Round Robin policy (Algorithm).

Configure the Round Robin policy:

ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } }

Configure the Pick First policy:

ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new PickFirstConfig() }, MethodConfigs = { methodConfig } }

This policy takes a list of addresses from the resolver then it tries to associate to the list of addresses until it finds a reachable one.

Custom Resolver

Sometimes we need to create a custom thing for problems, fortunately, .NET allows us to create a custom resolver, and we can implement (Resolver and ResolverFactory) objects to perform this goal. For example, we have an XML file (addresses.xml) on our local machine that contains a list of service addresses.


public class xmlFileResolver: Resolver
{


	private readonly Uri _uriAddress;

	private Action < ResolverResult > ? _listener;


	public xmlFileResolver(Uri uriAddress)
	{
		_uriAddress = uriAddress;
	}



	public override async Task RefreshAsync(CancellationToken cancellationToken)
	{


		await Task.Run(async () =>
		{

			using(var reader = new StreamReader(_uriAddress.LocalPath))
			{
				XmlSerializer serializer = new XmlSerializer(typeof(Root));

				var resultsDeserialize = (Root) serializer.Deserialize(reader);

				IReadOnlyList < DnsEndPoint > listAddresses = resultsDeserialize.Element.Select(r => new DnsEndPoint(r.HostName, r.Port)).ToList();

				_listener(ResolverResult.ForResult(listAddresses, serviceConfig: null));


			}


		});

	}


	public override void Start(Action < ResolverResult > listener)
	{
		_listener = listener;
	}
}

public class customFileResolver: ResolverFactory
{

	public override string Name => "file";

	public override Resolver Create(ResolverOptions options)
	{
		return new xmlFileResolver(options.Address);
	}
}

}

After the custom resolver has been created it needs to be registered in the dependency injection (DI) also it needs a path of the file like the following code.


builder.Services.AddSingleton<ResolverFactory, customFileResolver>();
 
builder.Services.AddSingleton(channelService => {

 return GrpcChannel.ForAddress("file:///c:/urls/addresses.xml", new GrpcChannelOptions

    {
        Credentials = ChannelCredentials.Insecure,
        ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() } },
        ServiceProvider = channelService

    });

});

DnsResolverFactory

In this approach, the DNS Resolver will depend on an external source to get a Service record that contains the address with the ports of the backend service (instance), to configure load balancing we’re using the Headless Service in Kubernetes that provides us all requirements for the gRPC client DnsResolver, for a better understanding of the idea we have the following interactive diagram.

“gRPC client-side load balancing in .NET”

First, we need to create gRPC service backend pods in Kubernetes.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-service-pod
  labels:
    app: grpc-service-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: grpc-service-app
  template:
    metadata:
      labels:
        app: grpc-service-app
    spec:
      containers:
      - name: grpcservice
        image: rebinoq/grpcservicek8s:v1
        resources:
            requests:
              memory: "64Mi"
              cpu: "125m"
            limits:
              memory: "128Mi" 
              cpu: "250m"
        ports:
        - containerPort: 80

Run the following command to create the service pod in Kubernetes.

kubectl create -f grpc-service-pod.yaml

To create the Headless service in Kubernetes, we run the following yaml code.

note: in Kuberetes environment to create a Headless service (svc) the clusterIP key must have ‘none’ value.

apiVersion: v1
kind: Service
metadata:
  name: grpc-headless-svc
spec:
  clusterIP: None 
  selector:
    app: grpc-service-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Run the following command to create the Headless service.

kubectl create -f grpc-headless-svc.yaml

It is time to check out the status of the Headless service and DNS Resolver we run the following command.

kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml

kubectl exec -i -t dnsutils -- nslookup grpc-headless-svc       
Server:         10.96.0.10
Address:        10.96.0.10#53

Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.163
Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.165
Name:   grpc-headless-svc.default.svc.cluster.local
Address: 10.1.1.167

So far everything going fine, finally, we got the service (grpc-headless-svc.default.svc.cluster.local) including the three Pods IP addresses

Again we create another Pod for the gRPC client and just run the following yaml code.

note: we pass dns:///grpc-headless-svc in the environment variable to the gRPC client.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: grpc-client-pod
  labels:
    app: grpc-client-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grpc-client-app
  template:
    metadata:
      labels:
        app: grpc-client-app
    spec:
      containers:
      - name: grpcclient
        image: rebinoq/grpcclientk8s:v1
        resources:
            requests:
              memory: "64Mi"
              cpu: "125m"
            limits:
              memory: "128Mi" 
              cpu: "250m"
        ports:
        - containerPort: 80
        env:
        - name: k8s-svc-url
          value: dns:///grpc-headless-svc

run the command

kubectl create -f grpc-client-pod.yaml

The last thing we need to create in Kubernetes is called Service Node this service enables us to access the gRPC client Pod from outside of the Kubernetes.

apiVersion: v1
kind: Service
metadata:
  name: grpc-nodeport-svc
spec:
  type: NodePort
  selector:
    app: grpc-client-app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
      nodePort: 30030

run the command

kubectl create -f grpc-nodeport-svc.yaml

Because of this port (30030), we can access the gRPC client app. for example http://localhost:30030

So far everything has been done on Kubernetes, let’s see the DnsResolver configure code in the gRPC client.

Here’s the dns resolver code

builder.Services.AddSingleton<ResolverFactory>(new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(25)));


builder.Services.AddSingleton(channelService => {


    var methodConfig = new MethodConfig
    {
        Names = { MethodName.Default },
        RetryPolicy = new RetryPolicy
        {
            MaxAttempts = 5,
            InitialBackoff = TimeSpan.FromSeconds(1),
            MaxBackoff = TimeSpan.FromSeconds(5),
            BackoffMultiplier = 1.5,
            RetryableStatusCodes = { Grpc.Core.StatusCode.Unavailable }
        }
    };

    var configuration = builder.Configuration;


return GrpcChannel.ForAddress(configuration.GetValue<string>("k8s-svc-url"), new GrpcChannelOptions

{
    Credentials = ChannelCredentials.Insecure,
    ServiceConfig = new ServiceConfig { LoadBalancingConfigs = { new RoundRobinConfig() }, MethodConfigs = { methodConfig } },
    ServiceProvider = channelService

});

      
});

Whenever any connection has disconnected the DNS resolver will refresh and immediately it tries to detect another Pod of the gRPS service (instance).

builder.Services.AddSingleton<ResolverFactory>(new DnsResolverFactory(refreshInterval: TimeSpan.FromSeconds(25)));

Here’s the gRPC client code.

public class HomeController : Controller
{
private readonly ILogger<HomeController> _logger;
private readonly GrpcChannel _grpcChannel;
private List<FeedViewModel>? _feedViewModel;
public HomeController(ILogger<HomeController> logger, GrpcChannel grpcChannel)
{
    _logger = logger;
    _grpcChannel = grpcChannel;
            
}

public async Task<IActionResult> Index()
{
             
    var client = new FeedService.FeedServiceClient(_grpcChannel);

    try
    {
        var replay = client.GetFeedAsync(new Google.Protobuf.WellKnownTypes.Empty());


        var responseHeaders = await replay.ResponseHeadersAsync;

        ViewData["hostAdress"] = responseHeaders.GetValue("host");

        var responseFeed = await replay;

                
        _feedViewModel = new List<FeedViewModel>();

        foreach (var item in responseFeed.FeedItems)
        {
                    

            _feedViewModel.Add(new FeedViewModel
            {

                title = item.Title,
                Summary = item.Summary,
                publishDate = DateTime.Parse(item.PublishDate),
                Link = item.Link

            });
        }
    }

    catch (RpcException rpcex)
    {

        _logger.LogWarning(rpcex.StatusCode.ToString());
        _logger.LogError(rpcex.Message);
    }

    return View(_feedViewModel);
}

       
}

The final result of the gRPC client-side load balancing.

At the end of this blog post, I hope you will find it useful. Note: The code is only for demonstration purposes! maybe it is not suitable for production use it should be reviewed and tested

The source code of the project can be found on this GitHub repository.