How to use Polly and IHttpClientFactory per url

Question

I am trying to find a way to use a HttpClient using IHttpClientFactory.CreateClient([name]) where [name] is an HttpClient that was configured using Polly in the Startup class using PolicyRegistry. I want that policy to apply for each of the urls that i make calls using that client.

So if I create a client called "myclient" and throughout my application I will make calls to N different urls , using that named client, I want to have the policy applied for each url individually , not as a whole.

Startup

PolicyRegistry registry = new PolicyRegistry();
AsyncPolicyWrap type1Policy = //somePolicy;
registry.Add("type1",type1Policy);
services.AddHttpClient("someClient").AddPolicyHandlerFromRegistry("type1");

Some Consumer Service

public class SomeService
{
    private IHttpClientFactory factory;
    public SomeService(IHttpClientFactory factory)
    {
        this.factory=factory;
    }
    public void SomeCall(string url)
    {
        var client=factory.GetClient("someClient");
        client.SendAsync(...);
    }
}

In the above case I want the policy set up in the startup class to be taken individually for each of the urls that I am calling.

Is that possible ?

Answer 1

In order to achieve that you need to register your HttpClient and your PolicyRegistry separately:

services.AddHttpClient("someClient");
services.AddPolicyRegistry(registry);

Then you can ask the DI to provide that registry for you:

public class SomeService
{
    private readonly IHttpClientFactory factory;
    private readonly IReadOnlyPolicyRegistry<string> policyRegistry;

    public SomeService(IHttpClientFactory factory, IReadOnlyPolicyRegistry<string> policyRegistry)
    {
        this.factory = factory;
        this.policyRegistry = policyRegistry;
    }
    ...
}

Finally inside your SomeCall you can retrieve the policy:

public async Task SomeCall(string url)
{
     var client = factory.GetClient("someClient");
     var policy = policyRegistry.Get<IAsyncPolicy>("type1");

     await policy.ExecuteAsync(async ct => await client.SendAsync(..., ct), CancellationToken.None);
}

---

UPDATE #1: Create new policy per url

In functional programming there is a well-known technique, called Memoization. It basically memorizes / caches the output of a method based on the received parameters. In other words if the method receives the same input then it emits the result from a cache rather than re-executing it.

This technique works fine for pure functions. They return the same value if they receive the exact same input in a side effect free way.

So, if we can have a solution which

• creates a new policy if the url is new
• or returns a cached policy if the url is an "already seen" one then we are good to go.

Here is a simple implementation, but you can find a lots of variations like 1, 2, 3, etc.

public interface IMemoizer<K, V> where K : IComparable
{
    Func<K, V> Memoize(Func<K, V> toBeMemoized);
}

public sealed class Memoizer<K, V> : IMemoizer<K, V> where K: IComparable
{
    public static readonly ConcurrentDictionary<K, Lazy<V>> memoizedValues;
    static Memoizer()
    {
        memoizedValues = new ConcurrentDictionary<K, Lazy<V>>();
    }

    public Func<K, V> Memoize(Func<K, V> toBeMemoized)
        => (K input) => memoizedValues.GetOrAdd(input, (K param) => new Lazy<V>(() => toBeMemoized(param))).Value;
}

• If you want to have TTL then you can replace the ConcurrenctDictionary to ASP.NET's MemoryCache

Here is a sample for usage:

static void Main()
{
    var memoizer = new Memoizer<string, IAsyncPolicy<HttpResponseMessage>>();
    var policyMemoizer = memoizer.Memoize(GetSamplePolicy);

    var gPolicy1 = policyMemoizer("https://google.com");
    var soPolicy = policyMemoizer("https://stackoverflow.com");
    var gPolicy2 = policyMemoizer("https://google.com");

    Console.WriteLine(gPolicy1 == gPolicy2); //true
    Console.WriteLine(gPolicy1 == soPolicy); //false
    
    var policyMemoizer2 = memoizer.Memoize(GetSamplePolicy);
    var gPolicy3 = policyMemoizer2("https://google.com");

    Console.WriteLine(gPolicy1 == gPolicy3); //true
}

static IAsyncPolicy<HttpResponseMessage> GetSamplePolicy(string _)
        => Policy<HttpResponseMessage>
        .Handle<HttpRequestException>()
        .WaitAndRetryAsync(3, _ => TimeSpan.FromSeconds(2));

You can register the memoizer as a singleton and you are ready to use it. :D

I hope it helps you to achieve the desired behaviour.