.Net 6 use option pattern to read json file with int as key

Question

I am trying to read a json file using a c# option pattern. However, I am running into a problem where a property of Dictionary<int, myClass> is not being mapped correctly unless I changed it the key to string like Dictionary<string, myClass>

This is my example json file: test.json

{
  "TestMap": [
    {
      "SenderID": "RIMC_EVAC_ZONES",
      "FeedType": null,
      "CategoryMappings": {
        "4": {
          "CategoryID": 4,
          "VccCategoryName": "Local Disaster",
          "ReiCategoryName": "Local Disaster",
          "SubcategoryMappings": {
            "177": [
              {
                "SubcategoryID": 177,
                "ParentCategoryID": 4,
                "VccSubcategoryName": "Evacuation",
                "ReiSubcategoryName": "Evacuation"
              }
            ]
          }
        }
      }
    },
    {
      "SenderID": "EARLY_HURRICANE",
      "FeedType": null,
      "CategoryMappings": {
        "16": {
          "CategoryID": 16,
          "VccCategoryName": "Tropical Storm",
          "ReiCategoryName": "Tropical Storm",
          "SubcategoryMappings": null
        }
      }
    }
  ]
}

This is my model:

public class Sender {
    public string SenderID { get; set; }
    public string FeedType { get; set; }
    
    //[JsonConverter(typeof(IntKeyDictionaryConverter))]    
    public Dictionary<int, CategoryMapping> CategoryMappings { get; set; }
}

public class CategoryMapping {
    public int CategoryID { get; set; }
    public string VccCategoryName { get; set; }
    public string ReiCategoryName { get; set; }
    
    public Dictionary<int, List<SubcategoryMapping>> SubcategoryMappings { get; set; }
}

public class SubcategoryMapping
{
    public int SubcategoryID { get; set; }
    public int ParentCategoryID { get; set; }
    public string VccSubcategoryName { get; set; }
    public string ReiSubcategoryName { get; set; }
}

How I register the page:

builder.Configuration.AddJsonFile("Properties/test.json", optional: true, reloadOnChange: true)

And How I called to retrive the data:

List<Sender> people = Configuration.GetSection("TestMap").Get<List<Sender>>();

Problem:: with my current model, The variable "People" will have its CategoryMapping as null but will retrieve other information such as SenderID or FeedType correctly.

However, if I change the "CategoryingMappings" to Dictionary<string, CategoryMapping>, then it will be mapped correctly. Same thing happened to "SubcategoryMapping" property of CategoryMapping class.

What I have tried: I tried to write custom converter, but it didn't work and cause the conversion to fail

public class IntKeyDictionaryConverter : JsonConverter<Dictionary<int, CategoryMapping>>
{
    public override Dictionary<int, CategoryMapping> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        var dictionary = new Dictionary<int, CategoryMapping>();
    
        if (reader.TokenType != JsonTokenType.StartObject)
        {
            throw new JsonException();
        }

        while (reader.Read())
        {
            if (reader.TokenType == JsonTokenType.EndObject)
            {
                return dictionary;
            }

            if (reader.TokenType == JsonTokenType.PropertyName)
            {
                int key = int.Parse(reader.GetString());
                reader.Read(); 
                CategoryMapping value = JsonSerializer.Deserialize<CategoryMapping>(ref reader, options);
                dictionary.Add(key, value);
            }
        }

        throw new JsonException();
    }

    public override void Write(Utf8JsonWriter writer, Dictionary<int, CategoryMapping> value, JsonSerializerOptions options) {
    }
}

What I need help with : I want integer as key when read the json into my class object. How do I achieve that?

Answer 1

Your problem is that Microsoft.Extensions.Configuration.Binder does not use System.Text.Json to deserialize JSON. Instead it manually parses the JSON to IConfigurationSection elements then binds those to POCOs using reflection. None of this seems to be documented, so for confirmation you may check the source code for ConfigurationBinder.

In the .NET 6, the method BindDictionary() is used to bind to a dictionary. It has a commented restriction on dictionary key types to string and enum types:

private static void BindDictionary(
   object dictionary,
    [DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicProperties | DynamicallyAccessedMemberTypes.NonPublicProperties)]
    Type dictionaryType,
    IConfiguration config, BinderOptions options)
{
    // IDictionary<K,V> is guaranteed to have exactly two parameters
    Type keyType = dictionaryType.GenericTypeArguments[0];
    Type valueType = dictionaryType.GenericTypeArguments[1];
    bool keyTypeIsEnum = keyType.IsEnum;

    if (keyType != typeof(string) && !keyTypeIsEnum)
    {
        // We only support string and enum keys
        return;
    }

In .NET 7 an additional method BindDictionaryInterface() exists which contains logic (added via PR #71609) that explicitly supports numeric keys:

private static object? BindDictionaryInterface(
    object? source,
    [DynamicallyAccessedMembers(DynamicallyAccessedMemberTypes.PublicProperties | DynamicallyAccessedMemberTypes.NonPublicProperties)]
    Type dictionaryType,
    IConfiguration config, BinderOptions options)
{
    // IDictionary<K,V> is guaranteed to have exactly two parameters
    Type keyType = dictionaryType.GenericTypeArguments[0];
    Type valueType = dictionaryType.GenericTypeArguments[1];
    bool keyTypeIsEnum = keyType.IsEnum;
    bool keyTypeIsInteger =
        keyType == typeof(sbyte) ||
        keyType == typeof(byte) ||
        keyType == typeof(short) ||
        keyType == typeof(ushort) ||
        keyType == typeof(int) ||
        keyType == typeof(uint) ||
        keyType == typeof(long) ||
        keyType == typeof(ulong);

    if (keyType != typeof(string) && !keyTypeIsEnum && !keyTypeIsInteger)
    {
        // We only support string, enum and integer (except nint-IntPtr and nuint-UIntPtr) keys
        return null;
    }

This explains why the public Dictionary<int, CategoryMapping> CategoryMappings { get; set; } property can be bound successfully only after moving to .NET 7.

If you cannot move to .NET 7 but still require the use of dictionaries with integer keys, then as a workaround you could use the adapter pattern to wrap your integer dictionaries in IDictionary<string, TValue> surrogates.

First, define the following IntegerDictionaryAdapter<TValue> class:

public class IntegerDictionaryAdapter<TValue> : AdapterDictionary<int, string, TValue>
{
    static int ToInt(string value) => int.Parse(value, NumberFormatInfo.InvariantInfo);
    static string ToString(int value) => value.ToString(NumberFormatInfo.InvariantInfo);

    public IntegerDictionaryAdapter() : base(new Dictionary<int, TValue>(), s => ToInt(s), i => ToString(i)) { }
    public IntegerDictionaryAdapter(IDictionary<int, TValue> dictionary) : base(dictionary, s => ToInt(s), i => ToString(i)) { }
}

public class AdapterDictionary<TKeyIn, TKeyOut, TValue> : IDictionary<TKeyOut, TValue>
{
    readonly IDictionary<TKeyIn, TValue> dictionary;
    readonly Func<TKeyIn, TKeyOut> mapKeyToOuter;
    readonly Func<TKeyOut, TKeyIn> mapKeyToInner;

    public AdapterDictionary(IDictionary<TKeyIn, TValue> dictionary, Func<TKeyOut, TKeyIn> mapKeyToInner, Func<TKeyIn, TKeyOut> mapKeyToOuter)
    {
        this.dictionary = dictionary ?? throw new ArgumentNullException(nameof(dictionary));
        this.mapKeyToInner = mapKeyToInner ?? throw new ArgumentNullException(nameof(mapKeyToInner));
        this.mapKeyToOuter = mapKeyToOuter ?? throw new ArgumentNullException(nameof(mapKeyToOuter));
    }

    public IDictionary<TKeyIn, TValue> UnderlyingDictionary => dictionary;

    KeyValuePair<TKeyIn, TValue> MapItemToOuter(KeyValuePair<TKeyOut, TValue> item) { return new KeyValuePair<TKeyIn, TValue>(mapKeyToInner(item.Key), item.Value); }
    KeyValuePair<TKeyOut, TValue> MapItemFromOuter(KeyValuePair<TKeyIn, TValue> item) { return new KeyValuePair<TKeyOut, TValue>(mapKeyToOuter(item.Key), item.Value); }
    public void Add(TKeyOut key, TValue value) { dictionary.Add(mapKeyToInner(key), value); }
    public bool ContainsKey(TKeyOut key) { return dictionary.ContainsKey(mapKeyToInner(key)); }
    public ICollection<TKeyOut> Keys => new CollectionAdapter<TKeyIn, TKeyOut>(() => dictionary.Keys, mapKeyToOuter, mapKeyToInner);
    public bool Remove(TKeyOut key) { return dictionary.Remove(mapKeyToInner(key)); }
    public bool TryGetValue(TKeyOut key, out TValue value) { return dictionary.TryGetValue(mapKeyToInner(key), out value); }
    public ICollection<TValue> Values { get { return dictionary.Values; } }

    public TValue this[TKeyOut key]
    {
        get { return dictionary[mapKeyToInner(key)]; }
        set { dictionary[mapKeyToInner(key)] = value; }
    }

    public void Add(KeyValuePair<TKeyOut, TValue> item) { dictionary.Add(MapItemToOuter(item)); }
    public void Clear() { dictionary.Clear(); }
    public bool Contains(KeyValuePair<TKeyOut, TValue> item) { return dictionary.Contains(MapItemToOuter(item)); }
    public void CopyTo(KeyValuePair<TKeyOut, TValue>[] array, int arrayIndex)  => this.CopyToArray(array, arrayIndex);
    public int Count { get { return dictionary.Count; } }
    public bool IsReadOnly { get { return dictionary.IsReadOnly; } }
    public bool Remove(KeyValuePair<TKeyOut, TValue> item) { return dictionary.Remove(MapItemToOuter(item)); }
    public IEnumerator<KeyValuePair<TKeyOut, TValue>> GetEnumerator() { return dictionary.Select(i => MapItemFromOuter(i)).GetEnumerator(); }
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() { return GetEnumerator(); }
}

public abstract class CollectionAdapterBase<TIn, TOut, TCollection> : ICollection<TOut> 
    where TCollection : ICollection<TIn>
{
    readonly Func<TCollection> getCollection;
    readonly Func<TIn, TOut> toOuter;

    public CollectionAdapterBase(Func<TCollection> getCollection, Func<TIn, TOut> toOuter)
    {
        this.getCollection = getCollection ?? throw new ArgumentNullException(nameof(getCollection));
        this.toOuter = toOuter ?? throw new ArgumentNullException(nameof(toOuter));
    }

    protected TCollection Collection { get { return getCollection(); } }
    protected TOut ToOuter(TIn inner) { return toOuter(inner); }
    public abstract void Add(TOut item);
    public abstract void Clear();

    public virtual bool Contains(TOut item)
    {
        var comparer = EqualityComparer<TOut>.Default;
        foreach (var member in Collection)
            if (comparer.Equals(item, ToOuter(member)))
                return true;
        return false;
    }

    public void CopyTo(TOut[] array, int arrayIndex) => this.CopyToArray(array, arrayIndex);
    public int Count { get { return Collection.Count; } }
    public abstract bool IsReadOnly { get; }
    public abstract bool Remove(TOut item);
    public IEnumerator<TOut> GetEnumerator() => Collection.Select(item => ToOuter(item)).GetEnumerator();
    IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); }
}

public class CollectionAdapter<TIn, TOut> : CollectionAdapterBase<TIn, TOut, ICollection<TIn>>
{
    readonly Func<TOut, TIn> toInner;

    public CollectionAdapter(Func<ICollection<TIn>> getCollection, Func<TIn, TOut> toOuter, Func<TOut, TIn> toInner)
        : base(getCollection, toOuter)
    {
        this.toInner = toInner ?? throw new ArgumentNullException(nameof(toInner));
    }

    protected TIn ToInner(TOut outer) { return toInner(outer); }
    public override void Add(TOut item) => Collection.Add(ToInner(item));
    public override void Clear() => Collection.Clear();
    public override bool IsReadOnly { get { return Collection.IsReadOnly; } }
    public override bool Remove(TOut item) => Collection.Remove(ToInner(item));
    public override bool Contains(TOut item) => Collection.Contains(ToInner(item));
}

public static class EnumerableExtensions
{
    internal static void CopyToArray<TItem>(this IEnumerable<TItem> collection, TItem[] array, int arrayIndex) 
    {
        ArgumentNullException.ThrowIfNull(collection);  
        ArgumentNullException.ThrowIfNull(array);   
        foreach (var item in collection)
            array[arrayIndex++] = item;
    }
}

Then modify your classes to use it as follows:

public class Sender {
    public string SenderID { get; set; }
    public string FeedType { get; set; }
    
    public IntegerDictionaryAdapter<CategoryMapping> CategoryMappings { get; set; } = new();
}

public class CategoryMapping {
    public int CategoryID { get; set; }
    public string VccCategoryName { get; set; }
    public string ReiCategoryName { get; set; }
    
    public IntegerDictionaryAdapter<CategoryMapping> SubcategoryMappings { get; set; } = new();
}

public class SubcategoryMapping
{
    public int SubcategoryID { get; set; }
    public int ParentCategoryID { get; set; }
    public string VccSubcategoryName { get; set; }
    public string ReiSubcategoryName { get; set; }
}

And now you will be able to bind your configuration data model in .NET 6 (and .NET 7 as well) as follows, using the UnderlyingDictionary property to access your integer-keyed dictionaries:

List<Sender> people = Configuration.GetSection("TestMap").Get<List<Sender>>();

people.First().CategoryMappings.UnderlyingDictionary.Add(10101, new CategoryMapping { CategoryID = 10101, VccCategoryName = "foo", ReiCategoryName = "bar" });

(Honestly however I'm not sure it's worth the trouble.)

Demo fiddle here.

Answer 2

The C# Configuration extensions use System.Text.Json to deserialize, which by default is fairly strict about types. I know there are some "Web Defaults" JsonSerializerOptions that allow reading numbers from strings, but I don't see an obvious way of how to pass the options to the configuration reader... But if there is a way, that's how you might solve this.

Otherwise, a workaround is to deserialize this JSON by hand, and don't use the options pattern.

public record Root(List<Sender> TestMap)
...
var options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
var result = JsonSerializer.Deserialize<Root>(File.OpenRead("Properties/test.json"), options);