Why dynamic call on ref return property throws exception?

Question

I have been looking into c# 7 ref return feature and came across unexpected scenario when running one of test snippets.

The following code:

namespace StackOverflow
{
    using System;

    public interface IXTuple<T>
    {
        T Item1 { get; set; }
    }

    public class RefXTuple<T> : IXTuple<T>
    {
        T _item1;

        public ref T Item1Ref
        {
            get => ref _item1;
        }

        public T Item1
        {
            get => _item1;
            set => _item1 = value;
        }
    }

    public struct ValXTuple<T> : IXTuple<T>
    {
        T _item1;

        public T Item1
        {
            get => _item1;
            set => _item1 = value;
        }
    }

    public class UseXTuple
    {
        public void Experiment1()
        {
            try
            {
                RefXTuple<ValXTuple<String>> refValXTuple = new RefXTuple<ValXTuple<String>> {Item1 = new ValXTuple<String> {Item1 = "B-"}};
                dynamic dynXTuple = refValXTuple;

                refValXTuple.Item1Ref.Item1 += "!";
                Console.WriteLine($"Print 1: {refValXTuple.Item1.Item1 == "B-!"}");
                Console.WriteLine($"Print 2: {dynXTuple.Item1.Item1 == "B-!"}");

                refValXTuple.Item1Ref.Item1 += "!";
                Console.WriteLine($"Print 3: {refValXTuple.Item1Ref.Item1 == "B-!!"}");
                Console.WriteLine($"Print 4: {dynXTuple.Item1Ref.Item1 == "B-!!"}");
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}   

gives the following printout:

Print 1: True
Print 2: True
Print 3: True
System.InvalidCastException: The result type 'StackOverflow.ValXTuple`1[System.String]&' of the dynamic binding produced by binder 'Microsoft.CSharp.RuntimeBinder.CSharpGetMemberBinder' is not compatible with the result type 'System.Object' expected by the call site.
   at System.Dynamic.DynamicMetaObjectBinder.Bind(Object[] args, ReadOnlyCollection`1 parameters, LabelTarget returnLabel)
   at System.Runtime.CompilerServices.CallSiteBinder.BindCore[T](CallSite`1 site, Object[] args)
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at StackOverflow.UseXTuple.Experiment1() in C:\Repo\TestBed.Lib\Features\ReturnRefByDynamic.cs:line 52

which is somewhat unexpected. I would expect to see the following line in the printout instead of exception:

Print 4: True

Exception is thrown when property which returns ref is called through dynamic variable. I have spent some time looking for the answer (e.g. here C# Reference) but could not find anything which could justify such behavior. I would appreciate your help on this.

It is clear that call via strong typed variable works just fine ("Print 3" line) whereas the same call via dynamic variable throws an exception. Can we consider calls via dynamic variable safe and predictable in this circumstances? Are there any other scenario where dynamic calls produce far different results then their strong typed counterparts?

Answer 1

dynamic is just object with a fancy hat on it that tells the compiler to generate type checks at run time. This gives us one of the fundamental rules of dynamic:

If you cannot use object in a location, then you cannot use dynamic in that location either.

You can't initialize an object variable with a ref something call; you have to assign it to a ref something variable.

More specifically: dynamic is designed for scenarios where you're interoperating with dynamic object models, and you care so little about performance that you're willing to start the compiler again at runtime. "Ref returns" are designed for strictly typesafe scenarios where you care so much about performance that you're willing to do something dangerous like passing around variables themselves as values.

They're scenarios that have opposite use cases; do not try to use them together.

More generally: this is a great example of how difficult modern language design is. It can be very, very difficult to make a new feature like "ref returns" work well with every existing feature added to the language in the previous decade. And when you add a new feature like "dynamic" it is hard to know what problems that is going to cause when you add all the features you're going to add in the future.

Are there any other scenario where dynamic calls produce far different results then their strong typed counterparts?

Sure. For example, since dynamic is object, and since there is no such thing as a "boxed nullable value type", you can run into odd situations when you have a T? and convert it to dynamic. You cannot then call .Value on it because it is no longer a T?. It's either null or T.

there is still one detail which does not fit. Probably I'm missing something. How is that the expression refValXTuple.Item1Ref.Item1 from the sample works just fine? It does not assign anything to ref variable either.

Excellent catch. Let me explain.

As you note, "ref returns" is a new feature for C# 7, but ref has been around since C# 1.0 in three ways. One you realized, and two you might not have known about.

The way you realized was that of course you can pass ref or out arguments to ref or out formal parameters; this creates an alias to the variable passed as the parameter, so the formal and the argument refer to the same variable.

The first way you perhaps might not realize that ref was in the language is actually an example of ref return; C# will sometimes generate operations on multidimensional arrays by calling helper methods that return a ref into the array. But there is no "user visible" surface to this in the language.

The second way is the this of a call to a method on a value type is a ref. That's how you can mutate the receiver of a call in a mutable value type! this is an alias for the variable which contains the call.

So now let's look at your call site. We'll simplify it:

bool result = refValXTuple.Item1Ref.Item1 == "whatever";

OK, what's going to happen at the IL level here? At a high level we need:

push the left side of the equality
push "whatever"
call string equality
store the result in the local

What are we going to do to compute the left side of the equality?

put refValXTuple on the stack
call the getter of Item1Ref with the receiver that's on the stack

What's the receiver? It's a reference. Not a ref. It's a reference to a perfectly ordinary object of reference type.

What does it return? When we are done, the reference is popped, and a ref ValXTuple<String> is pushed.

OK, what do we need to set up the call to Item1? It's a call to a member of a value type, so we'll need a ref ValXTuple<String> on the stack and... we have one! Hallelujah, the compiler doesn't have to do any additional work here to meet its obligation to put a ref on the stack before the call.

So that's why this works. You need a ref on the stack at this point and you have one.

Put it all together; suppose loc.0 contains a reference to our RefXTuple. The IL is:

// the evaluation stack is empty
ldloc.0
// a reference to the refxtuple is on the stack
callvirt instance !0& class StackOverflow.RefXTuple`1<valuetype StackOverflow.ValXTuple`1<string>>::get_Item1Ref()
// a ref valxtuple is on the stack
call instance !0 valuetype StackOverflow.ValXTuple`1<string>::get_Item1()
// a string is on the stack
ldstr "whatever"
// two strings are on the stack
call bool [mscorlib]System.String::op_Equality(string, string)
// a bool is on the stack
stloc.1
// the result is stored in the local and the stack is empty.

Now compare that to the dynamic case. When you say

bool result = dynXTuple.Item1Ref.Item1 == "whatever"

That basically does the moral equivalent of:

object d0 = dynXTuple;
object d1 = dynamic_property_get(d0, "Item1Ref");
object d2 = dynamic_property_get(d1, "Item1");
object d3 = "whatever"
object d4 = dynamic_equality_check(d2, d3);
bool result = dynamic_conversion_to_bool(d4);

As you can see, it is nothing but calls to helpers and assignments to object variables.

If you want to see something horrifying, take a look at the real generated IL for your dynamic expression; it is a lot more complex than I've laid out here, but morally equivalent.

---

I just thought of another way to express this concisely. Consider:

refValXTuple.Item1Ref.Item1

The refValXTuple.Item1Ref of this expression is classified as a variable, not a value because it is a ref to a variable; it's an alias. .Item1 requires that the receiver must be a variable -- because Item1 might (bizarrely!) mutate the variable, and so it's good that we have a variable in hand.

By contrast, with

dynXTuple.Item1Ref.Item1

the subexpression dynXTuple.Item1Ref is a value, and moreover, one that must be storable in an object so that we can do a dynamic invocation of .Item1 on that object. But at runtime it turns out to not be an object, and moreover, is not even anything we can convert to object. A value type you can box, but a ref-to-variable-of-value-type is not a boxable thing.