Why does a method returning &Self makes a trait object-unsafe?

Question

So I have been exploring/trying to understand static/dynamic dispatch, as well as trait objects in Rust. I think I get the gist of it. But there is still one thing that I cannot figure out: why does a method that returns &Self make a trait object-unsafe? When returning Self I get it, we don't know the size of what will be returned (because with a trait object, we lose the original typing of the data to only have something of type dyn MyTrait, so the compiler doesn't what space to assign for the return value, which is mandatory). But I tried with a method returning &Self and it does not work either. I was expecting it to work because a reference always has the same size. But then I thought that it might not be the case, as for instance &dyn MyTrait is twice as big (or something like that) as &i32 because the former also stores a pointer to a vtable.

But we could also argue that the compiler might know it. Indeed, in any vtable, the methods are de facto for trait objects only, so the compiler should know exactly what kind of smart pointer &Self is like (hence its size) right? Is this something the compiler could potentially do, but Rust has just not been made that way? Or am I missing something completely?

Example of what does not compile:

trait FooTrait {
    fn foo_fn(&self) -> &Self; // error: for a trait to be "object safe" it needs to allow
}                              // building a vtable to allow the call to be resolvable dynamically

struct FooStruct {
    x: Box<dyn FooTrait>,
}

Answer 1

This makes more sense when you have an implementation of the trait.

impl FooTrait for String {
    fn foo_fn(&self) -> &Self {
        self
    }
}

This is a function that returns &String. If you were able to create a trait object and call this function, you would then have &String.

let s = "hello".to_string();
let foo_trait: Box<dyn FooTrait> = Box::new(s);
let string_ref: &String = foo_trait.foo_fn();

Notice that this doesn't return &Box<dyn FooTrait> or &dyn FooTrait. The function is only capable of returning &String, so that's what it returns. This is the main issue, and it's also one reason why returning Self doesn't work.

The problem arises when you have another implementer of the trait.

impl FooTrait for u32 {
    fn foo_fn(&self) -> &Self {
        self
    }
}

let s = "hello".to_string();
let i = 5u32;
let foo_trait: Box<dyn FooTrait> = if rand::random() {
    Box::new(s)    
} else {
    Box::new(i)
};
let unknown = foo_trait.foo_fn();

Now you have a value with a runtime-dependent type, which is not allowed.

You may think, well that's fine as long as it's never used as anything other than FooTrait. But even that doesn't work.

Trait objects are callable because they consist of a pointer to the data and a pointer to a vtable of functions. Rust can compile this into instructions that say "call the function at this vtable pointer with this data pointer". But &String and &u32 are only pointers to data, with no pointer to a vtable, so Rust has no way of knowing whether to call String::foo_fn or u32::foo_fn at runtime.

And the final blow: references are not all the same size. References to sized types like String and u32 are one pointer, while references to unsized types like str and dyn FooTrait are two pointers.

impl FooTrait for str {
    fn foo_fn(&self) -> &Self {
        self
    }
}

So now, even the sizes of the return types are different.

---

the methods are de facto for trait objects only

I'm not totally sure about this, but I think the trait object vtable points to the same methods as the ones used statically. The static functions would often be inlined, but when they're not, I don't see a reason they couldn't be reused.

Answer 2

You are correct that it could work. Just like &Self in return type position, &self in receivers is also a fat pointer in the trait method while the concrete method expects a thin pointer. The compiler generates a shim that converts them. It could just as well convert the return type.

It just hasn't been implemented. Probably because it's not a common need.

But you can implement it yourself by returning &dyn Trait from the method.