How to set memento in memento pattern

Question

I need to write a Graph class which performs some sort of operations and gives possibility to save it using memento pattern. I am trying to implement that functionality, but I want to have IGraph interface which cannot return explicit type of ConcreteMemento so it returns Memento interface, but because of that I need to downcast it inside Graph class.

So my question is, is it a good practise or am I missing something in memento pattern? Also is it possible to implement ConcreteMemento in the way that only Graph can access some of its methods?

Grahp

class ConcreteMemento : Memento
{

}
internal class Graph : IGraph
{
    public Memento GetMemento()
    {
        return new ConcreteMemento();
    }

    public void SetMemento(Memento m)
    {
        var concreteMemento = m as ConcreteMemento;
    }
}

IGraph

public interface Memento
{

}
interface IGraph
{
    Memento GetMemento();
}