Make sure in-memory state sync with persisted data in Microsoft Orleans

Question

As I understand, each grain have an in-memory state (this act as a cache so we don't have to read everytime we need to write). The problem is what happen if we can't write?

Take an example of a conference management software:

public class ConferenceGrain : Grain<ConferenceState>, IConferenceGrain
{
   public async Task<Reservation> BookReservation(string audienceId)
   {
      // make sure the number of audience is less than 10
      if(State.Reservations.Count() < 10)
      {
          // add reservation
          var reservation = new Reservation(audienceId);
          State.Reservations.Add(reservation);
          await WriteStateAsync();
          return reservation;
      }
      throw new InvalidOperationException("Conference is full, can't make reservation");
   }
}

Now, if our db having problem, then the reservation is not being saved. The audience thought that he/she successfully reserve the conference, but turnout it is not saved.

Few hours later, the grain get deactivated, the in-memory data get lost, so as the in-memory reservation.

Could someone show me a different way to think about this?

I just couldn't wrap my head around the idea of my data could be lost at any time, or worse: validating the next request on "could be lost" data.

Thank you.

Answer 1

After having conversation with @Reuben, we have two options:

• option 1: Being optimistic (if we can) that the state will be saved in the next call.
• option 2: try to reload the state of the grain if we can't save it

I'm leaning toward option 2, so this is the code:

siloBuilder.AddIncomingGrainCallFilter(async context =>
    {
        try {
            await context.Invoke();
        } catch {
            // reload the grain when something went wrong
            (context.Grain as IGrainBase).DeactivateOnIdle();
            throw;
        }
    });

The code above using IncomingGrainCallFilter to wrap all method invoke with a try-catch statement.

If anything went wrong, we just restart the grain, which reload the in-memory state.

So how can I throw a validation exception, just like the one in the question?

• option 1: create a custom exception and does not catch that one
• option 2: does not throw exception at all

Option 2 sounds weird but it actually make more sense to me. We could just create a new structure and return that structure instead of throwing error. We only throwing error when something actually went wrong, something not expected. Never throw exception only because of some validation logic.

Instead, I try the approach in this video: https://www.youtube.com/watch?v=a1ye9eGTB98

Answer 2

You must write state to the database for it to be saved. In Orleans, this is accomplished using the WriteStateAsync() method. You would await a call to WriteStateAsync() before returning the reservation.