How to handle possible race-condition in overwritten Document.save method - Mongoengine

Question

I am using flask-mongoengine and think I am running in some kind of race conditions while trying to overwrite the Document.save method.

My models (simplified) look like this:

class User(Document):
    meta = {"collection": "users"}
    name = StringField()


class Group(Document):
    meta = {"collection": "groups"}
    name = StringField()


class History(EmbeddedDocument):
    key = StringField()
    oldValue = StringField()
    newValue = StringField()


class Asset(DynamicDocument):
    meta = {"collection": "assets"}
    c_id = SequenceField()
    name = StringField()
    history = ListField(EmbeddedDocumentField(History))
    user = ReferenceField('User')
    group = ReferenceField('Group', required=True, default=Group.objects.first())

    def save(self, **kwargs):
        for key, value in self._data.items():
            history_update = History(
                key=key,
                oldValue="",
                newValue=str(value)
            )
            self.history.append(history_update)
        return super(Asset, self).save(**kwargs)

What I am trying to achieve is:

When a new Document of type Asset is created, add an entry of type History for each Key/Value pair of the document that changed. (Here from None to some value, I have similar code in the update method for changes on existing assets). This history list should be something like a changelog of the particular asset through its lifetime.

My problem with the current implementation is that:

1. c_id of type SequenceField is None in my for-loop.
2. str(value) for the User object gives me the correct user-object (or the result of my custom __str__ method) but str(value) for the Group object gives me DBRef('groups', '<mongoidstring>') and does not trigger my customer str method
3. When debugging with a breakpoint beforehand, these two errors do not occur. c_id has its correct value and my group object is a group object and not a DBRef object

I've tried saving the Document once before and then adding my history which at least gives me a correct c_id but the group is still a DBRef.

I do think the SequenceField is populated in parallel and therefore still None when I try to access it but not when I come through the debugger. But the DBRef still gives me headaches. And that I don't really see a way to properly implement my ChangeHistory through overwriting the save method. Any ideas how to properly handle this?

Answer 1

So I find an answer myself (somewhat).

1. SequenceFields are only populated during a save(). When overwritting the save method we first have to make a super.save to get the SequenceField value or we have to assume its value by the helper collection that is created by mongoengine. I took the easy route and just added an super(Asset, self).save() and the c_id is set corectly for the changes afterwards.
2. ReferenceFields are avalaible as DBRef until you first access it from the Document object. Just add some kind of check beforehand to ensure its value is correctly resolved like:

    assert self.group is not None
    assert self.user is not None