Temporal Database with Django Rest Framework using perform_update

Question

We are trying to implement a temporal database so that we are able to track changes made

All our models have the following fields

vt = models.DateTimeField(db_column='VT', default=datetime(3000, 12, 31, 23, 00, 00, 000000))  # Value To
vflag = models.IntegerField(db_column='VFlag', default=1)  # Version Flag 1 = Current, 0 = Old   

When using the Django rest framework I’ve tried to modify the perform_update in my viewset to duplicate the existing record, make the updates and then set temporal fields appropriately.

It works when I have 1 record and the first update

However once I try and make a second update it fails and create a duplicate of the changes and overrides the very first record.

Original Record

Currency = AUD, VFlag = 1, VT = time1

Perform update - success

Currency = USD, VFlag = 1, VT = time2

Currency = AUD, VFlag = 0, VT = time1

Next perform update currently produces - fails

Currency = GBP, VFlag = 1, VT = time3

Currency = GBP, VFlag = 1, VT = time3

Currency = USD , VFlag = 0, VF = time2

Expected update output

Currency = GBP, VFlag = 1, VT = time3

Currency = USD, VFlag = 0, VT = time2

Currency = AUD, VFlag = 0, VT = time1

Is a temporal database a possibility in django rest?

Is anybody able to please point in the right direction

Below is my view set code

class OptionMasterViewSet(viewsets.ModelViewSet):
serializer_class = OptionMasterSerializer
paginate_by = None
queryset = OptionMaster.objects.filter(vflag=1)
# queryset = OptionMaster.objects.all()

def get_queryset(self):
    queryset = OptionMaster.objects.filter(vflag=1)
    contractcode = self.request.query_params.get('contractcode', None)
    if contractcode is not None:
        queryset = queryset.filter(contractcode=contractcode, vflag=1)
    return queryset

def perform_update(self, serializer):

    changes = serializer.validated_data
    original_object = serializer.instance

    vt = datetime.now()

    changes['vf'] = vt

    #Build the old record
    old_record = {}
    for field in original_object._meta.get_fields():           
        old_record[field.name] = getattr(original_object, field.name)

    old_record['vflag'] = 0                
    old_record['vt'] = vt

    old_record = OrderedDict(old_record)

    #Save the new rrecord
    serializer.save()

    #Create the old record
    obj = OptionMaster.objects.create(**old_record)

    return serializer

my serialize

class OptionMasterSerializer(TemporalModelSerializer):

class Meta:
    model = OptionMaster
    fields = '__all__'

potential temporal serializer

class TemporalHyperlinkedModelSerializer(serializers.HyperlinkedModelSerializer):
vt = serializers.HiddenField(default=datetime(3000, 12, 31, 00, 00, 00, 000000))
vflag = serializers.HiddenField(default=1)

class TemporalModelSerializer(serializers.ModelSerializer):
vt = serializers.HiddenField(default=datetime(3000, 12, 31, 23, 00, 00, 000000))
vflag = serializers.HiddenField(default=1)

class TemporalModel(models.Model):
vt = models.DateTimeField(db_column='VT')  # Field name made lowercase.
vflag = models.IntegerField(db_column='VFlag')  # Field name made lowercase..
class Meta:
        abstract = True

Answer 1

The solution to my problem was to FILTER and UPDATE using a dictionary data.

 self.Meta.model.objects.filter(pk=instance.pk, vflag=1).update(**new_record)

This was my final working TemporalModelSerializer below

    class TemporalModelSerializer(serializers.ModelSerializer):
        vf = serializers.HiddenField(default=timezone.now())
        vt = serializers.HiddenField(default=datetime(3000, 12, 31, 23, 00, 00, 000000))
        vflag = serializers.HiddenField(default=1)
        vu = serializers.HiddenField(default='Theodore')

        class Meta:
            model = None
            fields = '__all__'

        def update(self, instance, validated_data):

            time_now = timezone.now()

            old_record = {}
            new_record = {}

            for (key, value) in validated_data.items():
                old_record[key] = getattr(instance, key)
                new_record[key] = validated_data[key]
                setattr(instance, key, value)

            old_record['vt'] = time_now
            old_record['vflag'] = 0
            new_record['vf'] = time_now

            self.delete_me(old_record)

            self.Meta.model.objects.filter(pk=instance.pk, vflag=1).update(**new_record)

            return instance

        def delete_me(self, old_record):
            obj = self.Meta.model.objects.create(**old_record)

Answer 2

Tbh, I have never dealt with such a problem before, but this is somewhat more of database design question than a django-rest one. Hence, without knowing a little bit more about the purpose and relation of these models, I can merely make an estimated guess (as follows).

An intuitive approach of mine would be to implement an additional IntegerField called something along the lines of series_group_id, which would serve as an identifier of grouped time series and a created_add DateTimeField the auto_now_add flag set to True. That way, this field gets populated with the datetime value at the moment of creation.

class SomeModel(models.Model):
    # some other stuff like vflag and vt
    goup_id = models.IntegerField()
    created_at = models.DateTimeField(auto_now_add=True)

What's more, if you are keeping track of a time series, you are technically not updating your instances but creating new ones. Hence, my approach would be to move your entire logic to the create method. In fact, a ModelSerializer provides this kind of behaviour out of the box. The only thing you need to do is add the group_id to your serializer's fields tuple. As you are using 'all', the field will automagically appear in your fields as soon as you add it to your model.

In case you need more granular control, you can override the create method like so:

class OptionMasterViewSet(viewsets.ModelViewSet):
    serializer_class = OptionMasterSerializer
    paginate_by = None
    queryset = OptionMaster.objects.filter(vflag=1)


    def get_queryset(self):
        queryset = OptionMaster.objects.filter(vflag=1)
        contractcode = self.request.query_params.get('contractcode', None)
        if contractcode is not None:
            queryset = queryset.filter(contractcode=contractcode, vflag=1)
        return queryset

    def create(self, request):
        data = request.get('data', None)
        serializer = self.serializer_class(data=data)
        serializer.is_valid(raise_exception=True)

        # some custom logic here in case you need it..
        self.perform_create(serializer)
        return Response({'detail': 'ok'}, status=status.HTTP_201_CREATED)

Limitations

This approach expects that the arriving data comprises the group_id of the corresponding time series you are currently tracking.