Django problems doing equivalent of SQL 'GROUP BY' and then serializing. Error: 'dict' object has no attribute 'mymodelfield' in .values() aggregation

Question

I am trying to do aggregation of a model (Reply) by geographic area (Hexgrid_10km2). Similar to SQL 'group by'. I am using .values(Hexgrid_10km) to do this. I am then annotating some numbers onto this. I am getting the following error in the serialization to geojson:

'dict' object has no attribute 'hexgrid_10km2'

But Reply does a have field called 'hexgrid_10km2'.

I can't figure this out. I've tried other ways to serialize on stackoverflow, but always seem to get an error. Does anybody know I am doing wrong? Many thanks!!

I am using djangorestframework-gis

Models

class Hexgrid_10km2(models.Model):
    lng = models.FloatField()
    lat = models.FloatField()
    polygon = models.MultiPolygonField(srid=4326)
    centroid = models.PointField(default=Point(0,0), srid=4326)

    def __str__(self):
        return f'lng: { self.lng } lat: {self.lat }'


class Animal (models.Model):
    name = models.CharField(max_length=200)


class Reply(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    reply_date = models.DateTimeField()
    animal = models.ForeignKey(Animal, on_delete=models.CASCADE)
    ability = models.IntegerField(default = 0)
    hexgrid_10km2 = models.ForeignKey(Hexgrid_10km2, on_delete=models.CASCADE, null=True, blank=True)

    @property
    def polygon_10km2(self):
        return self.hexgrid_10km2.polygon

Views

from rest_framework.views import APIView
from rest_framework.response import Response
from.models import Animal, Reply, Hexgrid_10km2
from django.db.models import Avg, Sum

class ReplyHeatmapAPIView(APIView):
    def get(self, request, pk):
        animal = get_object_or_404(Animal, pk=pk)
        relevant = Reply.objects.filter(animal=animal)
        most_recent = relevant.filter(
            reply_date=Subquery(
                (Reply.objects
                    .filter(user=OuterRef('user'))
                    .values('user')
                    .annotate(most_recent=Max('reply_date'))
                    .values('reply_date')[:1]
                )
            )
        final = most_recent.values('hexgrid_10km2')
            .annotate(average_ability=Avg('ability'), sum_ability=Sum('ability'))
        serializer = ReplyHeatmapSerializer(final, many=True)
        return Response(serializer.data)

Serializer

from rest_framework_gis.fields import GeometrySerializerMethodField
from rest_framework_gis.serializers import GeoFeatureModelSerializer

class ReplyHeatmapSerializer(GeoFeatureModelSerializer):
    """ A class to serialize hex polygons as GeoJSON compatible data """

    average_ability = serializers.FloatField()
    sum_ability = serializers.FloatField()

    polygon = GeometrySerializerMethodField()
    def get_polygon(self, obj):
        return obj.hexgrid_10km2.polygon

    class Meta:
        model = Reply
        geo_field = 'polygon'
        depth = 1
        id_field = False
        fields = ('id', 'average_abilty', 'sum_ability')

Traceback

Traceback (most recent call last):
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
    response = get_response(request)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\django\views\decorators\csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\django\views\generic\base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\views.py", line 476, in raise_uncaught_exception
    raise exc
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\karate2\animals\api\views.py", line 109, in get
    return Response(serializer.data)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework_gis\serializers.py", line 20, in data
    return super(ListSerializer, self).data
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\serializers.py", line 260, in data
    self._data = self.to_representation(self.instance)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework_gis\serializers.py", line 28, in to_representation
    ("features", super(GeoFeatureModelListSerializer, self).to_representation(data))
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\serializers.py", line 677, in to_representation
    return [
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\serializers.py", line 678, in <listcomp>
    self.child.to_representation(item) for item in iterable
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework_gis\serializers.py", line 116, in to_representation
    feature["geometry"] = field.to_representation(geo_value)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework_gis\fields.py", line 101, in to_representation
    value = super(GeometrySerializerMethodField, self).to_representation(value)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\venv\lib\site-packages\rest_framework\fields.py", line 1905, in to_representation
    return method(value)
  File "C:\Users\anton\OneDrive\Documents\django\karate-project-2.2\karate2\animals\api\serializers.py", line 61, in get_polygon
    return obj.hexgrid_10km2.polygon
AttributeError: 'dict' object has no attribute 'hexgrid_10km2'

UPDATE Amended code. The reason I am filtering by Replys and not Hexgrid is because Users can make multiple Replys for the same Animal. So I need to filter the most recent reply that a user made for a particular Animal. I understand that .values() is not made for grouping by, but how else can I do it? I thought about grouping by hex grid but failed as even making the query to get the first Reply per user took me ages to figure out.

Answer 1

The traceback is already telling you the problem: you are trying to access an attribute of a dictionary. The .values() method creates a dictionary with the values of the fields of an instance in the queryset, and what you're passing to the serializer is a list of all these dictionaries of values instead of a normal queryset, so when it tries to access a field of an instance of the queryset, it raises an exception because dictionaries have no attributes. You can't pass a dictionary to the serializer, it must receive a queryset.

To solve this, you have to erase .values() from the view, and you can just use polygon_10km2 as a field in the serializer, since it is a property of the model, to obtain what you want.