Optimizing pagination in function-based views (in Django)

Question

In a class-based Django listview, when one allots a value to paginate_by, it ensures that the object_list available in get_context_data method is limited to only those objects that are needed for that particular page.

E.g. get_queryset might return 1000 objects, but if a single page is to show 20 objects, then only 20 objects are available as object_list in get_context_data. This is a good optimization when dealing with large querysets.

How can one create this same behavior in function-based views? Under typical implementation (and taking the aforementioned example), all 1000 objects would be evaluated and then used in pagination. That means that in this particular case, CBVs are definitively better than function-based views. Can function-based views provide the same functionality? An illustrative example would be great (or a 'no they can't' kind of answer works too).

---

Here's the actual CBV:

class PostListView(ListView):
    model = Post
    paginate_by = 20
    template_name = "post_list.html"

    def get_queryset(self):
        return all_posts()

    def get_context_data(self, **kwargs):
        context = super(PostListView, self).get_context_data(**kwargs)
        posts = retrieve_posts(context["object_list"]) #retrieves 20 posts
        return context

Here's the actual FBV:

def post_list(request, *args, **kwargs):
    form = PostListForm()
    context = {}
    obj_list = all_posts()
    posts = retrieve_posts(obj_list) #retrieves 1000 posts
    paginator = Paginator(posts, 20)
    page = request.GET.get('page', '1')
    try:
        page = paginator.page(page)
    except PageNotAnInteger:
        page = paginator.page(1)
    except EmptyPage:
        page = paginator.page(paginator.num_pages)

the retrieve_posts() function:

def retrieve_posts(post_id_list):
    my_server = redis.Redis(connection_pool=POOL)
    posts = []
    pipeline1 = my_server.pipeline()
    for post_id in post_id_list:
        hash_name="pt:"+str(post_id)
        pipeline1.hgetall(hash_name)
    result1 = pipeline1.execute()
    count = 0
    for hash_obj in result1:
        posts.append(hash_obj)
        count += 1
    return posts

the all_posts() function:

def all_posts():
    my_server = redis.Redis(connection_pool=POOL)
    return my_server.lrange("posts:1000", 0, -1) #list of ids of 1000 posts

---

Difference in response times of the two approaches (via newrelic):

The blue part is processing of the view.

Answer 1

Don't call retreive_posts with the original object list. Use the page's object list after paginating.

posts = retrieve_posts(page.obj_list)