How to profile django application with respect to execution time?

Question

My Django application is insanely slow, I want to figure out what is taking time :

I tried Django-debug-toolbar but was unable to find a panel that can give me the break-up of the load time.

My requirements:

• A stack-trace type output with time of execution for each module called to render the page.
• I want to realize what part of the whole page rendering process is taking the time ?
• Also, what part is consuming how much CPU [ MOST IMPORTANT ] ?

Can django-debug-toolbar do that ? [ What panel ? ]

Any other django-app that can do that ?

Answer 1

Finally figured out a way to profile my django webapp :

Following 2 django snippets provide middleware that profile the whole flow and outputs if request has prof in GET keys :

• http://djangosnippets.org/snippets/727/ [ Uses cProfile ]

• http://djangosnippets.org/snippets/186/ [ Uses hotshot ]

Plain and simple profiling - Saved my day !

Answer 2

I would recommend writing some integration tests instead, or at least using the built in testing client to automate requests and put lots of debugging statements in the views

Django has a built in testing client:

from django.test.client import Client
c = Client()
response = c.post('/slow_url/')

And then in your view:

def slow_url(request):
    start = time.time()
    print 'Started db query'
    result = SomeComplexModel.objects.all()
    print 'Finished db query, took ', time.time() - start
    return render('some_complex_template.html', {'result': result})  

Automating the process of making requests and being able to replicate them again and again while you make small changes is how you will improve your code. CPU time can be worked out if you measure the time it takes to run each function. It won't take you long to hone in on the part that is actually chewing up resources.

Answer 3

django-silk can help.

• Install it with: pip install django-silk
• Add it in settings.py:

MIDDLEWARE = [
    ...
    'silk.middleware.SilkyMiddleware',
    ...
]

INSTALLED_APPS = (
    ...
    'silk'
)

• Add the route in urls.py:

urlpatterns += [url(r'^silk/', include('silk.urls', namespace='silk'))]

• Finally, create the adequate tables in the database with:

python manage.py makemigrations
python manage.py migrate

You can then browse in your internet browser to /silk to find the page requested there.

Answer 4

It's not profiling , but i generally simply use a view to calculate the execution time , it works also for views that need user login, it displays execution time in a simple page

 def test(request):
     from django.test.client import Client
     import time

     c = Client()

     #client login if needed
     response = c.post('/login/', {'username': 'admin', 'password': 'password'})

     start = time.time()
     response = c.get('/pagetotest/')

     #print response
     #print 'Finished, time: ', time.time() - start # output to console
     end=time.time() - start
     return render(request,'loadingtime.html',{'time':end})

it's a good start i think , hope it will help someone

Answer 5

django-debug-toolbar 2.0

By default, django-debug-toolbar 2.0 includes 'debug_toolbar.panels.profiling.ProfilingPanel' in the settings DEBUG_TOOLBAR_PANELS. You can view this profiling information by ticking the "Profiling" checkbox in the toolbar and refreshing the page.

Old versions of django-debug-toolbar:

You can try the profiling panel of the django-debug-toolbar (make sure you use the application's latest version from github). Enable the panel like so in your settings.py:

DEBUG_TOOLBAR_PANELS = (
  'debug_toolbar.panels.version.VersionDebugPanel',
  'debug_toolbar.panels.timer.TimerDebugPanel',
  'debug_toolbar.panels.profiling.ProfilingDebugPanel',
)

This existence of this panel is not documented on the readme of django-debug-toolbar; that's why I answer here in the first place.