Measure time of numeric plot vs symbolic plot

Question

I am using Spyder (Python 3.9) and I am writing some scripts to show the importance of coding efficiently, by comparing the execution time of different lines of code that do the same thing with different ways.

Suppose we have some symbolic function whose graph we want to plot. We could use scipy.plot directly, or we could lambdify the function to obtain a numpy function f, generate an array x and use plot(x,f(x)) from matplotlib.pyplot. I'd like to compare the two options.

I investigated the proper ways to measure execution time in Python and I found the timeit module, which I used for other comparisons. Seems to me that is better than time.perf_counter() since it executes the code several times and gives more reliable results.

When I use it, seems that the symbolic is way faster, but I think this is not really the case. Here is my attemp at measuring the two methods. When I use time.perf_counter I obtain that the numeric method can be way quickier. So what did I do wrong with timeit?

import timeit

setup = '''
import numpy as np
import matplotlib.pyplot as plt
import sympy as sy

t = sy.Symbol('t')
expression = sy.exp(t)'''


code_sym = '''
sy.plot(expression, xlim = [0,5], ylim = [0,150], show = False) 
'''


code_num = '''
f= sy.lambdify(t, expression, 'numpy')
x = np.linspace(0,5,100)
plt.plot(x,f(x))
'''

num_exec = 10

time1 = timeit.timeit(stmt=code_sym, setup = setup, number = num_exec)
print('Time w/ symbolic:' , time1, ' sec')


time2 = timeit.timeit(stmt = code_num, setup = setup, number = num_exec)
print('Time w/ numeric:' , time2, ' sec')

Answer 1

There may be an issue with "warm-up" time that results in apparently longer time for the numeric approach. To ensure it does not weight on the results, you should check if the average time relation between the two approaches changes significantly when you increase the number of iterations.

To check that I added the following line at the end of your code:

print('Ratio:', time2/time1)

For 10 iterations I got:

Time w/ symbolic: 0.0009346000000001187  sec
Time w/ numeric: 0.1785722999999999  sec
Ratio: 191.06815750051064

For 100:

Time w/ symbolic: 0.00569429999999993  sec
Time w/ numeric: 0.2562344999999999  sec
Ratio: 44.998419472104224

For 1000:

Time w/ symbolic: 0.055028999999999995  sec
Time w/ numeric: 1.2229339999999997  sec
Ratio: 22.223445819477

For 10000:

Time w/ symbolic: 0.5119651000000001  sec
Time w/ numeric: 8.5127048  sec
Ratio: 16.627509961128208

As you can see the ratios between the timings are declining with the growing number of iterations (rounded: 191, 45, 22, 16) which confirms the suggestion that the warm-up code eats up a significant portion of time and the potential (we do not know if the eventual results would confirm it) advantage of numerical vs. symbolic approach would justify using the numeric even for very large numbers of iterations.

This is why it is important to measure. Our intuitions can be grossly wrong.

“It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. “ – Mark Twain