Plotting log likelihood as a function of theta1 and theta2

Question

I am asked to plot the log-likelihood for a logistic regression model as a function of theta1 and theta2 to assess if its a convex function (the model only has two parameters and thetas are the weights of each parameter). My understanding is that I need a cartesian graph with each axis denoting the values of the thetas. The negative log-likelihood is given as (in code):

sigma = sigmoid(np.dot(x, weight))
loss = -1/size * np.sum(y * np.log(sigma)) + (1 - y) * np.log(1-sigma)

How can I plot this function as a function of theta1 and theta2 in Python?

Answer 1

As you stated, you can visualize the loss landscape by plotting the log-likelihood as a function of the model parameters. The usual way to visualize this is to create a contour plot. To get a good solution, you'll want to make sure you have a good number of samples, and that your weights are optimal w.r.t. the negative log-likelihood. See below for some pseudo-code.

# Use numpy and matplotlib.
import numpy as np
import matplotlib.pyplot as plt

# The size of the neighborhood around the point to visualize.
nbh = 0.25

# Create a mesh grid of the parameters within the specified neighborhood.
w1_space = np.linspace(weight[0]-nbh, weight[0]+nbh, 100)
w2_space = np.linspace(weight[1]-nbh, weight[1]+nbh, 100)
w1, w2 = np.meshgrid(w1_space, w2_space)

# Assuming loss is a function of the weights, 
# compute the negative log-likehood for points within the given subspace.
nlls = np.array(list(map(loss, zip(w1.flatten(), w2.flatten())))).reshape(w1.shape)

# Plot the result using matplotlib.
plt.figure()
plt.contourf(w1, w2, nlls)
plt.show()