skopt: How to dynamically change bounds during optimization?

Question

I have just started using skopt so please feel free to redirect me to any basic tutorial that I might have missed. Anyway, here it goes:

I have an optimization problem where I calculate a 2-peaks spectrum via a complex physical model and then extract its Lorentzian profile (i.e. 6 parameters, 3 per peak). I then set up a cost function that calculates the squared difference between the calculated parameters and the experimental one, such that I get an f(x) that takes an array and returns a scalar (if I understood correctly that is what gp_minimize requires).

I set up the constraints of the problem using the following:

dim1=Real(name="A1", low=1, high=100) dim2=Real(name="A2", low=1, high=200)

dimensions = [dim1, dim2,...] but in my particular system A2 is bound by 2*A1. Is there a way to use this in constraints like shown above, to avoid searching a lot of "unphysical" parameter space? In my particular case evaluating the model is very time consuming so avoiding unnecessary calculations would be very helpful :)

Thanks in advance for your help!

Best, Chris

Answer 1

As of now, it does not seem possible to specify this type of dependence between dimensions of the search space. However, you could incorporate the constraint a2 <= 2*a1 into the objective passed to the optimizer.

In the example below, the "true" (and expensive) objective f is only evaluated if the constraint is met. Otherwise a high value is returned immediately.

In the figure everything to the left of the dashed line violates a2 <= 2*a1. The red cross is the true minimum of the target f and the black dots are the objective evaluations. Due to the high objective values to the left of the line, the search focus is on the valid part of the search space. Of course there might also be evaluations in the invalid sub space (during initial random sampling or when "exploring" instead of "exploiting"), but those will not involve evaluation of the expensive target f.

import numpy as np
from matplotlib import pyplot as plt

from skopt import gp_minimize
from skopt.space import Real
from skopt.utils import use_named_args

# search space
dimensions = [Real(name='a1', low=1, high=10),
              Real(name='a2', low=1, high=10)]


# expensive target function to minimize
def f(a1, a2):
    x1 = a1 - 4.5
    x2 = a2 - 7
    return 1.5*x1**2 + x2**2 - 0.9*x1*x2


# objective function passed to optimizer incorporates constraint
@use_named_args(dimensions)
def objective(a1, a2):

    # if constraint violated quickly return a high value ...
    if a2 > 2*a1:
        return 1e4

    # ... otherwise expensive target can be evaluated
    else:
        return f(a1, a2)


# run optimization
res = gp_minimize(objective, dimensions, n_calls=50, n_initial_points=20, random_state=92)

# evaluate f on regular grid for plotting
a1_grid = np.linspace(0, 10, 50)
a2_grid = np.linspace(0, 10, 50)
f_grid = np.array([f(i, j) for j in a2_grid for i in a1_grid]).reshape(
    len(a1_grid), len(a2_grid))

# visualize results
fig, ax = plt.subplots()
ax.set_xlabel('a1')
ax.set_ylabel('a2')

# contours of f
ax.contourf(a1_grid, a2_grid, f_grid, 20)

# true minimum
ax.plot(4.5, 7, 'rx')

# constraint
ax.plot([0, 5], [0, 10], 'k--')

# evaluations
for x in res.x_iters:
    ax.plot(x[0], x[1], 'k.')