How to write the constraints for scipy SLSQP properly

Question

I am trying to write an optimization code with constraints using SLSQP method. But I can't understand it enough to write the proper codes for constraints.

Lets say we have a function f, with p parameters, all with the same starting values (0.1) and arg is the array of these parameters:

arg = [.1] * p

Here I want sum(arg[2:]) to be less than 1, and arg[i] > 0 for the optimization. How can i write the constraints/boundaries for scipy.optimize.minimize:

from scipy.optimize import minimize

minimize( fun = f,
          x0  = arg,
          method = 'SLSQP',
          constraints = ?,
          bounds = ?
)

Answer 1

First of all, you can't use strict inequality constraints like g(x) < 1 or x > 0. Instead, you can model them by g(x) ≥ 1 - ε and x ≤ ε, where ε is a sufficiently small positive constant. In addition, if args is your starting point, i.e. the vector consisting of 1s only, then there's no constraint as sum(c) doesn't depend on your any optimization variable. I take it you'd like to add the constraint that the sum of all the optimization variables (except the first two ones) is smaller than 1:

Note that scipy expects each equality constraint in the form g(x) = 0 and each inequality constraint in the form g(x) ≥ 0. Therefore, you can define your first constraint by

def con_fun(x, eps = 1.0e-8):
    # np.sum(x[2:]) ≤ 1 - eps
    # (np.sum(x[2:]) - 1 + eps ≤ 0
    # 1 - eps - np.sum(x[2:]) ≥ 0
    return 1 - eps - np.sum(x[2:])

constrs = [{'type': 'ineq', 'fun': con_fun})

Your second constraints are simple bounds on the optimization variables, thus

eps = 1.0e-8
# 0 + eps <= x[i] < ∞
bounds = [(0 + eps, None) for _ in range(p)]

Finally, you can can call the minimize method:

minimize(your_obj_fun, x0=x0, method='SLSQP', constraints=constrs, bounds=bounds)