Scipy Differential Evolution - Extraction of percentage of invalid population energies (via callback?)

Question

I am currently working on an optimization problem and would like to extract the information of the fraction of invalid population energies from the package during each iteration step of the optimization.

Currently I am using a callback to get the best solution and the convergence:

    def __call__(self, xk: NDArray[float64], convergence: float) -> bool:

Is it possible to get the population energies via the callback?

I'm working with scipy 1.12.0 in python 3.11.2.

I already altered the dependency (scipy differential evolution) to print the fraction, but this is not a solution I would like to keep long term).

Answer 1

As os scipy 1.12.0 callbacks can contain intermediate_result: OptimizeResult, which is a dict structure, that can contain different parameters, depending on the scipy optimization it is returned from.

The contents of the OptimizeResult weren't clear from scipys documentation.

For the differential_evolution these contain additionally to the parameters named in the documentation: population, population_energies, constr, constr_violation and convergence.

I needed to replace xk and convergence in the callback with intermediate_result: OptimizeResult. The population_energies, x and the convergence can then be accessed from there.

def __call__(self, intermediate_result: OptimizeResult) -> bool:

The fraction of invalid population energies can be calculated by:

sum(np.isinf(intermediate_result.population_energies))/len(intermediate_result.population_energies)