Constraining conditional summation in Bounded Optimization in Python

Question

I am trying to minimize a the sum of least squares subject to certain constraints. For the most part, this problem seems fairly straightforward except for one constraint. I'm having a difficult time figuring out a way of modeling the constraint that the sum of all values greater than a given number cannot exceed a constant value.

I currently have a working solution that iterates through the possible combinations and picks the best solution with scipy's minimize function. However, this solution is not the most efficient in terms of time (or computing power). Is there another way of approaching this problem that makes it possible to solve in a single pass?

I've been looking at Gekko as it makes the general code much more readable, but once I add an equation that uses a summation, the efficiency is worse than looping.

Python (scipy) code:

import time
import numpy as np
from scipy.optimize import minimize

def _func(self, w, w_bar):
        """
        Objective function - minimize the difference of squares between
        w_bar and w
        """
        error = np.sum([(w_bar[i] - w[i]) ** 2 for i in range(len(w))])
        return error


def _func_deriv(self, w, w_bar):
    """
    Derivative of the objective function
    """
    gradiant = [-2 * (w_bar[i] - w[i]) for i in range(len(w))]
    return gradiant

display_solver_output = True
epsilon = 1E-13
tolerance = 1E-16

w_bar = <List of decimal values>

optimal_solution = []
optimal_tracking_error = float("inf")

total_time = 0.0

default_bounds = [(0, 0.3) for i in range(len(w_bar))]

initial_guess = [min(w_bar[i], 0.3) for i in range(len(w_bar))]

max_iterations = int(0.6 / 0.06) + 1
for j in range(max_iterations):
    constraints = (\
        {# Equality Constraint: where the equality is of the form C_j(x) = 0
            'type': 'eq',
            'fun' : lambda w: np.sum(w) - 1
        },
        {# Inequality Constraint: where the inequality is of the form C_j(x) >= 0
            'type': 'ineq',
            'fun' : lambda w: -1 * np.sum(w[-j:]) + 0.6
        })

    # create bounds from the default bounds
    bounds = default_bounds[:]
    # set upper bounds for all 'small' w
    for i in range(len(w_bar) - j):
        bounds[i] = (bounds[i][0], 0.06)

    try:

        start = time.time()

        res = minimize(_func,
                        initial_guess, # Initial Guess/Seed
                        args=w_bar,
                        jac=_func_deriv,
                        bounds=bounds,
                        constraints=constraints,
                        tol=tolerance,
                        options=\
                        {
                            #'maxiter': 100,
                            'disp': display_solver_output
                        })

        end = time.time()

        total_time += end - start

        if display_solver_output:
            print(f"Solution Time: {end-start} s")
            print(f"Total Time: {total_time} s\n")

        if res.success:
            # check to see if the result is within our acceptable range
            # (expected_value +- epsilon)
            passes_large = sum([wl for wl in res.x if wl > 0.06]) <= (0.6 + epsilon)
            passes_sum_weights_one = abs(sum(res.x) - 1) <= epsilon

            # current solution is best solution if it is better than
            # previous best solution
            if (passes_large and passes_sum_weights_one and res.fun < optimal_tracking_error):
                optimal_tracking_error = res.fun
                optimal_solution = res.x

    except Exception as err:
        print(err)

Gekko code:

import numpy as np
from gekko import GEKKO

w_bar = <List of decimal values>

model = GEKKO(remote=False)

w = model.Array(model.Var, len(w_bar), lb=0, ub=0.3)

# Sum of all values = 1
model.Equation(model.sum(w) == 1)

# Sum of all values greater than 6% cannot exceed 60%
# This is the part that degrades the speed greatly
model.Equation(model.sum([model.if3(w[i] - 0.06, 0, w[i]) for i in range(len(w))]) <= 0.6)

model.Minimize(model.sum([(w_bar[i] - w[i]) ** 2 for i in range(len(w_bar))]))

model.solve()

With scipy, I created a loop where I explicitly indicate if the value is subject to the constraint. It works but it is slow. For large inputs (300+), it takes about 10 sec.

In Gekko, I added an equation to constrain but this degrades the performance greatly. For small amount of input, it run relatively quickly (under 1 sec), but once there are more than 300 inputs the time to solve increases to about 90 seconds.

Am I thinking about the equations properly? Do I need to reshape this problem somehow?

Data set 1 - w_bar: (Works well)

0.0000290098720682487343868705,
0.0000723827066008387518299476,
0.0002072744176096173915822705,
0.0003672809435335959934840107,
0.0003752484336042253846688711,
0.0004405565779129329566220824,
0.0005525596498762783855459046,
0.0006728198725774592129480554,
0.0008351912653287541643826696,
0.0009825084042287753464578806,
0.0009948648777755642828519691,
0.0012517612267766885632838162,
0.0013546692913343326698875246,
0.0015307554230129220783373226,
0.0018079051888007421049402691,
0.0018273451279460959651328143,
0.0019106018091149384691427602,
0.0019397417543016734988324706,
0.0021105212741847910173045113,
0.0025270382452472284845495592,
0.0026030268201749619505495437,
0.0026381414030341878249837264,
0.0035891127645570676979108904,
0.0042890768176753193003852561,
0.0054266187080476129436017240,
0.0072342421507922666121696771,
0.0141269512168524993586417500,
0.0277835377270709998751732016,
0.0358112813055125623355637036,
0.4222060446840235199334703797,
0.4525019300404232987113785672

Data set 1 - Expected Results

0.00029009872068244083,
0.0007238270660083144,
0.002072744176096174,
0.003672809435335925,
0.003752484336042254,
0.004405565779129317,
0.0055255964987627705,
0.006728198725774593,
0.008351912653287517,
0.009825084042287735,
0.009948648777755588,
0.012517612267766834,
0.013546692913343296,
0.014123432102475512,
0.014400581868263323,
0.01442002180740866,
0.014503278488577536,
0.01453241843376423,
0.014703197953647362,
0.015119714924709763,
0.0151957034996375,
0.015230818082496761,
0.016181789444019656,
0.016881753497137882,
0.01801929538751016,
0.019826918830254826,
0.02671962789631506,
0.04037621440653355,
0.04840395798497512,
0.3,
0.3

Data set 2 - w_bar: (does not work well)

0.000007815875,
0.000008941521,
0.000009627394,
0.000010762907,
0.000011539940,
0.000011822949,
0.000012052014,
0.000012526334,
0.000013414645,
0.000013932927,
0.000013955431,
0.000015010769,
0.000015721501,
0.000015847039,
0.000017499261,
0.000017740886,
0.000022409018,
0.000023446383,
0.000023769321,
0.000025218211,
0.000026432503,
0.000029219318,
0.000030326871,
0.000030598632,
0.000030767053,
0.000032463081,
0.000034340788,
0.000039307435,
0.000041047637,
0.000042717492,
0.000043676212,
0.000044110066,
0.000044188176,
0.000044540981,
0.000045730008,
0.000047201637,
0.000048350816,
0.000049262536,
0.000052023657,
0.000053054453,
0.000056020848,
0.000058046508,
0.000063947874,
0.000067167499,
0.000069187263,
0.000071131270,
0.000071613326,
0.000072761127,
0.000073458337,
0.000074730372,
0.000079639189,
0.000084199573,
0.000084603561,
0.000089676063,
0.000090002613,
0.000090250728,
0.000090958865,
0.000091780483,
0.000094820916,
0.000096386941,
0.000097951562,
0.000098087729,
0.000098285781,
0.000099286808,
0.000100880981,
0.000112384492,
0.000113020079,
0.000119654507,
0.000119844927,
0.000121917556,
0.000125752295,
0.000127115137,
0.000128301202,
0.000128434254,
0.000131136836,
0.000136985654,
0.000137451057,
0.000137892405,
0.000138478238,
0.000138510178,
0.000140286393,
0.000141644129,
0.000143025370,
0.000145684057,
0.000146128282,
0.000149165998,
0.000152928119,
0.000158011782,
0.000163334650,
0.000163993685,
0.000165146865,
0.000165346542,
0.000165727164,
0.000166763187,
0.000167570306,
0.000167701282,
0.000168187959,
0.000168466851,
0.000171537576,
0.000174634817,
0.000175208593,
0.000176437493,
0.000177213028,
0.000178471925,
0.000179304250,
0.000181361576,
0.000181774556,
0.000182317630,
0.000184348532,
0.000185496791,
0.000186600111,
0.000186986147,
0.000187895217,
0.000189050126,
0.000190311524,
0.000191188576,
0.000192708902,
0.000193427219,
0.000194377568,
0.000195221574,
0.000198353309,
0.000199150740,
0.000200669882,
0.000201329466,
0.000202603999,
0.000202840576,
0.000205243564,
0.000205267618,
0.000206280289,
0.000207180107,
0.000207403834,
0.000207693968,
0.000208319663,
0.000210918430,
0.000213097112,
0.000213808958,
0.000216127148,
0.000219614612,
0.000219958232,
0.000220138310,
0.000222035657,
0.000222401415,
0.000222760998,
0.000227509090,
0.000227596620,
0.000228951334,
0.000229255220,
0.000234511344,
0.000236992905,
0.000237318329,
0.000237794112,
0.000241724793,
0.000244140378,
0.000245108625,
0.000245348717,
0.000248554774,
0.000248622886,
0.000249551658,
0.000249917107,
0.000251383144,
0.000252364867,
0.000252518485,
0.000253298440,
0.000253519628,
0.000253894898,
0.000254257568,
0.000254474702,
0.000255773762,
0.000258142697,
0.000258694891,
0.000259473672,
0.000263981063,
0.000265385129,
0.000265929045,
0.000267164327,
0.000273901777,
0.000277254717,
0.000278234965,
0.000278852360,
0.000283459901,
0.000289070649,
0.000291372423,
0.000295603226,
0.000301966065,
0.000303991670,
0.000304363077,
0.000305384930,
0.000311071976,
0.000311670131,
0.000312786000,
0.000314656110,
0.000316524766,
0.000325310734,
0.000331275522,
0.000336062292,
0.000338832600,
0.000340484394,
0.000342036718,
0.000357296061,
0.000357704298,
0.000357937771,
0.000358446109,
0.000359799583,
0.000359998968,
0.000361527417,
0.000362966939,
0.000364820190,
0.000374621447,
0.000386212038,
0.000387148965,
0.000387832733,
0.000387999956,
0.000388426165,
0.000388865023,
0.000389427459,
0.000389902480,
0.000393306999,
0.000396066587,
0.000396212999,
0.000400014580,
0.000400166845,
0.000404149671,
0.000411878348,
0.000414328908,
0.000417213521,
0.000420034700,
0.000423594466,
0.000424079830,
0.000426916906,
0.000427907197,
0.000428596411,
0.000447038030,
0.000465801486,
0.000466707081,
0.000486192543,
0.000492689659,
0.000497607736,
0.000499009449,
0.000502225035,
0.000509876067,
0.000512491653,
0.000526893703,
0.000544232360,
0.000551226109,
0.000553362405,
0.000554621217,
0.000555323671,
0.000566235583,
0.000568858650,
0.000585816740,
0.000595444454,
0.000595676538,
0.000596027082,
0.000601532816,
0.000603398396,
0.000603875830,
0.000604469221,
0.000618072836,
0.000618924385,
0.000633748912,
0.000638051121,
0.000638063264,
0.000646909261,
0.000648598715,
0.000650473629,
0.000681635898,
0.000688663096,
0.000695006176,
0.000698233309,
0.000698315251,
0.000705518120,
0.000714561173,
0.000732850381,
0.000751384305,
0.000759306513,
0.000762051797,
0.000762449955,
0.000763277577,
0.000777398792,
0.000804723486,
0.000806123103,
0.000811382583,
0.000826331403,
0.000863195543,
0.000885311967,
0.000888494491,
0.000905345436,
0.000906955857,
0.000913376383,
0.000919577290,
0.000945443427,
0.000953607106,
0.000960165048,
0.000967380780,
0.000986657560,
0.001013995795,
0.001035286884,
0.001040623463,
0.001042209681,
0.001054608040,
0.001097461414,
0.001123190067,
0.001151108539,
0.001174213778,
0.001179660877,
0.001236145549,
0.001262545378,
0.001319707705,
0.001407165160,
0.001408158461,
0.001449382306,
0.001505208285,
0.001525693275,
0.001545857137,
0.001548823775,
0.001604079926,
0.001631439516,
0.001662684683,
0.001664953145,
0.001670337051,
0.001681930216,
0.001721041622,
0.001772208339,
0.001814347121,
0.001857630106,
0.001999227026,
0.002000140185,
0.002059968093,
0.002073217095,
0.002076366149,
0.002087479476,
0.002212095863,
0.002213487143,
0.002226900538,
0.002240316849,
0.002245926943,
0.002397677381,
0.002409873183,
0.002433426457,
0.002533656116,
0.002554175483,
0.002702565798,
0.002725149141,
0.002862990738,
0.003012626379,
0.003106708313,
0.003166477317,
0.003311416707,
0.003376595359,
0.003389236791,
0.003394227968,
0.003447322975,
0.003564280055,
0.003693253982,
0.003760854305,
0.003789602821,
0.003812877852,
0.004221260332,
0.004573300544,
0.004682578946,
0.004872355381,
0.004894274971,
0.005046232120,
0.005583382792,
0.005638003076,
0.005814559667,
0.006184242380,
0.007007230181,
0.007892807911,
0.008176775716,
0.009132843479,
0.011816355808,
0.015919697299,
0.017167705703,
0.018201374196,
0.019553531484,
0.019770204022,
0.020634517885,
0.020672075448,
0.022999345332,
0.034482186306,
0.037620656622,
0.052245806915,
0.061315610525,
0.131015009271,
0.161722678961

Data set 2 - Expected Results

7.815875e-05,
8.941521e-05,
9.627393999999998e-05,
0.00010762907,
0.00011539939999999982,
0.00011822948999999979,
0.00012052014,
0.00012526333999999986,
0.00013414645,
0.00013932926999999957,
0.00013955431,
0.0001495835059119702,
0.000150294237911998,
0.00015041977591199324,
0.00015207199791202302,
0.000152313622912043,
0.00015698175491200684,
0.00015801911991201296,
0.00015834205791202864,
0.00015979094791202398,
0.00016100523991198663,
0.00016379205491198664,
0.0001648996079119817,
0.0001651713689119893,
0.00016533978991200703,
0.00016703581791202901,
0.00016891352491201118,
0.00017388017191200737,
0.00017562037391202252,
0.00017729022891198592,
0.00017824894891204157,
0.00017868280291200405,
0.00017876091291208266,
0.00017911371791199957,
0.00018030274491195414,
0.00018177437391201945,
0.000182923552912061,
0.00018383527291203298,
0.00018659639391205675,
0.00018762718991203147,
0.00019059358491195527,
0.0001926192449119648,
0.00019852061091196402,
0.00020174023591201703,
0.00020375999991193355,
0.00020570400691193437,
0.00020618606291200375,
0.00020733386391192797,
0.00020803107391205626,
0.0002093031089120231,
0.00021421192591195874,
0.00021877230991197577,
0.00021917629791203723,
0.0002242487999119585,
0.00022457534991197144,
0.00022482346491199185,
0.00022553160191199193,
0.0002263532199120038,
0.00022939365291196605,
0.0002309596779119836,
0.00023252429891199093,
0.00023266046591194687,
0.00023285851791200062,
0.00023385954491204357,
0.00023545371791194143,
0.0002469572289119646,
0.00024759281591200586,
0.0002542272439119975,
0.0002544176639119319,
0.00025649029291192915,
0.0002603250319119855,
0.0002616878739119835,
0.000262873938912035,
0.0002630069909120449,
0.0002657095729119685,
0.00027155839091206046,
0.00027202379391202676,
0.0002724651419120668,
0.0002730509749120375,
0.00027308291491199606,
0.00027485912991198353,
0.0002762168659120024,
0.0002775981069120444,
0.00028025679391200156,
0.0002807010189120066,
0.00028373873491202133,
0.0002875008559119559,
0.000292584518912039,
0.0002979073869119615,
0.00029856642191202376,
0.0002997196019119469,
0.00029991927891203,
0.0003002999009120056,
0.0003013359239119744,
0.0003021430429120131,
0.0003022740189120128,
0.0003027606959119818,
0.0003030395879120486,
0.00030611031291202025,
0.0003092075539120183,
0.00030978132991203765,
0.00031101022991204873,
0.0003117857649120417,
0.0003130446619119617,
0.000313876986911976,
0.00031593431291201474,
0.0003163472929119965,
0.0003168903669120422,
0.00031892126891197917,
0.00032006952791199296,
0.0003211728479119738,
0.0003215588839120256,
0.0003224679539119852,
0.00032362286291197145,
0.00032488426091201736,
0.0003257613129119702,
0.00032728163891202416,
0.0003279999559120312,
0.0003289503049119891,
0.00032979431091203777,
0.00033292604591197614,
0.0003337234769120466,
0.00033524261891200056,
0.0003359022029119755,
0.0003371767359120057,
0.0003374133129120084,
0.0003398163009119851,
0.0003398403549120571,
0.00034085302591201603,
0.0003417528439120266,
0.0003419765709120092,
0.000342266704911983,
0.00034289239991205385,
0.00034549116691198653,
0.00034766984891196443,
0.0003483816949120069,
0.0003506998849119761,
0.00035418734891205084,
0.00035453096891195537,
0.0003547110469119837,
0.00035660839391197575,
0.0003569741519120372,
0.0003573337349120194,
0.00036208182691197064,
0.0003621693569119687,
0.0003635240709120471,
0.0003638279569120289,
0.00036908408091199313,
0.00037156564191204633,
0.0003718910659120154,
0.00037236684891197506,
0.00037629752991198565,
0.0003787131149120248,
0.00037968136191196647,
0.0003799214539119949,
0.0003831275109120432,
0.0003831956229120269,
0.00038412439491202323,
0.00038448984391204334,
0.00038595588091199304,
0.00038693760391196466,
0.00038709122191198455,
0.0003878711769119594,
0.0003880923649119783,
0.0003884676349119532,
0.00038883030491203654,
0.000389047438912041,
0.00039034649891200916,
0.00039271543391198124,
0.0003932676279120183,
0.0003940464089119744,
0.0003985537999119681,
0.00039995786591199667,
0.00040050178191201354,
0.0004017370639120306,
0.0004084745139119775,
0.00041182745391203234,
0.0004128077019120334,
0.0004134250969120184,
0.0004180326379120275,
0.00042364338591197536,
0.00042594515991198806,
0.00043017596291201876,
0.00043653880191198266,
0.00043856440691201305,
0.0004389358139119734,
0.00043995766691204343,
0.0004456447129119721,
0.0004462428679119628,
0.0004473587369120003,
0.0004492288469120187,
0.0004510975029120407,
0.0004598834709119715,
0.0004658482589120257,
0.0004706350289119652,
0.000473405336911973,
0.00047505713091202815,
0.0004766094549119654,
0.0004918687979119956,
0.0004922770349119918,
0.0004925105079119638,
0.0004930188459120279,
0.0004943723199119822,
0.0004945717049119815,
0.0004961001539120082,
0.0004975396759119774,
0.0004993929269120327,
0.0005091941839120283,
0.0005207847749119795,
0.0005217217019120183,
0.0005224054699119622,
0.0005225726929119816,
0.0005229989019119875,
0.0005234377599120318,
0.0005240001959119908,
0.0005244752169120295,
0.0005278797359120326,
0.0005306393239119781,
0.0005307857359119724,
0.0005345873169119753,
0.0005347395819119778,
0.0005387224079120155,
0.000546451084912013,
0.00054890164491197,
0.0005517862579119822,
0.0005546074369120173,
0.0005581672029120285,
0.0005586525669119659,
0.0005614896429120325,
0.0005624799339119684,
0.0005631691479120259,
0.0005816107669119913,
0.0006003742229120048,
0.0006012798179119884,
0.0006207652799119849,
0.0006272623959120036,
0.0006321804729119897,
0.0006335821859119923,
0.000636797771912025,
0.0006444488039119931,
0.0006470643899120163,
0.0006614664399119813,
0.0006788050969119976,
0.0006857988459120204,
0.0006879351419119665,
0.0006891939539120166,
0.0006898964079119804,
0.0007008083199120062,
0.0007034313869120015,
0.0007203894769119833,
0.0007300171909120018,
0.00073024927491201,
0.0007305998189119847,
0.0007361055529120149,
0.0007379711329119846,
0.0007384485669120252,
0.0007390419579119909,
0.0007526455729120031,
0.0007534971219120086,
0.0007683216489120074,
0.0007726238579120136,
0.0007726360009119909,
0.0007814819979120094,
0.0007831714519120047,
0.0007850463659119877,
0.0008162086349119961,
0.0008232358329119806,
0.0008295789129119884,
0.0008328060459119852,
0.0008328879879120103,
0.000840090856911989,
0.0008491339099119937,
0.0008674231179120169,
0.0008859570419120222,
0.0008938792499119844,
0.0008966245339119852,
0.0008970226919120021,
0.0008978503139120199,
0.00091197152891202,
0.0009392962229119841,
0.0009406958399119855,
0.000945955319912013,
0.0009609041399120041,
0.0009977682799120165,
0.0010198847039119786,
0.0010230672279119826,
0.0010399181729120084,
0.0010415285939120026,
0.001047949119911996,
0.0010541500269119941,
0.0010800161639119834,
0.001088179842911993,
0.001094737784912012,
0.0011019535169120112,
0.001121230296912,
0.0011485685319119845,
0.001169859620911995,
0.001175196199911988,
0.0011767824179119955,
0.001189180776912008,
0.00123203415091198,
0.0012577628039120029,
0.001285681275911981,
0.0013087865149120157,
0.001314233613911996,
0.0013707182859119997,
0.0013971181149119995,
0.0014542804419119865,
0.0015417378969120117,
0.0015427311979119846,
0.0015839550429120077,
0.0016397810219120137,
0.0016602660119119995,
0.0016804298739120022,
0.0016833965119119854,
0.0017386526629119892,
0.0017660122529120081,
0.0017972574199120023,
0.0017995258819119972,
0.0018049097879120027,
0.0018165029529119917,
0.001855614358911993,
0.0019067810759119958,
0.0019489198579120096,
0.0019922028429119935,
0.0021337997629119866,
0.0021347129219119907,
0.002194540829911992,
0.002207789831911997,
0.0022109388859119913,
0.0022220522129120074,
0.0023466685999120026,
0.002348059879911988,
0.0023614732749120005,
0.0023748895859120045,
0.002380499679912001,
0.0025322501179119907,
0.0025444459199120014,
0.002567999193912005,
0.002668228852911991,
0.002688748219911994,
0.002837138534911992,
0.0028597218779120056,
0.0029975634749119964,
0.003147199115911993,
0.0032412810499120024,
0.0033010500539119992,
0.0034459894439119996,
0.0035111680959119995,
0.003523809527911996,
0.003528800704912003,
0.003581895711911993,
0.003698852791912005,
0.003827826718911997,
0.0038954270419120007,
0.003924175557911998,
0.003947450588911994,
0.004355833068911996,
0.004707873280912004,
0.004817151682911996,
0.005006928117911997,
0.005028847707912001,
0.005180804856911993,
0.005717955528911997,
0.005772575812911997,
0.005949132403911996,
0.006318815116911996,
0.007141802917911999,
0.008027380647911999,
0.008311348452911996,
0.009267416215911995,
0.011950928544912,
0.016054270035912,
0.017302278439912003,
0.018335946932911996,
0.019688104220911995,
0.019904776758911995,
0.020769090621911995,
0.020806648184911996,
0.023133918068911997,
0.034616759042912,
0.037755229358912,
0.052380379651911994,
0.061450183261912,
0.131149582007912,
0.161857251697912

---

with slightly tighter constraints

• w_bar are the input values
• w represents the absolute difference
• E(w) represents the least-squares difference

Answer 1

I recommend that you give up on the idea of least-squares, so that you can instead frame this as a mixed-integer linear program. It's slightly tricky, but it does work. Setup up variables for:

• |err|, the absolute value of w - w_bar, to be minimized
• w, to be bounded in (0, 0.3)
• whi, equal to w if it's above const1, otherwise 0
• ishi, a binary indicating whether w > const1

import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import milp, Bounds, LinearConstraint
import scipy.sparse as sp


def make_constraints(
    w_bar: np.ndarray,
    const1: float,
    const2: float,
    w_max: float,
) -> tuple[LinearConstraint, ...]:
    n = w_bar.size

    # Constrain error to absolute difference of w and wbar
    # err >= w - wbar: -err + w <= wbar
    # err >= wbar - w:  err + w >= wbar
    err_constraint = LinearConstraint(
        A=sp.bmat((
            (-sp.eye(n), sp.eye(n), sp.csc_array((n, 2 * n))),
            (sp.eye(n), sp.eye(n), sp.csc_array((n, 2 * n))),
        ), format='csc'),
        lb=np.concatenate((np.full(n, -np.inf), w_bar)),
        ub=np.concatenate((w_bar, np.full(n, np.inf))),
    )

    # sum w = 1
    one_constraint = LinearConstraint(
        A=sp.hstack((np.zeros((1, n)), np.ones(n), np.zeros(2 * n)), format='csc'),
        lb=1, ub=1,
    )

    # Activate ishi based on value of w and const1
    # ishi >= w - const1: w - ishi <= const1
    # ishi <= w/const1:   w/const1 - ishi >= 0
    ishi_constraint = LinearConstraint(
        A=sp.bmat((
            (sp.csc_array((n, n)), sp.eye(n), sp.csc_array((n, n)), -sp.eye(n)),
            (sp.csc_array((n, n)), sp.eye(n) / const1, sp.csc_array((n, n)), -sp.eye(n)),
        ), format='csc'),
        lb=np.concatenate((
            np.full(n, -np.inf),
            np.zeros(n),
        )),
        ub=np.concatenate((
            np.full(n, const1),
            np.full(n, np.inf),
        )),
    )

    # Constrain whi based on whether ishi is set
    # whi <= w                  w - whi          >= 0
    # whi <= ishi*M                -whi + M*ishi >= 0
    # whi >= w - M*(1 - ishi)   w - whi + M*ishi <= M
    M = 2 * w_max
    whi_constraint = LinearConstraint(
        A=sp.bmat((
            (sp.csc_array((n, n)), sp.eye(n), -sp.eye(n), sp.csc_array((n, n))),
            (sp.csc_array((n, n)), sp.csc_array((n, n)), -sp.eye(n), M * sp.eye(n)),
            (sp.csc_array((n, n)), sp.eye(n), -sp.eye(n), M * sp.eye(n)),
        ), format='csc'),
        lb=np.concatenate((
            np.zeros(n),
            np.zeros(n),
            np.full(n, -np.inf),
        )),
        ub=np.concatenate((
            np.full(n, np.inf),
            np.full(n, np.inf),
            np.full(n, M),
        )),
    )

    # Constrain whi sum based on value of const2
    # sum whi <= const2
    whisum_constraint = LinearConstraint(
        A=sp.hstack((
            sp.csc_array((1, n)), sp.csc_array((1, n)), np.ones(n), sp.csc_array((1, n)),
        ), format='csc'),
        lb=-np.inf, ub=const2,
    )

    # w[i] <= w[i + 1]: -w[i] + w[i + 1]  >= 0
    monotonicity = LinearConstraint(
        A=sp.hstack((
            sp.csc_array((n - 1, n)),
            sp.eye(m=n - 1, n=n, k=1) - sp.eye(m=n - 1, n=n, k=0),
            sp.csc_array((n - 1, n)),
            sp.csc_array((n - 1, n)),
        )),
        lb=0, ub=np.inf,
    )

    return (
        err_constraint, one_constraint,
        ishi_constraint, whi_constraint, whisum_constraint,
        monotonicity,
    )


def solve(
    w_bar: np.ndarray,
    const1: float = 0.06,
    const2: float = 0.6,
    w_max: float = 0.3,
) -> tuple[
    float,       # cost
    np.ndarray,  # err
    np.ndarray,  # w
    np.ndarray,  # whi
    np.ndarray,  # ishi
]:
    n = w_bar.size

    # Variables: n |err|, n w, n whi, n ishi
    c = np.concatenate((
        np.ones(n),   # minimize |err|
        np.zeros(3*n),  # don't optimize w, whi or ishi
    ))

    # err, w, whi continuous, ishi integral
    integrality = np.zeros(shape=4*n, dtype=np.uint8)
    integrality[-n:] = 1

    bounds = Bounds(
        #                  |err|,     w,    whi, ishi
        lb=np.repeat((-np.inf,     0,      0, 0), n),
        ub=np.repeat((+np.inf, w_max, np.inf, 1), n),
    )

    constraints = make_constraints(w_bar, const1, const2, w_max)

    result = milp(
        c=c, integrality=integrality, bounds=bounds, constraints=constraints,
    )
    assert result.success, result.message
    err, w, whi, ishi = np.split(result.x, (n, 2*n, 3*n))
    return result.fun, err, w, whi, ishi


def solve_and_output(w_bar: np.ndarray, ax: plt.Axes, label: str) -> None:
    const2 = 0.6
    cost, err, w, whi, ishi = solve(w_bar, const2=const2)

    print(f'Cost: {cost:.3f}')
    print(f'Sum: {w.sum():.2f} == 1')
    print(f'const2: {whi.sum():.2f} <= {const2}')
    print()

    ax.plot(w_bar, w, label=label)


def make_data() -> tuple[np.ndarray, ...]:
    data_1 = (
        0.0000290098720682487343868705,
        0.0000723827066008387518299476,
        0.0002072744176096173915822705,
        0.0003672809435335959934840107,
        0.0003752484336042253846688711,
        0.0004405565779129329566220824,
        0.0005525596498762783855459046,
        0.0006728198725774592129480554,
        0.0008351912653287541643826696,
        0.0009825084042287753464578806,
        0.0009948648777755642828519691,
        0.0012517612267766885632838162,
        0.0013546692913343326698875246,
        0.0015307554230129220783373226,
        0.0018079051888007421049402691,
        0.0018273451279460959651328143,
        0.0019106018091149384691427602,
        0.0019397417543016734988324706,
        0.0021105212741847910173045113,
        0.0025270382452472284845495592,
        0.0026030268201749619505495437,
        0.0026381414030341878249837264,
        0.0035891127645570676979108904,
        0.0042890768176753193003852561,
        0.0054266187080476129436017240,
        0.0072342421507922666121696771,
        0.0141269512168524993586417500,
        0.0277835377270709998751732016,
        0.0358112813055125623355637036,
        0.4222060446840235199334703797,
        0.4525019300404232987113785672,
    )

    data_2 = (
        0.000007815875,
        0.000008941521,
        0.000009627394,
        0.000010762907,
        0.000011539940,
        0.000011822949,
        0.000012052014,
        0.000012526334,
        0.000013414645,
        0.000013932927,
        0.000013955431,
        0.000015010769,
        0.000015721501,
        0.000015847039,
        0.000017499261,
        0.000017740886,
        0.000022409018,
        0.000023446383,
        0.000023769321,
        0.000025218211,
        0.000026432503,
        0.000029219318,
        0.000030326871,
        0.000030598632,
        0.000030767053,
        0.000032463081,
        0.000034340788,
        0.000039307435,
        0.000041047637,
        0.000042717492,
        0.000043676212,
        0.000044110066,
        0.000044188176,
        0.000044540981,
        0.000045730008,
        0.000047201637,
        0.000048350816,
        0.000049262536,
        0.000052023657,
        0.000053054453,
        0.000056020848,
        0.000058046508,
        0.000063947874,
        0.000067167499,
        0.000069187263,
        0.000071131270,
        0.000071613326,
        0.000072761127,
        0.000073458337,
        0.000074730372,
        0.000079639189,
        0.000084199573,
        0.000084603561,
        0.000089676063,
        0.000090002613,
        0.000090250728,
        0.000090958865,
        0.000091780483,
        0.000094820916,
        0.000096386941,
        0.000097951562,
        0.000098087729,
        0.000098285781,
        0.000099286808,
        0.000100880981,
        0.000112384492,
        0.000113020079,
        0.000119654507,
        0.000119844927,
        0.000121917556,
        0.000125752295,
        0.000127115137,
        0.000128301202,
        0.000128434254,
        0.000131136836,
        0.000136985654,
        0.000137451057,
        0.000137892405,
        0.000138478238,
        0.000138510178,
        0.000140286393,
        0.000141644129,
        0.000143025370,
        0.000145684057,
        0.000146128282,
        0.000149165998,
        0.000152928119,
        0.000158011782,
        0.000163334650,
        0.000163993685,
        0.000165146865,
        0.000165346542,
        0.000165727164,
        0.000166763187,
        0.000167570306,
        0.000167701282,
        0.000168187959,
        0.000168466851,
        0.000171537576,
        0.000174634817,
        0.000175208593,
        0.000176437493,
        0.000177213028,
        0.000178471925,
        0.000179304250,
        0.000181361576,
        0.000181774556,
        0.000182317630,
        0.000184348532,
        0.000185496791,
        0.000186600111,
        0.000186986147,
        0.000187895217,
        0.000189050126,
        0.000190311524,
        0.000191188576,
        0.000192708902,
        0.000193427219,
        0.000194377568,
        0.000195221574,
        0.000198353309,
        0.000199150740,
        0.000200669882,
        0.000201329466,
        0.000202603999,
        0.000202840576,
        0.000205243564,
        0.000205267618,
        0.000206280289,
        0.000207180107,
        0.000207403834,
        0.000207693968,
        0.000208319663,
        0.000210918430,
        0.000213097112,
        0.000213808958,
        0.000216127148,
        0.000219614612,
        0.000219958232,
        0.000220138310,
        0.000222035657,
        0.000222401415,
        0.000222760998,
        0.000227509090,
        0.000227596620,
        0.000228951334,
        0.000229255220,
        0.000234511344,
        0.000236992905,
        0.000237318329,
        0.000237794112,
        0.000241724793,
        0.000244140378,
        0.000245108625,
        0.000245348717,
        0.000248554774,
        0.000248622886,
        0.000249551658,
        0.000249917107,
        0.000251383144,
        0.000252364867,
        0.000252518485,
        0.000253298440,
        0.000253519628,
        0.000253894898,
        0.000254257568,
        0.000254474702,
        0.000255773762,
        0.000258142697,
        0.000258694891,
        0.000259473672,
        0.000263981063,
        0.000265385129,
        0.000265929045,
        0.000267164327,
        0.000273901777,
        0.000277254717,
        0.000278234965,
        0.000278852360,
        0.000283459901,
        0.000289070649,
        0.000291372423,
        0.000295603226,
        0.000301966065,
        0.000303991670,
        0.000304363077,
        0.000305384930,
        0.000311071976,
        0.000311670131,
        0.000312786000,
        0.000314656110,
        0.000316524766,
        0.000325310734,
        0.000331275522,
        0.000336062292,
        0.000338832600,
        0.000340484394,
        0.000342036718,
        0.000357296061,
        0.000357704298,
        0.000357937771,
        0.000358446109,
        0.000359799583,
        0.000359998968,
        0.000361527417,
        0.000362966939,
        0.000364820190,
        0.000374621447,
        0.000386212038,
        0.000387148965,
        0.000387832733,
        0.000387999956,
        0.000388426165,
        0.000388865023,
        0.000389427459,
        0.000389902480,
        0.000393306999,
        0.000396066587,
        0.000396212999,
        0.000400014580,
        0.000400166845,
        0.000404149671,
        0.000411878348,
        0.000414328908,
        0.000417213521,
        0.000420034700,
        0.000423594466,
        0.000424079830,
        0.000426916906,
        0.000427907197,
        0.000428596411,
        0.000447038030,
        0.000465801486,
        0.000466707081,
        0.000486192543,
        0.000492689659,
        0.000497607736,
        0.000499009449,
        0.000502225035,
        0.000509876067,
        0.000512491653,
        0.000526893703,
        0.000544232360,
        0.000551226109,
        0.000553362405,
        0.000554621217,
        0.000555323671,
        0.000566235583,
        0.000568858650,
        0.000585816740,
        0.000595444454,
        0.000595676538,
        0.000596027082,
        0.000601532816,
        0.000603398396,
        0.000603875830,
        0.000604469221,
        0.000618072836,
        0.000618924385,
        0.000633748912,
        0.000638051121,
        0.000638063264,
        0.000646909261,
        0.000648598715,
        0.000650473629,
        0.000681635898,
        0.000688663096,
        0.000695006176,
        0.000698233309,
        0.000698315251,
        0.000705518120,
        0.000714561173,
        0.000732850381,
        0.000751384305,
        0.000759306513,
        0.000762051797,
        0.000762449955,
        0.000763277577,
        0.000777398792,
        0.000804723486,
        0.000806123103,
        0.000811382583,
        0.000826331403,
        0.000863195543,
        0.000885311967,
        0.000888494491,
        0.000905345436,
        0.000906955857,
        0.000913376383,
        0.000919577290,
        0.000945443427,
        0.000953607106,
        0.000960165048,
        0.000967380780,
        0.000986657560,
        0.001013995795,
        0.001035286884,
        0.001040623463,
        0.001042209681,
        0.001054608040,
        0.001097461414,
        0.001123190067,
        0.001151108539,
        0.001174213778,
        0.001179660877,
        0.001236145549,
        0.001262545378,
        0.001319707705,
        0.001407165160,
        0.001408158461,
        0.001449382306,
        0.001505208285,
        0.001525693275,
        0.001545857137,
        0.001548823775,
        0.001604079926,
        0.001631439516,
        0.001662684683,
        0.001664953145,
        0.001670337051,
        0.001681930216,
        0.001721041622,
        0.001772208339,
        0.001814347121,
        0.001857630106,
        0.001999227026,
        0.002000140185,
        0.002059968093,
        0.002073217095,
        0.002076366149,
        0.002087479476,
        0.002212095863,
        0.002213487143,
        0.002226900538,
        0.002240316849,
        0.002245926943,
        0.002397677381,
        0.002409873183,
        0.002433426457,
        0.002533656116,
        0.002554175483,
        0.002702565798,
        0.002725149141,
        0.002862990738,
        0.003012626379,
        0.003106708313,
        0.003166477317,
        0.003311416707,
        0.003376595359,
        0.003389236791,
        0.003394227968,
        0.003447322975,
        0.003564280055,
        0.003693253982,
        0.003760854305,
        0.003789602821,
        0.003812877852,
        0.004221260332,
        0.004573300544,
        0.004682578946,
        0.004872355381,
        0.004894274971,
        0.005046232120,
        0.005583382792,
        0.005638003076,
        0.005814559667,
        0.006184242380,
        0.007007230181,
        0.007892807911,
        0.008176775716,
        0.009132843479,
        0.011816355808,
        0.015919697299,
        0.017167705703,
        0.018201374196,
        0.019553531484,
        0.019770204022,
        0.020634517885,
        0.020672075448,
        0.022999345332,
        0.034482186306,
        0.037620656622,
        0.052245806915,
        0.061315610525,
        0.131015009271,
        0.161722678961,
    )

    return np.array(data_1), np.array(data_2)


def main() -> None:
    data_1, data_2 = make_data()

    ax: plt.Axes
    fig, ax = plt.subplots()
    xmin = min(data_1.min(), data_2.min())
    xmax = max(data_1.max(), data_2.max())
    ax.plot(
        [xmin, xmax],
        [xmin, xmax],
        'g--', label='ideal',
    )

    for w_bar, label in (
        (data_1, 'Dataset 1'),
        (data_2, 'Dataset 2'),
    ):
        solve_and_output(w_bar, ax, label=label)

    ax.set_title('Linear fit, all datasets')
    ax.set_xlabel('w_bar')
    ax.set_ylabel('w_fit')
    ax.set_xscale('log')
    ax.set_yscale('log')
    ax.legend()
    plt.show()


if __name__ == '__main__':
    main()

Cost: 0.549
Sum: 1.00 == 1
const2: 0.60 <= 0.6

Cost: 0.052
Sum: 1.00 == 1
const2: 0.35 <= 0.6

Answer 2

Try using m.if2() instead of m.if3() to use continuous instead of discrete optimization. The m.if3() function can take much longer for a larger number of binary decision variables.

model.sum([model.if2(w[i] - 0.06, 0, w[i])

If it needs to fit at the lower magnitude values, try normalizing by w_bar:

model.sum([((w_bar[i] - w[i])/w_bar[i])** 2

Here is a full script that produces the results (not normalized).

import numpy as np
from gekko import GEKKO
import matplotlib.pyplot as plt

w_bar = [0.000007815875,0.000008941521,0.000009627394,0.000010762907,0.000011539940,\
0.000011822949,0.000012052014,0.000012526334,0.000013414645,0.000013932927,0.000013955431,\
0.000015010769,0.000015721501,0.000015847039,0.000017499261,0.000017740886,0.000022409018,\
0.000023446383,0.000023769321,0.000025218211,0.000026432503,0.000029219318,0.000030326871,\
0.000030598632,0.000030767053,0.000032463081,0.000034340788,0.000039307435,0.000041047637,\
0.000042717492,0.000043676212,0.000044110066,0.000044188176,0.000044540981,0.000045730008,\
0.000047201637,0.000048350816,0.000049262536,0.000052023657,0.000053054453,0.000056020848,\
0.000058046508,0.000063947874,0.000067167499,0.000069187263,0.000071131270,0.000071613326,\
0.000072761127,0.000073458337,0.000074730372,0.000079639189,0.000084199573,0.000084603561,\
0.000089676063,0.000090002613,0.000090250728,0.000090958865,0.000091780483,0.000094820916,\
0.000096386941,0.000097951562,0.000098087729,0.000098285781,0.000099286808,0.000100880981,\
0.000112384492,0.000113020079,0.000119654507,0.000119844927,0.000121917556,0.000125752295,\
0.000127115137,0.000128301202,0.000128434254,0.000131136836,0.000136985654,0.000137451057,\
0.000137892405,0.000138478238,0.000138510178,0.000140286393,0.000141644129,0.000143025370,\
0.000145684057,0.000146128282,0.000149165998,0.000152928119,0.000158011782,0.000163334650,\
0.000163993685,0.000165146865,0.000165346542,0.000165727164,0.000166763187,0.000167570306,\
0.000167701282,0.000168187959,0.000168466851,0.000171537576,0.000174634817,0.000175208593,\
0.000176437493,0.000177213028,0.000178471925,0.000179304250,0.000181361576,0.000181774556,\
0.000182317630,0.000184348532,0.000185496791,0.000186600111,0.000186986147,0.000187895217,\
0.000189050126,0.000190311524,0.000191188576,0.000192708902,0.000193427219,0.000194377568,\
0.000195221574,0.000198353309,0.000199150740,0.000200669882,0.000201329466,0.000202603999,\
0.000202840576,0.000205243564,0.000205267618,0.000206280289,0.000207180107,0.000207403834,\
0.000207693968,0.000208319663,0.000210918430,0.000213097112,0.000213808958,0.000216127148,\
0.000219614612,0.000219958232,0.000220138310,0.000222035657,0.000222401415,0.000222760998,\
0.000227509090,0.000227596620,0.000228951334,0.000229255220,0.000234511344,0.000236992905,\
0.000237318329,0.000237794112,0.000241724793,0.000244140378,0.000245108625,0.000245348717,\
0.000248554774,0.000248622886,0.000249551658,0.000249917107,0.000251383144,0.000252364867,\
0.000252518485,0.000253298440,0.000253519628,0.000253894898,0.000254257568,0.000254474702,\
0.000255773762,0.000258142697,0.000258694891,0.000259473672,0.000263981063,0.000265385129,\
0.000265929045,0.000267164327,0.000273901777,0.000277254717,0.000278234965,0.000278852360,\
0.000283459901,0.000289070649,0.000291372423,0.000295603226,0.000301966065,0.000303991670,\
0.000304363077,0.000305384930,0.000311071976,0.000311670131,0.000312786000,0.000314656110,\
0.000316524766,0.000325310734,0.000331275522,0.000336062292,0.000338832600,0.000340484394,\
0.000342036718,0.000357296061,0.000357704298,0.000357937771,0.000358446109,0.000359799583,\
0.000359998968,0.000361527417,0.000362966939,0.000364820190,0.000374621447,0.000386212038,\
0.000387148965,0.000387832733,0.000387999956,0.000388426165,0.000388865023,0.000389427459,\
0.000389902480,0.000393306999,0.000396066587,0.000396212999,0.000400014580,0.000400166845,\
0.000404149671,0.000411878348,0.000414328908,0.000417213521,0.000420034700,0.000423594466,\
0.000424079830,0.000426916906,0.000427907197,0.000428596411,0.000447038030,0.000465801486,\
0.000466707081,0.000486192543,0.000492689659,0.000497607736,0.000499009449,0.000502225035,\
0.000509876067,0.000512491653,0.000526893703,0.000544232360,0.000551226109,0.000553362405,\
0.000554621217,0.000555323671,0.000566235583,0.000568858650,0.000585816740,0.000595444454,\
0.000595676538,0.000596027082,0.000601532816,0.000603398396,0.000603875830,0.000604469221,\
0.000618072836,0.000618924385,0.000633748912,0.000638051121,0.000638063264,0.000646909261,\
0.000648598715,0.000650473629,0.000681635898,0.000688663096,0.000695006176,0.000698233309,\
0.000698315251,0.000705518120,0.000714561173,0.000732850381,0.000751384305,0.000759306513,\
0.000762051797,0.000762449955,0.000763277577,0.000777398792,0.000804723486,0.000806123103,\
0.000811382583,0.000826331403,0.000863195543,0.000885311967,0.000888494491,0.000905345436,\
0.000906955857,0.000913376383,0.000919577290,0.000945443427,0.000953607106,0.000960165048,\
0.000967380780,0.000986657560,0.001013995795,0.001035286884,0.001040623463,0.001042209681,\
0.001054608040,0.001097461414,0.001123190067,0.001151108539,0.001174213778,0.001179660877,\
0.001236145549,0.001262545378,0.001319707705,0.001407165160,0.001408158461,0.001449382306,\
0.001505208285,0.001525693275,0.001545857137,0.001548823775,0.001604079926,0.001631439516,\
0.001662684683,0.001664953145,0.001670337051,0.001681930216,0.001721041622,0.001772208339,\
0.001814347121,0.001857630106,0.001999227026,0.002000140185,0.002059968093,0.002073217095,\
0.002076366149,0.002087479476,0.002212095863,0.002213487143,0.002226900538,0.002240316849,\
0.002245926943,0.002397677381,0.002409873183,0.002433426457,0.002533656116,0.002554175483,\
0.002702565798,0.002725149141,0.002862990738,0.003012626379,0.003106708313,0.003166477317,\
0.003311416707,0.003376595359,0.003389236791,0.003394227968,0.003447322975,0.003564280055,\
0.003693253982,0.003760854305,0.003789602821,0.003812877852,0.004221260332,0.004573300544,\
0.004682578946,0.004872355381,0.004894274971,0.005046232120,0.005583382792,0.005638003076,\
0.005814559667,0.006184242380,0.007007230181,0.007892807911,0.008176775716,0.009132843479,\
0.011816355808,0.015919697299,0.017167705703,0.018201374196,0.019553531484,0.019770204022,\
0.020634517885,0.020672075448,0.022999345332,0.034482186306,0.037620656622,0.052245806915,\
0.061315610525,0.131015009271,0.161722678961]

model = GEKKO(remote=False)

w = model.Array(model.Var, len(w_bar), lb=0, ub=0.3)

# Sum of all values = 1
model.Equation(model.sum(w) == 1)

# Sum of all values greater than 6% cannot exceed 60%
# This is the part that degrades the speed greatly
model.Equation(model.sum([model.if2(w[i] - 0.06, 0, w[i]) for i in range(len(w))]) <= 0.6)

model.Minimize(model.sum([(w_bar[i] - w[i])** 2 for i in range(len(w_bar))]))

model.options.SOLVER=3
model.solve()

plt.figure(figsize=(8,5))
w_sol = [w[i].value[0] for i in range(len(w_bar))]
plt.semilogy(w_bar,label='w_bar')
plt.semilogy(w_sol,label='w')
plt.xlabel('index'); plt.ylabel('value[index]')
plt.tight_layout(); plt.grid(); plt.savefig('results.png')
plt.show()

The m.if2() function can have problems converging, especially if values are at the switching point because it creates a saddle point that is difficult for optimizers. If this doesn't work, I recommend calculating a solution with m.if2() and then use that to initialize a problem using m.if3(). There is additional information on the m.if2() function (MPCC) and m.if3() (binary switching).