I am trying to solve the following optimization problem, which is a maximum entropy formulation of logistic regression, m is the number of features in the data set, q is the number of target class in the target categorical variable, $\hat{y}_{ik}$ is the estimated probability that observation i belong to category k.
Here is part of my implementation
yik = cp.Variable(shape=(n,q))
obj = cp.Maximize(cp.sum(cp.entr(yik)))
constraints = [cp.sum(yik,axis=1)==np.ones(n),X[:,:-1].T@yik == X[:,:-1].T@y_mod]
prob = cp.Problem(obj, constraints)
prob.solve(solver=cp.MOSEK,constraints)
Solving this with MOSEK in some cases seems to yield an infeasible dual solution. With some data set it works just fine, with others it fails. Here is part of the output from MOSEK using cvxpy package for optimization.
I don't really understand what is going on here. It seems to me that the problem should never be infeasible. Am I mistaken? Here, the output seems to say that all variables are causing the problem to be infeasible. Can someone clarify this for me?
Note: The data are scaled.