For several very sparse and irregular time series of counts, I am trying to fit them (one at a time) to a range and extract the coefficients as a way to "describe" each time series. However, most of them are failing to converge, even when I filter out cases where there are only 1-2 unique values or 0-2 nonzero counts.
I'm not very familiar with Poisson regression. How can I check whether the model will converge prior to fitting? Or, better yet, what can I do to get it to converge?
def get_poisson_coefs(y):
# if y is doomed to not converge:
# return np.nan, np.nan
# elif y will converge if I do blah:
# do blah
x = add_constant(np.arange(len(y)))
model = ZeroInflatedPoisson(y, x).fit_regularized(
alpha=0.1, L1_wt=0.5, maxiter=1000, trim_mode="size", disp=False
)
return model.params[1], model.params[0]
For context, my data consists of N observations, where each observation is its own multivariate time series of shape (D, T), and I'm flattening this tensor into a matrix of shape (N, E) (where D*T>E>D) to use as input for a machine learning model. Each coefficient derived is a column of the matrix.