I have some experiment data where I have been using the CUPED methodology outlined in a microsoft paper to adjust the post-assignment data using the pre-assignment data.
One of my metrics is a composite metric where I have, for each user, the total purchase value in the experiment and the total purchase count in the experiment. I'm using the totals to calculate a population mean for the resulting metric (i.e. group means are SUM(numerator)/SUM(denominator).
I want to adjust this based on pre-experiment data, but I know that I have to adjust for the covariance between the numerator and denominator when I do the adjustment. Conveniently there's no case where there's a 0 denominator for a non-zero numerator.
For other metrics, I was using this approach:
theta = covariance(metric, covariate) / variance(covariate)
adjusted_metric = metric - (covariate - covariate_mean) x theta
I can calculate the covariance matrix between pre/post and num/denom easily enough, but it's unclear to me how I can use that to do a linear adjustment on the numerator/denominator, and if this is valid.
I tried doing a basic regression here, but the results I got seemed off.