Is it feasible to use weightthem - ebal (R statistics) on an interaction as a Treatment?

Question

I´m trying to find out if there are differences in outcomes between students that were treated in a given school type vs. students that were treated in any other school type.

I used PSM to find effects of the treatment on the outcome before - but since the treated group (given school type that was treated) becomes small and quite special (regarding covariates), I was wondering if I could use entropy balancing in the weightthem-package in R (since I also used MI to deal with missing values). First I was thinking on using a reduced dataset (only treated, all school types), but there I neither achieved balance via PSM nor did the ebal-weighting converge. One possibility might be to decrease the number of covariates to increase overlap, but then the conditional independence assumption might be problematic.

So now I´m using the full dataset (with the majority of students being untreated).

My take would be

weighted_data <- weightthem(Given-schooltype*Treatment ~ X1 + X2+ X3+ Xn, imputed.data, approach = "within", method = "ebal", estimand = "ATT").

My questions so far: Does it even make sense to try and balance a dataset on moments of an interaction? How would a subsequent analysis look like? I´m thinking of a regression with outcome ~ given-schooltype+ treatment+ given-schooltype*Treatment (and maybe further controls?)

Thanks in advance!

Edit: Treatment is binary (0/1), given-schooltype is also a binary dummy (0/1).

Edit2 (Reply/Clarification/answers): Hey, thank you for replying! Your answers (and questions) already help me very much to clarify some things! The way you write it, I see that for answering my RQ I should use the sample that only contains treated units (given-type x treated) and untreated units (other-type x treated) since I´m looking for ATT. So I only have a binary treatment. It acually might happen that I won´t find a sufficient balance since as stated the students in the given-school-type are more or less privileged. So I might not be able to conduct my analyses the way I intended to. For my paper that would be quite desastrous, but in itself it would be a interesting find that I would investigate further.

I tried different approaches (PSM, genetic matching; ebal). Acutally the balance isn´t too bad afterwards, but my effective sample sizes become super small so I wouldn´t want to draw inferences in these samples. I´ll try out further approaches as you suggested :)

My colleagues I brainstoremd with (and that I actually also trust to have some fundamental knowledge in statistical analyses) suggested to just run regression analyses in the full sample with main effects (school-type, treatment), interaction-term (school-type*treatment) and control variables, (that would have been used in weighting). But I don´t think this would be the best way to go (or even a plausible way to go, since the majority of my sample wasn´t treated (but attended the given school type)).

A further note is, that I work with non-experimental longitudinal data, so I have pretreatment values for the treatment (but not for the admission to the given school type - these I can only approximate, but not badly).

Answer 1

It sounds like you have a 4-categroy treatment, which you can use with entropy balancing using weightit() (weightit() is the engine behind weightthem() and contains all the documentation regarding what is possible; weightthem() just runs weightit() on each imputed dataset). Just create the 4-category variable and supply that as the treatment in the model formula.

I also see that you only seem to care about comparing two groups of students; those who were treated in one school type and those treated in another school type. So do you actually have a 4-category treatment, or just a 2-category treatment? How are untreated students supposed to come into this analysis?

Entropy balancing weights each group to resemble a target. By default, it weights each group to resemble the full sample. Do you want each group to resemble the full sample, just the treated units, or just one of the 4 categories? You need to decide that. It could have large implications on your ability to achieve balance, the estimated effect, and its interpretation. This is controlled both by the estimand argument and the units you include in your sample.

If entropy balancing failed because the balance constraints were too tight, just use a different method. weightit() has several other methods that might work in your dataset. Did you try propensity score weighting? Did you try CBPS? Did you try energy balancing? If no method can balance your dataset, then maybe it can't be balanced and you have fundamentally incomparable groups. What will you do then?