Correlation higher than 1 when performing a multivariate meta-analysis with metaSEM

23 Views Asked by At

I have simulated a dataset for a meta-analysis with 20 studies and 1000 observations each. Each participant presents data on two outcomes: a clinician rating (CR) and self-report (SR) that measure the same latent Illness (e.g., depression). The performance of each participant is measured at T0 and at EOT (Outcome). Then, for each study I estimated the treatment effect, its standard error and the correlation between treatment effects using Seemingly Unrelated Regressions.

I performed the meta-analysis with metaSEM as:
msem <- metaSEM::meta(y=cbind(EstCR, EstSR), v=cbind(varCR, Cov, varSR), data=dat)

This is the output:

95% confidence intervals: z statistic approximation (robust=FALSE)
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 3.015002 0.104818 2.809563 3.220441 28.7642 <2e-16 ***
Intercept2 3.153640 0.226073 2.710545 3.596734 13.9497 <2e-16 ***
Tau2_1_1 0.012351 0.068944 -0.122776 0.147479 0.1792 0.8578
Tau2_2_1 0.190127 0.134189 -0.072879 0.453133 1.4169 0.1565
Tau2_2_2 0.478727 0.321296 -0.151002 1.108456 1.4900 0.1362

Q statistic on the homogeneity of effect sizes: 49.16779
Degrees of freedom of the Q statistic: 38
P value of the Q statistic: 0.1059594

Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.0561
Intercept2: I2 (Q statistic) 0.4683\

Number of studies (or clusters): 20
Number of observed statistics: 40
Number of estimated parameters: 5
Degrees of freedom: 35
-2 log likelihood: 64.06444
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

Then, I wanted to know the correlation between the effects on CR and SR, but I found a strange 2.47.

cov2cor(vec2symMat(coef(msem, select="random")))

To tackle this problem I set a higher lower bound for the between study heterogeneity, but it doesn't change too much. How should I interpret this and where can I find the source of this "problem"?

msem <- metaSEM::meta(y=cbind(EstCR, EstSR), v=cbind(varCR, Cov, varSR), data=dat, RE.lbound = 1e-1)
summary(msem)

Call: metaSEM::meta(y = cbind(EstCR, EstSR), v = cbind(varCR, Cov, varSR), data = dat, RE.lbound = 0.1)

95% confidence intervals: z statistic approximation (robust=FALSE)
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 3.01466 0.12399 2.77165 3.25767 24.3145 <2e-16 ***
Intercept2 3.15429 0.25281 2.65879 3.64978 12.4770 <2e-16 ***
Tau2_1_1 0.10000 0.14807 -0.19021 0.39021 0.6754 0.4994
Tau2_2_1 0.33946 0.26822 -0.18624 0.86517 1.2656 0.2057
Tau2_2_2 0.73458 0.54414 -0.33191 1.80108 1.3500 0.1770

Q statistic on the homogeneity of effect sizes: 49.16779
Degrees of freedom of the Q statistic: 38
P value of the Q statistic: 0.1059594

Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.3251
Intercept2: I2 (Q statistic) 0.5748

Number of studies (or clusters): 20
Number of observed statistics: 40
Number of estimated parameters: 5
Degrees of freedom: 35
-2 log likelihood: 65.08753
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)

cov2cor(vec2symMat(coef(msem, select="random")))

[,1] [,2]
[1,] 1.000000 1.252487
[2,] 1.252487 1.000000

0

There are 0 best solutions below