New to Bayesian stats, using ordered categorical data, how to interpret/check model?

98 Views Asked by At

I am trying to fit a model to my ordered categorical data of rating (1,2,3) and group (1,2,3). Other predictors are age (18-23, 24-30), continent, and use of medication (y/n). I have been using the helpful tutorial: https://journals.sagepub.com/doi/full/10.1177/2515245918823199 to guide me through my first Bayesian regression but I am hoping someone can help explain how to ‘check’ the models?

I ran the following models:

modelall<- brm(formula = rating ~ 1 + cs(group) + medication + continent +agequartiles,
data = df,
family = acat(“cloglog”))

modelagroup<- brm(formula = rating  ~ 1 + cs(group),
data = df,
family = acat("cloglog"))

modelage<- brm(formula = rating  ~ 1 + cs(group)  + agequartiles,
data = df,
family = acat("cloglog"))

modelmeds<- brm(formula = rating  ~ 1 + cs(group)  + medication,
data = df,
family = acat("cloglog"))

I'm so used to working with P values I'm not actually sure how to interpret my output

> summary(model group)
 Family: acat 
  Links: mu = cloglog; disc = identity 
Formula: rating ~ 1 + cs(group) 
   Data: df (Number of observations: 2637) 
Samples: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup samples = 4000

Population-Level Effects: 
               Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept[1]       2.81      0.17     2.49     3.14 1.01      697     1156
Intercept[2]      -0.56      0.09    -0.73    -0.40 1.01      682     1341
group2[1]          0.81      0.20     0.42     1.20 1.01      765     1399
group2[2]         -0.47      0.12    -0.70    -0.24 1.01      855     1508
group3[1]          0.83      0.20     0.45     1.23 1.01      684     1328
group3[2]         -0.45      0.12    -0.68    -0.22 1.01      805     1533

Family Specific Parameters: 
     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
disc     1.00      0.00     1.00     1.00 1.00     4000     4000

Samples were drawn using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

I would be so grateful if someone can help, from my understanding That should be under 1.1 and the ESS as high as possible. As not everyone in my sample consented to answer each question, I cannot seem to do a LOO as they have different numbers of observations. I’m just not sure how to work out what should go in the model, other than looking at whether CI cross 0 or not. How can I check if my model is working?

Thank you

0

There are 0 best solutions below