Interrater agreement and CI calculation

Question

I have the following dataset

df <- data.frame(Nurse=c(1,2,4,3,5,4,3,2,4,3), Patient=c(2,3,5,4,4,4,3,3,2,3))

where for each row both a nurse rates and the patient self-rates the severity of their illness (from 1-5). Each row represents a different patient and different event.

How would I calculate kappa interrate agreement between nurse and patient and calculate associated CIs?

I have trialled

kappa2(df,weight = "equal")

but does not provide confidence intervals.

Answer 1

Try using the psych library, which gives the CI:

df <- data.frame(Nurse=c(1,2,4,3,5,4,3,2,4,3), Patient=c(2,3,5,4,4,4,3,3,2,3))

psych::cohen.kappa(df)

# Call: cohen.kappa1(x = x, w = w, n.obs = n.obs, alpha = alpha, levels = levels)
# 
# Cohen Kappa and Weighted Kappa correlation coefficients and confidence boundaries 
#                   lower estimate upper
# unweighted kappa -0.27    0.054  0.38
# weighted kappa    0.16    0.533  0.90
# 
# Number of subjects = 10