How to interpret output "Percent_Sig" in ParcelAllocaiton Function (Sem Tools)?

Question

In the parcel allocation function, there is an output called "Percent_Sig".

I interpreted this value as an averaged p-value acros allocations. (e.g. 1rst allocation p-value 0.56, 2nd allocation p-value 0.34, 3rd allocation p-value 0.54 -> averaged p-value = 0.48)

However in the description of the sem tool package it says, it represents the "proportion of allocations in which each test of fit was significant."

How do I interpet this value then?

For instance, if it is a value of Percent_Sig = 0.48. Okay, I know that in 48 % allocations there was a significant p-value. But when would I say the probability is low enough to say that my chi sqaure value (because low p-values mean a better model fit) is good.

Would be happy about an answer :)

Answer 1

there is an output called "Percent_Sig". I interpreted this value as an averaged p-value acros allocations

Nope.

it represents the "proportion of allocations in which each test of fit was significant."

Yup.

I know that in 48 % allocations there was a significant p-value. But when would I say the probability is low enough to say that my chi square value (because low p-values mean a better model fit) is good.

The proportion significant is not meant to help you test your model, but rather to provide an idea about how uncertain you should be about whether your model would be rejected if you chose a different random-allocation of items to parcels. This is the issue with arbitrary allocations, as discussed in the older papers listed among References on the ?parcelAllocation help page.

To obtain a single test statistic for your model (which appropriately accounts for the uncertainty due to random allocation), as well as tests for individual parameters, you can save the allocations as a list of data sets, then treat them as multiple imputations. This is discussed in the later papers among the References, and demonstrated in the ## POOL RESULTS section of the help-page Examples.