Using prior information to compare dissimilar groups

Bayes' theorem (Photo: Matt Buck, Wikimedia Commons)

How can you compare the development of an unusual group with that of the norm group?

In statistics, the more complex the model, the larger the sample needs to be in order to obtain effective estimates and reach statistically significant results. One example of a complex model is a latent growth model (LGM): a model for estimating growth that can also be used to compare groups. For research questions that fit this model it is not always possible to find sufficient numbers of participants. Sometimes the population is limited or there are ethical considerations that restrict the sample. For example: if you wish to compare the development of people with a rare illness with the development of people who do not have the illness. Or: if you want to compare the development of under-age girls who have committed murder with girls who have committed other crimes.

Bayesian statistics

Research has shown that a special branch of statistics, Bayesian statistics, sometimes offers greater options than conventional statistics. In Bayesian statistics, researchers involve prior information in the research.

If the sample size is small (for example for ethical reasons), Bayesian statistics can provide a solution.

Specific prior information

But what exactly are the limits of conventional statistics for comparisons between dissimilar groups? And how far can you go with Bayesian statistics?

To answer these questions, M&S researchers have created 1,000 datasets. They based these on values determined in advance, for example a minor difference in growth between the groups. They then checked how effectively the different statistical methods were able to identify the value and statistical significance of this effect. What did this reveal?

  • Conventional statistics regularly resulted in impossible values.
  • Both statistical methods had particular difficulty in demonstrating significance. Only if very specific prior information is involved in the Bayesian statistics, was the minor effect found to be statistically significant sufficiently frequently. 

Prior information can make an important contribution to statistical analysis, but applied research shows that this can sometimes be difficult to identify. In addition, it is important to understand the effect that the prior information has on the results. A major difference in prior information and data can justify conducting further research.