Permutation test for the structure of a covariance matrix

Abstract

Scope and Method of Study: Many statistical procedures, such as repeated measures analysis, time-series, structural equation modeling, and factor analysis, require an assessment of the structure of the underlying covariance matrix. The classical parametric method of testing such a hypothesis involves the use of a likelihood ratio test (LRT). These tests have many limitations, including the need for very large sample sizes and the requirement of a random sample from a multivariate normal population. The LRT is also undefined for cases in which the sample size is not greater than the number of repeated measures. In such situations, researchers could benefit from a non-parametric testing procedure. In particular, permutation tests have no distributional assumptions and do not require random samples of any particular size. This research involves the development and analysis of a permutation/randomization test for the structure of a covariance matrix. Samples of various sizes and number of measures on each subject were simulated from multiple distributions. In each case, the type I error rates and power were examined.

Findings and Conclusions: When testing for sphericity, compound symmetry, type H structure, and serial correlation, the LRT clearly performs best with regard to type I error rates for normally distributed data, but for uniform data, it is too conservative, and for double exponential data, it results in extremely large type I error rates. The randomization test, however, is consistent regardless of the data distribution and performs better than the LRT, in most cases, for non-normally distributed data. In most situations, the LRT is more powerful than the randomization test, but the power of the randomization test is comparable to that of the LRT in many situations.