en:statpqpl:norm:normmultipl

Multivariate normality tests

Many methods of multivariate analysis, including MANOVA, Hotelling tests, or regression models are based on the assumption of multivariate normality. If a set of variables is characterized by a multivariate normal distribution, then each variable can be assumed to have a normal distribution. However, when all individual variables are characterized by a normal distribution, their set does not have to have a multivariate normal distribution. Therefore, testing the unidimensional normality of each variable may be helpful, but cannot be assumed to be sufficient.

Different types of statistical analyses that assume normality are sensitive to different degrees to different types of departure from this assumption. Tests that refer to means in their hypotheses are generally taken as more sensitive to skewness, while tests comparing covariances depend more heavily on kurtosis.

The window with the test of multivariate normality of distribution settings is opened via Statistics→Normality tests→Multi-dimensional normality.

Mardia's test for multivariate normality

The test proposed by Mardia in 1970 ¹⁾ and modified in 1974 ²⁾ tests the normality of a distribution by analyzing separately the magnitude of multivariate skewness and multivariate kurtosis. Jarque and Bera ³⁾ proposed combining these two Mardia measures into a single test. A similar way of combining skewness and kurtosis information into a single test is provided by the method of Hanusz and Tarasinska ⁴⁾.

Mardia defined multivariate skewness and kurtosis as follows: $\begin{displaymath} skew=\frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^nm_{ij}^3, \quad kurt=\frac{1}{n}\sum_{i=1}^nm_{ii}^2 \end{displaymath}$

where

$m_{ij}=\left(X_i-\bar{X}\right)^TS^{-1}\left(X_j-\bar{X}\right)$ ,

$S=\frac{1}{n}\sum_{j=1}^n\left(X_i-\bar{X}\right)\left(X_i-\bar{X}\right)^T$ ,

$\bar{X}$ -mean, $S$ - covariance matrix.

For data derived from a sample rather than a population, the formulas for skewness and kurtosis are multiplied respectively: skewness by $\left(n/(n-1)\right)^3$ and kurtosis by $\left(n/(n-1)\right)^2$ .

Hypotheses:

$\begin{array}{cl} \mathcal{H}_0: & $population data distribution $=$ multivariate normal distribution$,\\ \mathcal{H}_1: & $population data distribution $\neq $ multivariate normal distribution,$ \end{array}$

Mardia test of skewness: When the sample is drawn from a population with a multivariate normal distribution (null hypothesis), the test statistic is in the form of (Mardia, 1970):

$\begin{displaymath} \chi^2(M)=\frac{n}{6}skew \end{displaymath}$

or with correction of exact moments for groups with smaller numbers (<20) (Mardia, 1974):

$\begin{displaymath} \chi_c^2(M)=\frac{n}{6}\frac{(n+1)(n+3)(k+1)}{n((n+1)(k+1)-6)}skew \end{displaymath}$

This statistic has asymptotically (for large numbers) Chi-square distribution with $df=f=\frac{k(k+1)(k+2)}{6}$ degrees of freedom.

Mardia test of kurtosis: When the sample is drawn from a population with a multivariate normal distribution (null hypothesis), the test statistic is in the form of (Mardia, 1974):

$\begin{displaymath} Z(M)=\frac{kurt-k(k+2)}{\sqrt{\frac{8k(k+2)}{n}}} \end{displaymath}$

or with correction (Mardia, 1974)

$\begin{displaymath} Z_c(M)=\frac{(n+1)kurt-k(k+2)(n-1)}{\sqrt{\frac{8k(k+2)(n-3)(n-k-1)(n-k+1)}{(n+3)(n+5)}}} \end{displaymath}$

This statistic has asymptotically (for large numbers) normal distribution.

The p-value, designated on the basis of the test statistic, for both tests i.e. of the skewness test and the kurtosis test, are compared with the significance level $\alpha$ :

$\begin{array}{ccl} $ if $ p \le \alpha & \Longrightarrow & $ reject $ \mathcal{H}_0 $ and accept $ \mathcal{H}_1, \\ $ if $ p > \alpha & \Longrightarrow & $ there is no reason to reject $ \mathcal{H}_0. \\ \end{array}$

Jarque-Bera test for multivariate normality

Jarque and Bera's (1987) ⁵⁾ test is based on the skewness and kurtosis statistics of the Mardia test. The test statistic is in the form of:

$\begin{displaymath} \chi^2(JB)=\chi^2(M)+(Z(M))^2 \end{displaymath}$

or with correction (Mardia, 1974)

$\begin{displaymath} \chi_c^2(JB)=\chi_c^2(M)+(Z_c(M))^2 \end{displaymath}$

This statistic has asymptotically (for large numbers) Chi-square distribution with $df=f+1$ degrees of freedom.

The p-value, designated on the basis of the test statistic, is compared with the significance level $\alpha$ :

Hanusz-Tarasinska test for multivariate normality

Zofia Hanusz and Joanna Tarasinska's (2014) ⁶⁾ test is based on the skewness and kurtosis statistics of the Mardia test. The test statistic is in the form of:

$\begin{displaymath} t_c(HT)=\frac{Z_c(M)}{\sqrt{\frac{\chi_c^2(M)}{f}}} \end{displaymath}$

The test statistic has t-Student distribution with $df=f$ degrees of freedom.

The p-value, designated on the basis of the test statistic, is compared with the significance level $\alpha$ :

Henze-Zirkler test for multivariate normality

Henze and Zirkler (1990) ⁷⁾ proposed a test to examine multivariate normality of the distribution extending the work of Baringhaus and Henze on the empirical characteristic function ⁸⁾. In the literature, this test is considered one of the strongest tests dedicated to multivariate normal distribution (Thode 2002) ⁹⁾. The test statistic has the form:

$\begin{displaymath} Z(HZ)_{\beta}=n\left(4I_E+D_{n,\beta}I_{E^c}\right) \end{displaymath}$

$I_E$ and $I_{E^c}$ are indicator functions that depend on the singularities of the covariance matrix,

$D_{n,\beta}=\frac{1}{n^2}\sum exp\left(\frac{-\beta^2||Y_j-Y_k||^2}{2}\right)+(1+2\beta^2)^{-p/2}-2(1+\beta^2)^{-p/2}\sum exp\left(\frac{-\beta^2||Y_j||^2}{2(1+\beta^2)}\right)$

$Y_i=S^{1/2}(X_i-\bar{X})$

$\beta*=2^{-1/2}\left(\frac{n(2k+1)}{4}\right)^{1/(k+4)}$ - optimum parameter value $\beta$

The statistic $Z(HZ)_{\beta}$ has an asymptotically (for large sizes) normal distribution based on the mean and variance described by Henze and Zirkler and read one-sided.

The p-value, designated on the basis of the test statistic, is compared with the significance level $\alpha$ :

EXAMPLE (Iris.pqs file)

We examine the normality of the distribution for the classical data set of R.A. Fisher 1936 ¹⁰⁾. The file can be found in the manual included with the program and contains measurements of the length and width of the petals and calyx sepals for 3 varieties of iris flower. The analysis will be performed separately for each variety. In the analysis window, select all tests and the graph, and set a multiple filter to repeat the analysis for each variety of iris. All the results will be returned to the same datasheet, so select Combine into one report.

All tests confirm the normality of the distribution for the versicolor and virginica varieties. For the setosa cultivar, the test results are on the verge of statistical significance, with the Mardia test for Kurtosis and the Henze-Zirkler test indicating deviations from the multivariate normal distribution. We can observe such deviations also in the first graph, where as the Mahalanobis distance increases, the points are further and further from the straight line.

¹⁾

Mardia K. V. (1970), Measures of multivariate skewness and kurtosis with applications, Biometrica 57, 519-530

²⁾

Mardia K. V. (1974), Applications of some measuresof multivariate skewness and kurtosis for testing normality and robustness studies, Sankhay B 36, 115-128

³⁾ , ⁵⁾

Jarque C. M., Bera A. K., (1987)., A test for normality of Observations and Regression Residuals, International Statistical Review 55, 163-172

⁴⁾ , ⁶⁾

Hanusz Z., Tarasińska J. (2014), On multivariate normality tests using skewness and kurtosis, Colloquium Biometricum 44, 139-148

⁷⁾

Henze N., Zirkler B. (1990), A class of invariant consistent tests for multivariate normality. Comm. Statist. Theory Methods. 1990;19:3595–3617

⁸⁾

Epps T.W., Pulley L.B. (1983), A test for normality based on the empirical characteristic function. Biometrika. 1983;70:723–726

⁹⁾

Thode H. C. (2002), Testing For Normality. CRC Press; 2002. 506 s.

¹⁰⁾

Fisher R.A. (1936), The use of multiple measurements in taxonomic problems. Annals of Eugenics 7 (2): 179–188