A Comparative Study of the Power of Parametric and Permutation Tests for a Multidimensional Two-sample Location Problem
DOI:
https://doi.org/10.15678/AOC.2020.2304Keywords:
permutation tests, comparing populations, power of test, Monte Carlo simulation, R softwareAbstract
Objective: A comparison of multidimensional populations is a very interesting and common statistical problem. It most often involves verifying a hypothesis about the equality of mean vectors in two populations. The classical test for verification of this hypothesis is the Hotelling’s T2 test. Another solution is to use simulation and randomization methods to test the significance of differences between the studied populations. Permutation tests are to enable statistical inference in situations where it is not possible to use classical parametric tests. These tests are supposed to provide comparable power to parametric tests with a simultaneous reduction of assumptions, e.g. regarding the sample size taken or the distribution of the tested variable in the population. The purpose of this study is a comparative analysis of the parametric test, the (usual) permutation test, and the nonparametric permutation procedure using two-stage ASL determination.
Research Design & Methods: The study considered the analysis of multivariate data. The paper presents theoretical considerations and refers to the Monte Carlo simulation.
Findings: The article presents a permutational, complex procedure for assessing the overall ASL (achieved significance level) value. The applied nonparametric statistical inference procedure uses combining functions. A simulation study was carried out to determine the size and power of the test under normality. A Monte Carlo simulation made it possible to compare the empirical power of this test with that of Hotelling’s T2 test. The most powerful test was the permutation test based on a two-stage ASL determination method using the Fisher combining function.
Implications/Recommendations: The advantage of the proposed method is that it can be used even when samples are taken from any type of continuous distributions in a population.
Contribution: The proposed test can be used in the analysis of multidimensional economic phenomena.
References
Anderson, M. J., Walsh, D. C. I., Clarke, K. R., Gorley, R. N., Guerra-Castro, E. (2017) “Permutational Multivariate Analysis of Variance (PERMANOVA)”. Statistics Reference Online: 1–15, https://doi.org/10.1002/9781118445112.stat07841.
Chang, C. H., Pal, N. (2008) “A Revisit to the Behrens-Fisher Problem: Comparison of Five Test Methods”. Communications in Statistics – Simulation and Computation 37(6): 1064–85, https://doi.org/10.1080/03610910802049599.
Fisher, R. A. (1932) Statistical Methods for Research Workers. 4 ed. Edinburgh: Oliver & Boyd.
Hotelling, H. (1931) “The Generalization of Student’s Ratio”. Annals of Mathematical Statistics 2(3): 360–78, https://doi.org/10.1214/aoms/1177732979.
Hotelling, H. (1947) “Multivariate Quality Control” in C. Eisenhart, M. W. Hastay, W. A. Wallis (eds) Techniques of Statistical Analysis. New York: McGraw-Hill.
Janssen, A., Pauls, T. (2005) “A Monte Carlo Comparison of Studentized Bootstrap and Bermutation Tests for Heteroscedastic Two-sample Problems”. Computational Statistics 20(3): 369–83, https://doi.org/10.1007/bf02741303.
Kończak, G. (2016), Testy permutacyjne. Teoria i zastosowania. Katowice: Wydawnictwo Uniwersytetu Ekonomicznego w Katowicach.
Krzyśko, M. (2009) Podstawy wielowymiarowego wnioskowania statystycznego. Poznań: Wydawnictwo Naukowe UMA.
Liptak, I. (1958) “On the Combination of Independent Tests”. Magyar Tudomanyos Akademia Matematikai Kutato Intezenek Kozlomenyei 3: 127–41.
Mahalanobis, P. C. (1930) “On Tests and Measures of Group Divergence”. Journal of the Asiatic Society of Bengal 26: 541–88.
Mahalanobis, P. C. (1936) “On the Generalized Distance in Statistics”. Proceedings of the National Institute of Science of India 12: 49–55, https://doi.org/10.1007/s13171-019-00164-5.
Marozzi, M. (2008) “The Lepage Location-scale Test Revisited”. Far East Journal of Theoretical Statistics 24: 137–55.
Pesarin, F. (2001) Multivariate Permutation Test with Applications in Biostatistics. Chichester: Wiley.
R Core Team (2016) R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing, https://www.R-project.org/.
Rencher, A. C. (2002) Methods of Multivariate Analysis. New York: John Wiley & Sons.
Tippett, L. H. C. (1931) The Methods of Statistics. London: Williams and Norgate.
