Robust Use of External Information in Statistical Analysis

Abstract

Scientific research involves different processes, including the accumulation, aggregation, and cumulation of knowledge. The latter is construed by using existing theories and empirical findings to obtain new results based on previous ones in further research. In psychology, the accumulation and aggregation of knowledge is employed in everyday research. This thesis sheds light on the cumulation of knowledge in psychology by focusing on statistical cumulation – the use of quantitative external information in statistical analyses. To prevent new results from being biased by misspecified external information, the uncertainties of the external information should be considered. This includes estimation and structural uncertainty, as external quantities are estimates and there are often structural differences between a new data set and external sources, such as different designs or populations. Furthermore, this thesis discusses a wide range of approaches for incorporating external information and considers how well they reflect present uncertainties. Previous approaches include generalized Bayesian analyses and inferential models that incorporate partial prior information about a parameter of interest. Careful consideration of previous approaches indicates that a frequentist approach had not been developed for this purpose in psychology. To address this issue, this thesis introduces an externally informed generalized method of moments approach, which was developed and outlined across the four attached papers. Within this novel approach, two uses of external information are possible: improving the statistical analysis of new data and testing the fit of external information and data to indicate structural differences between them. Furthermore, this approach can incorporate external information about variables in the form of statistical moment equations. Thus, it is a relevant addition to existing methods as generalized Bayesian and inferential model approaches have difficulty incorporating this type of external information. The main focus of the four attached papers was on the application of the externally informed generalized method of moments approach to multiple linear and repeated measures generalized linear models. Additionally, this PhD project provides software that allows applied researchers to use the developed approach with various models, such as multiple linear models, repeated measures generalized linear models, two-level mixed linear models, and structural equation models.