Review of Aviation Research: A Content Analysis of Articles Published in the Collegiate Aviation Review, 2007–2012
The purpose of the study was to conduct a content analysis of the methodological quality of articles published in the Collegiate Aviation Review (CAR)— a refereed journal of the University Aviation Alliance (UAA)—to determine if actual practices of aviation researchers were consistent with commonly recommended research methods and procedures. The accessible population consisted of the 76 articles published in CAR between 2007 and 2012 (Volumes 25–30). The sample consisted of N = 69 articles and excluded literature reviews, meta-analyses, studies that described the development or validation of an instrument, philosophical inquiries, position papers, or historical studies. The analysis focused on the fundamentals of research principles, measurement, and data analysis procedures including the extent to which authors gave attention to describing: purpose statements and research questions; sampling issues such the target and accessible populations, sampling strategy, sample representativeness, and sample size determination; instrumentation validity and reliability; research methodology and design; threats to internal validity; data analysis procedures; conclusions and recommendations; and limitations and delimitations. Using a coding form with a set of predetermined categories that corresponded to these methodological issues, two coders coded the articles independently and interrater reliability was established using percent agreement. Major findings included the following: (a) the majority of articles contained a purpose statement, but half did not include corresponding research questions; (b) about half of the articles did not contain information about the population, the most frequently used sampling strategy was convenience, more than half of the articles did not describe the sample, and nearly 90% of the articles did not address sample representativeness; (c) two-thirds of the articles did not give attention to instrumentation validity and reliability; (d) survey was the most common research methodology; (e) nearly 90% of the articles did not discuss at least one internal validity threat; (f) the most commonly used statistical procedures were descriptive; (g) only 13% of the articles gave attention to population generalizability; (h) 90% of the articles expressed conclusions by restating the study’s findings; and (i) two-thirds of the articles did not specify any study limitations or delimitations. The findings indicate that the methodological quality of articles published in CAR for the targeted 6-year period should be of concern to the aviation research community, particularly to the editors, authors, and readers of CAR. The lack of thoroughness with respect to methodological quality affects both generalizability and replication studies. The reader is cautioned not to overgeneralize the findings because they apply only to the targeted articles of CAR published in 2007–2012.