Investigating the (Neglected) Role of Personality in Testing

Institutional accountability assessments are common in higher education, and most have no personal consequences for students. Importantly, research has shown that in low-stakes testing environments, test-taking motivation is related to test performance (i.e., lower motivation is associated with lower test performance — e.g., Wolf & Smith, 1995). Liu, Bridgeman, and Adler’s (2012) recent work showed that the addition of personal consequences in a low-stakes testing environment increased both students’ test-taking motivation and student’s test performance. Students were told their scores were used in one of three ways: for research purposes only; in aggregate for institutional accountability purposes for potential employers; or for faculty and employers who would be shown individual scores. Students in the latter two conditions had higher motivation and test scores than students in the research purposes condition.  Liu et al. did not consider the impact of personality — specifically, conscientiousness, which has been related positively to academic performance (see O’Connor & Paunonen, 2007, and Poropat, 2009, for meta-analyses). Thus, we considered how test consequences, test-taking effort, and personality combine to predict test performance.

Like Liu, Bridgeman, and Adler, we designed testing conditions of increasing personal consequences. In our lowest consequence Control condition, students were told scores would be aggregated for institutional decisions, a realistic accountability testing scenario when compared to research-only use. The second condition increased the personal consequences slightly: Students were given the Control condition instructions and told that they would be able to receive score feedback. Finally, the third condition received all previous instructions plus notification that their personal scores would be shared with their faculty.

Neither test-taking effort nor performance differed across conditions (F (2) = 0.08, p = .923 and F (2) = 1.96, p = .142, respectively). Furthermore, a regression analysis predicting performance from effort, condition, conscientiousness, and all interactions among these three variables uncovered effort as the only significant predictor. Conscientiousness did not predict performance, which was reassuring: If personality factors influenced assessment of ability, validity of inferences from the test scores would be impacted.

Interestingly, our results contradicted the conclusions of Liu, Bridgeman, and Adler (2012) that motivation and performance differ across consequential conditions. However, our findings do align with theirs in that there was no difference between conditions where scores were used for purposes other than research. This is reassuring news with respect to the validity of inferences from test scores; we have increased confidence in the inferences made from scores obtained during low-stakes testing.

Matthew S. Swain, Laura M. Williams, Devon N. Hopkins, Donna L. Sundre & Sara J. Finney
James Madison University