Different Paths, same destination? Comparison between two approaches to developing situational judgment tests on cross cultural competency
Abstract
This dissertation focuses on developing SJTs to measure an individual’s cross-cultural
competency, and comparing the two SJT development approaches in terms of development
costs, reliability, validity, susceptibility to social desirability, and test-taker reactions. In
the first phase, the two 3C SJTs were developed with the model-based approach and the
SME-driven approach respectively. In the second phase, data were collected to examine the
reliability and validity of the two SJTs. Both 3C SJTs demonstrated acceptable reliability
(αSME = .72; αmodel =.70), and convergent to CQS (rSME = .35, p < .01; rmodel = . 24, p < .01).
The SJTs psychometric properties were further examined in the third phase, wherein the
SJTs displayed similar reliability and were convergent to CQS. Both SJTs predicted
satisfaction with overseas life (βSME = .24, p < .01; βmodel = .18, p < .05) and sociocultural
adaptability (βSME = -.20, p < .05; βmodel = -.21, p < .05), meanwhile, only having none or
small correlation with satisfaction with general life (rSME = .10, n.s. and rmodel = .19, p <
.05). The SME-driven SJT outperformed the model-based SJT and CQS in predicting the
actual multicultural team performance that was rated by peers (βSME = .26, p < .05; βmodel = -
.04, n.s.; βCQs = .01, n.s.). The utility of the two SJT development approaches, implications,
future research directions and limitations were discussed in the end.