Different Paths, same destination? Comparison between two approaches to developing situational judgment tests on cross cultural competency

Abstract

This dissertation focuses on developing SJTs to measure an individual’s cross-cultural competency, and comparing the two SJT development approaches in terms of development costs, reliability, validity, susceptibility to social desirability, and test-taker reactions. In the first phase, the two 3C SJTs were developed with the model-based approach and the SME-driven approach respectively. In the second phase, data were collected to examine the reliability and validity of the two SJTs. Both 3C SJTs demonstrated acceptable reliability (αSME = .72; αmodel =.70), and convergent to CQS (rSME = .35, p < .01; rmodel = . 24, p < .01). The SJTs psychometric properties were further examined in the third phase, wherein the SJTs displayed similar reliability and were convergent to CQS. Both SJTs predicted satisfaction with overseas life (βSME = .24, p < .01; βmodel = .18, p < .05) and sociocultural adaptability (βSME = -.20, p < .05; βmodel = -.21, p < .05), meanwhile, only having none or small correlation with satisfaction with general life (rSME = .10, n.s. and rmodel = .19, p < .05). The SME-driven SJT outperformed the model-based SJT and CQS in predicting the actual multicultural team performance that was rated by peers (βSME = .26, p < .05; βmodel = - .04, n.s.; βCQs = .01, n.s.). The utility of the two SJT development approaches, implications, future research directions and limitations were discussed in the end.