Accommodating Method Effects in Testing Construct Validity Through Confirmatory Factor Analysis
Mengakomodasi Efek Metode dalam Pengujian Validitas Konstruk Melalui Analisis Faktor Konfirmatori
-
(1) * Wahyu Widhiarso
Universitas Gadjah Mada
Indonesia
(*) Corresponding Author
Abstract
Literatures in the field of psychometrics recommend researchers to employvarious of methods on measuring individual attributes. Ideally,each methods are complementary and measuresthe construct designed to be measured. However, some problems arise when among the methods is unique and unrelated to the construct being measured. The uniqueness of method can lead what is called the method effect. In testing construct validity using confirmatory factor analysis, the emergence of this effect tend to reducing the goodness of fit indices of the model. There are many ways to solve these problem, one of themis controling the method effects and accommodate it to the model. This paper introduces how to accommodate method effecton the confirmatory factor analysis using structural equation modeling. In the application section, author identify the emergence of method effects due to the differences item writing direction (favorable-unfavorable). The analysis showed that method effectemerge from different writing direction.
References
Andrews, F.M. (1984). Construct Validity and Error Components of Survey Measures: A Structural Modeling Approach. The Public Opinion Quarterly, 48(2), 409-442. DOI: https://doi.org/10.1086/268840
Arbukle, J.L., & Wothke, W. (1999). AMOS 4.0 User’s Guide. Chichago: Smallwaters Corp. .
Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105. DOI: https://doi.org/10.1037/h0046016
Chen, Y.-H., Gobioff, G.R., & Dedrick, R.F. (2010). Factorial invariance of a chinese self-esteem scale for third and sixth grade students: Evaluating method effects associated with positively and negatively worded items The International Journal of Educational and Psychological Assessment, 6(1), 21-35.
Eid, M., Lischetzke, T., Nussbeck, F.W., & Trierweiler, L.I. (2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8(1), 38-60. DOI: https://doi.org/10.1037/1082-989X.8.1.38
Fiske, D.W. (1982). Convergent–discriminant validation in measurements and research strategies. In D. Brinbirg & L. H. Kidder (Eds.), Forms of validity in research. San Francisco: Jossey-Bass. Geiser, C., Eid, M., & Nussbeck, F.W. (2008). On the meaning of the latent variables in the CT-C(M-1) model: A comment on Maydeu-Olivares and Coffman (2006). Psychological Methods, 13(1), 49-57. DOI: https://doi.org/10.1037/1082-989X.13.1.49
Geiser, C., Eid, M., West, S.G., Lischetzke, T., & Nussbeck, F.W. (2012). A Comparison of Method Effects in Two Confirmatory Factor Models for Structurally Different Methods. Structural Equation Modeling: A Multidisciplinary Journal, 19(3), 409-436. DOI: https://doi.org/10.1080/10705511.2012.687658
Grayson, D., & Marsh, H. (1994). Identification with deficient rank loading matrices in confirmatory factor analysis: Multitrait-multimethod models. Psychometrika, 59(1), 121-134. DOI: https://doi.org/10.1007/BF02294271
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2010). Multivariate Data Analysis. New Jersey: Prentice Hall.
Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: User's Reference Guide. Chicago Scientific Software International. Kenny, D.A. (1976). An empirical application of confirmatory factor analysis to the multitrait-multimethod matrix. Journal of Experimental Social Psychology, 12(3), 247-252. DOI: https://doi.org/10.1016/0022-1031(76)90055-X
Kenny, D.A., & Kashy, D.A. (1992). Analysis of the multitrait-multimethod matrix by confirmatory factor analysis. Psychological Bulletin, 112(1), 165-172. DOI: https://doi.org/10.1037/0033-2909.112.1.165
Kline, R.B. (2011). Principles and Practice of Structural Equation Modeling. New York, NY: Guilford Publications, Inc.
Lance, C.E., Baranik, L.E., Lau, A.R., & A., S.E. (2009). If it ain’t trait it must be method: (mis)application of the multitrait-multimethod design in organizational research. In C. E. Lance & R. L. Vandenberg (Eds.). New York: Routledge
MacKenzie, S.B., & Podsakoff, P.M. (2012). Common Method Bias in Marketing: Causes, Mechanisms, and Procedural Remedies. Journal of Retailing, 88(4), 542-555. DOI: https://doi.org/10.1016/j.jretai.2012.08.001
Marsh, H.W. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors? Journal of Personality and Social Psychology, 70(4), 810-819. DOI: https://doi.org/10.1037/0022-3514.70.4.810
Marsh, H.W., Asci, F.H., & Thomas, I.M. (2002). Multitrait-multimethod analyses of two physical self-concept instruments: A cross-cultural perspective. [Empirical Study]. Journal of Sport & Exercise Psychology, 24(2), 99-119. DOI: https://doi.org/10.1123/jsep.24.2.99
McLaughlin, T.P., Khandker, R.K., Kruzikas, D.T., & Tummala, R. (2006). Overlap of anxiety and depression in a managed care population: prevalence and association with resource utilization. Journal of Clinical Psychiatry 67(8), 1187-1193. DOI: https://doi.org/10.4088/JCP.v67n0803
Muthen, L.K., & Muthen, B.O. (2005). Mplus: Statistical analysis with latent variables: User's guide. Los Angeles, CA.: Muthen & Muthen. Podsakoff, P.M.,
MacKenzie, S.B., & Podsakoff, N.P. (2011). Sources of Method Bias in Social Science Research and Recommendations on How to Control It. Annual Review of Psychology, 63(1), 539-569. DOI: https://doi.org/10.1146/annurev-psych-120710-100452
Pohl, S., & Steyer, R. (2010). Modeling Common Traits and Method Effects in Multitrait-Multimethod Analysis. Multivariate Behavioral Research, 45(1), 45-72. DOI: https://doi.org/10.1080/00273170903504729
Raykov, T., & Marcoulides, G.A. (2006). A first course in structural equation modeling. Mahwah, NJ: Lawrence Erlbaum Associates. Urbina, S. (2004). Essentials of psychological testing. Hoboken, NJ.: John Wiley & Sons, Inc.
Copyright (c) 2016 Wahyu Widhiarso
This work is licensed under a Creative Commons Attribution 4.0 International License.
