E-Screening Children Mental Health Problems: Psychometric Evaluation of Parent and Teacher Scales

Akbari-Zardkhaneh, Saeed; Daneshmand Kafteroudi, Amir Hossein; Zanganeh, Alimohammad; Mansurkiaie, Nader; Tahmasbi Garmtani, Siyamak; Mohsen, Jallalat-Danesh; Teymourifard, Seyed Einollah; Raoof, Kosar

doi:10.48308/apsy.2024.234689.1615

E-Screening Children Mental Health Problems: Psychometric Evaluation of Parent and Teacher Scales

Document Type : Original Article

Authors

¹ Assistant Professor, Department of Educational and Developmental Psychology, University of Shahid Beheshti, Tehran, Iran.

² M.A. Student of Educational Psychology, University of Shahid Beheshti, Tehran, Iran

³ Ph.D. in Management, Physician, Social and Crime Prevention Directorate Judiciary, Tehran, Iran.

⁴ Ph.D. in Communication Sciences, Department of Communication Studies, Faculty of Communication Sciences, Allameh Tabatabai University, Tehran, Iran

⁵ Ph.D. in Clinical Psychology, Department of Preschool Education, university of social welfare and rehabilitation sciences, Tehran, Iran

⁶ MSc., Social and Crime Prevention Directorate Judiciary, Tehran, Iran

⁷ Ph.D. in educational Psychoogy, Minesrty of Education, Tehran, Iran.

⁸ Ph.D. Student of Educational Psychology, University of Shahid Beheshti, Tehran, Iran.

10.48308/apsy.2024.234689.1615

Abstract

Aim: Given the importance of screening children’s psychological problems, the precision of multi-gate tools, and the need for fast and accurate data collection in the virtual realm, this study aimed to develop an electronic instrument for psychological screening of children, in both teacher and parent report formats.

Method: The study involved parents and teachers of elementary students from 31 provinces across the country. From each province, two cities were selected, and from each city, two schools were purposefully chosen. The sample included 2,420 parents and 6,972 teachers. The conceptual model for both instruments encompassed eight constructs: anxiety, depression, hyperactivity and inattention, bullying behaviors, academic and social impairments, self-regulation, self-harm, and risk of child abuse.

Results: Item analysis showed that 139 items in the parent version and 101 items in the teacher version demonstrated satisfactory adequacy. Exploratory and confirmatory factor analyses supported an eight-factor structure for the parent version and a six-factor structure for the teacher version. In the teacher version, two combined components emerged: aggressive impulsive behaviors with inattention and hyperactivity, and self-injurious behaviors with child abuse. Coefficient alphas for internal consistency ranged from 0.73 to 0.97, ordinal alphas from 0.86 to 0.98, and omegas from 0.75 to 0.97.

Conclusion: The electronic parent and teacher report versions demonstrated appropriate structural validity and reliability, making them suitable for both clinical and research applications.

Keywords

Main Subjects

Clinical Psychology

References

Achenbach, T. M., Rescorla, L. A., & Ivanova, M. Y. (2012). International epidemiology of child and adolescent psychopathology I: Diagnoses, dimensions, and conceptual issues. Journal of the American Academy of Child and Adolescent Psychiatry, 51(12), 1261–1272.[link]

Akbari-Zardkhaneh, S., Alebuieh, M., Zanganeh, A., Mansurkiaie, N., Jallalat-Danesh, M., & Mahdavi, M. (2018). Parent Form of Psychological Pathology for Children: Preliminary Study of Development and Psychomentric Properties [Research]. Quarterly Journal of Child Mental Health, 4(4), 152-164. [persian][link]

Akbari Zardkhaneh, S., Zanganeh, A.-M., Mansour Kiaee, N., Mahdavi, M., Alebuieh, M., Jallalat-e-Danesh, M., Taymouri, E., & Tahmasebi Garmtani, S. (2019). Priliminary Version of Psychological Problems Scale- Teacher Form: Development and Psychometric Evaluation [Applicable]. Quarterly Journal of Child Mental Health, 6(1), 1-13. [persian][link]

Akbari-Zardkhaneh, S., Shahvarani, S. M., Mansurkiaie, N., Zanganeh, A., Jallalat-Danesh, M., Tahmasebi, S., & Zareian, M. (2018). Psychometric Properties of Electronic Mental Health Assessment Device for High School Students: Teacher and Parent Form. Iranian Journal of Psychiatry and Clinical Psychology, 24(3), 324-339. [persian][link]

Akbari-Zardkhaneh, S., Soleimani, Z., Zanganeh, A. M., Mansour Kiaei, N., Mostafavi, M., Taimouri, A. A., Jallatdanesh, M., & Tahmasabi, S. (1400). Short-electronic scale of mental health problems of primary school students - teacher's version. Educational Measurement and Evaluation Studies, 33(11), 23-40. [persian][link]

Armor, D. J. (1971). Theta Reliability And Factor Scaling. Sociological Methodology, 5, 17-50.[link]

Aronen, E.T., Teerikangas, O. M., & Kurkela, S.A. (1999). The continuity of psychiatric symptoms from adolescence into young adulthood. Nordic Journal of Psychiatry, 53(5), 333-338.[link]

Atladottir, H. O., Gyllenberg, D., Langridge, A., Sandin, S., Hansen, S. N., Leonard, H., ...., & Parner, E. T. (2014). The increasing prevalence of reported diagnoses of childhood psychiatric disorders: A descriptive multinational comparison. European Child & Adolescent Psychiatry, 24 (2), 173–183.[link]

Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Psychology, 3(2), 77–85.[link]

Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606. [link]

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246. [link]

Booth-Kewley, S., Larson, G. E., & Miyoshi, D. K. (2007). Social desirability effects on computerized and paper-and-pencil questionnaires. Comput Human Behav, 23(1), 463-477. [link]

Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford.

Carlbring, P., Brunt, S., Bohman, S., Austin, D., Richards, J., öst, L-G., & Andersson, G. (2007). Internet vs. paper and pencil administration of questionnaires commonlyused in panic/agoraphobia research. Computers in Human Behavior, 23(3), 1421–1434. [link]

Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276. [link]

Cronbach, L. J. (1951). Coefficient Alpha And The Internal Structure Of Tests. Psychometrika, 16 (3), 297-334. [link]

DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Sage.

Di Girolamo, M., Giromini, L., Winters, C. L., Serie, C. M. B., & de Ruiter, C. (2019). The Questionnaire of Cognitive and Affective Empathy: A comparison between paper-and-pencil versus online formats in Italian samples. Journal of Personality Assessment, 101(2), 159–170. [link]

Eid, Michael & Geiser, Christian & Koch, Tobias & Heene, Moritz. (2016). Anomalous Results in G-Factor Models: Explanations and Alternatives. Psychological Methods. 22. 10.1037/met0000083. [link]

Flora, D. B., & Flake, J. K. (2017). The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioural Science / Revue canadienne des sciences du comportement, 49(2), 78–88. [link]

Giordano, Casey & Waller, Niels. (2019). Recovering bifactor models: A comparison of seven methods. Psychological Methods. 25. 10.1037/met0000227. [link]

Golino, H. F., & Epskamp, S. (2017). Exploratory graph analysis: A new approach for estimating the number of dimensions in psychological research. PlosOne, 12(6), 1–26. [link]

Gorsuch, R. L. (1983). Factor analysis (2nd ed.). Erlbaum.

Gunter, B., Nicholas, D., Huntington, P., & Williams, P. (2002). Online versus offline research: implications for evaluating digital media. Aslib Proceedings, 54(4), 229-239. [link]

Hays, R. D., Liu, H., & Kapteyn, A. (2015). Use of Internet panels to conduct surveys. Behavior Research Methods, 47(3), 685–690. [link]

Hirai, M., Vernon, L. L., & Clum, G. A. (2018). Factor structure and administration measurement invariance of the beliefs toward mental illness scale in Latino college samples: Paper–pencil versus internet administrations. Assessment, 25(6), 759–768. [link]

Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis. Psychometrika, 30(2), 179–185. [link]

Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453. [link]

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. [link]

Jamali, H. R., Sarafzadeh, M., and Asadi, S. (1387). Article: Electronic navigation. Monthly Book of Social Sciences, -12(12), 78-88. [persian] [link]

Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36. [link]

Lorenzo-Seva, U., & Ferrando, P. J. (2021). MSA: The Forgotten Index for Identifying Inappropriate Items Before Computing Exploratory Item Factor Analysis. Methodology, 17(4), 296-306. [link]

Masten, A. S., & Coatsworth, J. D. (1998). The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist, 53(2), 205-220. [link]

McCoy, S., & Marks, P. (2001). Using electronic surveys to collect data: experiences from the field. Seventh Americas Conference on Information Systems, Boston, USA.

McMahon, S. R., Iwamoto, M., Massoudi, M. S., Yusuf, H. R., Stevenson, J. M., Chu, S. Y., & Pickering, L. K. (2003). Comparison of E-mail, Fax, and Postal Surveys of Pediatricians. Pediatrics, 111(4), 299-303. [link]

Mitchell, M. L., & Jolley, J. M. (2012). Research Design explained. Cengage Learning.

Moses, T. (2017). A review of developments and applications in item analysis. Advancing Human Assessment, 19-46. [link]

Mvududu, N. H., & Sink, C. A. (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4(2), 75–98. [link]

Noyes, J. M., & J, Garland, K. (2008). Computer- vs. paper-based tasks: Are they equivalent? Ergonomics, 51(9), 1352–1375. [link]

Orçan, F., (2018). Exploratory and confirmatory factor analysis: which one to use first?. Journal of Measurement and Evaluation in Education and Psychology, 9(4), 414-421. [link]

Polanczyk, G. V., Salum, G. A., Sugaya, L. S., Caye, A., & Rohde, L. A. (2015). Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, 56(3), 345-65. [link]

R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

Revelle, W. (2021). psych: Procedures for Personality and Psychological Research (Version 2.2.9). [R package]. Evanston, Illinois: Northwestern University. Available from https://CRAN.R-project.org/package=psych

Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. [link]

Rust, J., Kosinski, M., Stillwell, D. (2020). Modern Psychometrics: The Science of Psychological Assessment. United Kingdom: Taylor & Francis.

Sellbom, M., & Tellegen, A. (2019). Factor analysis in psychological assessment research: Common pitfalls and recommendations. Psychological Assessment, 31(12), 1428–1441. [link]

Shannon, D. M., Johnson, T. E., Searcy, S., & Lott, A. (2002). Using electronic surveys: advice from survey professionals. Practical Assessment, Research & Evaluation, 8(1), 1-9. [link]

Schwartz, S. J., Mullis, R. L., & Dunham, R. M. (1998). Effects of authoritative structure in the measurement of identity formation: Individual computer-managed versus group paper-and pencil testing. Computers in Human Behavior, 14(2), 239–248. [link]

Shokuhi-Yekta, M., & Prand, A. (2009). Behavioral disorder of children and Teenager [Persian]. Tehran: Institute Cultural Publication Tiemorzadeh-Tayeb, 2(3), 25-27.

Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180. [link]

Stiffler, M. C., & Dever, B. V. (2015). Mental health screening at school: Instrumentation, implementation, and critical issues. Contemporary issues in psychological assessment. Cham, Switzerland: Springer International Publishing.

Thurstone, L. L. (1947). Multiple factor analysis. University of Chicago Press.

Thangavel, S., & Subramaniam, R. (2016). Investigating the Benefits and Challenges with E-Surveying- A Review. International Journal of Science and Research Methodology, 4(1), 32-44. [link]

Vallejo. M,A., Jordán, M.C., Díaz, M. I., Comeche, M. I., & Ortega, J. (2007). Psychological Assessment via the Internet: A Reliability and Validity Study of Online (vs Paper-and-Pencil) Versions of the General Health Questionnaire-28 (GHQ-28) and the Symptoms Check-List-90-Revised (SCL-90-R). Journal of Medical Internet Research. 9(1), 14-23. [link]

Van Gelder, M. M., Bretveld, R. W., & Roeleveld, N. (2010). Webbased questionnaires: The future in epidemiology? American Journal of Epidemiology, 172, 1292–1298. [link]

Velicer, W. F. (1976). Determining the number of components from the matrix of partial correlations. Psychometrika, 41(3), 321–327. [link]

Watkins, Marley. (2020). A Step-by-Step Guide to Exploratory Factor Analysis with R and RStudio. 10.4324/9781003120001.

Watson, J. C. (2017). Establishing evidence for internal structure using exploratory factor analysis. Measurement and Evaluation in Counseling and Development, 50(4), 232–238. [link]

Williams, J. E., & McCord, D. M. (2006). Equivalence of standard and computerized versions of the Raven Progressive Matrices test. Computers in Human Behavior, 22(5), 791–800. [link]

Williams, S.T. (2008). Mental Health Screening and Assessment Tools for Children. UCDAVIS EXTENTION, University Of California.

Yang S. Liu, Jeffrey Hankey, Nigel Mantou Lou, Pratap Chokka & Jason M. Harley (2021): Usability and Emotions of Mental Health Assessment Tools: Comparing Mobile App and Paper-and-Pencil Modalities, Journal of Technology in Human Services, 39:2, 193-211. [link]

Zeiler, M., Peer, S., Philipp, J., Truttmann, S., Wagner, G., Karwautz, A., & Waldherr, K. (2021). Web-based versus paper-pencil assessment of behavioral problems using the Youth Self-Report. European Journal of Psychological Assessment, 37(2), 95–103. [link]

Zhang, Y. (1999). Using the Internet for Survey research: A Case Study. Journal of the American Society for Information Science. 51(1), 57– 68. [link]

Zumbo, B. D., Gadermann, A. M., & Zeisser, C. (2007). Ordinal Versions of Coefficients Alpha And Theta For Likert Rating Scales. Journal of Modern Applied Statistical Methods, 6, 21-29. [link]