Development and Preliminary Validation of a Self-Report Scale on Generative AI Literacy in Learning Among High School Students
Online First: 20/08/2026
Corressponding author's email:
hoaqm.hcmue@gmail.comDOI:
https://doi.org/10.54644/jte.2026.2502Keywords:
GenAI literacy, General education, Preliminary scale validation, Scale development, Self-assessmentAbstract
The use of generative AI (GenAI) in learning creates a need for assessment instruments that are appropriate to learners’ developmental characteristics, learning purposes, and educational contexts. Existing scales, however, have largely been developed for higher education or non-Vietnamese contexts, whereas high school students differ in learning autonomy, academic integrity experience, and technology-use conditions. This study develops and preliminarily validates a self-report scale assessing high school students’ GenAI literacy in learning. The scale was developed from existing AI/GenAI literacy frameworks, revised Bloom’s taxonomy, and Vietnam’s digital competence framework for learners. Survey data from 1,003 students at eight high schools in Ho Chi Minh City were analyzed using Cronbach’s alpha and Exploratory Factor Analysis. The findings provide initial evidence of reliability and a multidimensional structure consisting of 23 observed items across five dimensions: Knowledge & Understanding, Use & Application, Evaluation, Create, and Ethics. The psychometric evaluation yielded a KMO value of 0.896, a significant Bartlett’s test (p < 0.05), and a total variance explained of 59.07%, while internal consistency was confirmed with Cronbach’s alpha coefficients ranging from 0.691 to 0.851 across the factors. The study proposes a contextually grounded instrument that may help schools identify students’ self-perceived GenAI literacy. Beyond its methodological contributions, this scale serves as a pivotal diagnostic tool for educational stakeholders to tailor AI integration strategies and for policymakers to refine digital literacy standards within the general education sector, while emphasizing the need for further validation through CFA, convergent and discriminant validity testing, and measurement invariance before broader use.
Downloads: 0
References
L. Casal-Otero, A. Catala, C. Fernández-Morante, M. Taboada, B. Cebreiro, and S. Barro, “AI literacy in K-12: A systematic literature review,” Int. J. STEM Educ., vol. 10, no. 1, Art. no. 29, 2023, doi: 10.1186/s40594-023-00418-7. DOI: https://doi.org/10.1186/s40594-023-00418-7
J. Su, Y. Zhong, and D. T. K. Ng, “A meta-review of literature on educational approaches for teaching AI at the K-12 levels in the Asia-Pacific region,” Computers and Education: Artificial Intelligence, vol. 3, Art. no. 100065, 2022, doi: 10.1016/j.caeai.2022.100065. DOI: https://doi.org/10.1016/j.caeai.2022.100065
Y. Chen, K. K. Chai, V. Wijeratne, U. Naeem, J. Loo, X. Zhou, and S. Lucas, “A GenAI competence framework for engineering curriculum enhancement in higher education,” Intelligent Technologies in Education, 2025, doi: 10.53761/ITED/1.2. DOI: https://doi.org/10.53761/ITED/1.2
G. K. Adalı and A. Bilgili, “Generative AI in higher education: Students’ perspectives on adoption, ethical concerns, and academic impact,” Acta Infologica, vol. 9, no. 1, pp. 147–166, 2025, doi: 10.26650/acin.1670197. DOI: https://doi.org/10.26650/acin.1670197
A. Fock and H. S. Siller, “Generative artificial intelligence in secondary STEM education in the light of Human Flourishing: A scoping literature review,” Int. J. STEM Educ., 2025, doi: 10.21203/rs.3.rs-6923010/v1. DOI: https://doi.org/10.21203/rs.3.rs-6923010/v1
UNESCO, AI Competency Framework for Students. Paris, France: UNESCO, 2024, doi: 10.54675/JKJB9835. DOI: https://doi.org/10.54675/JKJB9835
S. N. A. Sampah, M. Nabang, E. K. Krampa, M. Issah, F. Koduah, H. B. Essel, and J. A. Agyem, “Generative AI competence and student engagement: The mediating effects of GenAI utilization, perceived autonomy, and AI formal learning among KNUST students,” Research Square, 2025, doi: 10.21203/rs.3.rs-7096161/v1. DOI: https://doi.org/10.21203/rs.3.rs-7096161/v1
S. C. Kong, J. Zhu, and Y. N. Yang, “Developing and validating a scale of empowerment in using artificial intelligence for problem-solving for senior secondary and university students,” Computers and Education: Artificial Intelligence, vol. 8, Art. no. 100359, 2025, doi: 10.1016/j.caeai.2024.100359. DOI: https://doi.org/10.1016/j.caeai.2024.100359
Z. Ahmad, A. Sultana, N. A. Latheef, N. Siby, A. Sellami, and S. A. Abbasi, “Measuring students’ AI competence: Development and validation of a multidimensional scale integrating educational psychology perspectives,” Acta Psychologica, vol. 259, Art. no. 105446, 2025, doi: 10.1016/j.actpsy.2025.105446. DOI: https://doi.org/10.1016/j.actpsy.2025.105446
T. K. F. Chiu, M. Çoban, I. T. Sanusi, and M. A. Ayanwale, “Validating student AI competency self-efficacy (SAICS) scale and its framework,” Educ. Technol. Res. Dev., vol. 73, no. 4, pp. 2785–2807, 2025, doi: 10.1007/s11423-025-10512-y. DOI: https://doi.org/10.1007/s11423-025-10512-y
H. Chee, S. Ahn, and J. Lee, “A competency framework for AI literacy: Variations by different learner groups and an implied learning pathway,” Br. J. Educ. Technol., vol. 56, no. 6, pp. 2146–2182, 2025, doi: 10.1111/bjet.13556. DOI: https://doi.org/10.1111/bjet.13556
D. T. Thanh, P. D. Van, and N. V. Hien, “Exploring AI integration in biology self-learning: A TAM-based analysis among high school students in Ho Chi Minh City,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 21, no. 10, Art. no. em2716, 2025, doi: 10.29333/ejmste/17158. DOI: https://doi.org/10.29333/ejmste/17158
T. Lintner, “A systematic review of AI literacy scales,” npj Sci. Learn., 2024, doi: 10.1038/s41539-024-00264-4. DOI: https://doi.org/10.1038/s41539-024-00264-4
Q. T. Thanh, “The impact of Artificial Intelligence (AI) applications on teaching quality and learner satisfaction at foreign language centers in Ho Chi Minh City,” Journal of Knowledge Learning and Science Technology, vol. 4, no. 3, pp. 156–164, 2025.
A. Carolus, M. J. Koch, S. Straka, M. E. Latoschik, and C. Wienrich, “MAILS—Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies,” Computers in Human Behavior: Artificial Humans, vol. 1, no. 2, Art. no. 100014, 2023, doi: 10.1016/j.chbah.2023.100014. DOI: https://doi.org/10.1016/j.chbah.2023.100014
D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, and M. S. Qiao, “Conceptualizing AI literacy: An exploratory review,” Computers and Education: Artificial Intelligence, vol. 2, Art. no. 100041, 2021, doi: 10.1016/j.caeai.2021.100041. DOI: https://doi.org/10.1016/j.caeai.2021.100041
R. Annapureddy, A. Fornaroli, and D. Gatica-Perez, “Generative AI literacy: Twelve defining competencies,” Digital Government: Research and Practice, vol. 6, no. 1, pp. 1–21, 2025, doi: 10.1145/3685680. DOI: https://doi.org/10.1145/3685680
X. Zhou and L. Schofield, “Developing a conceptual framework for Artificial Intelligence (AI) literacy in higher education,” Journal of Learning Development in Higher Education, no. 31, 2024, doi: 10.47408/jldhe.vi31.1354. DOI: https://doi.org/10.47408/jldhe.vi31.1354
Ministry of Education and Training, Circular No. 02/2025/TT-BGDĐT dated January 24, 2025 on the Digital Competency Framework for Learners. Hanoi, Vietnam, 2025.
R. F. DeVellis and C. T. Thorpe, Scale Development: Theory and Applications, 5th ed. Thousand Oaks, CA, USA: SAGE Publications, 2022.
H. M. N. Le, T. K. A. Nguyen, and M. H. Quan, “A scale for assessing collaborative competence of lower secondary school students,” Ho Chi Minh City University of Education Journal of Science, vol. 22, no. 4, pp. 669–679, 2025, doi: 10.54607/hcmue.js.22.4.4431.
T. T. Ta, “Proposing a process for developing student competency scales based on structural equation modeling,” Ho Chi Minh City University of Education Journal of Science, vol. 20, no. 8, pp. 1337–1352, 2023, doi: 10.54607/hcmue.js.20.8.3812.
J. F. Hair, W. C. Black, B. J. Babin, and R. E. Anderson, Multivariate Data Analysis, 8th ed. Boston, MA, USA: Cengage Learning, 2019.
I. Bernstein and J. Nunnally, “The assessment of reliability,” in Psychometric Theory, 3rd ed. New York, NY, USA: McGraw-Hill, 1994, pp. 248–292.
H. F. Kaiser, “An index of factorial simplicity,” Psychometrika, vol. 39, no. 1, pp. 31–36, 1974, doi: 10.1007/BF02291575. DOI: https://doi.org/10.1007/BF02291575
F. B. Bryant and P. R. Yarnold, “Principal-components analysis and exploratory and confirmatory factor analysis,” in Reading and Understanding Multivariate Statistics, L. G. Grimm and P. R. Yarnold, Eds. Washington, DC, USA: American Psychological Association, 1995, pp. 99–136.
D. W. Gerbing and J. C. Anderson, “An updated paradigm for scale development incorporating unidimensionality and its assessment,” J. Mark. Res., vol. 25, no. 2, pp. 186–192, 1988, doi: 10.2307/3172650. DOI: https://doi.org/10.1177/002224378802500207
E. Knekta, C. Runyon, and S. Eddy, “One size doesn’t fit all: Using factor analysis to gather validity evidence when using surveys in your research,” CBE—Life Sciences Education, vol. 18, no. 1, Art. no. rm1, 2019, doi: 10.1187/cbe.18-04-0064. DOI: https://doi.org/10.1187/cbe.18-04-0064
Y. Y. Wang and Y. W. Chuang, “Artificial intelligence self-efficacy: Scale development and validation,” Educ. Inf. Technol., vol. 29, no. 4, pp. 4785–4808, 2024, doi: 10.1007/s10639-023-12015-w. DOI: https://doi.org/10.1007/s10639-023-12015-w
B. Wang, P. L. P. Rau, and T. Yuan, “Measuring user competence in using artificial intelligence: Validity and reliability of artificial intelligence literacy scale,” Behav. Inf. Technol., vol. 42, no. 9, pp. 1324–1337, 2023, doi: 10.1080/0144929X.2022.2072768. DOI: https://doi.org/10.1080/0144929X.2022.2072768
A. Schepman and P. Rodway, “The General Attitudes towards Artificial Intelligence Scale (GAAIS): Confirmatory validation and associations with personality, corporate distrust, and general trust,” Int. J. Hum.-Comput. Interact., vol. 39, no. 13, pp. 2724–2741, 2023, doi: 10.1080/10447318.2022.2085400. DOI: https://doi.org/10.1080/10447318.2022.2085400
M. C. Laupichler, A. Aster, and T. Raupach, “Delphi study for the development and preliminary validation of an item set for the assessment of non-experts’ AI literacy,” Computers and Education: Artificial Intelligence, vol. 4, Art. no. 100126, 2023, doi: 10.1016/j.caeai.2023.100126. DOI: https://doi.org/10.1016/j.caeai.2023.100126
S. C. Kong, M. Y. W. Cheung, and O. Tsang, “Developing an artificial intelligence literacy framework: Evaluation of a literacy course for senior secondary students using a project-based learning approach,” Computers and Education: Artificial Intelligence, vol. 6, Art. no. 100214, 2024, doi: 10.1016/j.caeai.2024.100214. DOI: https://doi.org/10.1016/j.caeai.2024.100214
Downloads
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2026 Journal of Technical Education Science
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright © JTE.
