Abstract

Q-matrix construction is a foundational yet challenging step in cognitive diagnostic assessment (CDA), which is traditionally reliant on labor-intensive and subjective methods like expert judgment and verbal report analysis. This study explores the potential of generative artificial intelligence (GenAI) to optimize this critical process within the domain of EFL reading. By applying three GenAI models (DeepSeek-V3.2, Kimi 2.5, and Doubao 2.0), three purely GenAI-informed Q-matrices (Qmat-DS, Qmat-K, and Qmat-DB) were generated, and through expert revision, a human–AI collaborative Q-matrix (Qmat-DS-H) was obtained. These were compared with an expert-constructed Q-matrix (Qmat-E) and a student-derived Q-matrix (Qmat-S). Using a simulated dataset (N = 1000) and empirical response data from 1083 EFL learners on a diagnostic reading test, the psychometric performance of the six Q-matrices was estimated via the G-DINA model, ACDM model, and RRUM model. Results demonstrated that the human–AI collaborative Q-matrix consistently outperformed the other five Q-matrices, achieving the best absolute model-data fit, the highest classification accuracy, the most stable item parameters, and the most balanced attribute correlation structure. The purely GenAI-informed Q-matrices showed mixed results: there were some improvements in relative fit and slip stability compared to manually constructed Q-matrices, but variable absolute fit and attribute correlation patterns. The findings substantiate GenAI as a feasible pathway for enhancing the efficiency, consistency, and psychometric quality of Q-matrix construction. This study offers a preliminary framework for advancing CDA development, addressing a key methodological bottleneck in language assessment.

IPC Classification

Keywords