Cognitive diagnostic models show great promise in language assessment for providing rich diagnostic information. The lack of a full understanding of second language listening subskills made model selection difficult. In search of optimal CDM that could provide a better understanding of L2 listening subskills and facilitate accurate classification, this study carried a two-layer model selection. At the test level, A-CDM, LLM, and R-RUM had an acceptable and comparable model fit, suggesting mixed inter-attribute relationships of L2 listening subskills. At the item (...) level, Mixed-CDMs were selected and confirmed the existence of mixed relationships. Mixed-CDMs had better model and person fit than G-DNIA. In addition to statistical approaches, the content analysis provided theoretical evidence to confirm and amend the item-level CDMs. It was found that semantic completeness pertaining to the attributes and item features may influence the attribute relationships. Inexplicable attribute conflicts could be a signal of suboptimal model choice. Sample size and the number of multi-attribute items should be taken into account in L2 listening cognitive diagnostic modeling studies. This study provides useful insights into the model selection and the underlying cognitive process for L2 listening tests. (shrink)

Computer multistage adaptive test combines the advantages of paper and pencil-based test and computer-adaptive test. As CAT, MST is adaptive based on modules; as P&P, MST can meet the need of test developers to manage test forms and keep test forms parallel. Cognitive diagnosis can accurately measure students’ knowledge states and provide diagnostic information, which is conducive to student’s self-learning and teacher’s targeted teaching. Although MST and CD have a lot of advantages, many factors prevent MST from applying to CD. (...) In this study, we first attempt to employ automated test assembly to achieve the objectives of MST in the application of CD via heuristic algorithms. The mean correct response probability of all KSs for each item is used to describe the item difficulty of CD. The attribute reliability in CD is defined as the test quantitative target. A simulation study with the G-DINA model was carried out to investigate the proposed CD-MST, and the results showed that the assembled panels of CD-MST satisfied the statistical and the non-statistical constraints. (shrink)