Full-Text Search:
Home|Journal Papers|About CNKI|User Service|FAQ|Contact Us|中文
《Journal of Psychological Science》 2018-06
Add to Favorite Get Latest Update

Three Methods for the Interval Estimation of AttributeLevel Classification Consistency in Cognitive Diagnostic Assessment

Wang Wenyi;Fang Xiaoting;Ye Baojuan;School of Computer and Information Engineering,Jiangxi Normal University;Center of Mental Health Education and Research,School of Psychology,Jiangxi Normal University;  
As is well known, point estimate contains limited information about a population parameter and cannot give how far it can be from the population parameter. The confidence interval of the parameter can provide more information about the precision of estimated parameters. In evaluating the quality of a test, the confidence interval of reliability has received more and more attention in recent years. However, there is no research on the estimation of the reliability interval of the cognitive diagnostic assessment. The researchers often report the reliability point estimation only, and no one is concerned about the confidence interval in the report. It is far from enough to report the reliability of the cognitive diagnostic assessment. The confidence interval of reliability in cognitive diagnostic assessment should be taken into account. We can estimate error range from the confidence interval.This study investigated and compared three reliability interval estimation methods of the cognitive diagnostic assessment based on DINA model and attribute-level classification consistency(Wang, et al., 2015). There are three approaches to estimate the confidence interval of the cognitive diagnostic assessment: Bootstrap method, parallel test method, and parallel test pairing method. Each of the three approaches produces the standard error, average and confidence interval about attribute-level classification consistency reliability. These factors were considered in the simulation design:(a) The number of test attributes(K=5, and 7);(b) The number of test items(5 attributes is T=15, and 31; 7 attributes is T=21, and 42);(c) The quality of test items [U(0.05,0.25), and U(0.25,0.45)];(d) The number of sample size(n=500, 1000 and 1500);(e) The method for calculating the confidence intervals of attribute-level classification consistency reliability(Bootstrap, parallel test method, and parallel test pairing method). Totally, 72 treatment conditions were generated in terms of the above 5-factor simulation design(i.e., 72=2×3×2×2×3).The simulation results indicated:(1) Whether the tests contained 5 or 7 attributes of the independent attribute hierarchical relationship, the standard error and reliability interval obtained by three interval estimation methods estimated(Bootstrap method, parallel test method and parallel test matching method) were all affected by the quality of test, sample size or test length. With the increase of the number of test items and subjects, the standard error and the length of confidence interval tended to decrease. As test quality degraded, the standard error and the length of confidence interval increased;(2) Whether the tests contained 5 or 7 attributes of the independent attribute hierarchical relationship, the average of the reliability estimated by three interval estimation methods was affected by the quality of test, sample size or test length. With the increase of test length, the average of the reliability showed a larger increase. With the increase of sample size, the change of the average of the reliability was small; as test quality degraded, the average of the reliability showed a larger decline;(3) The parallel test method and the Bootstrap method were close to the standard errors and confidence intervals estimated when test length and sample size was small. However, with the increase of sample size, the estimation accuracy of the Bootstrap method was improved rapidly. When the amount of test items and the sample size was large, the result was basically close to the parallel test matching method. Bootstrap method requires the least time, while parallel test matching method is difficult to achieve in practice. Therefore, Bootstrap method is recommended when estimating the confidence interval of cognitive diagnosis.
【Fund】: 国家自然科学基金项目(31600909 31500909);; 全国教育科学规划教育部重点课题(DHA150285);; 江西省自然科学基金项目(20161BAB212044);; 江西省教育科学2013年度一般课题(13YB032);; 国家社会科学基金项目(16BYY096);; 江西省高校人文社科重点研究基地——心理健康教育研究中心项目(项目编号:JD17078);; 江西省社会科学规划项目(17JY10);; 江西师范大学青年成长基金和江西师范大学博士启动基金的资助
【CateGory Index】: B842.1
Download(CAJ format) Download(PDF format)
CAJViewer7.0 supports all the CNKI file formats; AdobeReader only supports the PDF format.
【Citations】
Chinese Journal Full-text Database 2 Hits
1 Wang Wenyi1 Ding Shuliang2 Song Lihong1 Xiao Linru3 1 College of Psychology, Jiangxi Normal University, Nanchang, Jiangxi, 330022. 2 College of Computer Information Engineering, Jiangxi Normal University, Nanchang, Jiangxi,330022. 3 ZhongZhou Middle School ,Zhangshu,Jiangxi, 331211.;A Review of Methods of Knowledge States Classification in Cognitive Diagnostic Assessment[J];考试研究;2012-03
2 Guan Dandan , Zhang Houcan (School of Psychology, Beijing Normal University, Beijing, 100875);A Redefinition of Reliability and the Study of Reliability Generalization[J];心理科学;2004-02
【Co-citations】
Chinese Journal Full-text Database 9 Hits
1 JIANG Xuelai;School of Physical Education,Sanming University;;Methodological Exploration on the Reliability Test of Questionnaire in Sports Social Investigation—An Analysis Based on 163 Papers Published in Chinese Major Sport Academic Journals[J];上海体育学院学报;2015-06
2 DU Jiang-hong;WANG Hua-xin;LI Yong-xin;Research Institute of Psychology and Behavior,Henan University;;The Revision of Adult Attachment Scale(AAS) in Chinese College Students Sample[J];湖州师范学院学报;2015-01
3 XIE Jing-Yi;HUANG Qin-Jin;WANG Chun-Xiao;Key Laboratory of Health Technology Assessment,School of Public Health,Fudan University;;Design of job satisfaction scale for primary maternal and child health care staff and validity and reliability evaluation[J];中国妇幼保健;2015-01
4 Wang Wenyi;Song Hongli;Ding Shuliang;College of Computer Information and Engineering,Jiangxi Normal University;Elementary Education College,Jiangxi Normal University;;A Review of Reliability and Validity Indices in Cognitive Diagnostic Assessments[J];考试研究;2014-05
5 LUO Li;ZHOU Qiu-wen;XUE Min;Public Health Security Key Laboratory of Ministry of Education, School of Public Health, Fudan University;;Internal Reliability of Psychosocial Function Evaluation for Mental Disabilities[J];中国康复理论与实践;2013-11
6 LUO Jie1,ZHAO Shou-Ying1,PAN Yun1,DAI Xiao-Yang2 1School of Educational Science,Guizhou Normal University,Guiyang 550001,China 2Department of Psychology,Shenzhen University Normal College,Shenzhen 518060,Guangdorg Province,China;A reliability generalization analysis of the Adolescence Time Management Disposition Inventory[J];中国心理卫生杂志;2013-04
7 Jiao Can~(2,1),Zhang Minqiang~1,Zhang Jieting~1,Wu Li~1,Zhang Wenyi~3 (1 Psychological Application Research Center,South China Normal University,Guangzhou,510631) (2 Department of Psychology,Shenzhen University,Shenzhen,518000) (3 School of Science,Australian National University,Canberra,2617);A Cross-cultural Comparison of the Reliability Generalization Studies on the Eysenck Personality Questionnaire and Its Implications[J];心理科学;2011-06
8 JIAO Can,ZHANG Jie-ting,WU Li,ZHANG Min-qiang;Scores Reliability from the Minnesota Multiphasic Personality Inventory in China[J];华南师范大学学报(社会科学版);2010-04
9 Jiao Can,Wu Li,Zhang Minqiang and Zhang Wenyi;A Comment on the New Points of the Studies of Reliability Generalization[J];学术研究;2009-02
【Secondary Citations】
Chinese Journal Full-text Database 10 Hits
1 SUN Jia-Nan1,3;ZHANG Shu-Mei1;XIN Tao2;BAO Yu1(1 School of Mathematical Sciences,Beijing Normal University,Laboratory of Mathematics and Complex Systems,Ministry of Education,Beijing 100875,China)(2 Institute of Developmental Psychology,Beijing Normal University,Beijing 100875,China)(3 School of Psychology,Beijing Normal University,Beijing 100875,China);A Cognitive Diagnosis Method Based on Q-Matrix and Generalized Distance[J];心理学报;2011-09
2 Ding Shuliang1,Wang Wenyi2,Yang Shuqun3(1 College of Computer Information Engineering,Jiangxi Normal University,Nanchang,330022)(2 School of Psychology,Jiangxi Normal University,Nanchang,330022) (3 Faculty of Software,Fujian Normal University,Fuzhou,35007);The Design of Cognitive Diagnostic Test Blueprints[J];心理科学;2011-02
3 YU Xiao-Feng1,2;DING Shu-Liang1;QIN Chun-Ying2;LU Yun-Na1,3(1 Computer Information Engineer College,Jiangxi Normal University,Nanchang 330027,China)(2 Computer Department,Bozhou teachers College,Bozhou 233500,China)(3 Nanchang No.1 high school,Nanchang 330003,China);Application of Bayesian Networks to Identify Hierarchical Relation Among Attributes in Cognitive Diagnosis[J];心理学报;2011-03
4 DING Shu-liang1,YANG Shu-qun2,WANG Wen-yi1(1.College of Computer Information Engineering,Jiangxi Normal University,Nanchang Jiangxi 330027,China;2.College of Software,Fujian Normal University,Fuzhou Fujian 350007,China);The Importance of Reachability Matrix in Constructing Cognitively Diagnostic Testing[J];江西师范大学学报(自然科学版);2010-05
5 XU Zhiyong,DING Shuliang,WANG Wenyi and TU Dongbo;Explore the Test Development to Improve Learning[J];中国考试;2010-07
6 Zhao Dingwei~1,Dai Haiqi~1,Guo Siping~2 (1 psychology College,Jiangxi Normal University,Nanchang,330022) (2 Education College,Guangzhou University,Guangzhou,510006);The Developing of the Syllogistic Reasoning Test and Classifying of Individual's Attribute Master Pattern Under AHM[J];心理科学;2010-03
7 Chen Dezhi1,2,Dai Haiqi1,Zhao Dingwei1 (1 Education College of Jiangxi Normal University,Nanchang 33002)(2 Zhejiang Normal University,Hangzhou,310012);Comparing the Diagnostic Accuracy Between the Rule-Space Approach and the Attribute Hierarchy Method[J];心理科学;2009-02
8 Zhu Yu Fang1,2,Ding Shu Liang1(1Computer Information Engineering College of Jiangxi Normal University,Nanchang 330027,China) (2 Shangrao Normal School,Shangrao 334000,China);A Polytomous Extension of Attribute Hierarchy Method Based on Graded Response Model[J];心理学报;2009-03
9 DING Shu-Liang1,ZHU Yu-Fang1,2 LIN Hai-Jing3,CAI Yan4,(1Computer Information Engineering College of Jiangxi Normal University,Nanchang 330027,China)(2Shangrao Normal School,Shangrao,Jiangxi 334000,China)(3JiangXi Vocational Technical College of Industry & Trade,Nanchang 330100,China)(4Mathematics and information college of Jiangxi Normal University,Nanchang 330027,China);Modification of Tatsuoka's Q Matrix Theory[J];心理学报;2009-02
10 Liu Yuanwo (Personnel Testing Authorities,Minisity of Personnel P.R.C.,Beijing 100054) Zhang Houcan (Beijing Normal University. Beijing 100875);APPLICATION OF GENERALIZABILITY THEORY IN COMPOSITION SCORING[J];心理学报;1998-02
©2006 Tsinghua Tongfang Knowledge Network Technology Co., Ltd.(Beijing)(TTKN) All rights reserved