When it comes to improving assessment procedures in the field of education, one key area of focus is the use of student-response data. Item-discrimination indices, such as the commonly employed item-difficulty index (p value), play a crucial role in this process.
The item-difficulty index (p value) is calculated by dividing R, the number of students who responded correctly to an item, by T, the total number of students who responded to that item. This index tells us the proportion of students who answered an item correctly.
Another important index is the item-discrimination index, which tells us how frequently an item is answered correctly by those who perform well on the total test and their responses to a particular test item. This index can range from 0 to 1.00, with higher p values indicating items that more students answered correctly.
The item-discrimination index is calculated by calculating a correlation coefficient between students’ total test scores and their performance on a particular item. This helps us to identify items that are good indicators of overall student performance.
It's important to keep in mind that these indices are primarily designed to discriminate between high-scorers and low-scorers, and may not be appropriate for evaluating the quality of items in teacher-made tests if those tests are being used for instructional purposes.
Distractor analyses can also provide insights into the effectiveness of each item’s alternatives. Additionally, two indices of item quality for criterion-referenced measurement can be used in settings where many effectively taught students perform well on examinations.
By using these indices and techniques, educators can gain valuable insights into the effectiveness of their assessment procedures, identify areas for improvement, and ultimately provide better educational experiences for their students.