NR523 Assessment and Evaluation Techniques in Education
Week 7 Assignment
Application Exercise 3: Conducting an Item Analysis
Purpose
The purpose of this assignment is to help students apply information regarding item analysis. Students are provided an item analysis as well as test questions. Upon considering the item analysis information, each students makes a decision to either retain the item or revise it. Students revise the item while demonstrating best practices regarding test item construction.
Due Date: Sunday 11:59 p.m. MT at the end of Week 7
Total Points Possible: 175
REQUIREMENTS:
Description of the assignment:
A printout of an item analysis for a 45-item test on HIV/AIDs, along with the corresponding test questions is found in Course Resources. This assignment has two elements. First the overall reliability of the examination is considered and then the item analysis of specific questions. Within the item analysis document, there are nine highlighted test questions (Questions 1 – 8 and 45). These highlighted test questions are the items to be individually analyze. The analysis of selected test questions consists of two major elements:
• Using item analysis numerical results
• Appling best practices regarding item construction
Using the Item Analysis Template (found in Course Resources), the student, first, examines the overall reliability of the test and make suggestions for improvement. Following the required review of overall test reliability, the student will examine the required test items (Questions 1 – 8 and 45); individually. The analysis includes consideration of numerical item analysis results AND if the selected test question demonstratedbest practices regarding item construction. The student will then provide a decision whether he/she would retain or revise the item. If the student’s decision is that the item should be revised, the student identifies suggestion (s) to improve the item as well as re-writing the test question to be consistent with base practices of item construction. For this assignment the desired difficulty index is .70 – .80 for most questions. The desired discrimination index value is .30 or greater. There are no mastery items on this examination. A cover page and running head are required. If a reference is used, you must provide a reference list.
Criteria for content:
1. Review of the reliability of the entire examination: In this section the MSN student reviews the reliability results of the entire examination. It includes the following required elements:
• Explain what is meant by test reliability
• Identify the name of the statistical test used in the item analysis to determine overall test reliability
• Identify the overall test reliability
• Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
• Identify 2 (two) ways/methods to improve the overall test reliability
2. Analysis for Individual Test Questions: For each of the nine identified test questions (items 1 – 8 and 45), the student will identify and discuss:
• If the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
• If the question has a poor difficulty index
• If the question has a poor discrimination index
• If the question should be retained as written or revised
• If you selected “revision”, describe ALL of the revisions that are needed
• If you selected “remain”, state why the question is good to use
PREPARING THE PAPER
Criteria for Format and Special Instructions:
1. The use of the Item Analysis Template is required. It is found within Course Resources.
2. A reference page is required if a reference is cited. APA (6th ed.) is required.
.
