Deck 7: Item Analysis: Methods for Fitting the Right Items to the Right Test

Question 1

An effective distracter should be selected by:

A)at least some examinees and demonstrate positive discrimination.
B)at least some examinees and demonstrate negative discrimination.
C)no examinees and demonstrate zero discrimination.
D)all examinees and demonstrate positive discrimination.

Answer 1

at least some examinees and demonstrate negative discrimination.

Question 2

In a class of 100 students,70 answer item #4 correctly and 30 answer it incorrectly.What is the p value of this item?

A)0.30
B)0.40
C)0.49
D)0.70

Answer 2

0.70

Question 3

A test item with p = .80 and D = .40 is:

A)relatively easy and discriminates well.
B)relatively easy and does not discriminate well.
C)relatively difficult and does discriminate well.
D)relatively difficult and does not discriminate well.
E)of intermediate difficulty and does not discriminate well.

Answer 3

relatively easy and discriminates well.

Question 4

A test item with p = .50 and D = .40 is:

A)relatively easy and discriminates well
B)relatively easy and does not discriminate well.
C)relatively difficult and does discriminate well.
D)relatively difficult and does not discriminate well.
E)of intermediate difficulty and discriminates well.

Question 5

While the item difficulty index is only applicable for _____________,the percent endorsement statistic can be calculated for ____________.

A)maximum performance tests; constructed-response tests
B)maximum performance tests; typical-response tests
C)typical-response tests; constructed-response tests
D)typical-response tests; maximum performance tests

Question 6

Item 10 on an exam had an item difficulty index equal to .00.From this information,one knows that:

A)the item was very difficult and all students answered it incorrectly.
B)the item was very easy and all students answered it correctly.
C)the item was of medium difficulty and half of the students answered it correctly.
D)nothing since there is not enough information provided.

Question 7

What is the approximate optimal Item Difficulty Index for a multiple-choice item with 4 choices?

A)0.0
B)0.25
C)0.50
D)0.75
E)1.0

Question 8

On ___________ tests,measures of item difficulty largely reflects the position of the item in the test.

A)power
B)typical response
C)multidimensional
D)speed

Question 9

A test item with a p = .70 and D = .45 is:

A)relatively easy and discriminates well.
B)relatively easy and does not discriminate well.
C)relatively difficult and discriminates well.
D)relatively difficult and does not discriminate well.
E)of intermediate difficulty and does not discriminate well.

Question 10

A test item with a p = .30 and D = .15 is:

A)relatively easy and discriminates well.
B)relatively easy and does not discriminate well.
C)relatively difficult and does discriminate well.
D)relatively difficult and does not discriminate well.
E)of intermediate difficulty and does not discriminate well.

Question 11

Reliability relates to test ___________.

A)items
B)length
C)scores
D)constructs

Question 12

Items with a difficulty index of _______ do not contribute to the measurement characteristics of a test.

A)0.25
B)0.50
C)0.75
D)1.0

Question 13

A general recommendation is to use items with p values that have a range of approximately _________ around the optimal value.

A)0.05
B)0.10
C)0.15
D)0.20
E)0.25

Question 14

The item-total correlation is typically calculated using which correlation?

A)Coefficient Alpha
B)Pearson product moment correlation
C)Point-biserial correlation
D)Spearman rank order correlation

Question 15

Your employer wants a test that will help him to select the upper 30% of employees to consider for new positions.It would be beneficial for the item difficulty index to average around:

A)0.20
B)0.30
C)0.70
D)0.80
E)0.90

Question 16

As a general rule,the authors of the chapter suggest that items with D values greater than ______ are acceptable.

A)0.25
B)0.30
C)0.50
D)0.70

Question 17

On a maximum performance test administered to 100 students,60 students correctly answer item #4.The item difficulty index equals:

A)0.40
B)0.60
C)40
D)60

Question 18

On a reading comprehension exam,the proportion of examinees in the top group that answered item 5 correctly equaled 0.70 and the proportion of examinees in the bottom group that answered item 5 correctly equaled 0.10.What is the discrimination index for item 5?

A)0.36
B)0.49
C)0.60
D)0.80

Question 19

What is the optimal Item Difficulty Index on a test consisting of only constructed response items?

A)0.0
B)0.25
C)0.50
D)0.75
E)1.0

Question 20

When a test item has a discrimination index ______,it is considered to be acceptable by the chapter authors.

A)greater than 0.30
B)less than 0.30
C)greater than 0.60
D)less than 0.60

Question 21

Items with a value of _____ and ______ do not contribute to the variability of test scores.

A)-1.0; 0.00
B)0.00; 1.0
C)-1.0; 1.0

Question 22

The one-parameter IRT model assumes that items only differ in:

A)the chance of a random correct answer.
B)difficulty.
C)discrimination.
D)scree point.

Question 23

Which of the following was described as a qualitative approach to item analysis?

A)Set the test aside and review at a later time.
B)Have a colleague review the test.
C)Allow examinees to provide feedback.
D)All of the above.

Question 24

The ___________ IRT model takes into consideration the possibility of an examinee with no 'ability' answering the item correctly by chance.

A)one-parameter
B)Rasch model
C)two-parameter
D)three-parameter

Question 25

In a class of 100 students,only 20 answered item 10 correctly.The p value for this item equals:

A)0.20
B)0.40
C)0.64
D)0.80

Question 26

On an Item Characteristic Curve,the point halfway between the lower and upper asymptotes is referred to as the __________ and represents the difficulty of the item.

A)medial apex
B)modal beta
C)beta index
D)inflection point

Question 27

The optimal p value for a true/false items is approximately:

A)0.55.
B)0.65.
C)0.75.
D)0.85.

Question 28

How can reliability be reduced with many items close to a p value of 1.0?

A)The total score is transformed in an oblique manner.
B)The total score is an area transformation.
C)This results in a restricted range of scores.
D)This results in an artificially inflated validity coefficient.

Question 29

__________ illustrates the reliability of measurement at different points along the distribution.

A)Classical Test Theory
B)Item Characteristic Curve
C)Reliability Curve
D)Test Information Function

Question 30

An Item Characteristic Curve is a graph with ____________ reflected on the horizontal axis and ______________reflected on the vertical axis.

A)ability; probability of a correct response
B)probability of a correct response; ability
C)ability; probability of an incorrect response
D)probability of an incorrect response; ability

Question 31

Item difficulty indexes on mastery tests tend to be ________ item difficulty indexes on tests designed to produce norm-referenced scores.

A)larger than
B)lower than
C)equal to

Question 32

For item 21,pT is 0.80 and pB is 0.30.What is the value of D for this item?

A)0.20
B)0.25
C)0.50
D)0.70

Question 33

Item difficulty and distracter analysis are related to:

A)Classical Test Theory.
B)Factor Analytic Theory.
C)Item Characteristic Curve Theory.
D)Item Response Theory.

Question 34

Which of the following statements regarding p values is correct?

A)They can range from -1.0 to 1.0.
B)Items with a high p value are more difficult.
C)Items with a low p value are more difficult.
D)Items with a p value of 1.0 were answered incorrectly by all students.

Question 35

A test analysis is displayed below.Based on this information,what is the most obvious problem? $p = 0.40 D = 0.54$
$\begin{array}{lcccc}&\quad\quad\quad\text {Options}\\\text { Item \#7 } & \mathrm{A} & \mathrm{B} & \mathrm{C}^{*} & \mathrm{D} \\\hline \text { Number in top group } & 0 & 1 & 10 & 4 \\\\\text { Number in bottom group } & 4 & 3 & 2 & 6 \\\hline \text {* correct answer } & & &\end{array}$

A)All distracters are performing well but the item is too easy.
B)Distracter B demonstrates negative discrimination and should be revised.
C)Distracter A demonstrates negative discrimination and should be revised.
D)No problem,retain the item as is.

Question 36

The optimal p value for a selected-response item with two choices is approximately:

A)0.55.
B)0.65.
C)0.75.
D)0.85.

Question 37

A test analysis is displayed below.Based on this information,what is the most obvious problem? $p = 0.40 D = 0.13$
$\begin{array}{lcccc}&\quad\quad\quad\text {Options}\\\text { Item \#3 } & \mathrm{A} & \mathrm{B} & \mathrm{C}^{*} & \mathrm{D} \\\hline \text { Number in top group } &6 & 1 & 1 & 7 \\\\\text { Number in bottom group } & 2 & 4 & 4 & 5 \\\hline \text { *correct answer } & & & \end{array}$

A)All distracters are performing well.
B)Distracter A displays positive discrimination and should be revised.
C)Distracter B displays positive discrimination and should be revised.
D)No problem,retain the item as is.