Deck 6: Writing and Evaluating Test Items

Question 1

Under what circumstance is it NOT to your advantage to guess on a multiple-choice exam?

A) when you are making a "wild guess" and a correction formula is being used
B) in any test situation where you are making a "wild guess"
C) when you can rule out one or more of the alternatives as being incorrect
D) when the guessing threshold is low

Answer 1

A

Question 2

The following is an item from an attitude scale: Physical punishment is essential in order to control children.
Strongly disagree
Disagree
Neither agree or disagree
Agree
Strongly agree
This item is in the

A) category format.
B) Likert format.
C) dichotomous format.
D) polytomous format.

Answer 2

B

Question 3

Which item format can best be factor analyzed to find which ones group together?

A) multiple-choice
B) Likert
C) dichotomous
D) forced-choice

Answer 3

B

Question 4

Suppose you got 75 items correct on a 100-item,six alternative,multiple-choice exam.What would your score be after we corrected for guessing?

A) 50
B) 57
C) 63
D) 70

Question 5

When distractors are likely to be selected as alternative responses on multiple-choice tests,

A) validity is increased.
B) item reliability is increased.
C) item reliability is decreased.
D) guessing is reduced.

Question 6

One problem with the use of category rating scales is that

A) many respondents are confused by dichotomous formats.
B) responses are sometimes influenced by the context in which objects are rated.
C) rating scales must be at least 100 points in order to be meaningfully interpreted.
D) category rating scale data do not have ordinal scale property.

Question 7

In order to correct for guessing

A) a correction formula can be used.
B) distractors should be eliminated.
C) the number of items should be increased.
D) distractors should be increased.

Question 8

In multiple choice examinations,incorrect alternatives are called

A) flags.
B) non-categories.
C) distractors.
D) miss rates.

Question 9

The expected level of chance performance,for a 200-item multiple-choice exam with four choice alternatives,is

A) 25 correct.
B) 50 correct.
C) 75 correct.
D) 100 correct.

Question 10

True-false examinations use

A) a dichotomous format.
B) a polytomous format.
C) a Likert format.
D) a category format.

Question 11

A test format that is typically used for attitude measurement is the

A) checklist format.
B) dichotomous format.
C) category format.
D) Likert format.

Question 12

What describes the chances that a low-ability test taker will obtain each score?

A) acquiescence response set
B) the miss rate
C) guessing threshold
D) the moments method

Question 13

Describing the chances that low-ability test takers will obtain each score is called the

A) dichotomous format.
B) polytomous format.
C) guessing threshold.
D) 50% threshold.

Question 14

Suppose that you are taking a multiple choice test where there is no correction for guessing.If you aren't sure of the answer,

A) only guess if you have some confidence you are correct.
B) you should always guess on a speed test.
C) you should always guess.
D) you should never guess.

Question 15

What format do some personality tests use because it requires an absolute judgment?

A) multiple-choice
B) Likert
C) dichotomous
D) category

Question 16

The tendency for test takers to agree on most of the items is called a(n)

A) guessing threshold.
B) acquiescence response set.
C) item difficulty.
D) the miss rate.

Question 17

This test item is an example of a

A) polytomous format.
B) dichotomous format.
C) Likert format.
D) category format.

Question 18

Distractors that are obviously incorrect

A) lower the reliability of the test.
B) increase the reliability of the test.
C) have no impact on the reliability of the test.
D) reduce the likelihood of correct guessing.

Question 19

The difference between Likert scales and category formats is that

A) category formats are used only in health settings.
B) category formats tends to be dichotomous while Likert scales tends to be polytomous.
C) category formats tend to have a smaller number of choices.
D) Likert scales tend to have a smaller number of choices.

Question 20

One method for measuring chronic pain asks the respondent to group statements according to how accurately they describe his/her discomfort.This would be an example of the

A) Q-sort format.
B) checklist format.
C) Likert format.
D) category format.

Question 21

A multiple-choice test with five options has a chance performance level of

A) .50.
B) .25.
C) .20.
D) .10.

Question 22

Which of the following item writing recommendations has research support?

A) All answer options should be plausible.
B) Items should cover important concepts and objectives.
C) All parts of an item or exercise should appear on the same page.
D) There should be an equal number of true and false statements.

Question 23

Which method involves scoring that is very time consuming?

A) dichotomous format.
B) visual analogue scale.
C) Likert scale.
D) multiple-choice format.

Question 24

The optimum level of item difficulty for a five-alternative multiple choice item is

A) .50.
B) .60.
C) .70.
D) .80.

Question 25

For most tests,the maximum amount of information about differences between individuals can be obtained from items in the difficulty range of

A) .30 to .70.
B) .40 to .80.
C) between .55 and .85.
D) above .90.

Question 26

In the extreme group method of item analysis,

A) point-biserial correlations are used.
B) data from some test-takers are not used in the analysis.
C) only the performance of those who scored extremely well is studied.
D) distractors are eliminated.

Question 27

Which type of item tends to lose reliability and become obsolete over time?

A) factual items
B) skill-based items
C) items based on abstract concepts
D) simple items

Question 28

When Lupe argued that one of the questions on the five-alternative test was unfairly difficult,the teacher simply replied by saying that the item difficulty was optimal at

A) .50.
B) .60.
C) .625.
D) .70.

Question 29

The optimal item difficulty of a six-alternative test is

A) .50.
B) .585.
C) .60.
D) .625.

Question 30

If 50% of the individuals taking a particular test get a certain item correct,the difficulty (or easiness)level of that item would be

A) .05.
B) .25.
C) .50.
D) .10.

Question 31

How do Likert format tests differ from tests made of dichotomous and polytomous items?

A) Likert format tests require far fewer items to achieve reliability and validity.
B) Likert format items quantify characteristics rather than classifying responses as correct or incorrect.
C) Likert format tests cannot be validated whereas dichotomous and polytomous item tests can be validated.
D) Likert format items require higher literacy levels than do dichotomous and polytomous items.

Question 32

If the five applicants for the chief financial officer position of ABC Company are highly qualified,the company should use a test that

A) has easier items.
B) discriminates 20% of the time.
C) contains mostly difficult items.
D) contains items ranging in difficulty from .30 to .70.

Question 33

Which testing method is popular for measuring self-rated health?

A) q-sort technique
B) visual analogue scale
C) checklists
D) category formats

Question 34

What is the impact of adding distractors on polytomous item reliability?

A) The number of distractors is inversely related to item reliability.
B) Large numbers of distractors can greatly increase reliability.
C) Adding distractors may not increase reliability if the distractors are implausible.
D) Reliability is optimized when there are 8 to 10 distractors.

Question 35

Which of the following increases the likelihood that students will be likely to guess when they are not sure of the correct response on a multiple choice item?

A) when they expect a low grade
B) when the items are easy
C) when the course is a required course
D) when they dislike the subject

Question 36

Which of the following is a disadvantage of true-false tests?

A) They are typically only useful with simple information.
B) They encourage memorization without understanding.
C) They are difficult to administer.
D) They encourage rapid responding.

Question 37

The method of item analysis which looks at the correlation between performance on an item (correct or incorrect)and total test score is

A) the extreme group method.
B) the tetrachoric method.
C) the point-biserial method.
D) the item characteristic curve method.

Question 38

As the proportion of people who get an item on a test correct increases,the measure of item difficulty

A) decreases.
B) remains the same.
C) increases.
D) approaches chance.

Question 39

When teachers are initially told that the students they will be teaching are either not very imaginative or are very imaginative,ratings using an adjective checklist will tend to reflect this original assessment.This is an example of

A) the effect of context.
B) visual analogue.
C) low sample size.
D) forced choice effect.

Question 40

Why have checklists fallen out of favor?

A) They are simplistic.
B) They are prone to error.
C) They are difficult to write well.
D) They cannot be validated.

Question 41

In experimental psychology,the proportion of the top third of the class that correctly answered the last question of the final was .93 while .89 of the bottom third of the class answered correctly.The professor should decide not to include this question in the next final because the discrimination index indicates

A) negative discrimination.
B) chance level performance.
C) that students were incorrectly prepared.
D) that the item does not discriminate well.

Question 42

Exhibit 6-1

Refer to Exhibit 6-1.Which item discriminates well at low levels of performance but not at high levels?

A) item a
B) item b
C) item c
D) item d
E) item e

Question 43

In item analysis,the internal criteria against which items are evaluated refers to the

A) discrimination index.
B) total test score.
C) criterion.
D) predictor.

Question 44

The least frequent score in a frequency polygon is the

A) negative discriminator.
B) discrimination point.
C) antimode.
D) criterion.

Question 45

When test items are evaluated against total test score,we use a(n)

A) internal criterion.
B) external criterion.
C) multivariate analysis.
D) criterion referenced test.

Question 46

The approach to test construction in which the item characteristic curve for each individual item is analyzed is called

A) prophecy theory.
B) classical test theory.
C) item response theory.
D) item analysis theory.

Question 47

Exhibit 6-1

Refer to Exhibit 6-1.Which item is unrelated to total test score performance?

A) item a
B) item b
C) item c
D) item d
E) item e

Question 48

Exhibit 6-1

Refer to Exhibit 6-1.Which item discriminates at various levels of performance?

A) item a
B) item b
C) item c
D) item d
E) item e

Question 49

Dr.H likes to start off his tests with a few easier items in order to boost the confidence of the test takers.This is an example of

A) human factors.
B) the psychometric properties of the test.
C) optimum item difficulty.
D) item difficulty.

Question 50

The extreme group method and the point biserial method are both used to estimate

A) reliability.
B) validity.
C) discriminability.
D) difficulty.

Question 51

Professor Plum created class intervals from the test scores for his class.He made a line graph using these intervals on the X-axis and the proportion of students who answered a particular question correctly on the Y-axis.The result is

A) a discrimination index.
B) a correlation index.
C) an item characteristic curve.
D) a histogram.

Question 52

One of the major advantages of tests developed using item response theory is that they

A) can be easily adapted for computer administration.
B) are longer.
C) are easier to administer.
D) can be developed with little effort.

Question 53

In order to evaluate a criterion referenced test,the test was administered to a group of students who had studied a learning unit and to another group who had not studied the learning unit.For each item on the test,the criterion for mastery would be

A) the point-biserial correlation.
B) below the antimode.
C) above the antimode.
D) the validity coefficient.

Question 54

Proponents of criterion-referenced tests have criticized item analysis procedures because they

A) cannot be used for criterion-referenced tests.
B) have statistical flaws.
C) do not provide information about the type of errors that students make.
D) have no relevance for educational tests.

Question 55

The proportion of test takers that get a "good" item correct increases as a function of the

A) item characteristic curve.
B) total test score.
C) validity of the test.
D) item difficulty.

Question 56

When 100% of the test-takers get an item correct,the item will have a

A) low difficulty index (0%).
B) high discriminability index.
C) discriminability index of approximately .5.
D) very low discriminability index.

Question 57

Exhibit 6-1

Refer to Exhibit 6-1.Which item is inversely related to performance on the test?

A) item a
B) item b
C) item c
D) item d
E) item e

Question 58

An employment test attempted to find out if individuals who scored high on specific items that assessed an individual's 'ability to work well in a team' related strongly to the test as a whole.The purpose of the study was to evaluate

A) human factors.
B) optimum item difficulty.
C) item discriminability.
D) categories.

Question 59

An item characteristic curve that rises gradually and then turns down for people at the highest levels of performance

A) is likely to occur when students are making a wild guess.
B) can happen when 'none of the above' is one of the multiple choice options.
C) turns down at a point referred to as the antimode.
D) indicates an item with a high level of difficulty.

Question 60

The average of a series of item characteristic curves is known as

A) the average characteristic curve.
B) the standard error of the item characteristic.
C) a test characteristic curve.
D) the variance ratio curve.

Question 61

In order to choose questions for a final version of a test,the examiners created a graph with difficulty on one axis and discriminability on the other.The examiners should use the questions that

A) fall above the .50-point on the discriminability axis.
B) fall below the .50 point on the discriminability axis.
C) fall between .30 and .70 on difficulty and above .30 on discriminability.
D) fall above the .50 point on discriminability and difficulty.

Question 62

In most situations,a good test should contain items

A) from a wide range of difficulty levels.
B) at optimum difficulty levels.
C) at levels appropriate for the test taker.
D) mostly at or near average difficulty levels.

Question 63

Which of the following is true of the characteristic curve for a "good" test item?

A) It is normally distributed.
B) It is bimodal and positively skewed.
C) It has a gradual,positive slope.
D) It is negatively accelerated.

Question 64

Which of the following methods is used in the analysis of item discriminability?

A) test-retest
B) extreme group
C) characteristic curves
D) factor analysis

Question 65

Which type of test is especially helpful for evaluating progress in individualized programs of instruction?

A) peaked conventional
B) criterion-referenced
C) rectangular-referenced
D) dichotomous

Question 66

The proportion of responses that are expected to be correct for each level of ability can be represented by

A) an item characteristic curve.
B) a discrimination index.
C) a test characteristic curve.
D) the optimum difficulty.

Question 67

Your text presents empirical evidence that indicates school children tend to repeat the same kind of errors when given problems of a particular type.This highlights the

A) importance of ranking students.
B) need to provide corrective feedback.
C) concept of 'teaching to the test'.
D) ability of IRT procedures to initiate guidance.

Question 68

Peaked conventional tests present items

A) from a wide range of difficulty levels.
B) at optimum difficulty levels.
C) at levels appropriate for the test taker.
D) mostly at or near average difficulty levels.

Question 69

Item analysis is different from the classical method of reliability because

A) a smaller number of items leads to higher reliability in item analysis.
B) item analysis ignores total score.
C) item analysis describes results for individual items,whereas reliability describes results for the scale as a whole.
D) item analysis generally considers the validity of the test.

Question 70

The difference in proportion of correct responses for each item between the top third of the class and the bottom third of the class is an example of a(n)

A) point biserial correlation.
B) discrimination index.
C) item difficulty.
D) guessing threshold.

Question 71

What is the first step in developing criterion-referenced tests?

A) Getting expert agreement about how to construct the test
B) Clearly specifying the objectives in precise statements about what is to be learned
C) Identifying the types of people who are likely to take the test and either score very well or very poorly
D) Creating items with a wide range of difficulty and administering them to experts in the field

Question 72

Why should tests include a items from a variety of difficulty levels?

A) Students are less likely to guess when there is a wide range of item difficulty.
B) Good tests encourage students to do their best and a range of difficulty helps less confident students.
C) Good tests discriminate at a variety of difficulty levels.
D) Students like to have some easy items because they are more likely to respond correctly by chance.