Deck 4: Item Analysis and Standardization of Tests

Question 1

Item analyses are usually not worth the time and effort in terms of the improvements they make in test effectiveness.

Answer 1

False

Question 2

The point-biserial formula is useful in evaluating items on a test for predicting job performance.

Answer 2

True

Question 3

The difficulty of an item on a criterion-referenced test is the criterion score at which 50% or more of the examinees pass the item.

Answer 3

False

Question 4

The purpose of a criterion-referenced achievement test is to compare a student?s achievement with the achievement of other students.

Question 5

The optimum difficulty index (p) of a classroom achievement test varies with the number of item options.

Question 6

The difficulty index of a test item equals the difference in the proportions of examinees in the upper and lower groups on total test score who get the item right.

Question 7

The discrimination index of a test item equals the proportion of examinees in the upper and lower groups on total test score who get the item right.

Question 8

Every distracter in a multiple-choice item should be selected by approximately the same number of examinees who do not know the answer.

Question 9

The procedure for computing the discrimination indexes of items on criterion-referenced tests is the same as that for classroom achievement tests.

Question 10

An item response curve relates the performance of examinees on test items to their estimated standings on a hypothetical latent ability continuum.

Question 11

The difficulty level on an item characteristic curve is the score made by 50% of the examinees.

Question 12

Group tests of achievement are usually standardized on smaller samples than individual tests.

Question 13

Achievement tests are typically standardized on larger samples than other types of tests.

Question 14

An important purpose of test standardization is to determine the frequency distribution of raw scores in the standardization group.

Question 15

A standardized test has standard directions for administration and norms based on a representative sample of the target population.

Question 16

The most accurate and easily understood of all types of test norms are standard score norms.

Question 17

Simple random sampling is less likely than stratified random sampling to yield a sample that is representative of the target population.

Question 18

The traditional and most common method of interpreting a person?s score on a standardized test is to compare it with the scores of other people who have taken the test.

Question 19

The most basic of all standard scores are z scores.

Question 20

Norms are standards of acceptable performance for evaluating test scores.

Question 21

Norms obtained from stratified random samples are preferred to those obtained from simple random samples.

Question 22

Age and grade norms have equal units throughout the range of ages and grades.

Question 23

Percentile ranks are percentages, but percentiles are scores.

Question 24

With regard to whatever attribute is being measured, the difference between the 50th and the 55th percentiles is less than the difference between the 90th and 95th percentiles.

Question 25

Standard scores represent a higher-level, more precise form of measurement than age, grade, or percentile norms.

Question 26

Classroom teachers generally prefer age and percentile norms to standard score norms.

Question 27

Unlike other types of norms, stanine scores represent specific score points rather than score intervals.

Question 28

In item sampling, different sets of items are administered to different samples of people.

Question 29

Parallel forms of a test contain the same kinds of items of equal difficulty and have a high correlation with each other.

Question 30

The equipercentile method does not actually equate scores on two different tests.

Question 31

The validity of an item on a job selection test may be determined by relating responses to the item to a

A) critical incident.
B) job analysis.
C) performance criterion.
D) predictor variable.

Question 32

The purpose of an item analysis of a classroom achievement test is to

A) determine the external validity of the test.
B) identify the abilities and skills measured by the test.
C) improve the quality and validity of the test.
D) provide a basis for assigning test scores.

Question 33

An item analysis of a test is conducted to determine if the items are

A) correlated with the criterion measure.
B) functioning properly.
C) adequately standardized.
D) reliable but valid.

Question 34

The range of p (difficulty index) on a set of items comprising a four -option multiple-choice test should be approximately

A) -1.00 to 1.00.
B) .00 to 1.00.
C) .10 to .90.
D) .40 to .80.

Question 35

Fifteen people in a upper group consisting of 25 people get an item right and 5 people in a lower group of 25 people get the item wrong. The item has a difficulty index of __________ and a discrimination index of __________.

A) .40, .20
B) .40, .40
C) .60, .20
D) .80, .40

Question 36

Assume that of 55 persons who take a test, 10 of the 15 people falling in the upper group on total test score and 5 of the 15 falling in the lower group on total test score pass a given item. What is the item difficulty index?

A) .17
B) .33
C) .50
D) 1.00

Question 37

Assume that of 55 persons who take a test, 10 of the 15 people falling in the upper group on total test score and 5 of the 15 falling in the lower group on total test score pass a given item. What is the item discrimination index?

A) .17
B) .33
C) .50
D) 1.00

Question 38

Which of the following test items is best (a, b, c, or d), where p is the item difficulty index and D the item discrimination index?

A) p = .30, D = .50
B) p = .50, D = .50
C) p = .70, D = .20
D) p = 1.00, D = .50

Question 39

Assume that, for a given item, U = L = 10 and 80% of the examinees in the upper group but only 30% of those in the lower group get the item right. Then

A) p = .55, D = -.50.
B) p = .45, D = .60.
C) p = .55, D = .50.
D) p = 1.10, D = .25.

Question 40

The optimum item difficulty index for a multiple-choice item

A) is unaffected by the number of options.
B) varies directly with the number of options.
C) varies inversely with the number of options.
D) ranges from .30 to .70 for true-false tests.

Question 41

[Hint: D = (Up - LP)/U, and p = (Up + LP)/(U + L). Consequently, since the value of Up can never be greater than U, when D = 1.00, Up = U and LP = 0.)
When the number of examinees in the upper and lower groups are equal and the item discrimination index (D) equals 1.00, the item difficulty index (p) must be equal to

A) .00.
B) .50.
C) .75.
D) 1.00.

Question 42

If all the items on a test have high discrimination indexes, the test of necessity has a high

A) concurrent validity.
B) internal consistency.
C) level of difficulty.
D) predictive validity.

Question 43

Suppose that 15 of the 20 examinees who score above criterion level on a criterion-referenced test get an item right, but only 3 or the 30 scoring below criterion level get the item right. What is the item?s D index?

A) .40
B) .48
C) .60
D) .65

Question 44

The purpose of a criterion-referenced test is to determine

A) how effective scores are in identifying the differential strengths and weaknesses of examinees.
B) how predictive the scores on the test are of later performance.
C) how people score in relation to the scores of other people.
D) the standing of each person on certain educational objectives.

Question 45

On a multiple-choice item, the number of examinees who select each distracter should be approximately

A) equal for each distracter.
B) equal to zero if possible.
C) normally distributed across distracters.
D) the same as the number choosing the correct answer

Question 46

An item response curve is a plot of the

A) frequency distribution of examinees at each total test score range who get the item right.
B) point-biserial correlations between item and total test scores.
C) proportion of examinees who get an item right against the proportion who get it wrong.
D) proportion of examinees who get an item right against an external or internal criterion.

Question 47

If the response curve for item A has a steeper slope than the response curve for item B, then item A has a higher __________ index than item B.

A) characteristic
B) difficulty
C) discrimination
D) distracters uniformity

Question 48

The Rasch item-response model is a __________-parameter model.

A) one
B) two
C) three
D) four

Question 49

The most appropriate method for selecting a standardization group for an achievement test is

A) area sampling within blocks.
B) sampling from lists.
C) simple random sampling.
D) stratified random sampling.

Question 50

Which of the following types of norms represents measurement on an interval level?

A) age norms
B) grade norms
C) percentile norms
D) standard score norms

Question 51

Equal numerical differences imply equal differences in whatever is being measured only with

A) age norms.
B) grade norms.
C) percentile norms.
D) standard score norms.

Question 52

The mean and standard deviation of z scores are __________ and __________.

A) 0, 1
B) 5, 2
C) 50, 10
D) 100, 15
E) 500, 100

Question 53

Percentile is to percentile rank as

A) decile is to quartile.
B) percentage is to decile.
C) score is to percentage.
D) score is to quartile.

Question 54

The deviation IQs on the Wechsler Intelligence scales are

A) CEEB scores.
B) percentile ranks.
C) standard scores.
D) T scores.

Question 55

The use of either age norms or grade norms assumes that the rate of increase in achievement or ability is

A) greater during the early years or grades.
B) greater during the later years or grades.
C) highly variable from year to year.
D) the same from year to year or grade to grade.

Question 56

From which of the following are all the others derived?

A) AGCT scores
B) CEEB scores
C) T scores
D) z scores

Question 57

Given that the mean WAIS IQ is 100 and the standard deviation is 15, if Johnny?s WAIS IQ is 125 what is his corresponding z score?

A) -1.67
B) 1.67
C) 2.00
D) 2.33

Question 58

The problem of unequal score units is least troublesome in the case of __________ norms.

A) age
B) grade
C) percentile
D) standard score

Question 59

Approximately 68% of a normal distribution of test scores falls between a z score range of

A) -.5 to +.5.
B) -1.0 to +1.0.
C) -1.5 to +1.5.
D) -2.0 to +2.0.

Question 60

Which of the following types of scale is an ?open-ended? standard score scale?

A) CEEB
B) stanine
C) Wechsler IQ
D) z

Question 61

Wechsler deviation IQ is to Wechsler subtest scaled scores as __________ is to __________.

A) 15, 2
B) 15, 10
C) 100, 3
D) 100, 10

Question 62

Which of the following standard scores is equivalent to a percentile rank of 50?

A) z = 0
B) T = 50
C) AGCT = 100
D) all of the above
E) none of the above

Question 63

Wechsler deviation IQs range from approximately __________ to __________.

A) 20, 120
B) 40, 160
C) 55, 145
D) 25, 200

Question 64

What do the equipercentile, item response, and linear score transformation methods have in common? They are all ways of

A) analyzing test items.
B) equating tests.
C) scoring tests.
D) standardizing tests.

Question 65

How does the item analysis procedure with an internal criterion measure differ from that in using an external criterion measure?

Question 66

Describe the similarities and differences in item-analysis procedures for traditional classroom tests and criterion-referenced tests. Include a discussion of the item discrimination index, item difficulty index, point-biserial correlation, and other appropriate measures and methods.

Question 67

What is the difference between an item characteristic curve and an item response curve? Include in your discussion a description of differences among the various item response models.

Question 68

Describe three procedures for selecting a standardization sample for a test and the advantages and disadvantages of each.

Question 69

Describe the procedure for standardizing an achievement test, including sample selection, test administration, scoring, and types of norms computed.

Question 70

List and describe four kinds of norms, the types of tests with which each is used, and the merits and demerits of each kind of norm.