Deck 7: Reliability of Selection Measures

Question 1

A split-half reliability estimate is NOT a pure measure of internal consistency.

Answer 1

True

Question 2

The higher the test-retest reliability coefficient, the greater the true score and the less the error.

Answer 2

True

Question 3

A selection measure is internally consistent or homogeneous when individuals' responses on one part of the measure are unrelated to their responses on other parts.

Answer 3

False

Question 4

With a long time interval between administrations of a measure, test-retest reliability may underestimate reliability.

Question 5

To achieve a parallel forms reliability estimate, at least two equal versions of a measure must exist.

Question 6

Because of the way it is calculated, a higher reliability coefficient is desirable.

Question 7

If all respondents on a selection measure remember their previous answers to an initial administration of a measure and then on the retest respond according to their memory, the reliability coefficient will decrease.

Question 8

Surprisingly, increasing the lengths of time between administrations does not reduce the impact of memory effects on reliability.

Question 9

Reliability coefficients computed between parallel forms tend to be conservative estimates.

Question 10

Reliability of measurement in selection is synonymous with dependability, consistency, or stability of measurement.

Question 11

The higher the value of a reliability coefficient, the less the measurement error.

Question 12

To control the effects of memory on test-retest reliability estimates, the same measure should be used the second time.

Question 13

Selection measures that are designed to assess job-related characteristics are more precise than measures of physical characteristics.

Question 14

With increasing time intervals, test-retest reliability coefficients will generally decrease.

Question 15

Error score represents errors of measurement.

Question 16

When a measure is perfectly reliable, its obtained score is higher than its true score.

Question 17

A split-half reliability overestimates actual reliability.Therefore, a special formula, the Spearman-Brown prophecy formula, is used to make the correction.

Question 18

In general, the amount of measurement error has little effect on how high the reliability of measurement error will be.

Question 19

Selection measures involving traits of personality, attitudes, or interests are usually considered to be fairly static yielding high reliability coefficients.

Question 20

Reliability is generally determined by examining the relationship between two sets of measures measuring the same thing.

Question 21

Reliability is a group-based statistic.

Question 22

If coefficient alpha reliability is unacceptably low, then the items on the selection measure may be assessing more than one characteristic.

Question 23

In the context of personnel selection, the reliability of criterion measures need not be as high as predictor measures.

Question 24

The standard error of measurement is affected by variability within the group of respondents to whom a measure has been administered.

Question 25

A good rule of thumb is that reliability must be .90 or higher.

Question 26

If our standard error is 3.16 and the difference between two applicants' scores is 3, then it is possible that the difference in scores is due to chance.

Question 27

Reliability is a necessary but not sufficient condition for validity.

Question 28

Kuder-Richardson reliability estimates are usually lower than those obtained from split-half estimates.

Question 29

Interrater agreement indices are generally restricted to interval or ratio data.

Question 30

Unreliable performance by a respondent on a reliable measure is possible, but reliable performance on an unreliable measure is impossible.

Question 31

Although interrater agreement indices have their limitations, they are still widely used in selection research.

Question 32

In general, as the length of a measure decreases, its reliability increases.

Question 33

Split-half reliability procedures tend to produce a conservative estimate of reliability.

Question 34

Selection measures are not simply "reliable" or "not reliable," there are degrees of reliability.

Question 35

The standard error of measurement is another approach for estimating reliability.

Question 36

Kuder-Richardson reliability procedures are rarely used.

Question 37

Interrater reliability estimates test the hypothesis that ratings are determined by characteristics of the rater rather than by what is being rated.

Question 38

If variability or individual differences increase among respondents while variation within individuals remains the same, reliability will increase.

Question 39

Tests with many items that are very difficult are more reliable than tests containing many items of moderate difficulty.

Question 40

As the number of response options or categories on a measure increases, reliability also increases.

Question 41

In order to calculate an intraclass correlation, how many raters are necessary?

A)2
B)2 or more
C)more than 2
D)any number will do

Question 42

Generally speaking, the greater the variability or standard deviation of scores on the characteristic measured, the higher the reliability of the measure of that characteristic.

Question 43

What is a correlation coefficient calculated between two sets of scores over time called?

A)coefficient of stability
B)coefficient alpha
C)coefficient of equivalence
D)coefficient of dependability

Question 44

Test-retest reliability estimation is most appropriate for which of the following?

A)mental ability
B)attitudes
C)self-esteem
D)self-concept

Question 45

What is a true score?

A)the score obtained for a person under normal conditions
B)the score obtained because of the presence of external factors
C)the mean/average score made by a person on many different administrations of tests
D)the standard deviation on many different administrations of the same test on the same individual

Question 46

With a long time interval between administrations of a measure (test-retest), what could cause scores to change resulting in an underestimate of the reliability?

A)reasoning
B)thinking
C)memory
D)learning

Question 47

An obtained score consists of which two components?

A)controllable and uncontrollable
B)systematic and unsystematic
C)true and error
D)true and predictive

Question 48

Calculation of reliability estimates results in a coefficient ranging from _____ to _____.

A)0, 1.96
B)0.00; 1.00
C)-1.00; 1.00
D)-1.00; 0.00

Question 49

What reliability estimate consists of administering the same selection measure twice and correlating the two sets of scores?

A)parallel forms
B)internal consistency
C)split-half
D)test-retest

Question 50

Which of the following is not a method for estimating internal consistency reliability?

A)parallel or equivalent forms reliability
B)Kuder-Richardson reliability
C)Cronbach's coefficient alpha reliability
D)split-half reliability

Question 51

Among the most popular internal consistency methods are all of these EXCEPT:

A)Cronbach's coefficient alpha reliability
B)Kuder-Richardson reliability
C)Split-half reliability
D)Guion's measurement

Question 52

For a test with a time limit (i.e., a speed test), which reliability estimation procedure is not appropriate?

A)test-retest
B)split-half
C)parallel or equivalent forms (immediate administration)
D)parallel or equivalent forms (long-term administration)

Question 53

What is the difference between interclass and intraclass correlations (reliability estimates)?

A)minimum number of targets being rated
B)minimum number of equivalent forms being used
C)minimum number of raters needed for calculation
D)minimum number of attributes measured

Question 54

As the coefficient approaches 1.00, the set of measures is viewed as:

A)equivalent.
B)identical.
C)very different.
D)unrelated.

Question 55

In order to calculate an interclass correlation, how many raters are necessary?

A)2
B)2 or more
C)more than 2
D)more than 3

Question 56

Which of the following would NOT be a likely cause of interrater disagreement?

A)Raters view the same behavior differently.
B)Raters interpret the same behavior differently.
C)Error in rating or recording each impression.
D)Length of time behavior is displayed.

Question 57

What impact does memory have on a test-retest reliability estimate?

A)It is not possible to determine its effect on reliability.
B)It has no effect on test-retest reliability.
C)It will underestimate the true reliability of obtained scores.
D)It will overestimate the true reliability of obtained scores.

Question 58

For which of the following selection measures is it most appropriate to use equivalent forms for reliability estimation?

A)vocabulary
B)biographical inventory
C)personality inventory
D)physical fitness

Question 59

How many test administrations do you need in order to calculate a split-half reliability estimate?

A)1/2
B)1
C)2
D)1/4

Question 60

Which of the following is NOT one of the categories of statistical procedures for estimating interrater reliability?

A)interclass correlation
B)intraclass correlation
C)interrater agreement
D)underclass correlation

Question 61

If we have a test called "x" and rxx = .80, this means

A)80% of the differences in test scores is due to error and only 20% is due to true variance.
B)20% of the test scores were used to obtain the reliability estimate.
C)20% of the differences in test scores is due to error and 80% is due to true variance.
D)the test average is in the low 'B' range.

Question 62

If rxx = .85, and the standard deviation of x is 10, then the standard error of measurement for measure x is

A)3.873
B)3.16
C)3.50
D)3.30

Question 63

Research has shown that the reliability of rating scales can be improved by offering from _____ to _____ rating categories:

A)1, 4
B)1, 5
C)3, 7
D)5, 9

Question 64

.The difference between two individuals' scores should not be considered significant unless the difference is at least ___________ the standard error of measurement of the measure.

A)equal to
B)twice
C)three times
D)four times