Deck 4: Reliability

Question 1

In psychological testing, which is the best synonym for reliability?

A) consistency
B) validity
C) continuity
D) difficulty

Answer 1

A

Question 2

Test reliability deals primarily with ___.

A) long-term stability
B) short-term stability
C) both long and short-term stability
D) neither long-term nor short-term stability

Answer 2

B

Question 3

What is the effect of "constant errors" on test reliability?

A) They increase reliability.
B) They decrease reliability.
C) They have no effect on reliability.
D) They increase reliability for low scores but decrease it for high scores.

Answer 3

A

Question 4

The scattergram is also known as a ___.

A) standard deviation
B) bivariate distribution
C) coefficient
D) test chart

Question 5

A numerical summary of the relationship depicted in a bivariate distribution is called a ___.

A) scattergram
B) linear regression
C) standard error of estimate
D) correlation coefficient

Question 6

The most widely used type of correlation is the ___.

A) point biserial r
B) contingency coefficient
C) Pearson correlation coefficient
D) Spearman rank order correlation

Question 7

Homoscedasticity is the assumption that the degree of scatter is _____ at any point along the prediction line.

A) greater
B) equal
C) less
D) varied

Question 8

Which of these correlation coefficients indicates the weakest relationship?

A) r = -.199
B) r = +.60
C) r = -.79
D) r = +.007

Question 9

The standard error of estimate for a predicted Y value will be zero, that is, there will be no error in prediction, when ___.

A) r is + or - 1.00
B) r is .00
C) r is .50
D) SD_y is 1.00

Question 10

The possible range of r is from ___.

A) -1 to 0
B) 0 to 1
C) -2 to 2
D) -1 to 1

Question 11

If the standing on X is of no help in predicting Y, then r is ___.

A) 0
B) -1.00
C) less than -1.00
D) indeterminate

Question 12

If the relationship between X and Y is curvilinear, then the Pearson correlation coefficient will ___.

A) always underestimate the true degree of relationship
B) sometimes underestimate the true degree of relationship
C) always overestimate the true degree of relationship
D) sometimes overestimate the true degree of relationship

Question 13

The standard error of estimate is really a type of ___.

A) mean
B) correlation
C) z-score
D) standard deviation

Question 14

When we are estimating a Y score (i.e., Y') based on knowledge of the relationship between X and Y, the basic definition of the prediction equation is ___.

A) Y' = a + bX
B) Y' = (b)(X)(a)
C) Y' = a + X + b
D) Y' = (X/b) + a

Question 15

Which of these correlation coefficients indicates the strongest relationship?

A) r = +.007
B) r = +.60
C) r = -.79
D) r = -.199

Question 16

Which is NOT one of the major sources of unreliability identified in the text?

A) content
B) scoring
C) personal conditions
D) format

Question 17

A chemistry professor uses four different forms of a test covering chapters 1-5. To what extent does performance depend on which form you take? This is a question about unreliability due to ___.

A) content
B) scoring
C) administrative conditions
D) personal conditions

Question 18

Which of the following is NOT a major source of unreliability?

A) test selection
B) test scoring
C) test administration conditions
D) test content

Question 19

Which source of unreliability does test-retest reliability NEVER address?

A) test content
B) test scoring
C) personal conditions
D) test administration conditions

Question 20

The intraclass correlation coefficient would be used to express reliability in which of the following situations?

A) The sample size is very large.
B) There is more than one form of the test.
C) There are more than two raters for the test.
D) The test was given at two different times.

Question 21

Variations in health and mood relate primarily to what source of unreliability?

A) content
B) scoring
C) administrative conditions
D) personal conditions

Question 22

Which expresses the correct relationship among true score (T), observed score (O), and error score (E)?

A) T = O $\pm$ E
B) E = T + O
C) O = T/ E
D) O = T x E

Question 23

A person's test score with all sources of unreliability removed is called the _____ score.

A) observed
B) true
C) error
D) actual

Question 24

The summation of all unsystematic influences on a person's test score is called the _______ score.

A) observed
B) true
C) error
D) actual

Question 25

A person's raw score on a test is the _________ score.

A) true
B) error
C) reliable
D) observed

Question 26

Consider the hypothetical distribution of many observed scores around the true score for an individual. If the test is highly reliable, then the observed scores will be ______ around the true score.

A) tightly clustered
B) widely scattered
C) moderately scattered
D) can't tell without more information

Question 27

Which is another name for test-retest reliability?

A) timing coefficient
B) stability coefficient
C) homogeneity coefficient
D) split-half coefficient

Question 28

Practice effects are mainly a concern for which type of reliability determination?

A) test-retest
B) split-half
C) Kuder-Richardson
D) alternate form

Question 29

Which would be a typical time interval for determining test-retest reliability?

A) 2 minutes
B) 2 hours
C) 2 weeks
D) 2 years

Question 30

The intraclass correlation coefficient (ICC) is applied when determining which type of reliability?

A) intra-form
B) test-retest
C) inter-scorer
D) split-half

Question 31

When determining inter-scorer reliability, it is important for the two (or more) scorers to work ___.

A) cooperatively
B) independently
C) partly cooperatively and partly independently
D) It depends on the type of test.

Question 32

Which of the following is the least frequently used method of expressing test reliability?

A) internal consistency reliability
B) test-retest reliability
C) inter-scorer reliability
D) alternate form reliability

Question 33

The most common way of dividing items in order to determine split-half reliability is to form two groups consisting of the _____ and _____ items.

A) odd…even
B) first half…last half
C) easy…hard
D) correct…incorrect

Question 34

The extent to which items in the test are measuring the same construct(s) or trait(s) is indicated by ___.

A) standard error of difference
B) Spearman-Brown correction
C) coefficient alpha
D) standard error of measurement

Question 35

What is the main reason that alternate form reliability is not used very often?

A) It is difficult to calculate.
B) Few tests have alternate forms.
C) It overestimates reliability.
D) Many tests are too short for this type of reliability.

Question 36

If true variance equals observed variance, then reliability (r) will be ___.

A) +1.00
B) .00
C) +.50
D) -.50

Question 37

Which is NOT one of the internal consistency methods of determining reliability?

A) coefficient alpha
B) split-half
C) odd-even
D) inter-scorer

Question 38

The odd-even method of determining reliability is a particular application of the _______ method.

A) split-half
B) coefficient alpha
C) inter-scorer
D) test-retest

Question 39

What correction is applied after determining the correlation between two halves of a test in order to express the reliability of a full-length test?

A) Alternate form
B) Cronbach's
C) Spearman-Brown
D) Pearson's

Question 40

We know the reliability of a 90-item test. We wish to estimate the reliability of the test if we use only 30 items from it. What formula will help make this estimation?

A) McGowan's
B) Cronbach's
C) Spearman-Brown
D) Pearson's

Question 41

What generalization is incorporated into the Spearman-Brown formula.

A) Test length is unrelated to reliability
B) Short tests are generally more reliable.
C) Longer tests are generally more reliable.
D) Validity and reliability are really the same thing.

Question 42

You compare a student's scores on reading and math tests. So, what you are really interpreting is a difference. What would you expect the reliability of the difference to be in comparison to the reliabilities of the reading and math tests? Reliability of the difference is likely ___

A) The average of the reliabilities of the two original tests
B) Noticeably less than reliabilities of the two original tests
C) Noticeably more than reliabilities of the two original tests
D) Somewhat higher than the reliabilities of the two original tests

Question 43

Which source of unreliability does coefficient alpha measure?

A) scoring
B) content
C) administrative conditions
D) personal conditions

Question 44

Coefficient alpha is also known as ___.

A) Cronbach's alpha
B) Hogan's alpha
C) coefficient beta
D) the ABC factor

Question 45

The value of coefficient alpha depends on ___.

A) item intercorrelations and number of items
B) average test score and number of examinees
C) number of items and number of examinees
D) item intercorrelations and average (mean) test score

Question 46

In general, internal consistency reliability is maximized when the average item difficulty is -

A) .10
B) .50
C) .75
D) .95

Question 47

The standard error of measurement is a type of ___.

A) correlation coefficient
B) mean (average)
C) standard deviation
D) significance test

Question 48

The mean and SD for a test are 80 and 10, respectively. The test-retest reliability is .84. The standard error of measurement (SEM) is ___.

A) 1
B) 4
C) 8
D) 8.4

Question 49

The standard error of measurement (SEM) is the standard deviation of a hypothetically infinite number of obtained scores around the test-taker's ___.

A) raw score
B) error score
C) observed score
D) true score

Question 50

What generalization did the text state about the reliability of differences between two test scores? Reliability of the difference is often ____

A) Quite low
B) Very high
C) Difficult to calculate
D) Completely unknown

Question 51

Which is used to create a confidence band around an individual's test score?

A) standard error of estimate
B) standard error of the mean
C) standard error of measurement
D) standard error of reliability

Question 52

If test reliability is perfect (+1.00), then the standard error of measurement equals ___.

A) -.50
B) -1.00
C) +.50
D) 0 (zero)

Question 53

As test reliability approaches 0 (zero), the standard error of measurement approaches ___.

A) infinity
B) 0 (zero)
C) the test's mean
D) the test's standard deviation

Question 54

What units should be used to report the standard error of measurement for a test?

A) always z-scores
B) always raw scores
C) whatever score system is used for interpretation
D) whatever score system yields the lowest standard error

Question 55

The standard deviation of hypothetical distribution of obtained scores around the true score is called the ___.

A) standard error of true
B) standard error of measurement
C) standard deviation of true
D) standard deviation of the mean

Question 56

What generalization does the text give about the reliability of clusters of 3 or 4 items? The reliability of such clusters is usually ___.

A) surprisingly high
B) moderate
C) notoriously low
D) negative

Question 57

What generalization does the text make about the reliability of profiles? Profile reliability ___.?

A) is less than the reliabilities of the individual tests in the profile
B) is greater than the reliabilities of the individual tests in the profile
C) is the same as the reliabilities of the individual tests in the profile

Question 58

What generalization does the text make about the reliability of subscores within total scores? Subscore reliability is almost always ___ the reliability of the total score.

A) less than
B) greater than
C) the same as

Question 59

The precision of measurement index obtained in item response theory is a type of _________ reliability.

A) test-retest
B) inter-scorer
C) internal consistency
D) alternate form

Question 60

The technical term for the precision of measurement index obtained in item response theory is ___.

A) pm-irt
B) SE (M)
C) PoM
D) SE ( $\Theta$ )

Question 61

Which statistical method provides the framework for describing generalizability theory?

A) factor analysis
B) analysis of variance
C) multiple correlation
D) descriptive statistics

Question 62

The principal purpose of generalizability theory is to investigate ___.

A) several examinees at once
B) several different tests at once
C) several sources of unreliability at once
D) several types of validity at once

Question 63

A test must be _____ in order to be _____.

A) short…reliable
B) machine-scored…reliable
C) published…useful
D) reliable…valid

Question 64

Using the techniques of analysis of variance, _____ attempts to apportion several sources of unreliability in a single study.

A) item response theory
B) classical test theory
C) generalizability theory
D) standard score theory

Question 65

Of the several factors the text lists as affecting the magnitude of correlation coefficients, which one is most frequently important for interpreting reliability coefficients?

A) correlation is a matter of relative position
B) curvilinearity
C) heteroscedasticity
D) group variability

Question 66

What level of reliability is recommended for tests that affect important decisions about individuals?

A) .65
B) .75
C) .85
D) .95

Question 67

Which of these levels of reliability would be considered "moderate?"

A) .50
B) .65
C) .80
D) .95

Question 68

Applying the "correction for range restriction" has the greatest affect on what magnitude of correlation coefficient?

A) Very low coefficients
B) Moderate coefficients
C) Very high coefficients

Question 69

We observe that a correlation coefficient was determined on a group much more homogeneous than the group in which we are interested. To estimate the correlation in the group in which we are interested, we would apply the formula for _______.

A) Correction for range restriction
B) Correction for unreliability
C) Correction for continuity
D) Correction for small size