Deck 11: Measurement Reliability

Question 1

An assessment tool with high measurement reliability:

A) Decreases the amount of error variance in an observed score
B) Decreases the amount of systematic variation in an observed score
C) Has a low correlation coefficient
D) Has no relation between the true and observed scores

Answer 1

Decreases the amount of error variance in an observed score

Question 2

Which of the following is true about measurement reliability?

A) An r of +.90 indicates a strong relation between two scores
B) An r of .65 is not acceptable for measures used to make personnel decisions
C) A high negative correlation is not acceptable
D) All of the above are true

Answer 2

All of the above are true

Question 3

Dr. Allport wants to establish test-retest reliability for a new measurement tool on personality. Which of the following is true?

A) Dr. Allport should use a group of people similar to his target population to test the instrument
B) Dr. Allport should split the participants into two groups. He gives the test to one group, waits two weeks, and gives the test to the second group
C) An r of .55 is acceptable for a personality measurement tool
D) Dr. Allport should divide the high item pool in half and give these two different measures to the same people a week apart
E) None of the above

Answer 3

Dr. Allport should use a group of people similar to his target population to test the instrument

Question 4

Which of the following is true of test-retest reliability?

A) r is established during the study
B) An investigator should always collect test-retest data during a study
C) As the length of time increases between test administrations, r decreases
D) b and c
E) All of the above

Question 5

Which of the following is true of parallel forms reliability?

A) A relation (r) is established between two forms of the same test
B) At least three weeks should elapse between administration of the two test forms
C) Different populations of people (similar to the target population) should receive the two different test forms
D) b and c
E) All of the above

Question 6

A researcher wants to determine internal consistency for the items on a personality assessment tool he has developed. Which of the following is true?

A) He must correlate different administrations of the same instrument
B) He can correlate two halves of the same test
C) He can compare the odd items with the even items
D) b and c

Question 7

Dr. Dread has collected data on aggression using a measure with true/false items. She wants to determine the instrument's internal consistency. Which statistic should be used?

A) Spearman-Brown formula
B) Kuder-Richardson 20
C) Cronbach's alpha
D) Either b or c

Question 8

Dr. Dread is developing a measure of aggression that uses Likert scales. She wants to determine the instrument's internal consistency. Which statistic should be used?

A) Spearman-Brown formula
B) Kuder-Richardson 20
C) Cronbach's alpha
D) Standard error of measurement

Question 9

Dr. Feelgood reports a Cronbach's alpha of .95 for a scale of 6 Likert items that measures compassion. She may legitimately make which of the assumptions or conclusions?

A) Overall item intercorrelations are high
B) The items may be measuring more than one construct
C) Another index of reliability should be provided
D) All of the above.

Question 10

Dr. Feelgood is developing an instrument on tender loving care. She obtains a reliability coefficient of .85. The standard error of measurement is 6.2. A nursing student takes the test and has a score of 80 (on a 0 to 100 scale). Which of the following is true?

A) With a confidence interval of 95%, the true score falls between approximately 68 and 92
B) With a confidence interval of 95%, the true score falls between 73.8 and 86.2
C) The precision with which the true score is estimated is decreased due to a low reliability coefficient
D) b and c

Question 11

Dr. Morgan wants to use the intraclass correlation coefficient (ICC) to assess interrater reliability in rating mastery motivation. Which of the following is true?

A) There can only be two observers' scores to correlate
B) The Kappa statistic should be used to correlate observer's scores
C) The mastery ratings should use an interval scale
D) a and c

Question 12

Measurement reliability refers to consistency throughout a series of measurements.

Question 13

Measurement reliability influences the accurate assessment of a study's results.

Question 14

When r is close to -1.00 or to +1.00, there is evidence for an instrument's reliability.

Question 15

Carryover effects are a problem when establishing test-retest reliability for a measurement tool.

Question 16

Parallel forms reliability is also known as the coefficient of stability.

Question 17

An r of .65 is acceptable when establishing parallel forms reliability.

Question 18

Interitem reliability can only be used when there are several test items combined to make a composite score.

Question 19

The Spearman Brown formula must be used to adjust r when determining split-half reliability.

Question 20

Cronbach's alpha is the most common means of establishing internal consistency in educational and psychological research.

Question 21

All other things being equal, a shorter test is more reliable than one with more items.

Question 22

Interobserver reliability is frequently used in research to establish agreement between two or more observers.

Question 23

The Kappa statistic is used to determine intraclass correlation coefficients when the data are ordinal level.

Question 24

If a measurement tool has high reliability, it must have high validity as well.

Question 25

Is Cronbach's alpha an indicator of whether a scale is unidimensional? Why or why not?

Question 26

What factors affect the size of a reliability coefficient? Cronbach's alpha?

Question 27

What is the relationship between the observed score, the true score, and error?

Question 28

In studies, what two reliability coefficients need to be reported?

Question 29

Can a reliability coefficient be negative? Why or why not?