Deck 6: Just the Truth: An Introduction to Understanding Reliability and Validity

Question 1

What would the interrater reliability be for a 50-item measure in which the number of agreements between Rater 1 and Rater 2 was 45?

A) 0.45
B) 80%
C) 0.90
D) 95%

Answer 1

C

Question 2

What is the key to establishing criterion validity?

A) Number of criteria
B) Quality of the criterion
C) Type of correlation
D) Predictive ability

Answer 2

B

Question 3

What are the four levels of measurement?

A) Rank, ordinal, nominal, ratio
B) Ratio, interval, categorical, nominal
C) Nominal, ordinal, interval, ratio
D) Nominal, rank, interval, ordinal

Answer 3

C

Question 4

If you correlate scores from your test with some other valid measure that assesses the same set of abilities, what type of validity evidence are you collecting?

A) Criterion validity
B) Content validity
C) Construct validity
D) Internal consistency

Question 5

If you want to know that a test measures some underlying psychological construct, what type of validity evidence would you want to collect?

A) Content validity
B) Construct validity
C) Predictive validity
D) Criterion validity

Question 6

If the measures associated with a test are said to be consistent, you might conclude that the measure is which of the following?

A) Standard
B) Valid
C) Reliable
D) Concurrent

Question 7

Test-retest reliability is an example of which of the following?

A) Measure of consistency
B) Measure of stability
C) Measure of agreement
D) Measure of equivalence

Question 8

A score that you would actually record is an example of which of the following?

A) Observed score
B) True score
C) Error score
D) False score

Question 9

A measure of how stable a test is over time is an example of which of the following?

A) Interrater reliability
B) Test-retest reliability
C) Parallel forms reliability
D) Internal consistency

Question 10

Interrater reliability is an example of which of the following?

A) Measure of consistency
B) Measure of stability
C) Measure of agreement
D) Measure of equivalence

Question 11

Which of the following is the Greek letter associated with Cronbach's alpha, sometimes known as coefficient alpha?

Question 12

If I can say that my weekly statistics quiz fairly assesses the material covered, what source of validity evidence should I have collected?

A) Criterion validity
B) Content validity
C) Construct validity
D) Concurrent validity

Question 13

Which coefficient reflects the occurrence of perfect reliability?

A) +1.00
B) +.50
C) +.100
D) +2.00

Question 14

Parallel forms reliability is an example of which of the following?

A) Measure of consistency
B) Measure of stability
C) Measure of agreement
D) Measure of equivalence

Question 15

Two trained professionals observe the behavior of children in a classroom. They each rate observed behaviors using the same form, and the number of items that were rated the same is calculated. This is an example of which type of reliability?

A) Parallel reliability
B) Test-retest reliability
C) Interrater reliability
D) Internal consistency

Question 16

Which of the following is concerned with monitoring estimates of present performance and predictions of future performance?

A) Content validity
B) Criterion validity
C) Internal validity
D) Construct validity

Question 17

What type of validity is concerned with the adequate representation of test items?

A) Content validity
B) Predictive validity
C) Concurrent validity
D) Construct validity

Question 18

How does interrater reliability measure consistency?

A) Over time
B) From form to form
C) Across different tests
D) From rater to rater

Question 19

When we calculate reliability, we know the observed score. What are the two unknown components of the reliability equation?

A) Method and error scores
B) Means and standard deviations
C) Test-retest and interrater scores
D) True and error scores

Question 20

High school class rank is highly correlated with college GPA. This is an example of what type of validity?

A) Concurrent validity
B) Predictive validity
C) Content validity
D) Construct validity

Question 21

If you were interested in describing the order of variables along a continuum, what level of measurement would you use?

Question 22

What is the difference between reliability and validity?

Question 23

If in the underlying continuum we are measuring assumed equal intervals, at what level of measurement is the associated variable being measured?

Question 24

What are the traditional types of validity evidence?

Question 25

Which level of measurement is concerned only with the characteristics of an outcome that fits into a category?

Question 26

Explain the classical test theory formula for understanding observed scores. What is included in any observed score?

Question 27

What are the scales of measurement?

Question 28

Which level of measurement provides the most information about a variable?

Question 29

Name four types of reliability and what each type measures.

Question 30

How does error affect reliability? Write the general equation for reliability that demonstrates this concept.