Question 1

Test-retest reliability is an example of which of the following?

Accepted Answer

Test-retest reliability specifically measures the consistency or stability of a measurement over time, making it a measure of reliability over time.

Question 2

For a total of 18 possible agreements,there are seven times where Duck and Cow agreed on whether students in college fell asleep at their desks or not.What is the interrater reliability coefficient?

Accepted Answer

The interrater reliability coefficient can be calculated using the formula: 
(number of agreements) / (number of agreements + number of disagreements)
In this case, the number of agreements is 7 and the number of disagreements is 18 - 7 = 11. 
So the coefficient is: 
7 / (7 + 11) = 7 / 18 = 0.39
Therefore, the answer is B.

Question 3

What would the interrater reliability be for a 100-item measure in which the number of agreements between Rater 1 and Rater 2 was 80?

Accepted Answer

Interrater reliability in this context is calculated as the number of agreements between raters divided by the total number of items. Here, it is 80 agreements out of 100 items, which equals 0.80 or 80%.

Question 4

Parallel forms reliability is an example of which of the following?

Accepted Answer

Parallel forms reliability measures the consistency of results between two versions of a test, indicating equivalence.

Question 5

Interrater reliability is an example of which of the following?

Accepted Answer

Interrater reliability measures the degree of agreement between two or more raters or judges. Therefore, it is a measure of agreement.

Question 6

What is the difference between the TRUE score and the observed score?

Accepted Answer

The true score is the theoretical score that accurately reflects a person's ability or trait. The observed score, on the other hand, is the actual score obtained on a test or measure. The difference between the two is often referred to as the error score, which can be caused by various factors such as measurement error, situational factors, or personal factors. Therefore, B (error score) is the best choice as it distinguishes the difference between the two scores.

Question 7

The maximum level of validity possible is equal to ______.

Accepted Answer

The maximum level of validity possible is equal to the square root of the reliability coefficient. This is because validity cannot exceed the extent to which a measure is consistent (reliable), and the square root of the reliability coefficient mathematically represents this upper limit.

Question 8

If you want to know if several different forms of a test are reliable,what type of reliability do you use?

Accepted Answer

Parallel forms reliability assesses whether different versions of a test produce similar results. This is the appropriate type of reliability to use when evaluating the consistency of multiple forms of a test.

Question 9

For a total of 80 possible agreements,there are 60 times where Teacher A and Teacher B agreed on whether students in college checked their cellphone in the classroom.What is the interrater reliability coefficient?

Accepted Answer

The interrater reliability coefficient can be calculated using the formula: (number of agreements)/(total possible agreements) = 60/80 = 0.75. Therefore, the answer is D, 0.75.

Question 10

If your test is not reliable,you must try to lower your error.

Accepted Answer

If a test is not reliable, it means that there is a significant amount of error or inconsistency in the results. To improve reliability, efforts must be made to minimize the sources of error and increase consistency. Therefore, the statement is true.

Question 11

What would the interrater reliability be for a 50-item measure in which the number of agreements between Rater 1 and Rater 2 was 45?

Accepted Answer

To calculate interrater reliability, the number of agreements between raters is divided by the total number of items. In this case, 45 agreements divided by 50 items equals 0.90 or 90%. Therefore, the interrater reliability would be 0.90 or Choice C.

Question 12

Which of the following is the Greek letter associated with Cronbach's alpha,sometimes known as coefficient alpha?

Accepted Answer

Cronbach's alpha is represented by the Greek letter alpha (α).

Question 13

If you want to know if the items on a test assess one and only one dimension,what type of reliability do you use?

Accepted Answer

Internal consistency reliability is used to determine if the items on a test assess one and only one dimension. This is done by examining the correlation between the items on the test. If the items are highly correlated, it suggests that they are measuring the same construct and have internal consistency. This is different from parallel forms reliability, which is used to determine if two tests are equivalent, and test-retest reliability, which is used to determine if a test produces consistent results over time. Interrater reliability is used to determine if different raters or observers produce consistent ratings or observations.

Question 14

If you want to know whether a test is reliable over time,what type of reliability do you use?

Accepted Answer

Test-retest reliability is the type of reliability that measures the consistency of a test or measure over time. It is done by administering the same test or measure on two separate occasions, and then comparing the scores to see if they are consistent. This type of reliability is often used in fields like psychology or education, where it is important to know if a test or measure is consistent over time.

Question 15

Interrater reliability measures consistency ______.

Accepted Answer

Interrater reliability measures the consistency between different raters, so the correct choice is D, from rater to rater. Choices A, B, and C all refer to different types of reliability measures (test-retest reliability, alternate-forms reliability, and cross-test reliability, respectively) but not specifically to interrater reliability.

Question 16

For a total of 10 possible agreements,there are eight times where Bear and Cat agreed on whether children in ice cream commercials closed their eyes while eating ice cream or not.What is the interrater reliability coefficient?

Accepted Answer

The interrater reliability coefficient can be calculated by dividing the number of agreements by the total number of possible agreements. In this case, there were 8 agreements out of 10 possible, resulting in a coefficient of 0.80.

Question 17

When we calculate reliability,we know the observed score.What are the two unknown components of the reliability equation?

Accepted Answer

The two unknown components of the reliability equation are the true scores and the error scores. The reliability equation is: observed score = true score + error score. Therefore, in order to calculate reliability, we need to estimate the amount of variance in the observed scores that is due to true differences between individuals (i.e. the true scores) and the amount of variance that is due to measurement error (i.e. the error scores).

Question 18

If you want to know whether there is consistency in the rating of some outcome,what type of reliability do you use?

Accepted Answer

Interrater reliability is used to assess the degree of agreement among raters, making it the appropriate choice for determining if there is consistency in the rating of some outcome.

Question 19

Observed scores are commonly exactly the same as true scores in the social and behavioral sciences.

Accepted Answer

Observed scores can be influenced by measurement error, which means they can deviate from the true scores.

Question 20

For high reliability,you want reliability coefficients to be positive and as close as possible to 1.

Accepted Answer

This statement is true. Reliability coefficients range from 0 to 1, with higher values indicating greater reliability. Positive coefficients indicate a direct relationship between the observed scores and the true scores, which is desirable for high reliability.

Quiz 6: Understanding Reliability and Validity: Just the Truth