Question 1

Jennifer took an aptitude test and is interested in whether the score on the verbal aptitude subscale is significantly higher than her mathematical aptitude subscale score.What statistic would you be interested in to answer her question?

Accepted Answer

To compare the scores on two different subscales of an aptitude test, we would use the standard error of difference to determine if the difference between the scores is statistically significant. The standard error of estimate and standard error of measurement would not be appropriate in this situation, as they are used for different purposes. The standard deviation alone would not tell us if there is a significant difference between the two scores.

Question 2

The Spearman-Brown formula is used:

Accepted Answer

The Spearman-Brown formula is specifically used to correct a split-half reliability coefficient.

Question 3

The ABC Self-Concept Inventory has a split-half reliability coefficient of .90, and the XYZ Self-Concept Inventory has a Spearman-Brown reliability coefficient of .90.If all other factors are equal, which instrument would you choose to use?

Accepted Answer

The answer of The ABC Self-Concept Inventory has a split-half...

Question 4

In generalizability or domain sampling theory, the focus is on:

Accepted Answer

Generalizability theory (or domain sampling theory) seeks to identify specific sources of variation that affect the reliability of an assessment in order to determine the conditions under which the results obtained can be generalized. This theory is concerned with identifying how the assessment performs across different conditions, such as different raters, different tasks, or different settings. Therefore, the focus is on identifying specific sources of variation under defined conditions. Measures of internal consistency and standard error of measurement are important in other types of reliability analyses but are not central to generalizability theory. Identifying an individual's true score would be part of the interpretation of the results, but it is not the main focus of generalizability theory.

Question 5

Which of the following is appropriate for determining the reliability of a criterion-referenced instrument?

Accepted Answer

Criterion-referenced instruments are not designed to be reliable in the traditional sense. They are designed to measure a person's performance against a set of criteria, not against other people.

Question 6

As the reliability of an instrument increases, the standard error of measurement _______.

Accepted Answer

The standard error of measurement (SEM) is inversely related to reliability. As reliability increases, the SEM decreases. This means that there is less error in the measurement of scores, and therefore scores are more accurate and precise.

Question 7

In evaluating an instrument's reliability, a counselor should:

Accepted Answer

The counselor should consider how the instrument is going to be used because the context and purpose of the assessment can influence the acceptable level of reliability. Different uses may require different levels of reliability.

Question 8

If an instrument requires some professional judgments in scoring, then the manual should also include information on:

Accepted Answer

Interrater reliability information is necessary if professional judgment is needed in scoring an instrument to ensure that different raters would come up with consistent results. KR 20s and KR 21s are coefficients of internal consistency, which may not be relevant for instruments with professional judgments in scoring. The Spearman-Brown formula is used to estimate the reliability of a test when its length is changed, which is also not relevant in this case. Test-retest reliability is not necessarily needed when professional judgment is required for scoring.

Question 9

The reliability of Test LMN was estimated by three methods: 1) Spearman-Brown, 2) test-retest, and 3) coefficient alpha.Which method probably yielded the lowest reliability coefficient?

Accepted Answer

The answer of The reliability of Test LMN was estimated...

Question 10

Measurement experts generally suggest that counselors should use ________ in interpreting a client's test score results.

Accepted Answer

Using standard error of measurement in interpreting a client's test score results is considered best practice among measurement experts. This helps counselors understand the amount of error present in the test results and whether a client's score falls within an acceptable range.

Question 11

Joe had a score of 72 on the Counseling Aptitude Scale, and the standard error of measurement of the scale is 3.Where would we expect Tom's true score to fall 99.5 percent of the time?

Accepted Answer

The true score is expected to fall within a range determined by the standard error of measurement. For a 99.5% confidence interval, we use approximately 3 standard errors (since 99.5% is close to 3 standard deviations in a normal distribution). Thus, the range is 72 ± 3*3, which is 63 to 81.

Question 12

Standard error of measurement is designed to:

Accepted Answer

Standard error of measurement provides an estimate of the probable range of scores for an individual, taking into account the errors in measurement. It does not directly indicate the reliability of an instrument or the statistical significance of the reliability coefficient. The percentage of error in the reliability coefficient is not related to standard error of measurement.

Question 13

The decision to use either the Kuder-Richardson 20 (KR 20) or the Kuder-Richardson 21 (KR 21) is based on whether:

Accepted Answer

The decision to use either KR 20 or KR 21 depends on whether the items in the test measure a homogeneous or heterogeneous behavior domain. KR 20 is used for tests that measure a homogeneous behavior domain (items are similar in content and difficulty level), while KR 21 is used for tests that measure a heterogeneous behavior domain (items are different in content and difficulty level). Therefore, the best choice is C.

Question 14

If the reliability coefficient of an instrument is .91, and the standard deviation is 10, then a client's score of 59 could be interpreted that 95 percent of the time his/her true score will fall between _____ and _____ using standard error of measurement

Accepted Answer

To find the interval within which the true score of the client falls, we use the formula:
True Score = Observed Score +/- (SEM x z-score)
SEM = standard deviation * sqrt(1 - reliability coefficient)
Where z-score for a 95% confidence interval is 1.96
Plugging in the values given, we get:
SEM = 10 * sqrt(1 - 0.91) = 3.16
True Score = 59 +/- (3.16 x 1.96) = 59 +/- 6.21
Therefore, the interval within which the true score falls is 52.79 to 65.21. The closest choice is B, 53 and 65.

Question 15

The standard error of measurement is best used for _______ and the reliability coefficient is best used for _______.

Accepted Answer

The standard error of measurement is used to interpret the precision of an individual score, while the reliability coefficient is used to compare the consistency of scores across different measurements or instruments. Therefore, option B is the best fit for these concepts.

Question 16

What is the most appropriate way to determine reliability with Likert scales, where different answers receive different weightings?

Accepted Answer

Coefficient alpha (or Cronbach's Alpha) is the most appropriate way to determine reliability with Likert scales, where different answers receive different weightings. This method calculates the internal consistency of the items within the scale and provides a coefficient ranging from 0 to 1. A coefficient alpha of 0.7 or higher is typically considered acceptable for research purposes.

Question 17

A different reliability model from the "true score" or "classical" model is the:

Accepted Answer

The generalizability or domain sampling model is a different reliability model from the true score or classical model. This model allows for the examination of reliability across different sources of variation, such as different domains or contexts, and provides a more comprehensive measure of reliability. The other options listed (internal consistency, performance evaluation, and standard error of measurement models) are all variations or extensions of the true score model, rather than distinct reliability models in their own right.

Question 18

The standard deviation of a particular subtest of the WISC-IV is 3, and the reliability coefficient is .84.What is the standard error of measurement of that subtest?

Accepted Answer

The formula to calculate the Standard Error of Measurement (SEM) is:
SEM = Standard Deviation * sqrt(1 - reliability coefficient)

Plugging in the values given in the question, we get:
SEM = 3 * sqrt(1 - 0.84)
SEM = 3 * sqrt(0.16)
SEM = 3 * 0.4
SEM = 1.2

Therefore, the standard error of measurement of the subtest is 1.20. Hence, the correct answer is B.

Question 19

In general, the reliability coefficients for instruments designed for infants and young children are _____________ those of instruments designed for adolescents and adults.

Accepted Answer

The reliability coefficients for instruments designed for infants and young children are typically lower than those of instruments designed for adolescents and adults. This is due to the challenges in measuring cognitive and behavioral abilities in very young children, such as limited attention spans and communication skills. Therefore, researchers and clinicians need to be mindful of the limitations of the instruments used for assessing young children and interpret the results with caution.

Question 20

According to the Standards for Educational and Psychological Testing, which statistic should be used for the interpretation of group test scores?

Accepted Answer

The Standards for Educational and Psychological Testing recommend using the standard error of the observed score means for the interpretation of group test scores. This statistic reflects the amount of error present in the group test scores and can help determine the degree of confidence in the mean score as a representation of the group's true ability level.

Quiz 3: Reliability