Question 1

Cronbach's coefficient alpha is used for

Accepted Answer

Cronbach's coefficient alpha is used to determine the reliability of tests having items with answers that are weighted differently.

Question 2

A researcher who wants to establish score reliability in an observational study typically determines

Accepted Answer

The most common method for establishing score reliability in observational studies is to determine the agreement among multiple observers. This is typically done by having the observers independently assign scores to the same set of observations, and then calculating the extent to which their scores are in agreement. A common measure of agreement is Cohen's kappa coefficient. The other options given, correlating observer-assigned scores with an external criterion and examining the predictive validity of the observation, are both related to validity rather than reliability.

Question 3

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below.
-Refer to Exhibit 9-10.Direct comparison of item scores by formula.

Accepted Answer

Direct comparison of item scores by formula refers to the Kuder-Richardson method, which is a measure of internal consistency reliability for measures with dichotomous choices.

Question 4

Exhibit 9-9: Indicate the type of validity evidence that is represented in the following situations.
-Refer to Exhibit 9-9.Freshman college students who take a German placement test during orientation perform well in the German class in which they are placed by the test.

Accepted Answer

This situation represents predictive validity because it shows that the test scores (from the German placement test) are able to predict future performance (how well students perform in the German class they are placed in).

Question 5

The ability of a music aptitude test to differentiate among individuals who are known to differ in music ability provides evidence of the test's

Accepted Answer

The ability of a music aptitude test to differentiate among individuals with differing music abilities shows evidence of its validity. Validity refers to the extent to which a test measures what it is intended to measure. Objectivity refers to the extent to which a test is free from subjective bias, while reliability refers to the extent to which a test yields consistent results over time or across different raters. Consequential usefulness refers to the practical implications or consequences of test results.

Question 6

A test-retest reliability coefficient does not identify effects due to

Accepted Answer

A test-retest reliability coefficient only assesses the consistency of scores over time and does not identify effects due to practice, day-to-day fluctuations, or long-term change in behavior. However, it is possible for changes in the sample of items used on the test to impact the results, hence D is the correct choice.

Question 7

Exhibit 9-9: Indicate the type of validity evidence that is represented in the following situations.
-Refer to Exhibit 9-9.In validating a teacher-constructed test of writing ability, a researcher correlates the test scores with the grades the students are receiving on their writing exercises in class.

Accepted Answer

This situation represents concurrent validity evidence as the researcher is comparing the test scores with current performance measures (grades) rather than predicting future performance.

Question 8

Which of the following test characteristics would be most affected by using a short test rather than a longer one?

Accepted Answer

Reliability of a test often increases with the length of the test. Short tests may not cover the content or skills being assessed as comprehensively as longer tests, leading to greater variability in scores due to chance rather than true differences in ability or knowledge.

Question 9

Which of the following summarizes the relationship between the standard error measurement and a test's score reliability?

Accepted Answer

The standard error of measurement and a test's score reliability have an inverse relationship. The higher the test's reliability, the smaller the standard error of measurement. This is because a reliable test has less measurement error and therefore, the standard error of measurement is smaller.

Question 10

The most appropriate reliability index to use for a comprehensive multiple-choice final examination in an introductory college course is

Accepted Answer

The answer of The most appropriate reliability index to use...

Question 11

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below.
-Refer to Exhibit 9-10.Scores on odd items correlated with scores on even items across test takers.

Accepted Answer

The procedure described is split-half reliability, where the set of odd items is compared to the set of even items. This measures the internal consistency of the test by evaluating how well the two halves of the test items are related to each other.

Question 12

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below.
-Refer to Exhibit 9-10.Scores on Form A correlated with scores on Form B.

Accepted Answer

The procedures described indicate the use of alternate form reliability since the correlation is between two different forms of the test.

Question 13

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments.
-Refer to Exhibit 9-11.A personality test designed to assess an individual's self-concept.

Accepted Answer

To determine the score validity of a personality test designed to assess an individual's self-concept, the type of validity evidence needed is evidence based on measures of construct. This involves assessing whether the test accurately measures the construct of self-concept that it is intended to measure.

Question 14

Which of the following would represent discriminant evidence for the score validity of a test of numerical aptitude?

Accepted Answer

Discriminant evidence for validity is shown when a test does not correlate highly with measures of different constructs. A low correlation with reading comprehension indicates that the numerical aptitude test is measuring a different construct, supporting its discriminant validity.

Question 15

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below.
-Refer to Exhibit 9-10.Scores at time A correlated with scores at time B.

Accepted Answer

The described procedure indicates a test-retest reliability, which measures the consistency of scores over time by administering the same test twice and correlating the scores.

Question 16

One would not use the Kuder-Richardson 20 coefficient to estimate the score reliability of a comprehensive multiple choice final examination in a beginning psychology course because

Accepted Answer

The answer of One would not use the Kuder-Richardson 20...

Question 17

Exhibit 9-9: Indicate the type of validity evidence that is represented in the following situations.
-Refer to Exhibit 9-9.A respected university professor of American literature examines a test designed to measure student knowledge of American literature of the 19th century and states that, in his opinion, the test measures such knowledge.

Accepted Answer

The validity evidence represented in this situation is content validity. The university professor's opinion that the test measures knowledge of American literature of the 19th century is a form of content validity evidence.

Question 18

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments.
-Refer to Exhibit 9-11.A math test designed to predict achievement in mathematics.

Accepted Answer

Evidence based on the relationship to other variables is needed for a math test designed to predict achievement in mathematics because the validity of the test is determined by how well the test scores correlate with or predict future mathematics achievement, which is an external criterion.

Question 19

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments.
-Refer to Exhibit 9-11.A test designed to measure what fourth graders know about long division.

Accepted Answer

To determine the score validity of a test designed to measure what fourth graders know about long division, evidence based on content is the best choice. The test items should be reviewed by experts in the field of long division to ensure they are measuring what they are intended to measure.

Question 20

Low correlations with tests measuring different constructs provide ____ evidence for the score validity of an instrument.

Accepted Answer

Discriminant validity is demonstrated by low correlations between the measure and other tests measuring different constructs. This indicates that the measure is not simply measuring the same construct as other tests, and supports the validity of the measure. Convergent validity refers to high correlations between the measure and other tests measuring the same construct, concurrent validity refers to correlations with a criterion measure collected at the same time, and predictive validity refers to correlations with a criterion measure collected at a later time.

Quiz 9: Validity and Reliability

Cronbach's coefficient alpha is used for

A researcher who wants to establish score reliability in an observational study typically determines

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below. -Refer to Exhibit 9-10.Direct comparison of item scores by formula.

Exhibit 9-9: Indicate the type of validity evidence that is represented in the following situations. -Refer to Exhibit 9-9.Freshman college students who take a German placement test during orientation perform well in the German class in which they are placed by the test.

The ability of a music aptitude test to differentiate among individuals who are known to differ in music ability provides evidence of the test's

A test-retest reliability coefficient does not identify effects due to

Which of the following test characteristics would be most affected by using a short test rather than a longer one?

Which of the following summarizes the relationship between the standard error measurement and a test's score reliability?

The most appropriate reliability index to use for a comprehensive multiple-choice final examination in an introductory college course is

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below. -Refer to Exhibit 9-10.Scores on odd items correlated with scores on even items across test takers.

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below. -Refer to Exhibit 9-10.Scores on Form A correlated with scores on Form B.

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments. -Refer to Exhibit 9-11.A personality test designed to assess an individual's self-concept.

Which of the following would represent discriminant evidence for the score validity of a test of numerical aptitude?

Exhibit 9-10: Indicate the type of reliability that would result from the procedures described below. -Refer to Exhibit 9-10.Scores at time A correlated with scores at time B.

One would not use the Kuder-Richardson 20 coefficient to estimate the score reliability of a comprehensive multiple choice final examination in a beginning psychology course because

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments. -Refer to Exhibit 9-11.A math test designed to predict achievement in mathematics.

Exhibit 9-11: Choose the type of validity evidence that a researcher would need to determine the score validity of the following kinds of instruments. -Refer to Exhibit 9-11.A test designed to measure what fourth graders know about long division.

Low correlations with tests measuring different constructs provide ____ evidence for the score validity of an instrument.