Question 1

Test scores may be affected in pre- and post-testing by
A) therapy.
B) medication.
C) education.
D) All of these

Accepted Answer

Therapy, medication, and education can all have an impact on test scores. Therefore, all of these can affect pre- and post-testing results.

Question 2

Rating errors
A) may be unintentional.
B) may be intentional.
C) may involve a tendency to be lenient in rating.
D) All of these

Accepted Answer

Rating errors can occur both intentionally and unintentionally, and can involve a tendency to be lenient or harsh in rating. Therefore, all of the given options are possible rating errors.

Question 3

A supervisor unintentionally rates his supervisees less favorably than they really deserve.Which type of error is at work here?
A) unconscious error
B) severity error
C) random error
D) vocational error

Accepted Answer

Severity error occurs when a rater consistently rates employees lower than they actually deserve. This error may be due to a supervisor's tendency to focus on employees' mistakes rather than their accomplishments, leading them to undervalue their performance.

Question 4

If a test is a valid measure of a particular construct,we would expect that
A) groups of people who differ with respect to the construct will obtain different test scores.
B) groups of people who differ with respect to the construct will obtain similar test scores.
C) groups of people who obtain similar scores will have similar personalities.
D) None of these

Accepted Answer

If a test is a valid measure of a particular construct, it means that the test is accurately measuring what it is intended to measure. Therefore, individuals who have higher levels of the construct should score higher on the test, while individuals who have lower levels of the construct should score lower on the test. This implies that groups of people who differ with respect to the construct will obtain different test scores.

Question 5

Which term is used to refer to the tendency of a rater to evaluate ratees higher than they objectively deserve because of the rater's inability to discriminate between aspects of the ratee's behavior?
A) halo effect
B) random error
C) generosity error
D) severity error

Accepted Answer

The halo effect is the tendency of a rater to evaluate ratees higher than they objectively deserve because of the rater's inability to discriminate between aspects of the ratee's behavior. This occurs when a rater lets one positive attribute of a ratee influence his or her judgment of all other aspects of the ratee's behavior.

Question 6

A statistically insignificant correlation between scores on a new test of depression and a well established measure of satisfaction with life may be construed as which type of validity evidence with regard to the test of depression?
A) criterion-related validity
B) convergent evidence of construct validity
C) discriminant evidence of construct validity
D) None of these because there was an insignificant relationship.

Accepted Answer

A statistically insignificant correlation between scores on a new test of depression and a well-established measure of satisfaction with life may be interpreted as discriminant evidence of construct validity, as it suggests that the two constructs being measured are distinct and not related. Criteria-related validity requires a significant association between test scores and external criteria, while convergent validity requires a significant correlation between two measures of the same construct.

Question 7

Which is the MOST useful tool in evaluating convergent and discriminant validity evidence?
A) the Rulon formula
B) a multitrait-multimethod matrix
C) a Greco-Latin squares design
D) an abacus

Accepted Answer

A multitrait-multimethod matrix is the most useful tool in evaluating convergent and discriminant validity evidence. This matrix examines the correlations between two or more constructs measured using different methods (such as self-report and observation) to determine if they converge (correlate highly) or discriminate (correlate low or not at all). This approach is considered the gold standard for evaluating construct validity. The Rulon formula, Greco-Latin squares design, and abacus are not relevant tools for evaluating validity evidence.

Question 8

Which statistic is appropriate for use to estimating the heterogeneity of a test composed of multiple-choice items?
A) point-biserial correlation coefficient
B) Pearson-product moment correlation coefficient
C) coefficient alpha
D) chi square

Accepted Answer

Coefficient alpha (also known as Cronbach's alpha) measures the internal consistency of a test composed of multiple-choice items, providing a good estimate of the heterogeneity of the test. It ranges from 0 to 1, with higher values indicating greater reliability and consistency of the test scores. The other options (point-biserial correlation coefficient, Pearson-product moment correlation coefficient, and chi square) are not appropriate for estimating the heterogeneity of a multiple-choice test.

Question 9

Evidence of the homogeneity of a test can be found in the
A) correlation between a test and some criterion.
B) correlation between test items and total test scores.
C) correlation between subtest scores and total scores.
D) Both b and c

Accepted Answer

The correlation between test items and total test scores (option B) and the correlation between subtest scores and total scores (option C) both provide evidence of homogeneity in a test. Option A (correlation between a test and some criterion) is not directly related to homogeneity, but rather to validity. Therefore, the best choice is D (both b and c).

Question 10

Which is TRUE regarding a rating?
A) It refers only to a numerical judgment that places a person or an attribute along a continuum.
B) It refers only to a verbal judgment that places a person or an attribute along a continuum.
C) It tends not to involve a judgment.
D) It refers to either a numerical or a verbal judgment that places a person or an attribute along a continuum.

Accepted Answer

A rating can involve either a numerical or a verbal judgment that places a person or an attribute along a continuum, depending on the context and purpose of the rating. Therefore, option D is the correct answer.

Question 11

Which type of error has occurred when a music critic's review of Lady GaGa's latest album is more positive than most person on the planet believe was warranted?
A) fashion error
B) central tendency error
C) severity error
D) halo effect

Accepted Answer

The halo effect occurs when an individual's overall impression of a person, brand, or product influences their feelings and thoughts about that entity's character or properties. In this case, the music critic's overall positive impression of Lady Gaga might have led to an overly positive review of her latest album, regardless of its actual quality.

Question 12

A significant,positive relationship exists between scores on a new test of intelligence and scores on the fourth edition of the Stanford-Binet intelligence scale.These data may be viewed as supportive of which type of validity evidence for the new test?
A) criterion-related validity
B) content validity
C) convergent evidence of construct validity
D) discriminant evidence of construct validity

Accepted Answer

Convergent evidence of construct validity is supported when scores on a new test correlate positively with scores on an established test measuring the same construct, in this case, intelligence.

Question 13

A test is considered to contain a bias if
A) 50% of the test-takers fail the test.
B) one group, such as males, consistently performs better than another group, such as females.
C) a factor inherent in the test systematically prevents accurate measurement.
D) the test developer was found to harbor prejudice against some group.

Accepted Answer

A biased test is one that has a systematic factor or flaw that causes inaccurate measurement. It may not necessarily result in a certain percentage of test-takers failing or succeeding, nor does it have to involve one group performing better or worse than another. Prejudice on the part of the test developer may be a factor, but it is not the defining characteristic of a biased test.

Question 14

A rater systematically assigns ratings in the middle range,thus avoiding extremely positive and negative ratings.Which type of error BEST characterizes this rater's ratings?
A) leniency error
B) central tendency error
C) severity error
D) halo effect

Accepted Answer

The rater is exhibiting a central tendency error by consistently assigning ratings in the middle range and avoiding extreme ratings. This type of error can occur when raters are unsure about how to evaluate the performance or lack confidence in their ability to differentiate between ratings.

Question 15

Quotas may be viewed as one type of remedy for
A) low reliability of selection tests.
B) previously unfair practices.
C) low validity of selection tests.
D) All of these

Accepted Answer

Quotas are generally used as a remedy for previously unfair practices, such as discrimination against certain protected groups in employment. They are not typically used to address issues of test validity or reliability.

Question 16

The names attributed to different factor loadings in a factor analysis are
A) dictated by the factors themselves.
B) subject to change as new analyses occur.
C) thoroughly validated against dictionary definitions.
D) dependent on the researcher's judgment.

Accepted Answer

The names attributed to different factor loadings in a factor analysis are dependent on the researcher's judgment. The researcher selects the names that best describe the underlying factor being measured, and these names may vary depending on the context and purpose of the analysis. The names are not dictated by the factors themselves, nor are they subject to a fixed or validated set of definitions. The factor loadings themselves are the quantitative indicators of the underlying factor, and they are used to determine the strength and direction of the relationship between the observed variables and the latent factor.

Question 17

Issues of "fairness" as applied to tests
A) are seldom discussed in the popular media.
B) may be determined through mathematical procedures.
C) are generally agreed on.
D) are rooted in moral and philosophical issues.

Accepted Answer

Issues of "fairness" as applied to tests are not only about technical procedures, but also involve ethical considerations such as the values and beliefs that underlie fairness in relation to educational opportunities and outcomes.

Question 18

Which of the following is TRUE of test bias as compared to test fairness?
A) Test bias is dependent on statistical analyses while test fairness relates to values.
B) Test bias is dependent on values while test fairness relates to statistical analyses.
C) Whether a test is fair can be answered with certainty while whether a test is biased cannot.
D) None of these statements are true.

Accepted Answer

Test bias refers to how accurate the test is across different groups, while test fairness relates to the ethical and moral values of the test itself. Test bias can be measured and analyzed statistically, while test fairness involves a more subjective evaluation of the test's outcomes. Therefore, test bias is dependent on statistical analyses, while test fairness relates to values.

Question 19

In the context of test bias,a biased test
A) may be used fairly.
B) may be used unfairly.
C) may be used either fairly or unfairly.
D) is only used by biased test users.

Accepted Answer

A biased test can be used in ways that are either fair or unfair, depending on how it is applied and in what context. The bias in the test does not necessarily dictate how it will be used by test administrators or interpreters.

Question 20

Any definition of test fairness as used in a psychometric context would be likely to include reference to
A) the percent of items answered correctly by members of different groups.
B) the mean scores earned by various groups on a particular test.
C) the degree to which a test is used in an impartial, just and equitable way.
D) All of these

Accepted Answer

Test fairness in a psychometric context refers to the degree to which a test is used in an impartial, just, and equitable way. While measures of item difficulty and group mean scores are important components of test analysis, they do not directly address issues of fairness. Thus, choice C is the best answer.

Quiz 6: Validity

Test scores may be affected in pre- and post-testing by

Rating errors

A supervisor unintentionally rates his supervisees less favorably than they really deserve.Which type of error is at work here?

If a test is a valid measure of a particular construct,we would expect that

Which term is used to refer to the tendency of a rater to evaluate ratees higher than they objectively deserve because of the rater's inability to discriminate between aspects of the ratee's behavior?

A statistically insignificant correlation between scores on a new test of depression and a well established measure of satisfaction with life may be construed as which type of validity evidence with regard to the test of depression?

Which is the MOST useful tool in evaluating convergent and discriminant validity evidence?

Which statistic is appropriate for use to estimating the heterogeneity of a test composed of multiple-choice items?

Evidence of the homogeneity of a test can be found in the

Which is TRUE regarding a rating?

Which type of error has occurred when a music critic's review of Lady GaGa's latest album is more positive than most person on the planet believe was warranted?

A significant,positive relationship exists between scores on a new test of intelligence and scores on the fourth edition of the Stanford-Binet intelligence scale.These data may be viewed as supportive of which type of validity evidence for the new test?

A test is considered to contain a bias if

A rater systematically assigns ratings in the middle range,thus avoiding extremely positive and negative ratings.Which type of error BEST characterizes this rater's ratings?

Quotas may be viewed as one type of remedy for

The names attributed to different factor loadings in a factor analysis are

Issues of "fairness" as applied to tests

Which of the following is TRUE of test bias as compared to test fairness?

In the context of test bias,a biased test

Any definition of test fairness as used in a psychometric context would be likely to include reference to