Question 1

A vice president (VP)of personnel employs a "Corporate Screening Test" in the hiring process.For future testing purposes,the VP maintains records of scores achieved by __________ as opposed to ___________ in order to avoid restriction-of-range effects.
A) job applicants; hired employees
B) hired employees; job applicants
C) successful employees; hired employees
D) successful employees; other corporate officers

Accepted Answer

The VP is maintaining records of scores achieved by job applicants rather than hired employees. This is to avoid restriction-of-range effects, where scores of hired employees might be artificially inflated due to the selection process. By maintaining records of job applicants' scores, the VP can ensure that the test remains a valid measure of the desired skills and abilities.

Question 2

Because of the unique problems in assessing very young children,which of the following would be the BEST practice when attempting to estimate the reliability of tests designed to measure cognitive and motor abilities in infants?
A) Use relatively short test-retest intervals.
B) Use relatively long test-retest intervals.
C) Do not use the test-retest method for estimating reliability of the test.
D) Use only inter-scorer reliability estimates.

Accepted Answer

Using relatively short test-retest intervals would be the best practice when attempting to estimate the reliability of tests designed to measure cognitive and motor abilities in infants as this approach would minimize the effects of developmental changes, maturation, and practice effects. Long test-retest intervals could result in confounding variables potentially influencing the results, making it difficult to establish test-retest reliability. Not using the test-retest method would not provide reliability estimates, and relying on only inter-scorer reliability estimates may not be sufficient as different scorers may have different interpretations of the child's behavior or responses.

Question 3

In the context of IRT,the term discrimination best refers to
A) the degree to which an item differentiates people with respect to some cultural variable.
B) the utility of a test used in personnel selection.
C) the degree to which internal consistency compares to selection precision.
D) the degree to which an item differentiates people with respect to a trait being measured.

Accepted Answer

In IRT, discrimination refers to the degree to which an item is able to differentiate between individuals with respect to the trait being measured. It is a measure of how well an item can distinguish between high and low scorers on the construct being measured. It is not related to cultural variables or test utility. Internal consistency and selection precision are also not directly related to discrimination.

Question 4

According to generalizability theory,a variable such as "number of items in the test" is a description of one
A) facet of the universe.
B) true element of the dominion.
C) dominion in the domain.
D) None of these

Accepted Answer

Generalizability theory defines facets as independent sources of variation that contribute to measurement error. The number of items in a test is an example of a facet, which affects the reliability and validity of test scores. Therefore, A is the correct choice as it correctly identifies the number of items in the test as a facet of the universe. B and C are incorrect as they do not accurately describe the concept of facets in generalizability theory.

Question 5

IRT is a term used to refer to
A) a model that has many parameters.
B) a parameter that has many models.
C) a family of models for data analysis.
D) a dysfunctional family of models.

Accepted Answer

IRT (Item Response Theory) is a family of models for data analysis that is used to analyze the relationships between test items and a latent trait. It is not a model with many parameters (option A) or a parameter with many models (option B), nor is it a dysfunctional family (option D).

Question 6

Why isn't IRT used more by "mom-and-pop" test developers such as classroom teachers?
A) most classroom teachers were trained in generalizability theory
B) IRT has no application in classroom tests
C) applying IRT requires statistical sophistication
D) All of these

Accepted Answer

Applying IRT requires statistical sophistication, which can be a barrier for "mom-and-pop" test developers such as classroom teachers who may not have the necessary training or resources to use this method effectively. While some teachers may have been trained in generalizability theory, this is not necessarily the most common or practical approach for classroom assessment. Additionally, IRT can be a useful tool for improving the quality and reliability of classroom tests, particularly for assessing student growth over time, but its advantages may not always be widely recognized or understood by educators.

Question 7

If the variance of either variable in a correlational analysis is restricted by the sampling procedure used,then the resulting correlation coefficient tends to be
A) higher.
B) lower.
C) unaffected.
D) unstable.

Accepted Answer

If the variance of either variable is restricted, it means that there is less variability in the scores. This results in a weakened relationship and a lower correlation coefficient.

Question 8

The directions for scoring a particular motor ability test instruct the examiner to "Give credit if the child holds his hands open most of the time." Because what constitutes "most of the time" is not specifically defined,directions such as these could result in lowered reliability estimates for
A) test-retest reliability.
B) alternate-form reliability.
C) inter-rater reliability.
D) parallel forms reliability.

Accepted Answer

Inter-rater reliability refers to the degree of agreement among different examiners or raters in scoring the test. Since the instructions are not specifically defined, it is possible that different examiners may interpret "most of the time" differently, which could result in inconsistent scoring and lowered inter-rater reliability. Test-retest reliability refers to the consistency of scores over time, alternate-form reliability refers to the consistency of scores on different versions of the same test, and parallel forms reliability refers to the consistency of scores on equivalent forms of the same test. None of these types of reliability are directly affected by vague scoring instructions.

Question 9

Who are the primary users of IRT?
A) classroom teachers
B) commercial test developers
C) instructors at universities in Departments of Education
D) life insurance actuaries

Accepted Answer

Commercial test developers are the primary users of IRT, as it is a statistical method commonly used in the development of standardized tests. While classroom teachers and instructors in Education departments may also use IRT, they are not the primary users. Life insurance actuaries do not typically use IRT.

Question 10

Which is the BEST example of a dynamic characteristic?
A) the stress level of a trapeze flyer at a circus
B) the intelligence of a college student during Spring Break
C) the anti-authority attitude of an inmate serving a life term
D) None of these

Accepted Answer

The stress level of a trapeze flyer at a circus is a dynamic characteristic as it can change moment to moment based on various factors such as the height, speed, and difficulty of the flyers' tricks. The intelligence of a college student or the anti-authority attitude of an inmate may have some variance, but they are not as likely to change as quickly or dramatically as the stress level of a trapeze flyer.

Question 11

Test items with little discriminative ability prompt the test developer to consider the possibility that
A) the content of the item does not match the construct measured by the other items in the scale.
B) the item is poorly worded and needs to be rewritten.
C) the item is too complex for the educational level of the population.
D) All of these

Accepted Answer

Test items with little discriminative ability could be due to a variety of reasons. It could be because the content of the item does not match the construct measured by the other items in the scale (A), the item is poorly worded and needs to be rewritten (B), or the item is too complex for the educational level of the population (C). Therefore, the best choice is D, which includes all of these as possible reasons for the lack of discriminative ability.

Question 12

The Rasch model
A) was developed by a Danish mathematician named Rasch.
B) is an IRT model with specific assumptions about the underlying distribution.
C) was devised from generalizability theory.
D) Both was developed by a Danish mathematician named Rasch and is an IRT model with specific assumptions about the underlying distribution.

Accepted Answer

The Rasch model was indeed developed by a Danish mathematician named Georg Rasch. It is also an item response theory (IRT) model that makes specific assumptions about the underlying distribution of item difficulties and person abilities. Therefore, both statements A and B are true, making D the correct answer. Generalizability theory, on the other hand, does not pertain to the Rasch model.

Question 13

In the term latent trait theory,"latent" is a synonym for
A) invisible.
B) state.
C) undeveloped.
D) dormant.

Accepted Answer

"Latent" means "hidden" or "unseen," so it is a synonym for "invisible."

Question 14

If the variance of either variable in a correlational analysis is inflated by the sampling procedure used,then the resulting correlation coefficient tends to be
A) higher
B) lower.
C) unaffected.
D) unstable.

Accepted Answer

When the variance of either variable in a correlational analysis is inflated by the sampling procedure, the resulting correlation coefficient tends to be higher. This is because correlation coefficients measure the strength and direction of a linear relationship between two variables, and an increase in the variance of one or both variables can artificially inflate the perceived strength of this relationship.

Question 15

The fact that cultural factors may be operating to weaken an item's ability to discriminate between groups is evident from
A) a general lack of reliability in culture-specific tests.
B) latent trait theory.
C) Georg Rasch's unauthorized biography, You Can Never Be Too Rich or Too "Rasch."
D) None of these

Accepted Answer

The options provided do not directly relate to the influence of cultural factors on an item's discriminative ability. Cultural factors affecting test discrimination would involve biases or cultural specificity in test items, not the general reliability of culture-specific tests, latent trait theory, or content from a biography.

Question 16

Which is TRUE about reliability in the psychometric sense?
A) reliability is an all-or-none measurement
B) a test may be reliable in one context and unreliable in another
C) a reliability coefficient may not be derived for personality tests
D) alternate forms reliability may not be derived for personality tests

Accepted Answer

Reliability in the psychometric sense refers to the consistency of a measure over time, across different raters, or within a test. It is not an all-or-none measurement, as there can be varying degrees of reliability. A test may be reliable in one context and unreliable in another due to factors such as differences in the sample or test administration. A reliability coefficient may be derived for personality tests, but it may be more difficult to establish due to the subjective nature of personality traits. Alternate forms reliability may also be derived for personality tests.

Question 17

A polytomous test item is a test item
A) that has multiple tomous's attached.
B) that may have more than one choice keyed correct.
C) that is exemplified by this test question.
D) None of these

Accepted Answer

The answer of A polytomous test item is a test...

Question 18

According to generalizability theory,a variable such as "number of items in the test" is a description of one
A) facet of the universe.
B) true element of the dominion.
C) dominion in the domain.
D) None of these

Accepted Answer

Generalizability theory proposes that the variability in a measurement can be attributed to different facets or sources of variation, such as the person being measured, the items in the test, the raters scoring the test, and the occasion/time of measurement. Therefore, the number of items in the test is one facet of the universe that contributes to the overall variability of the measurement. Choice B (true element of the dominion) and C (dominion in the domain) are not relevant concepts in generalizability theory.

Question 19

As used in Chapter 5 of your text,the term inflation of variance is synonymous with
A) restriction of variance.
B) restriction of range.
C) inflation of range.
D) None of these

Accepted Answer

Inflation of variance refers to the phenomenon where the variance of a sample is higher than the variance of the population from which it was drawn. This can happen when the sample includes extreme scores that are not representative of the population's variance. The term "inflation of range" is not used in this context, and the term "restriction of range" refers to the opposite phenomenon where the range of a sample is smaller than the range of the population. Therefore, the best choice is C, inflation of range.

Question 20

Advocates of generalizability theory prefer the use of which of the following terms as an alternative to the use of the term "reliability"?
A) generalizability
B) universality
C) regularity
D) dependability

Accepted Answer

Generalizability theory uses the term "generalizability" to describe the extent to which findings or measurements can be generalized across different conditions or populations, as an alternative to "reliability."

Quiz 5: Reliability

A vice president (VP) of personnel employs a "Corporate Screening Test" in the hiring process.For future testing purposes,the VP maintains records of scores achieved by as opposed to _ in order to avoid restriction-of-range effects.

Because of the unique problems in assessing very young children,which of the following would be the BEST practice when attempting to estimate the reliability of tests designed to measure cognitive and motor abilities in infants?

In the context of IRT,the term discrimination best refers to

According to generalizability theory,a variable such as "number of items in the test" is a description of one

IRT is a term used to refer to

Why isn't IRT used more by "mom-and-pop" test developers such as classroom teachers?

If the variance of either variable in a correlational analysis is restricted by the sampling procedure used,then the resulting correlation coefficient tends to be

The directions for scoring a particular motor ability test instruct the examiner to "Give credit if the child holds his hands open most of the time." Because what constitutes "most of the time" is not specifically defined,directions such as these could result in lowered reliability estimates for

Who are the primary users of IRT?

Which is the BEST example of a dynamic characteristic?

Test items with little discriminative ability prompt the test developer to consider the possibility that

The Rasch model

In the term latent trait theory,"latent" is a synonym for

If the variance of either variable in a correlational analysis is inflated by the sampling procedure used,then the resulting correlation coefficient tends to be

The fact that cultural factors may be operating to weaken an item's ability to discriminate between groups is evident from

Which is TRUE about reliability in the psychometric sense?

A polytomous test item is a test item

According to generalizability theory,a variable such as "number of items in the test" is a description of one

As used in Chapter 5 of your text,the term inflation of variance is synonymous with

Advocates of generalizability theory prefer the use of which of the following terms as an alternative to the use of the term "reliability"?

Quiz 5: Reliability

A vice president (VP) of personnel employs a "Corporate Screening Test" in the hiring process.For future testing purposes,the VP maintains records of scores achieved by __________ as opposed to ___________ in order to avoid restriction-of-range effects.

Because of the unique problems in assessing very young children,which of the following would be the BEST practice when attempting to estimate the reliability of tests designed to measure cognitive and motor abilities in infants?

In the context of IRT,the term discrimination best refers to

According to generalizability theory,a variable such as "number of items in the test" is a description of one

IRT is a term used to refer to

Why isn't IRT used more by "mom-and-pop" test developers such as classroom teachers?

If the variance of either variable in a correlational analysis is restricted by the sampling procedure used,then the resulting correlation coefficient tends to be

The directions for scoring a particular motor ability test instruct the examiner to "Give credit if the child holds his hands open most of the time." Because what constitutes "most of the time" is not specifically defined,directions such as these could result in lowered reliability estimates for

Who are the primary users of IRT?

Which is the BEST example of a dynamic characteristic?

Test items with little discriminative ability prompt the test developer to consider the possibility that

The Rasch model

In the term latent trait theory,"latent" is a synonym for

If the variance of either variable in a correlational analysis is inflated by the sampling procedure used,then the resulting correlation coefficient tends to be

The fact that cultural factors may be operating to weaken an item's ability to discriminate between groups is evident from

Which is TRUE about reliability in the psychometric sense?

A polytomous test item is a test item

According to generalizability theory,a variable such as "number of items in the test" is a description of one

As used in Chapter 5 of your text,the term inflation of variance is synonymous with

Advocates of generalizability theory prefer the use of which of the following terms as an alternative to the use of the term "reliability"?

A vice president (VP) of personnel employs a "Corporate Screening Test" in the hiring process.For future testing purposes,the VP maintains records of scores achieved by as opposed to _ in order to avoid restriction-of-range effects.