Deck 5: Validity

Question 1

Which of the following is true of the relationship between reliability and validity?

A)It is possible to create a highly reliable test that lacks validity.
B)If a test lacks reliability, it is important to emphasize its validity.
C)High levels of validity are often associated with low reliability.
D)There is an inverse relationship between reliability and validity.

Answer 1

A

Question 2

Under what circumstances is construct validation essential?

A)when the test has some type of novel approach
B)when no criterion or universe of content is accepted as adequate to define the quality
C)when insurance reimbursement is going to be sought
D)when a single, large-scale validation study is to be performed

Answer 2

B

Question 3

Asking patients where their injured back hurts illustrates ____ validity.

A)predictive
B)face
C)construct
D)content

Answer 3

B

Question 4

Why does the current edition of Standards for Educational and Psychological Testing reject face validity?

A)By definition, all tests have face validity.
B)Face validity in not technically a type of validity.
C)No reliable measures of face validity have been found.
D)Face validity is irrelevant.

Question 5

Evidence for validity should include what information about construct, criterion, and content validation?

A)any form suffices
B)two forms of necessary
C)two forms are necessary plus information about face validity
D)all three need be presented in unified form.

Question 6

Which of the following was not one of the major categories recognized in the booklet on Standards for Educational and Psychological Testing?

A)content validity
B)criterion validity
C)convergent validity
D)construct validity

Question 7

According to Campbell and Fiske, what types of evidence are required for a test to be meaningful?

A)reliability and validity
B)content and construct
C)convergent and divergent
D)quantitative and qualitative

Question 8

Which type of validity is unique because it is not statistically-based?

A)predictive
B)concurrent
C)content
D)construct

Question 9

Validity refers to

A)the stability of a score.
B)the alpha level.
C)the agreement between a test score and the quality it is believed to measure.
D)item to total correlation.

Question 10

If the results obtained in a validity study are not specific to the validation population, what quality does the validation have?

A)differential validation
B)convergence
C)population crossover
D)generalizability

Question 11

When Ms. Anderson applied for an administrative assistant job in a factory, she was given a test that examined her typing ability, political affiliation, and filing ability. This test clearly violates what kind of validity?

A)predictive
B)construct
C)criterion
D)content

Question 12

The agreement between a test score and the construct it is presumed to measure is referred to as its

A)stability.
B)alpha level.
C)reliability.
D)validity.

Question 13

Which type of validity is most important for educational tests?

A)face
B)content
C)concurrent
D)predictive

Question 14

Which of the following should be considered when evaluating evidence for validity?

A)Whether the cause of the relationship between the test and the criterion has changed since the test was validated.
B)Excessively high validity may indicate a need for improving reliability.
C)Validity is only of concern when legal challenges are expected.
D)Tests that are highly valid for at least two distinct populations are likely to be valid for all populations.

Question 15

As a result of the Griggs v. Duke Power decision, employers must provide evidence that a test used for employee selection or promotion has

A)test-retest reliability.
B)inter-rater reliability.
C)specific meaning.
D)internal consistency.

Question 16

Why is evidence for criterion-related validity gathered?

A)It is legally required.
B)It is critical for reliability.
C)Tests serve as a substitute for the real measure of interest.
D)Criterion validity subsumes all other validity.

Question 17

"Face validity" is

A)the correlation between a test score and some poorly established criterion.
B)the best evidence for validity.
C)acceptable for research studies but not for tests used in clinical practice.
D)not really a form of validity because it offers no evidence to support conclusions.

Question 18

Concurrent and predictive validity are both subcategories of

A)criterion validity.
B)empirical validity.
C)face validity.
D)convergent validity.

Question 19

Which of the following is an important validation consideration in industrial settings?

A)pay grade
B)occupational classification
C)attrition
D)education

Question 20

How does the current edition of Standards for Educational and Psychological Testing conceptualize validity?

A)Validity is a non-statistical measure for non-quantitative tests.
B)Validity should only be assessed when reliability is low.
C)Validity is a unitary concept that represents all evidence.
D)Only one type of validity must be high and the others can be low.

Question 21

Many psychologists no longer believe this to be a separate type of evidence for validity because of the lack of clear-cut boundaries between it and other types of evidence for validity. What is it?

A)content
B)criterion
C)constructive
D)predictive

Question 22

Job samples exemplify the use of

A)criterion validity.
B)predictive validity.
C)concurrent validity.
D)content validity.

Question 23

If the test score forecasts some criterion, the test has

A)face validity.
B)content validity.
C)predictive validity.
D)concurrent validity.

Question 24

Which of the following is a legal requirement for the validity of a test that is used to select employees for a job?

A)The test must focus on job-related tasks.
B)The test must include multiple choice, short-answer, and essay questions.
C)The test must include items taken directly from other, related tests.
D)The test must be uncorrelated with similar tests.

Question 25

Which type of validity requires that test items provide an adequate representation of the conceptual domain they are designed to cover?

A)face validity
B)content validity
C)predictive validity
D)construct validity

Question 26

If scores on a test of intelligence are influenced by eye color, this is an example of

A)face validity
B)construct underrepresentation.
C)aggregate validity.
D)construct-irrelevant variance.

Question 27

Both convergent and discriminant evidence are essential for

A)content validity.
B)predictive validity.
C)concurrent validity.
D)construct validity.

Question 28

Which of the following is one of the eight recommended criteria for evaluating validity coefficients?

A)Determine the statistical sophistication of the researchers who conducted the validation.
B)Do not confuse the criterion with the predictor.
C)Make sure the results obtained from the validity study are situation specific.
D)Perform a small-scale validation study with the population of interest to you.

Question 29

A careful test constructor obtained the correlation between her test and a well-defined validity criterion. She then repeated the study to determine whether this same relationship held in another sample. This exercise is known as

A)correction for attenuation.
B)construct validation.
C)cross validation.
D)criterion/predictor confusion.

Question 30

Some studies have failed to demonstrate that the Graduate Record Exam (GRE)is a good predictor of grade point average in graduate school. One of the difficulties with these studies is that

A)the Graduate Record Exam is rarely used as a predictor of graduate school success.
B)the Graduate Record Exam was validated on a very small sample of students.
C)both grade point average and GRE scores have a restricted range for graduate students.
D)the Graduate Record Exam has no reliability.

Question 31

A test that has a high correlation with some well-accepted outcome measure provides evidence for

A)face validity.
B)criterion validity.
C)construct validity.
D)content validity.

Question 32

Seaman Sanchez had a hearing impairment, but passed the test to become a radio operator because the test did not measure hearing acuity. What does this represent?

A)criterion/predictor confusion
B)construct underrepresentation
C)construct-irrelevant variance
D)criterion-related evidence for validity

Question 33

The advantage of using a test that is "face valid" is that

A)the appearance of validity can help motivate test-takers because they can see that the test is relevant.
B)it assures that the items are properly sampled from the domain of content.
C)face validity is an important component of criterion validity.
D)it boosts the construct validity coefficient.

Question 34

The evidence is logical rather than statistical for

A)face validity and predictive validity.
B)criterion validity and content validity.
C)content validity and face validity.
D)content validity and concurrent validity.

Question 35

A validity coefficient is

A)the correlation between a test and the validity criterion.
B)the squared correlation between the test and the validity criterion.
C)the square root of the correlation between the test and the validity criterion.
D)one minus the correlation between a test and the validity criterion.

Question 36

The extent to which a test is valid for making statements about the criterion is represented by its

A)content validity.
B)validity coefficient.
C)squared validity coefficient.
D)construct-irrelevant variance.

Question 37

Which of the following would be a cause for concern when assessing the validity of an employment test?

A)The validity study was done on a somewhat small sample.
B)The test correlates highly with other tests of known validity.
C)The test was only validated on an all white, male population.
D)There is great variability in both predictor and criterion.

Question 38

The proportion of variation in the criterion that is knowable in advance from knowledge of the test scores can be found by

A)construct-irrelevant variance.
B)the validity coefficient.
C)squaring the validity coefficient.
D)the square root of the validity coefficient.

Question 39

A student confronts a professor and says, "You assigned Chapters 7 through 10 but nearly all of the items came from Chapter 7. How can you evaluate whether we know anything about the other material we were supposed to read?" The student is challenging the test on the basis of

A)face validity.
B)content validity.
C)criterion validity.
D)construct validity.

Question 40

Hypothetically, 9% of the variation of skill among dentists is accounted for by the test that they took in order to pass the dental board examination. What is the validity coefficient of the test?

A)0.03
B)0.3
C)0.9
D)3.0

Question 41

Cronbach and other authors have argued that all types of validity are really categories of

A)face validity.
B)criterion validity.
C)content validity.
D)construct validity.

Question 42

An exclusive preschool program has strict enrollment criteria and only accepts children with very high marks on the enrollment test. Unfortunately, they have found that the enrollment test does not correlate highly with the grades the children make. The probable reason is

A)predictor/criterion confusion.
B)an invalid criterion.
C)convergent evidence.
D)range restriction.

Question 43

If a variable has a "restricted range", it is difficult to estimate a validity coefficient due to a lack of

A)criterion validity.
B)variability.
C)adequate sample size.
D)cross validation.

Question 44

Evidence that indicates a measure does not represent a construct other than the one for which it was devised is ____ validity.

A)convergent
B)discriminant
C)content
D)predictive

Question 45

The type of validity that subsumes all other types of validity is

A)construct.
B)discriminant.
C)predictive.
D)content.

Question 46

In Rubin's study on the Love Scale, the amount of time lovers spent gazing into each other's eyes was used as evidence for the scale's

A)face validity.
B)criterion validity.
C)convergent validity.
D)content validity.

Question 47

At what point would we assume that the construct validity of a test has been established and no more studies should be done?

A)when we have shown that it correlates with several other measures designed to tap the same construct
B)when we have shown that it does not correlate with measures designed to tap different constructs
C)never - construct validation is an ongoing process
D)when the validity coefficients are above .8

Question 48

Suppose you wanted to develop a measure of jealousy, but were unable to define precisely the concept. What type of validation studies would you most likely use?

A)face validation
B)reliability validation
C)criterion validation
D)construct validation

Question 49

Construct validity is used when

A)a test can be validated against a single criterion.
B)we cannot clearly define what we want to measure.
C)we want to construct a new test for a specific purpose.
D)we draw samples of items from some clearly defined.

Question 50

Which of the following illustrates lack of convergent validity?

A)demonstrating that tests A, B, and C all measure the same things for the same purpose
B)demonstrating a particular relationship between a test of job satisfaction and an employee's stated job satisfaction
C)demonstrating a relationship between the results of a physical exam and a health index score
D)demonstrating that a score on the SAT and ACT have no correlation

Question 51

Discriminant and convergent evidence provide evidence for what type of validity?

A)predictive
B)criterion
C)content
D)construct

Question 52

To develop a test with construct validity, what types of validity evidence are needed?

A)face validity and predictive validity
B)discriminant validity and predictive validity
C)face validity and content validity
D)discriminant validity and convergent validity

Question 53

If a validity coefficient of .3 is found between a written test of baseball rules and actual skills playing baseball, what percentage of the variation in baseball skills would be accounted for?

A)3%
B)6%
C)9%
D)30%

Question 54

Several measures are already available that assess coping skills. In order to prove that a new measure of coping is needed, what type of validity evidence should be provided?

A)convergent
B)discriminant
C)discrete
D)predictive

Question 55

A test with low reliability

A)will not have a high validity coefficient.
B)may or may not have a high validity coefficient.
C)is likely to be face valid.
D)can be used in clinical practice but not research.

Question 56

Which of the following statements is true ?

A)We can always demonstrate that a reliable test has meaning.
B)It is logically impossible to demonstrate that a totally unreliable test is valid.
C)A test cannot be reliable unless it is also valid.
D)Reliability and validity are basically unrelated concepts.

Question 57

In construct validation,

A)a single variable can serve as the criterion.
B)multiple studies are not required.
C)no single variable serves as the criterion.
D)the criterion remains static once defined.

Question 58

A researcher seeking to develop a measure of depression cites a moderate correlation between her measure and the Beck Depression Inventory as evidence of the test's validity. What type of validity documentation does this provide?

A)convergent validity
B)discriminant validity
C)content validity
D)face validity

Question 59

The problem with using the GRE to determine whether students who have already been admitted to graduate school should be given their degrees is that

A)the criterion validity of the GRE is not well enough established.
B)the sample size is too small.
C)the causes of performance at the school where this procedure is used might be different than the causes of performance at the schools where the GRE was validated.
D)this policy confuses the predictor and the criterion.

Question 60

Dr. Tong is trying to define 'life satisfaction' and to create a test that will assess the level of a person's satisfaction with their life. She is planning to find out how well her test correlates with other tests believed to measure life satisfaction. This is an example of

A)convergent evidence.
B)divergent evidence.
C)discriminant evidence.
D)differential prediction.

Question 61

According to your book, in some settings-particularly in education-tests have been criticized simply because of improper interpretation or over-reliance on a single measure. Discuss the topic and suggest ways to improve the situation.

Question 62

A professor considers the difference of validity of his mid-term exam between male and female students. This professor is dealing with

A)range restriction.
B)confusion between predictor and criterion.
C)differential prediction.
D)face validity.

Question 63

Discuss the factors a university should consider in deciding whether or not to eliminate the SAT or ACT as an admissions requirement.

Question 64

A new measure for anxiety correlates highly with the Beck Anxiety Inventory. This new measure is demonstrating

A)discriminant validity.
B)convergent validity.
C)construct validity.
D)predictive validity.

Question 65

The validity coefficient of an aggression scale is .5. Using a score on this scale as a predictor of domestic violence, how much of the variability in domestic violence can be attributed to scores on the aggression scale?

A)25%
B)50%
C)none
D)indeterminate

Question 66

Suppose you are consulting for a company that uses a test to predict how successful job applicants might be on the job.  The test is being challenged in the courts.  Discuss the types of evidence you would use to defend the test.

Question 67

A professor is developing a final exam for her course. She goes through each chapter of the textbook for the semester and develops questions from each to ensure they are all sufficiently covered in the exam. Which type of validity is this professor ensuring for her exam?

A)content validity
B)construct validity
C)predictive validity
D)criterion validity

Question 68

Explain how reliability is related to validity, and/or vice versa.

Question 69

The different categories of validity

A)are important distinctions with little overlap.
B)have little use when determining overall validity.
C)must each be considered in order to determine validity of a scale.
D)are convenient grouping methods, but not distinctly different from each other.

Question 70

It is important to consider several important factors when interpreting the meaning of a validity coefficient. Outline and discuss what those factors are. Give examples when appropriate.

Question 71

Which of the following is a predictor variable for the criterion of college GPA?

A)College test scores
B)High school graduation rates
C)The SAT
D)Graduate school admittance rates

Question 72

When your colleague tries to develop a scale to measure stress, what recommendations should you give the colleague in terms of validity? Give examples.

Question 73

Assembling evidence about what a test means is referred to as

A)cross validation
B)prediction
C)range restriction
D)construct validation

Question 74

Which of the following is equivalent to the percentage of variation in the criterion that we can expect to know in advance because of our knowledge of the test scores?

A)the validity coefficient
B)the validity coefficient squared
C)the correlation coefficient
D)the correlation coefficient squared

Question 75

Explain the process of construct validation.

Question 76

Criterion-related validity is meaningful when

A)the predictor is stable.
B)the criterion is valid and reliable.
C)the criterion is valid but not necessarily reliable.
D)the predictor is reliable.

Question 77

When evidence obtained in one situation is not found in other similar situations, the evidence lacks

A)generalizability.
B)criterion.
C)predictor.
D)inter-situation reliability.

Question 78

A student stays up all night studying for a test. She is so exhausted that she does not perform well even though she has learned the content for the test. This is an example of

A)poor construct validity.
B)predictive validity evidence.
C)construct-irrelevant invariance.
D)construct underrepresentation.

Question 79

Most aspects of validity can be seen in terms of these categories.

A)face-related, predictive-related, and construct-related
B)construct-related, criterion-related, and content-related
C)concurrent-related, discriminant-related, and content-related
D)convergent-related, concurrent-related, and discriminant-related