Deck 6: Critically Appraising Quasi-Experiments: Nonequivalent Comparison Groups Designs

Question 1

Which of the following would be a nonequivalent comparison groups design?

A) Compare two different sites that seem comparable except for the type of intervention, program, or policy provided in each.
B) Compare the degree of change achieved by service recipients and clients on a waitlist for those services.
C) Refer clients to a new treatment or to routine treatment. depending on caseload openings at the time of referral.
D) All of the above.

Answer 1

All of the above.

Question 2

Which of the following could be cited to support the comparability of groups in a nonequivalent comparison groups design?

A) Similarity in client characteristics.
B) The use of multiple pretests.
C) The use of switching replications.
D) All of the above.

Answer 2

All of the above.

Question 3

Which is the best interpretation for the following results using multiple pretests in a nonequivalent comparison groups design study evaluating the effects of an innovative intervention to improve self esteem?

A) The intervention was effective.
B) The results plausibly can be attributed to a selectivity bias.
C) Unlike the comparison group, the group receiving the innovative intervention appears to have been undergoing change before the start of the intervention.
D) Both b and c are correct.

Answer 3

Both b and c are correct.

Question 4

Which is the best interpretation for the following results using switching replications in a nonequivalent comparison groups design study evaluating the effects of an innovative intervention to reduce temper tantrums?

A) The intervention was effective.
B) The results plausibly can be attributed to a selectivity bias.
C) Unlike the comparison group, the group receiving the innovative intervention appears to have been undergoing change before the start of the intervention.
D) Both b and c are correct.

Question 5

Which statement is the most appropriate interpretation of the following results of a study evaluating the effectiveness of an innovative program that aims to prevent recidivism among released prisoners?

A) The innovative program was effective.
B) The innovative program was NOT effective.
C) The innovative program appears to have been effective, but the number of prior arrests is also a factor.
D) None of the above.

Question 6

Which statement is the most appropriate interpretation of the following results of a study evaluating the effectiveness of an innovative program that aims to increase victim empathy scores among prisoners?

A) The innovative program was effective.
B) The innovative program was NOT effective.
C) The innovative program appears to have been effective for first-time offenders only.
D) None of the above.

Question 7

Which of the following statements is/are true about pilot studies?

A) Even with weak internal validity they can have value in stimulating further investigation or in planning a more highly controlled investigation.
B) They usually have a high degree of internal validity.
C) Because they lack adequate internal validity, they are worthless.
D) None of the above.

Question 8

Which of the following statements is/are true about propensity score matching?

A) It is an advanced statistical procedure.
B) It utilizes longitudinal data or administrative datasets.
C) It creates treatment and comparison groups that appear to be comparable in light of a large number of attributes.
D) All of the above.

Question 9

Random assignment is often infeasible in real-world practice settings.

Question 10

The key issue in judging whether a study using a nonequivalent comparison groups design achieves a credible degree of internal validity is the extent to which its authors provide a persuasive case that the groups being compared are really comparable.

Question 11

The plausibility of a selectivity bias can be reduced by employing multiple pretests.

Question 12

The plausibility of a selectivity bias can be reduced by employing switching replications.

Question 13

The plausibility of a selectivity bias can be reduced by employing nonequivalent dependent variables.

Question 14

The plausibility of a selectivity bias can be assessed by employing multivariate data analysis procedures.

Question 15

Pilot studies with serious design limitations can have utility.

Question 16

Pilot studies typically have high degrees of internal validity.

Question 17

A limitation of propensity score matching is that it might be difficult to ascertain when exactly an intervention was implemented, especially if the data were collected infrequently.

Question 18

Explain the value of multiple pretests in studies using nonequivalent comparison groups designs.

Question 19

Explain the value of switching replications in studies using nonequivalent comparison groups designs.

Question 20

Construct a multivariate statistical table that illustrates an outcome that is attributable to a selectivity bias.

Question 21

Discuss the limitations as well as the possible value of pilot studies that use designs with weak internal validity.