Question 1

In terms of internal validity (i.e., least vulnerable to internal validity threats is first, most vulnerable is last), experimental methods, correlational methods, and quasi-experimental methods would be organized as:

Accepted Answer

Experimental methods have the highest internal validity as they involve manipulation and control of variables. Quasi-experimental methods also have some degree of control, but often involve pre-existing groups which can threaten internal validity. Correlational methods have the lowest internal validity as they do not involve control or manipulation of variables and cannot establish causality. Therefore, experimental > quasi-experimental > correlational is the correct order from least to most vulnerable to internal validity threats.

Question 2

Program evaluation research differs from strictly experimental research in which of the following ways? (circle all that apply)

Accepted Answer

Evaluation research often involves political considerations as it assesses programs that may have policy implications. It also tends to be more flexible to accommodate stakeholder needs and program adjustments.

Question 3

What are the main goals of "program efficacy" studies? (circle all that apply)

Accepted Answer

Program efficacy studies aim to determine if the intervention produces the intended effects (A) and test the intervention under ideal conditions to maximize treatment effectiveness (C).

Question 4

This type of evaluation is used to determine whether a planned program can be implemented and delivered as expected, given existing restraints.

Accepted Answer

Program feasibility studies are conducted in the initial stages of a program to determine whether it is possible to implement the program as planned, given existing constraints such as time, resources, and organizational capacity. The focus is on whether the program can be delivered as planned, rather than on evaluating its effectiveness or impact. B) Program development studies focus on refining the program design and developing program materials. C) Program efficacy studies evaluate the effectiveness of a program in achieving its intended outcomes. D) Cost-benefit analyses assess whether the benefits of a program outweigh its costs.

Question 5

A nonprofit organization wanted to implement its new, revolutionary program for addressing homelessness. As part of the series of evaluation studies that assessed this planned program, evaluators conducted a pilot study that resulted in several suggestions for revising the general procedures for introducing the program to homeless populations, changing some of the wording of questionnaires, and adding another treatment condition to provide greater insight into the nuances of the treatment effects. This type of evaluation study would be considered:

Accepted Answer

This pilot study conducted by evaluators resulted in suggestions for revising the program and improving its implementation. This suggests that the evaluation was aimed at improving and refining the program as it was being developed, making it a formative evaluation. A program feasibility evaluation would assess whether the program is likely to succeed or not, a needs assessment evaluation would assess the needs of the target population before designing the program, and a summative evaluation would assess the overall impact of the program after it has been implemented.

Question 6

What conditions must be met to be able to appropriately conduct an impact (or outcome) evaluation? (circle all that apply)

Accepted Answer

The answer of What conditions must be met to be...

Question 7

The reliability coefficient for a measure can reflect: (circle all that apply)

Accepted Answer

The reliability coefficient reflects the extent to which observed scores at different times for the same individuals are due to true score differences or random error (A) and the correlation between these scores (C). Option B is describing the concept of validity, not reliability. Option D is not relevant to the concept of reliability.

Question 8

Which of the following is true regarding regression toward the mean?

Accepted Answer

Regression toward the mean is a statistical phenomenon that occurs when there is imperfect correlation between two sets of scores. It means that extreme scores in one set are likely to be less extreme when measured again. This is true regardless of whether the scores are high or low. Thus, statement A is not entirely accurate as it only refers to high scores in the first set. Statement B is partially accurate as regression toward the mean is not an inevitable consequence, but it is a statistical property. Statement C is also partially accurate as regression toward the mean is a problem whenever extreme scores are selected, whether they are high or low. Therefore, the best answer is E, as it accurately captures the limitations and properties of regression toward the mean.

Question 9

Which of the following techniques is an accepted method of controlling for regression artifacts?

Accepted Answer

Both techniques of comparing observed posttest scores with a predicted posttest score and comparing outcomes for participants in a treatment group to those from a control group using a score that reflects the pattern of responses on several covariates are accepted methods of controlling for regression artifacts. Matching posttest scores in the treatment group to scores in a separate "control group" after assignment to treatments has been determined is not a commonly accepted method of controlling for regression artifacts. Therefore, the correct choice is E.

Question 10

With propensity scores analyses, the difference between observed and counterfactual scores reflects changes attributable to:

Accepted Answer

Propensity scores are used to estimate the causal effect of a treatment on an outcome when there is a confounding variable present. The observed propensity score represents the likelihood of receiving the treatment based on observed covariates. The counterfactual propensity score is the probability of receiving the treatment if the subject had received the other treatment. The difference between the observed and counterfactual propensity scores reflects the treatment effect, as it represents the change in probability of treatment due to the intervention. Therefore, the correct answer is C, as the difference between observed and counterfactual scores reflects treatment effects. The other options are incorrect because they do not reflect the purpose of propensity score analysis.

Question 11

Propensity scoring is often preferred when using a matching design for all of the following reasons, EXCEPT:

Accepted Answer

Propensity score matching does not control for extraneous variables that are not measured. It only accounts for observed and measured covariates when matching participants from different treatment groups.

Question 12

Not all social programs are amenable to basic comparison group designs, particularly if they involve an entire population. One alternative design that assesses the effect of a quasi-experimental treatment is a time series design. Researchers used a time series design to assess the impact of a new tax on fatty foods. They concluded that prior to the tax implementation on January 1, fatty food consumption was very high; after the tax went into effect, fatty food consumption declined considerably. Data were collected every four months. Before their conclusion can be accepted, what other factors are particular concerns for time series designs that must be taken into account? (circle all that apply)

Accepted Answer

The answer of Not all social programs are amenable to...

Question 13

Which of the following graphs reflect possible outcomes of a growth curve modeling design? (circle all that apply)

Accepted Answer

The answer of Which of the following graphs reflect possible...

Question 14

There are many types of longitudinal or quasi-experimental designs that assess data patterns over time. _____(a)_____ are concerned with differences in change between those who fall directly below a cutoff point to those who fall directly above the cutoff in terms of an outcome; _____(b)_____ are concerned with the shape or pattern of the data across multiple time points.

Accepted Answer

- Regression discontinuity designs are used to compare groups based on a cutoff point, while growth curve modeling designs focus on the pattern of data changes over time.

Question 15

The "Stroop" Task is an example of:

Accepted Answer

The Stroop Task is a measure of attention, as it involves identifying the color of a word while ignoring the semantic meaning of the word itself, which requires mental effort and control over automatic responses.

Question 16

If a participant's eye gaze is directed toward one particular stimulus rather than other stimuli involved in the study, this is likely an indicator of:

Accepted Answer

The answer of If a participant's eye gaze is directed...

Question 17

Research that uses recall tasks:

Accepted Answer

The answer of Research that uses recall tasks:
...

Question 18

Having participants complete a series of sentence unscrambling tasks containing words meant to induce a certain mood is an example of:

Accepted Answer

The answer of Having participants complete a series of sentence...

Question 19

The underlying logic of _________ tasks is that positive primes should facilitate the evaluation of positively viewed target stimuli, while positive primes should inhibit judgment of negatively viewed stimuli.

Accepted Answer

Automatic evaluation tasks rely on the idea that positive primes facilitate the evaluation of positively viewed target stimuli, while negative primes inhibit judgment of negatively viewed stimuli.

Question 20

Which of the following cognitive processes or components is NOT involved in the processing experienced by a participant completing the Implicit Association Test (IAT)?

Accepted Answer

Cognitive busyness is not typically considered a factor in the processing involved in completing the IAT. The other three factors listed (response interference, processing speed, and automatic associations) are all key components of the test.

Quiz 10: Quasi-Experiments and Evaluation Research

In terms of internal validity (i.e., least vulnerable to internal validity threats is first, most vulnerable is last) , experimental methods, correlational methods, and quasi-experimental methods would be organized as: