Question 1

Impact program evaluation differs from other types of evaluation in that the focus is on changes in program participants and not on changes in organizations and communities.

Accepted Answer

Impact program evaluation specifically aims to assess the outcomes or changes that can be attributed to a particular program, focusing on the effects on its participants rather than on broader organizational or community changes.

Question 2

An impact program evaluation seeks to establish a cause-and-effect relationship between a program and its outcome (result).

Accepted Answer

Impact program evaluation specifically aims to determine whether and how a program causes changes or outcomes, establishing a causal link between the program's implementation and its effects.

Question 3

When a human service services program fails to achieve its desired outcomes (result) due to theory failure, the results are invalid for any type of program evaluation.

Accepted Answer

Theory failure indicates a flaw in the underlying assumptions about how the program is supposed to work, not necessarily in the methods used to evaluate its outcomes. Program evaluation can still provide valuable insights into what aspects of the program did not work as expected and why.

Question 4

The essence of impact program evaluation is comparison.

Accepted Answer

Impact program evaluation fundamentally involves comparing outcomes with and without the intervention to determine its effectiveness.

Question 5

The pretest and posttest design in a single group pretest/posttest design does not need to be created. It may be any test, observation, or measurement:

Accepted Answer

In a single group pretest/posttest design, the pretest and posttest can indeed be any form of test, observation, or measurement that is relevant to the research question, as long as they are consistent and accurately measure the variables of interest before and after the intervention or treatment.

Question 6

To be able to say with more assurance that the program caused the change, we must identify threats to internal validity.

Accepted Answer

Identifying threats to internal validity is crucial for establishing a causal relationship between the program and the observed change, as it helps to rule out alternative explanations for the observed effects.

Question 7

The strongest impact research design and the one that deals most strongly with threats to internal validity is the single group pretest/posttest design.

Accepted Answer

The single group pretest/posttest design is vulnerable to many threats to internal validity, such as history, maturation, and regression to the mean. Stronger designs, like randomized controlled trials (RCTs), better address these threats by including control groups and random assignment.

Question 8

Ethical issues such as withholding participation in a program for experimental purposes and legal requirements such as withholding participation if a client meets eligibility requirements often cause human service administrators to rely on less religious designs when conducting impact program evaluations.

Accepted Answer

Ethical issues and legal requirements can limit the use of experimental designs, leading administrators to rely on less rigorous designs.

Question 9

Random assignment to groups is the least powerful in controlling for selection bias.

Accepted Answer

Random assignment is one of the most powerful methods for controlling selection bias because it ensures that each participant has an equal chance of being assigned to any group, thus minimizing pre-existing differences between groups.

Question 10

Statistically similar means that the characteristics of the individuals that comprise the comparison group are the same or similar to the characteristics of the individuals that comprise the experimental group on all relevant characteristics.

Accepted Answer

Statistically similar means that the groups being compared (e.g., experimental and comparison groups) have been matched or are equivalent on key characteristics relevant to the study, ensuring that differences observed can be attributed more confidently to the intervention or variable being studied rather than to pre-existing differences between the groups.

Question 11

Impact program evaluation's intent is to answer the question: Does the program actually have an impact and does it achieve positive outcomes (results) for _____________.

Accepted Answer

The answer of Impact program evaluation's intent is to answer...

Question 12

Impact program evaluation attempts to demonstrate that an outcome (result) is ______________ to the program and not some other variable.

Accepted Answer

The answer of Impact program evaluation attempts to demonstrate that...

Question 13

Using a logic model terminology, a successful program is one where there is ___________ between the program hypothesis (theory) and implementation of the program design (cause) that produces the desired outcomes (result).

Accepted Answer

The answer of Using a logic model terminology, a successful...

Question 14

When a program fails to achieve the desired outcome (result) due to ________________ the program has not been properly tested and we cannot make inferences about the success or failure of the program.

Accepted Answer

The answer of When a program fails to achieve the...

Question 15

Comparisons in impact program evaluation are usually made in one of two ways: two different groups are compared or one group is compared to ___________.

Accepted Answer

The answer of Comparisons in impact program evaluation are usually...

Question 16

Define theory success, theory failure. and program failure, describe the differences between them and give an example of each within a human service program.

Accepted Answer

The answer of Define theory success, theory failure. and program...

Question 17

Describe the two ways comparison in impact program evaluation is achieved and give an example of each.

Accepted Answer

The answer of Describe the two ways comparison in impact...

Question 18

Impact program evaluation attempts to determine if a human service program actually works. In order to make a cause-and-effect statement a counterfactual has to be created that would identify what would have happened to individuals had they not participated in the program?
You are the program director of a program that works with young women to improve their parenting skills and you are charged to evaluate the program to determine if the program has made a difference in knowledge and behaviors of the women who participated in the program. Describe 1) the program hypothesis, 2) which of the two ways that you will use to compare two groups, 3) create the cause-and-effect statement and, if necessary, create a counterfactual that identifies what would happen if participants would not have participated in the program, and 4) describe the desired result that should be the outcome of the program and how you would determine if it had been achieved.

Accepted Answer

The answer of Impact program evaluation attempts to determine if...

Quiz 10: Impact Program Evaluation and Hypothesis Testing