Deck 12: Evaluation Research

Question 1

Formative evaluation research often uses all of the following EXCEPT:

A)needs assessments.
B)focus groups.
C)trial runs.
D)all of the answers might be used.

Answer 1

D

Question 2

Nonpositivist approaches to evaluation research have come to recognize that:

A)there is consensus about what social ills are and how to achieve social betterment.
B)social programs cannot be evaluated unless they have only a few stakeholders.
C)randomized experiments are the preferred method for evaluating social programs and policies.
D)social ills and their betterment are not objective conditions.

Answer 2

D

Question 3

Which of the following is a (are)difference(s)between evaluation research and basic research

A)The basic researcher has more control over the project.
B)Evaluation research is more likely to encounter conflicting demands.
C)Results from basic research are more likely to receive wider dissemination.
D)All of the answers represent such differences.

Answer 3

D

Question 4

When it comes to measuring variables for evaluation research:

A)multiple indicators are suggested.
B)existing measures should be used where possible.
C)measures do not need to be of as high validity and reliability as for basic research purposes.
D)both multiple indicators are suggested and existing measures should be used where possible.

Question 5

As a part of formative program evaluation, program monitoring refers to:

A)controlled experiments to assess program outcomes.
B)assessing the ongoing operation of existing programs
C)using focus groups to help develop programs.
D)using needs assessments to help develop programs.

Question 6

The study by Rossi, Berk, and Lenihan, in which they compared six released inmates who received weekly monetary allowances with twenty others who received no payments, is an example of:

A)a summative evaluation.
B)a randomized block design.
C)the use of available data in evaluation research.
D)a formative evaluation.

Question 7

Which of the following is NOT an element of a logic model used to understand a program evaluation

A)resources/inputs of the program
B)activities/interventions of the program
C)a cost-effective analysis of the program
D)outputs/outcomes of the program

Question 8

Which of the following is NOT a valid reason to conduct evaluation research

A)to make the program look good
B)for administrative purposes
C)to test hypotheses regarding practice approaches
D)all of the answers represent valid reasons

Question 9

Proximate goals in evaluation research are:

A)goals of the researchers but not the sponsors.
B)goals of the sponsors but not the researchers.
C)short run goals that are related to long term goals.
D)goals that the evaluated program achieves successfully.

Question 10

Formative program evaluations would likely focus on all of the following EXCEPT:

A)program outcomes.
B)program planning.
C)program implementation.
D)all of the answers represent likely focuses of formative program evaluation.

Question 11

In program evaluations, the measurable criteria that are used to determine the success of a program are called the program:

A)objectives.
B)goals.
C)benefits.
D)opportunities.

Question 12

Summative evaluation research:

A)is usually conducted by in house personnel.
B)assesses the impact of a program.
C)is concerned with the planning, development, and implementation of a program.
D)is concerned with the mundane daily operations of a program.

Question 13

The dependent variables in evaluation research are most commonly:

A)program inputs.
B)too vague to be used for evaluation.
C)the goals of the program.
D)determined solely by the researcher.

Question 14

Which of the following is a (are)part(s)of an evaluability assessment

A)determining the purpose of the evaluation from the standpoint of the sponsor
B)studying the program as operated
C)identify those aspects of the program that are sufficiently unambiguous that they can be evaluated
D)all of these are part of an evaluability assessment

Question 15

In a summative program evaluation, the program inputs would most likely be:

A)the independent variables in the evaluation.
B)the dependent variables in the evaluation.
C)the control variables in the evaluation.
D)the proximate goals in the evaluation.

Question 16

The term "stakeholder" refers to:

A)the people who are being observed in evaluation research.
B)the people who have an interest in a particular social program or policy.
C)the people who assist the researcher in conducting evaluation research.
D)the sponsors of evaluation research.

Question 17

Evaluation research refers to:

A)any research that draws conclusions.
B)any research conducted for a sponsor.
C)research that helps to plan, monitor, or assess the effectiveness of programs or clinical practices.
D)research using an experimental design.

Question 18

Formative evaluation research:

A)involves an assessment of a program's impact after service has been provided to a group of clients.
B)is usually conducted by outside consultants.
C)is concerned with the planning, development, and implementation of a program.
D)always employs a randomized experimental design.

Question 19

The 30 year follow up study of the participants in the Cambridge Somerville Youth Study found:

A)no long term differences between the experimental and control groups.
B)that the experiment produced several negative effects on the experimental group.
C)that the control group suffered from not experiencing the experimental condition.
D)inconclusive results because few of the participants could be found after so many years.

Question 20

In recent decades, evaluation researchers have come to recognize that any given social program or policy:

A)has only one stakeholder.
B)has a small number of stakeholders.
C)has a large variety of stakeholders.
D)may or may not have stakeholders.

Question 21

Barriers to the effective utilization of evaluation results include:

A)poor communication of results by evaluators.
B)vested interests can be difficult to overcome.
C)failure of evaluators to press for implementation of their findings.
D)all of the answers are mentioned in the text as barriers.

Question 22

Which of the following are problems encountered in using randomized experiments for evaluation research

A)ethical problems resulting from randomized denial of service
B)staff resistance
C)legal barriers
D)all of the answers represent such problems

Question 23

The most difficult, and often unreliable, part of a cost benefit analysis is usually:

A)estimating the dollar value of benefits
B)estimating the dollar value of direct costs
C)estimating the costs to program participants
D)determining the costs of ongoing programs

Question 24

Evaluators themselves sometimes create barriers to the utilization of evaluation research by:

A)refusing to participate in evaluation research.
B)designing evaluations that do not produce clear-cut results.
C)failing to communicate results clearly to program administrators or policymakers.
D)designing evaluations that do not produce clear-cut results and failing to communicate results clearly to program administrators or policymakers.

Question 25

Which of the following is NOT one of the perspectives used in calculating costs and benefits in cost-benefit analysis

A)program participants
B)researchers
C)society as a whole
D)all of the answers represent such perspectives

Question 26

In cost benefit analysis, opportunity costs are:

A)the direct costs of funding a given program.
B)the value of other programs on which the resources could have been used.
C)the cost of pursuing unanticipated opportunities that arise while doing research.
D)social costs to the participants in programs.

Question 27

Which of the following was NOT listed in the text as a valid alternative to the use of randomized experiments in evaluation research

A)quasi experimental designs
B)matching
C)client testimonials
D)cohort groups

Question 28

Research in Practice 12.1 describes a study of a program to reduce recidivism among released prison inmates. The research design used in this study is:

A)a survey.
B)a time-series quasi-experiment.
C)a randomized experiment.
D)cohort groups.

Question 29

In conducting a cost benefit analysis, benefits and costs are usually considered from which perspectives

A)the participants, the funding source, and society as a whole
B)the participants, the funding source, and the researchers
C)the researchers, the funding source, and society as a whole
D)the participants, the researchers, and society as a whole

Question 30

In cost benefit analysis, the "discount rate" is:

A)the amount deducted from welfare recipients' payments for each dollar they have in earnings.
B)a reduction in the cost of doing evaluation research when several programs are evaluated at the same time.
C)that part of the costs of a program that is used for administrative purposes.
D)the amount future costs and benefits are reduced to account for inflation.

Question 31

The major difference between cost benefit analysis and cost effective analysis is:

A)cost benefit considers only direct costs.
B)cost effective monetizes only costs not benefits.
C)cost effective considers only direct costs.
D)there is no difference and the two terms are interchangeable.

Question 32

The nonpositivist evaluation of a juvenile gang intervention program described in Research in Practice 12.2 utilized which research methodology

A)field experiment
B)field research
C)focus groups
D)archival records

Question 33

In the cost benefit analysis of early childhood intervention programs discussed in the text, the lower costs for welfare payments to mothers that result from such programs is most clearly a benefit to:

A)society.
B)the participants.
C)the funding source.
D)all of these clearly benefit from this.

Question 34

In regard to deciding whether particular elements of a program are a cost or a benefit for purposes of conducting a cost-benefit analysis of the program, the text concludes that:

A)only benefits should be included in a cost-benefit analysis.
B)people can disagree over whether an element is a cost or a benefit.
C)government regulations determine whether a particular element is a cost or a benefit.
D)only those costs and benefits about which everyone agrees should be included in a cost-benefit analysis.

Question 35

Major weaknesses of statistical controls in evaluation research include:

A)an over reliance on matching. b.an over reliance on randomization.
C)a tendency to underadjust for differences between groups.
D)the threat to validity from history.

Question 36

The major limitation of cost effective analysis as compared with cost benefit analysis is that:

A)cost effective analysis cannot determine which of several programs is the most efficient.
B)cost effective analysis is far more complex and difficult to apply.
C)cost effective analysis is only good for determining which of several programs is most efficient.
D)cost effective analysis consistently underestimates costs.

Question 37

In cost-benefit analysis, monetizing benefits refers to:

A)reducing their costs.
B)increasing their costs.
C)attaching a dollar value to them.
D)measuring the costs on some scale other than dollars.

Question 38

The text cites which of the following as things that might improve the utilization of evaluation research

A)increased dissemination of results
B)involve potential users in the evaluation research itself
C)utilize research designs other than randomized experiments
D)increased dissemination of results and involve potential users in the evaluation research itself

Question 39

Which of the following are NOT among the assumptions made about summative program evaluations by those who use nonpositivist approaches

A)There are multiple stakeholders in social interventions.
B)Interpretation and social meanings are at the core of social interventions.
C)Social interventions reflect and reinforce inequitable distributions of power.
D)All of the answers represent assumptions made by these approaches.

Question 40

The major weakness of cohorts in evaluation research is:

A)an over reliance on randomization.
B)a lack of controls for the effects of history.
C)an over reliance on statistical controls.
D)staff resistance.

Question 41

What is the purpose of formative evaluation and to what questions does it provide answers

Question 42

"Randomized experiments may be best from a researcher's perspective, but they are simply too impractical and unwieldy for evaluation in the real world." How would you respond to such a claim

Question 43

Using the logic model as a guide, describe the elements that are necessary in a program before it can be assessed through a summative evaluation

Question 44

Compare and contrast evaluation research with basic research.

Question 45

What alternatives are available when a true randomized design cannot be used in a program valuation Identify three options and indicate the potential hazards of using each.

Question 46

How are cost benefit analysis and cost effective analysis alike and how are they different

Question 47

Describe the nonpositivist approach to evaluation research. Discuss as many of the assumptions of this approach as you can and compare them with the assumptions that positivists might make. How does all this relate to the "politics of evaluation research"

Question 48

Name and distinguish between the two main types of evaluation research.

Question 49

A community has a volunteer foster grandparent program in which retirees over the age of 55 are matched with children under 15 in families without accessible grandparents. The program, according to its brochure, is supposed to promote understanding between generations, provide positive role models, reduce loneliness, and make life more satisfying for young and old alike. Can these goals be evaluated Why Suggest how you might operationalize one or more of these goals in order to conduct an evaluation.

Question 50

One complaint about evaluation research is that it's not worth the time and trouble because it simply doesn't get used. Identify four barriers to the utilization of evaluation research and suggest what might be done to break them down.