Question 1

Program evaluation is critical in order to:&#10;A) Write the report to the funding agency&#10;B) Adjust the program to meet participants' needs during implementation&#10;C) Guarantee program sustainability&#10;D) Garner additional grant funding

Accepted Answer

Program evaluation is critical to any program to demonstrate its outcomes and adjust the program to meet needs during implementation. Program evaluation differs from an occupational therapy evaluation, but the process is similar in that the intent is to demonstrate progress and outcomes. The application of program evaluation to community programs requires a program development and population health perspective, with thinking that differs from a typical patient evaluation.

Question 2

Which type of program evaluation tends to be most cost effective?&#10;A) Internal evaluation&#10;B) External evaluation&#10;C) Participatory evaluation&#10;D) Outcomes evaluation

Accepted Answer

Internal evaluation is typically more cost-effective but can be affected by bias, as the evaluator is closely tied to the program. If the internal evaluator is an occupational therapy practitioner, the understanding of the impact of occupation-based interventions may be advantageous. External evaluation ensures rigor and bias reduction. This option may come with a higher cost, and the evaluator may potentially lack familiarity with the program, which may create a barrier to assessing the program accurately and efficiently.

Question 3

To ensure program evaluation is meaningful and relevant to a target population, program evaluators rely on:&#10;A) Internal evaluation&#10;B) External evaluation&#10;C) Formative assessment&#10;D) Stakeholder input

Accepted Answer

Evaluators rely on stakeholders to help develop the essential questions that will be answered in the evaluation. Stakeholders are "individuals, groups or organizations that can affect or are affected by an evaluation process and/or its findings" (Bryson, Patton, & Bowman, 2011, p. 1). The degree of stakeholder involvement in the evaluation process is often dependent on the approach and can range from helping generate questions to full involvement in the collection, analysis, and reporting of results.

Question 4

A type of program evaluation that occurs at intervals throughout the program and acts as an advisory process is:&#10;A) Outcome evaluation&#10;B) External evaluation&#10;C) Formative evaluation&#10;D) Participatory evaluation

Accepted Answer

Formative evaluation is conducted to inform decisions or actions to be taken for either program implementation or program improvement. Typically, formative evaluation occurs at intervals throughout the program and acts as an advisory process. Thus, formative evaluation provides credible and relevant information concerning a program's theoretical framework, design, activities, and operation.

Question 5

Which type of evaluation focuses on the &#34;4 W's&#34; approach (who, what, when, and where)?&#10;A) Outcome evaluation&#10;B) Process evaluation&#10;C) Formative evaluation&#10;D) Participatory evaluation

Accepted Answer

Process evaluation helps determine if the program is being delivered according to its design; it is truly focused on what went well in program implementation and what needs improvement so that program modification is possible. It is best evaluated through a "4 W's" approach (i.e., who, what, when, and where) (Salabarria-Peña et al., 2007). For example, imagine a program focused on fall prevention in the community. The program consists of screening, intervention, and follow-up. Process evaluation would occur during implementation of the program to ensure all three elements of the program are occurring as outlined when the program was developed.

Question 6

Impact evaluation provides the foundation for:&#10;A) Policy making&#10;B) Programmatic modification&#10;C) Program sustainability&#10;D) Return on investment

Accepted Answer

A well-conducted impact evaluation provides the foundation for sound policy-making. It describes whether the program had an impact, as well as how much of an impact and on whom. For example, impact evaluation on a new free health clinic would determine if and how the program goals were achieved and use this information to advocate for additional funding to continue to meet those program goals in the future.

Question 7

The most effective strategy for evaluating the financial elements of a program is to conduct a:&#10;A) Return on investment&#10;B) Impact evaluation&#10;C) Cost-benefit analysis&#10;D) Sustainability plan

Accepted Answer

One way to include financial factors in an evaluation plan is to include a cost-benefit analysis. Here, the evaluator compares the benefits of the program, determined in a benefits analysis, to the actual costs of the program. When conducting a cost analysis, evaluators must examine all the costs associated with running the program, not simply the costs associated with direct services to the client (e.g., material resources, operation expenses, salaries and benefits of employees, loss of work days for clients in the program).

Question 8

The type of programmatic evaluation that focuses on changes in knowledge, skills, behaviors, attitudes, and perceptions, as well as awareness of a program, is:&#10;A) Outcomes evaluation&#10;B) Impact evaluation&#10;C) Process evaluation&#10;D) Formative evaluation

Accepted Answer

Outcome evaluation can explore knowledge, skills, behaviors, attitudes, perceptions, and awareness (Salabarria-Peña et al., 2007). Survey tools are often used in a pretest-posttest approach to collect outcome data. Typically, outcome evaluation identifies a baseline for the target population and the change to this baseline, if any, after program implementation. Outcome evaluation determines whether or not the expected effect is occurring (Salabarria-Peña et al., 2007).

Question 9

Which type of evaluation best answers this question: What effect does the program have on the health or social need(s) experienced by the population?&#10;A) Outcomes evaluation&#10;B) Impact evaluation&#10;C) Process evaluation&#10;D) Formative evaluation

Accepted Answer

Impact evaluation explores the overall impact of the program. It explores whether the program met its goals (Salabarria-Peña et al., 2007). It typically occurs toward or at the end of program implementation (Salabarria-Peña et al., 2007). Impact evaluation considers the broad outcomes of the program on the community. The questions that drive impact evaluations include:
• What effect does the program have on the health or social needs(s) experienced by the population?
• To what extent can program outcomes be directly linked to program activities?
A well-conducted impact evaluation provides the foundation for sound policy-making. It describes whether the program had an impact, as well as how much of an impact and on whom. For example, impact evaluation on a new free health clinic would determine if and how the program goals were achieved and use this information to advocate for additional funding to continue to meet those program goals in the future.

Question 10

The following is an example of which approach to program evaluation: data on the efficacy of a bike safety program would likely include the number of participants who wear helmets with the correct fit.

A) Decision-oriented approach
B) Objectives approach
C) Participatory approach
D) Appreciative inquiry approach

Accepted Answer

The objectives approach emphasizes the determination of the achievement of stated goals and objectives in the program design. The main activities in this approach are determining the goals and objectives of the program, including the development of specific operational definitions (i.e., specific behavioral or observational definitions) of the objectives. Whereas stakeholders may be involved in determining the objectives and providing operational definitions, typically in this approach the objectives are determined based on the mission and goals of the program, and in some cases specific objectives may already exist as part of the program documentation (Fitzpatrick, Sanders, & Worthen, 2011).
Once the objectives are determined, a method for measuring the objectives is agreed upon. The evaluator then collects the data from these measurements and compares the data collected to the specific goals and objectives. If a specific criterion of success has been established for an objective, then the data collected are compared to the predetermined criterion and directly tied to the program goals or objectives. For example, objective data on the efficacy of a bike safety program would likely include the number of participants who wear helmets with the correct fit. The criterion for success for this objective may be "75% of participants wear helmets that fit correctly."

Question 11

Which of the following is a weakness of the decision-oriented approach to program evaluation?&#10;A) Important decisions may be made in advance, which may ignore the organic nature of programs&#10;B) Emphasis on qualitative methods may cause concerns of evaluator bias&#10;C) Program merit or worth is limited to program objectives&#10;D) Focus on stakeholders limits participant involvement

Accepted Answer

The answer of Which of the following is a weakness...

Question 12

Program evaluation that focuses on organizational and program assets, as opposed to the identification of problems and deficits, is:&#10;A) Decision-oriented approach&#10;B) Objectives approach&#10;C) Participatory approach&#10;D) Appreciative inquiry approach

Accepted Answer

The answer of Program evaluation that focuses on organizational and...

Question 13

As part of the evaluation framework from the Centers for Disease Control and Prevention, program evaluation should engage stakeholders who bring:&#10;A) A strong belief in the program's mission&#10;B) An understanding of program evaluation&#10;C) Both positive and negative perspectives&#10;D) Funding resources

Accepted Answer

The answer of As part of the evaluation framework from...

Question 14

A common strategy in program evaluation to visually demonstrate the program description is a:&#10;A) Logic model&#10;B) Mission statement&#10;C) 4 W's approach&#10;D) Logo

Accepted Answer

The answer of A common strategy in program evaluation to...

Question 15

According to the CDC framework, in which phase should stakeholders be able to articulate the evaluation results correctly and meaningfully?&#10;A) Engage stakeholders&#10;B) Justify conclusions&#10;C) Ensure use and share lessons learned&#10;D) Gather credible evidence

Accepted Answer

The answer of According to the CDC framework, in which...

Question 16

Which approach to data collection in evaluation can illuminate what happened, to whom, and with what outcomes based on the program?&#10;A) Logic model&#10;B) Quantitative data&#10;C) Qualitative data&#10;D) Credible evidence

Accepted Answer

The answer of Which approach to data collection in evaluation...

Question 17

Which of the following is an important purpose for disseminating program evaluation results?&#10;A) Generating support for continuing or expanding programs&#10;B) Promoting one's professional reputation&#10;C) Writing an end-of-year report&#10;D) Gaining media attention through a success story

Accepted Answer

The answer of Which of the following is an important...

Question 18

Which strategy for disseminating the outcomes of program evaluation focuses on the situation, the response, the results, and the evidence?&#10;A) Logic model&#10;B) Quantitative data&#10;C) Qualitative data&#10;D) Success story

Accepted Answer

The answer of Which strategy for disseminating the outcomes of...

Question 19

A community bike program develops an educational survey to determine what the local community knows about bike safety. The community includes many immigrants and refugees. When designing the survey, the program evaluator should consider which of the following?

A) Community desire for bike riding
B) Reading level
C) Funding
D) Creating an online survey

Accepted Answer

The answer of A community bike program develops an educational...

Deck 7: Program Evaluation Introduction