Question 1

Which of the following is not one of the four questions addressed in program evaluation?
A)assessment of needs
B)evaluation of process
C)final decision making
D)evaluation of outcome

Accepted Answer

The four questions addressed in program evaluation are assessment of needs, evaluation of process, evaluation of outcome, and efficiency of program implementation. Final decision making is not one of the four questions.

Question 2

The one-group pretest-posttest design is
A)a bad experiment.
B)an important quasi-experimental design.
C)the best design for clinical research.
D)least affected by threats to internal validity.

Accepted Answer

The answer of The one-group pretest-posttest design is
A)a bad experiment.
B)an...

Question 3

The relationship between basic research and applied research can best be described as
A)an independent relationship.
B)a reciprocal (or circular)relationship.
C)a hierarchical relationship with basic research taking precedence over applied research.
D)a top-down relationship with applied research taking precedence over basic research.

Accepted Answer

Basic research and applied research have a reciprocal relationship. Basic research is conducted to gain knowledge and understanding of a particular phenomenon or concept, while applied research uses that knowledge to solve practical problems or develop products. The findings from applied research can also lead to new questions and further basic research. This cycle continues, with each type of research informing and influencing the other.

Question 4

If the residents,staff,and facilities of a nursing home where a quasi-experiment was done are likely to be different from those in other nursing homes,then the findings of the study are likely to lack
A)reliability.
B)statistical significance.
C)internal validity.
D)external validity.

Accepted Answer

The lack of similarity in residents, staff, and facilities of the nursing home creates a threat to external validity, which refers to the generalizability of the study findings to other settings or populations. If the study participants are not representative of the population of interest, the results cannot be generalized to other nursing homes or groups of older adults. Therefore, the correct answer is D, external validity.

Question 5

After implementing a program to improve admitting procedures in a hospital emergency room,a researcher seeks to determine whether the program is being implemented as proposed.Which research method will best serve this purpose?
A)survey method
B)observational method
C)experimental design
D)quasi-experimental design

Accepted Answer

The observational method would be the best choice for determining if the program is being implemented as proposed, as it involves observing the actual behavior or actions of individuals in the setting where the program is being implemented. This method allows the researcher to directly observe and document any deviations from the proposed protocol, and can provide insight into potential barriers or challenges to implementation. Surveys may not provide the same level of detail or accuracy, while experimental or quasi-experimental designs may not be necessary for simply assessing implementation fidelity.

Question 6

Program evaluation is a hybrid discipline whose professional activity
A)relies almost exclusively on the experimental method.
B)has as its basic goal providing feedback to providers of human service activities.
C)is confined almost entirely to the discipline of psychology.
D)focuses primarily on conducting basic research.

Accepted Answer

Program evaluation involves gathering data to assess the effectiveness and efficiency of programs or interventions, with the goal of providing feedback to providers or decision-makers. While some program evaluations may use experimental methods, other research designs may also be used. Program evaluation is an interdisciplinary field, drawing on various disciplines including psychology, sociology, implementation science, and other related fields. The primary goal of program evaluations is not to conduct basic research, but rather to inform program improvement and decision-making.

Question 7

Among the following designs,which allows researchers to rule out the most threats to internal validity?
A)times series design with nonequivalent control group
B)one group pretest-posttest design
C)simple interrupted time series design
D)nonequivalent control group design

Accepted Answer

A time series design with a nonequivalent control group allows researchers to compare changes over time in both the treatment and control groups, helping to rule out many threats to internal validity such as history, maturation, and testing effects.

Question 8

When individuals in one group of a nonequivalent control group design experience natural changes at a faster rate than individuals in the other group,there is danger of a threat to internal validity called
A)additive effects of selection and history.
B)additive effects of selection and instrumentation.
C)additive effects of selection and maturation.
D)differential statistical regression.

Accepted Answer

This situation is known as differential maturation, where the two groups in a nonequivalent control group design experience natural changes (such as maturity level, experience, or biological changes) at different rates, leading to an internal validity threat.

Question 9

The effect of a treatment in a simple interrupted time-series design is indicated by
A)a clear discontinuity (abrupt increase or decrease)in the dependent variable at the point the treatment is administered.
B)a decreasing trend in the dependent variable that is present both before and after the treatment.
C)an increasing trend in the dependent variable that is present both before and after the treatment.
D)a gradual change in the dependent variable that begins just before the treatment is implementeD.

Accepted Answer

A clear discontinuity (abrupt increase or decrease) in the dependent variable at the point the treatment is administered indicates the effect of the treatment in a simple interrupted time-series design. This indicates that the treatment had an immediate and significant impact on the dependent variable.

Question 10

Social policy research (like the research done by Ellen Langer on the effects of living in a nursing home among the elderly)serves the purpose(s)of
A)informing policy decisions only.
B)advancing scientific theory and knowledge only.
C)advancing scientific theory and knowledge or informing policy decisions but not both.
D)advancing scientific theory and knowledge and informing policy decisions.

Accepted Answer

Social policy research serves the dual purpose of advancing scientific theory and knowledge and informing policy decisions. It has the potential to inform policymakers about the impact of social policies and programs on individuals and communities, as well as contribute to the development of theoretical frameworks that can guide future research and practice. Therefore, both advancing scientific theory and knowledge and informing policy decisions are important outcomes of social policy research.

Question 11

A human services agency was interested in finding out whether establishing a food distribution center in their community would be providing a service the community would use.Which of the following types of assessments in program evaluation would this agency find useful for this question?
A)evaluation of process
B)evaluation of outcome
C)assessment of needs
D)evaluation of efficiency

Accepted Answer

The assessment of needs would be the most useful in this situation as it would help identify whether there is a need for a food distribution center in the community and whether the community would utilize such services. Evaluation of process would focus on how the program is being implemented, evaluation of outcome would focus on the results achieved by the program, and evaluation of efficiency would focus on the cost-effectiveness of the program, which may not be as relevant at this stage.

Question 12

Which of the following threats to internal validity is controlled in the nonequivalent control group design?
A)differential regression
B)additive effect of selection and testing.
C)additive effect of selection and history.
D)testing

Accepted Answer

The answer of Which of the following threats to internal...

Question 13

A researcher was asked to evaluate the outcome of a program implemented to enhance recycling in a community.Which method will the researcher likely select to answer the question of whether the program was effective?
A)time series with nonequivalent control group design
B)participant observation
C)random groups design
D)cross-sectional survey design

Accepted Answer

A time series with nonequivalent control group design would allow the researcher to compare the recycling rates before and after the program was implemented in the intervention group (the community where the program was implemented) and the control group (a similar community where the program was not implemented). This would help to determine whether the program was effective in increasing recycling rates in the intervention group compared to the control group.

Question 14

A student used a nonequivalent control group design to examine the effectiveness of a computer module the library uses to introduce first year students to the library's resources.Which threat to internal validity could she potentially rule out by examining the pretest scores for both groups?
A)a maturation threat
B)a regression threat
C)a selection threat
D)an instrumentation threat

Accepted Answer

By examining the pretest scores for both groups, the student can assess whether the groups were equivalent at the start, thus potentially ruling out a selection threat.

Question 15

Even when pretest scores are the same,on average,for treatment and comparison groups in a nonequivalent control group design,the two groups may not be equivalent because
A)the pretest measure is unlikely to be relevant to the dependent variable.
B)the fact that the two groups are equal on the pretest measure does not ensure that the groups are equivalent on other characteristics relevant to the study.
C)the posttest measure is unlikely to be the same as the pretest measure,and the two groups must be equivalent at the posttest.
D)the natural growth rates of two groups from different populations are likely to be the same,but the pretest estimate of equality may still be in error.

Accepted Answer

Pretest scores being the same does not necessarily ensure equivalence on other characteristics that may affect the outcome of the study. Therefore, despite the pretest scores being equal, the groups may still not be equivalent.

Question 16

A clinical psychologist examined the effect of a treatment designed to reduce the amount of cigarette smoking in the day room of an inpatient psychiatric facility.His treatment consisted of a several components to maximize its potential effectiveness (e.g. ,increasing the number of available activities;giving patients responsibility for care of the unit's plants,fish,etc. ;the opportunity to exchange cigarettes for positive reinforcers).At the end of three months,his data indicated that cigarette smoking had declined significantly.Based on this,he is able to conclude that
A)any of the components,implemented by itself,would reduce smoking in the day room.
B)the treatment will work only when all of the components are present.
C)both (A)and (B)
D)neither (A)nor (B)

Accepted Answer

The psychologist cannot conclude that any of the components, implemented by itself, would reduce smoking in the day room, nor can he conclude that the treatment will work only when all of the components are present. He can only conclude that the treatment, as a whole, was effective in reducing smoking in the day room.

Question 17

When two groups in a nonequivalent control group design do not differ on the pretest,the researcher can conclude that
A)the two groups are equivalent only on the dependent variable measured on the pretest.
B)the two groups are not equivalent on any dependent variables.
C)the two groups will show the same natural growth rate over the course of the study.
D)the two groups are equivalent.

Accepted Answer

The two groups are equivalent only on the dependent variable measured on the pretest, as nonequivalent control group designs do not ensure equivalence on other variables.

Question 18

A simple interrupted time series design provides a reliable measure of the effect of the treatment because there are many observations both before and after the treatment.The major threat to internal validity in this design is
A)maturation.
B)history.
C)selection.
D)additive effect of selection and history.

Accepted Answer

The major threat to internal validity in a simple interrupted time series design is history, as external events occurring between the pre-treatment and post-treatment observations can affect the outcome, making it difficult to attribute changes solely to the treatment.

Question 19

Strong evidence of a treatment effect in a time series with nonequivalent control group design is indicated by
A)discontinuity in neither the treatment group nor the comparison group.
B)discontinuity in both the treatment group and the comparison group.
C)discontinuity in the comparison group but not in the treatment group.
D)discontinuity in the treatment group but not in the comparison group.

Accepted Answer

Discontinuity in the treatment group but not in the comparison group would indicate a treatment effect. This is because if there is a change or discontinuity in the treatment group, but not in the group that did not receive treatment, it suggests that the treatment had an impact. Discontinuity in neither group or both groups does not provide evidence of a treatment effect, and discontinuity only in the comparison group suggests something else may be affecting the results.

Question 20

Program evaluation represents perhaps the extreme case of
A)basic research.
B)applied research.
C)theory development.
D)experimental control.

Accepted Answer

Program evaluation is a form of applied research, as it involves assessing the effectiveness of a specific program or intervention in a real-world setting. It is distinct from basic research, which seeks to expand knowledge in a particular field, and from theory development, which seeks to generate new models and concepts to explain phenomena. While experimental control may be important in program evaluation, it is not the primary focus.

Quiz 10: Quasi-Experimental Designs and Program Evaluation

Which of the following is not one of the four questions addressed in program evaluation?

The one-group pretest-posttest design is

The relationship between basic research and applied research can best be described as

If the residents,staff,and facilities of a nursing home where a quasi-experiment was done are likely to be different from those in other nursing homes,then the findings of the study are likely to lack

After implementing a program to improve admitting procedures in a hospital emergency room,a researcher seeks to determine whether the program is being implemented as proposed.Which research method will best serve this purpose?

Program evaluation is a hybrid discipline whose professional activity

Among the following designs,which allows researchers to rule out the most threats to internal validity?

When individuals in one group of a nonequivalent control group design experience natural changes at a faster rate than individuals in the other group,there is danger of a threat to internal validity called

The effect of a treatment in a simple interrupted time-series design is indicated by

Social policy research (like the research done by Ellen Langer on the effects of living in a nursing home among the elderly) serves the purpose(s) of

A human services agency was interested in finding out whether establishing a food distribution center in their community would be providing a service the community would use.Which of the following types of assessments in program evaluation would this agency find useful for this question?

Which of the following threats to internal validity is controlled in the nonequivalent control group design?

A researcher was asked to evaluate the outcome of a program implemented to enhance recycling in a community.Which method will the researcher likely select to answer the question of whether the program was effective?

A student used a nonequivalent control group design to examine the effectiveness of a computer module the library uses to introduce first year students to the library's resources.Which threat to internal validity could she potentially rule out by examining the pretest scores for both groups?

Even when pretest scores are the same,on average,for treatment and comparison groups in a nonequivalent control group design,the two groups may not be equivalent because

When two groups in a nonequivalent control group design do not differ on the pretest,the researcher can conclude that

A simple interrupted time series design provides a reliable measure of the effect of the treatment because there are many observations both before and after the treatment.The major threat to internal validity in this design is

Strong evidence of a treatment effect in a time series with nonequivalent control group design is indicated by

Program evaluation represents perhaps the extreme case of