Deck 10: Quasi-Experimental Designs Applied Research

Question 1

Consider the methodological principles you have learned about so far. The Hollingworth's Coca-Cola study incorporating several of these principles. Which of the following did they not use in their study?

A) a factorial design
B) a double blind
C) a placebo control condition
D) counterbalancing

Answer 1

a factorial design

Question 2

What makes a nonequivalent control group design a quasi-experimental design?

A) it uses a pretest
B) random assignment is not possible
C) descriptive statistics can be used, but inferential analyses cannot be done
D) only a single group is tested

Answer 2

random assignment is not possible

Question 3

Early psychologists often tried to adapt laboratory methods to solve applied problems. For instance, Stanford psychologist Walter Miles tried to help the football team by adapting which method?

A) maze learning
B) mental testing
C) reaction time
D) trend analysis

Answer 3

reaction time

Question 4

Quasi-experimental designs can include all of the following except

A) random assignment
B) independent variables
C) subject variables
D) nonequivalent groups

Question 5

Which of the following is true about the Hollingworth's Coca-Cola study?

A) their research outcome was damaging enough to cause the company to remove caffeine from the drink
B) they used just a few subjects, and each subject was tested repeatedly, but they failed to use counterbalancing procedures
C) they used placebo controls and a double blind procedure
D) they were not allowed to publish their results unless they were favorable to Coca-Cola

Question 6

A nonequivalent control group design usually (but not necessarily) includes

A) pretests
B) random assignment
C) causal conclusions
D) a single group given pretest, then treatment, then posttest

Question 7

What is the relationship between applied research and theory testing?

A) the results of applied research bear directly on the evaluation of a theory (e.g., equity theory)
B) because applied research is focused on solving immediate problems, its results have no effect on theory evaluation
C) applied research is more appropriate for theory testing than is basic research
D) the primary purpose of applied research is to test theory

Question 8

In a nonequivalent control group study,

A) the experiment group gets a pretest, then the experimental treatment, then a posttest
B) subjects have to be assigned to the control group by means of a matching procedure
C) to interpret the results, the pretest scores have to be identical for the two groups
D) random assignment can be used if sample size is large enough

Question 9

Experiment is to quasi-experiment as _____ is to _____.

A) internal validity; external validity
B) equivalent groups; nonequivalent groups
C) positive correlation; negative correlation
D) control group; no control group

Question 10

In a study comparing two nonequivalent groups, a selection by history confound occurs when

A) some event intervenes between pretest and posttest and affects just one of the groups
B) some event intervenes between pretest and posttest and affects both selected groups equally
C) selection influences one of the groups and history influences the other
D) some historical event causes a failure of the random assignment procedure

Question 11

Basic research is to applied research as ______ is to _______.

A) field; laboratory
B) experimental; correlational
C) knowledge; problem solving
D) between-subjects; within-subjects

Question 12

Which of the following was true about the cognitive interview study?

A) the cognitive interview was first tested in the laboratory with college students
B) it showed that applied research results are not always consistent with basic research results
C) it showed that applied research has to be correlational rather than experimental
D) it showed that applied research can disprove the results of basic research

Question 13

Applied research

A) examines the basic laws of human behavior
B) is just as likely to occur in the laboratory as in the field
C) focuses on a specific real world problem
D) has no connection with theory

Question 14

The study by Fisher et al. on the "cognitive interview" shows that

A) applied research can be completed in the laboratory just as well as in the field
B) the results of applied research can provide empirical support for basic psychological phenomena (e.g., effect of reinstating context on memory)
C) applied research can solve a problem while at the same time providing no information about basic psychological phenomena
D) basic research phenomena sometimes fail when tried in the field in an applied research study

Question 15

When psychology first emerged as an independent discipline at the end of the nineteenth century, American psychologists

A) were interested in basic but not applied research
B) felt it necessary to show that their new science was useful to society
C) were initially interested in applied research, but quickly abandoned it for basic research
D) were philosophers and generally uninterested in any scientific research

Question 16

Applied research done in a field setting

A) is never able to accomplish random assignment
B) will find it impossible to accomplish informed consent
C) will be stronger in external than in internal validity
D) cannot manipulate independent variables

Question 17

The following questions relate to this study:
Teachers in a local school district are given a test of computer literacy (0-100, with 100 being perfect). Then during the school year they have monthly in-service programs designed to improve their computer literacy. Their computer knowledge is assessed again at the end of the year. Their average pretest score is 40 and their average posttest score is 80. Teachers in a comparable school district aren't given the training, but they do get the pretest and posttest.
This study uses a _________ design.

A) mixed factorial design
B) nonequivalent control group
C) interrupted time series
D) time series with a control group

Question 18

The following questions relate to this study:
Teachers in a local school district are given a test of computer literacy (0-100, with 100 being perfect). Then during the school year they have monthly in-service programs designed to improve their computer literacy. Their computer knowledge is assessed again at the end of the year. Their average pretest score is 40 and their average posttest score is 80. Teachers in a comparable school district aren't given the training, but they do get the pretest and posttest.
What pretest-posttest combination for the second group would make you believe that the treatment program was effective?

A) 50 - 50
B) 30 - 70
C) 40 - 80
D) 80 - 80

Question 19

In a nonequivalent control group study,

A) before the study begins, the two groups are different from each other in some systematic way
B) one group is treated while the second group is not treated
C) the major comparison between the groups is in terms of the amount of change from pretest to posttest
D) all of the above

Question 20

Applied research

A) was important to American psychologists from the beginning of psychology in the U.S.
B) became important to American psychologists after World War II
C) has never been as important to American psychologists as basic research
D) has only recently become popular among American psychologists

Question 21

Interrupted time series designs

A) enable researchers to draw causal conclusions
B) are well suited for analyzing trends
C) aren't able to include control groups
D) require a minimum of 10 pretest scores and 10 posttest scores

Question 22

Using the system developed by Campbell and Stanley, which design is this?

A) P x E factorial design
B) an archival experiment
C) nonequivalent control group design
D) interrupted time series design

Question 23

What is the defining feature of an interrupted time series with switching replications?

A) a program is implemented in two different situations at two separate times
B) a control group is added to the treatment group
C) the effects of a program on one dependent variable is compared with the effects on a second dependent variable
D) a program is implemented then taken away ("switched")

Question 24

Archival research has to deal with all of the following problems except

A) experimenter bias
B) nonrepresentative information
C) subject reactivity
D) missing data

Question 25

The following questions relate to this study:
Teachers in a local school district are given a test of computer literacy (0-100, with 100 being perfect). Then during the school year they have monthly in-service programs designed to improve their computer literacy. Their computer knowledge is assessed again at the end of the year. Their average pretest score is 40 and their average posttest score is 80. Teachers in a comparable school district aren't given the training, but they do get the pretest and posttest.
When evaluating the in-service programs, you determine that the second group scored 40 on both the pretest and posttest. You also discover that teachers in the first group were given home computers one month after the program began. How would you interpret the results?

A) the change in the first group demonstrates a regression to the mean
B) the in-service training might have had some effect, but there is a serious selection by history confound here
C) the in-service training program is the only factor that could have produced the effect
D) the second group obviously didn't try, so any comparison is meaningless

Question 26

The following questions relate to this study:
Teachers in a local school district are given a test of computer literacy (0-100, with 100 being perfect). Then during the school year they have monthly in-service programs designed to improve their computer literacy. Their computer knowledge is assessed again at the end of the year. Their average pretest score is 40 and their average posttest score is 80. Teachers in a comparable school district aren't given the training, but they do get the pretest and posttest.
What pretest-posttest combination for the second group would make you suspect that regression produced at least some of the change in the first group?

A) 50 - 50
B) 30 - 70
C) 40 - 80
D) 80 - 80

Question 27

Each of the following studies illustrates a nonequivalent control group design except

A) the study on the effects of flextime on two manufacturing plants, one in Cleveland and one in Pittsburgh
B) the study evaluating a training program for coaches
C) the study examining the influence of the California earthquake on nightmares
D) the study examining the effects of introducing an incentive plan on productivity in an iron foundry

Question 28

Ulrich's "room with a view" study includes all of the following except

A) archival data
B) an independent variable
C) limits on the type of patients studied
D) random room assignment

Question 29

In Wood and Bootzin's study of nightmares following earthquakes, students living in the area of the earthquake and who experienced it were in the experimental group. The nonequivalent control group consisted of

A) comparable students who heard of the earthquake but lived at a university far away from it
B) students living in the quake area who slept through it
C) parents of the students
D) comparable students who lived at a distant university and had not heard about the earthquake

Question 30

If a researcher decides to insure through matching that two nonequivalent groups start with the same pretest score, results could very likely be affected by

A) history
B) testing
C) regression
D) maturation

Question 31

Using the system developed by Campbell and Stanley, which design is this?

A) P x E factorial design
B) a multiple case study
C) nonequivalent control group design
D) interrupted time series design

Question 32

Wagner et al. examined worker productivity as the result of the institution of a worker incentive plan. They used a(n) ________ design.

A) interrupted time series with a control group
B) nonequivalent control group
C) needs analysis
D) interrupted time series

Question 33

Wagner et al. used an interrupted times series design to examine changes in worker productivity after the institution of a worker incentive plan. In addition to measuring productivity, they also examined other information in order to rule out alternative explanations of their results. They were able to rule out _______ because _______.

A) history; no significant event occurred around the time of the program's start
B) instrumentation; the manner of measuring productivity didn't change
C) selection; of an absence of significant worker turnover
D) all of the above

Question 34

In the above item, if worker absenteeism improved after flextime was installed and was generally lower after than before the program, what could be concluded?

A) the program worked
B) some uncontrolled historical event might have brought about the change
C) a regression to the mean could have occurred
D) all of the above are possibilities

Question 35

To see if a flextime program improved worker absenteeism, records were examined for six months prior to and six months following the program's institution. What design is this?

A) time series with control group
B) nonequivalent control group
C) multiple pretest-posttest
D) interrupted time series

Question 36

In Wood and Bootzin's study of nightmares following earthquakes, what design was used?

A) interrupted time series
B) time series with a control group
C) formative evaluation
D) nonequivalent control group design

Question 37

Which of the following is a problem with archival research?

A) it is not possible to include an independent variable
B) there is no way to control any extraneous factors
C) the researcher is limited by the manner is which data were originally collected
D) there is a very limited amount of archival data available to researchers

Question 38

The time series study on California's "three strikes and you're out" program to reduce crime used which of the following variations on the basic time series design?

A) the researchers added an "interruption" to the sequence of events
B) the researchers added a switching replication, comparing California with another state that implemented the program 6 months after California
C) the researchers added a control group, comparing California with Texas
D) the researchers added a second dependent variable, comparing felonies with misdemeanors

Question 39

Which of the following is true about Smith and Smoll's study on the training program for coaches?

A) it used a nonequivalent control group design
B) it was an example of an interrupted time series design
C) it was a formative evaluation
D) unlike similar studies, there was no pretest given

Question 40

Initial evaluations of the Head Start program seemed to question program effectiveness. An alternative explanation of the failure to find improvement is that

A) forced matching might have created an unfavorable regression to the mean
B) a selection by history confound probably occurred
C) fade-out effects occurred
D) none of the above - there is general agreement that the program in fact does not work

Question 41

Nonequivalent control group designs and interrupted time series designs are most likely to be a part of a

A) cost analysis
B) needs analysis
C) formative evaluation
D) summative evaluation

Question 42

Which type of program evaluation is needed when trying to choose between two programs, both of which seem to be equally effective?

A) needs analysis
B) cost-effectiveness analysis
C) formative analysis
D) summative analysis

Question 43

Campbell's "reforms as experiments" article includes a description of an evaluation of a state police crackdown on speeding in Connecticut. There were 324 fatalities in the year before the crackdown and 284 in the year after. According to Campbell, the program was partially the cause of the drop, but ______ also occurred.

A) selection
B) maturation
C) instrumentation
D) regression to the mean

Question 44

Campbell's "reforms as experiments" article includes a description of an evaluation of a state police crackdown on speeding in Connecticut. In conducting this study, Campbell used

A) an interrupted time series design, with comparisons made to similar states
B) a simple one-group interrupted time series design
C) a nonequivalent control group design, with comparisons made to similar states
D) a pretest posttest design (no control group)

Question 45

The purpose of a _________ is to examine what is actually occurring in a program to see if it is running as designed.

A) program audit
B) summative evaluation
C) survey of existing programs
D) cost analysis

Question 46

What is the advantage of doing a formative evaluation?

A) it enables the agency to decide whether or not to implement the program
B) it allows program changes to be made before it is too late
C) it allows the researcher to complete an experimental study instead of a quasi-experimental study
D) it allows for a final "formed" evaluation of the success or failure of the program

Question 47

It is sometimes said that program evaluation research is susceptible to Type II errors. What does this mean?

A) significant effects are found, which later turn out to be phony effects
B) small program effects might truly exist, but the measuring tools aren't sensitive enough to detect them
C) program evaluators often use only descriptive statistics and ignore inferential ones
D) program evaluators often fail to take costs into account in their analyses

Question 48

Formative evaluation is to summative evaluation as ________ is to ________ .

A) before; after
B) before; during
C) during; before
D) during; after

Question 49

There were a record 324 automobile fatalities in Connecticut in 1955, which prompted a statewide crackdown on speeding. The next year there were 284 fatalities. What did Campbell conclude from his evaluation of the state's program?

A) regression to the mean occurred and it accounted for the reduction from 324 to 284
B) because the program was so highly publicized, it accounted for the drop from 324 to 284
C) the drop from 324 to 284 was partly due to regression, but the program was also successful to a degree
D) the results could be attributed partly to regression and partly to instrumentation (they defined "traffic fatality" different in 1956)

Question 50

According to Campbell, why is regression to the mean such a likely explanation when evaluating the effects of social programs (e.g., speeding crackdowns)?

A) programs to fix some problem are usually begun after an especially bad year
B) program evaluation is always correlational rather than experimental
C) it is usually easy to rule out all other confounds
D) regression is the one confound that cannot be evaluated by comparison with a control group

Question 51

A needs analysis is designed to answer the question:

A) Is this program currently working the way it is supposed to work?
B) Did this program produce the effects it was supposed to produce?
C) Should we develop this program?
D) Which of these two very effective programs should we continue?

Question 52

Surveys of existing programs, census data, and surveys of residents all contribute to which phase of program evaluation?

A) cost analysis
B) needs analysis
C) formative evaluation
D) summative evaluation

Question 53

Cost-effectiveness analysis occurs at the end of a program, but assessing cost is also an important part of

A) program audit
B) needs analysis
C) summative evaluation
D) none of the above

Question 54

A cost-effectiveness analysis is designed to answer the question:

A) Is this program working the way it is supposed to work?
B) Did this program produce the effects it was supposed to produce?
C) Should you develop this program?
D) Which of these two very effective programs should we continue?

Question 55

In Ulrich's "room with a view" study, what was the independent variable?

A) whether recovering patients saw trees or a brick wall when looking out their windows
B) whether recovering patients had a room with a window or not
C) whether or not patients requested pain medication
D) from nursing records, whether or not nurses liked the patients

Question 56

As part of a program to inform clients about how to manage credit card expenses, they are counseled, and at the end of an opening session, given an expense log to fill out over the next six months. Examining these logs after a month would constitute part of a

A) cost analysis
B) needs analysis
C) formative evaluation
D) summative evaluation

Question 57

Summative evaluations sometimes fail to detect genuine program effects. These Type _____ errors often happen because _______.

A) II; the measuring tools aren't very good
B) II; the program usually doesn't work
C) I; the measuring tools aren't very good
D) I; the program really doesn't work

Question 58

Census data is most likely to be used during which phase of program evaluation?

A) needs analysis
B) summative evaluation
C) formative evaluation
D) cost-effectiveness analysis

Question 59

A summative evaluation is designed to answer the question:

A) Is this program currently working the way it is supposed to work?
B) Did this program produce the effects it was supposed to produce?
C) Should you develop this program?
D) Which of these two very effective programs should we continue?

Question 60

A formative evaluation is designed to answer the question:

A) Is this program currently working the way it is supposed to work?
B) Did this program produce the effects it was supposed to produce?
C) Should you develop this program?
D) Which of these two very effective programs should we continue?

Question 61

In program evaluation research, what is meant by the problem of "leakage?"

A) people in one group discover how people in another group are being treated
B) people in one group switch to another group
C) it's another name for attrition
D) stakeholders give away the purpose of the study in order to sabotage it

Question 62

Wood and Bootzin used a(n) __________ design to evaluate the effects of experiencing an earthquake on the quality and quantity of nightmares.

Question 63

__________ often includes a survey of people who will be affected by the program in question.

Question 64

The needs analysis done at the Du Pont Company included all of the following except

A) a survey of existing company programs for enhancing health
B) an analysis of employee health records
C) census data on the proportion of people who were overweight
D) worker surveys (on their knowledge of healthy behavior)

Question 65

Collectively, persons with a direct interest in a program are known as

A) a focus group
B) stakeholders
C) key informants
D) a community forum

Question 66

A _________ is a systematic examination of qualitative information in terms of predefined categories.

Question 67

Informed consent can be a problem in program evaluation research. Why?

A) there tends to be a high rate of attrition in this research
B) powerless clients might believe that a failure to give consent would have adverse consequences for them
C) most subjects in these studies are under the age of 18
D) subjects often cannot be reached before the study begins

Question 68

A _________ is an person with special knowledge about the feasibility of a program.

Question 69

Describe the defining feature a quasi-experimental design.

Question 70

In a(n) ___________ design, both groups get pretests and posttests, but just one group gets the experimental treatment.

Question 71

In _________ research, the investigator relies on the gathering together of information already collected for another purpose.

Question 72

A(n) __________ design occurs whenever a lack of control makes it impossible to draw causal conclusions.

Question 73

Program audits are most likely to be associated with which type of program evaluation?

A) needs analysis
B) cost analysis
C) summative evaluations
D) formative evaluations

Question 74

Use the cognitive interview study to show (a) the general purpose of applied research, and (b) how applied research can also shed light on basic phenomena.

Question 75

Explain why early American psychologists were so interested in applied psychology and give a "football" example of applied work from psychology's early years.

Question 76

The overall evaluation of program effectiveness involves conducting a(n) __________.

Question 77

Trends can be evaluated by using a(n) ___________ design.

Question 78

In a(n) _____________, a researcher examines a program-in-progress to see if it is running as planned.

Question 79

Stolzenberg and D'Alessio et al. used a(n) __________ design to evaluate the effects of a change in a felony sentencing law in California.

Question 80

Deciding between two equally effective programs can be accomplished by completing a(n) ____________.