Deck 12: Experimental Research: Designs, Part 1

Question 1

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-A pretest similar to the posttest is administered before the experimental treatment.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Answer 1

Testing

Question 2

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-Students below the 15th percentile in reading achievement who volunteer are placed in the experimental group, and nonvolunteers at the same reading level are placed in the control group.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Answer 2

Differential selection

Question 3

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-During the experiment students are exposed to special instructional experiences that are not part of the experimental treatment.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Answer 3

History

Question 4

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-Low achievers drop out at a higher rate than other students from an individualized reading program designed to improve reading achievement.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Question 5

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-A group of students who fall below the 15th percentile on a reading achievement test earns higher scores on the posttest following an experimental treatment, whereas a group of students who fall above the 85th percentile earns lower scores on the same posttest following the same experimental treatment.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Question 6

Match each the situation that illustrates it with type of extraneous variable that threatens the internal validity of an experiment.

-Students who attend summer camp three years in a row show annual increases on a measure of physical strength.

A) History
B) Maturation
C) Testing
D) Experimental treatment diffusion
E) Statistical regression
F) Differential selection
G) Experimental mortality

Question 7

An experiment is internally valid to the extent to which it

A) yields findings that are generalizable to a defined population.
B) uses valid measurement instruments.
C) controls for extraneous variables.
D) all of the above.

Question 8

A group of teachers receives access to the Internet as part of their participation in an experiment. The school district's administrators decide that this is a good idea, and provide Internet access for all teachers in the district, including control group teachers. The administrators' action weakens internal validity through

A) resentful demoralization of the control group.
B) compensatory equalization of treatments.
C) compensatory rivalry by the control group.
D) differential selection.

Question 9

If the same observers rate students' on-task behavior before and after an experimental intervention, their ratings are susceptible to an internal-validity threat known as:

A) statistical regression.
B) experimental treatment interaction.
C) an instrumentation effect.
D) a testing effect.

Question 10

Multiple-treatment interference refers to the situation in which

A) research participants' performance on the dependent variable reflects the combined effect of more than one treatment.
B) extraneous variables rather than the experimental treatment are responsible for the experimental outcome.
C) high correlations between pretest and posttest scores reduce the chances of significant gain scores.
D) research participants in the control group are affected by their knowledge of the experimental treatment.

Question 11

Students' racial attitudes are measured by a paper-and-pencil test immediately before and after viewing a film denouncing racial prejudice. What threat to the external validity of the experiment does this procedure raise?

A) Statistical regression
B) Attribute-treatment interaction
C) Multiple-treatment interference
D) Pretest sensitization

Question 12

In the Hawthorne effect, individuals who participate in an experiment improve their performance because

A) they are aware that they are participating in an experiment.
B) the experimental treatment leads to greater learning.
C) they like the experimenter.
D) the posttest is easier than the pretest.

Question 13

The external validity of an experiment refers to whether

A) practitioners view its findings as credible.
B) the experiment's findings generalize to individuals and settings beyond those that were studied.
C) an independent review board approves the experiment's design and statistical procedures.
D) the posttest measures used in the experiment have good construct validity.

Question 14

Which one of the following is not a threat to the external validity of an experiment?

A) Novelty and disruption effects
B) Pretest sensitization effects
C) Measurement of the dependent variable
D) Experimental treatment diffusion

Question 15

In determining the population validity of an experiment, one needs to know the researchers'

A) experimentally accessible population.
B) target population.
C) personological characteristics of the sample that might interact with the experimental treatment.
D) all of the above.

Question 16

A researcher provides an extensive orientation to introduce research participants to the experimental treatment. The purpose of this orientation is probably to

A) control for experimental treatment diffusion.
B) control for history.
C) strengthen the representativeness of the experimental design with respect to real-life environments.
D) control for the Hawthorne effect.

Question 17

An experiment that incorporates principles of representative design

A) controls more rigorously for differential selection.
B) takes more variables into account in the research design than an experiment that has a systematic design.
C) controls more rigorously for experimenter effects.
D) sacrifices population validity for increased internal validity.

Question 18

In an experiment that follows principles of representative design, it is

A) acceptable to observe what students are doing during the experiment.
B) not acceptable to observe what students are doing during the experiment.
C) acceptable to observe the research participants, but not the social context in which the experiment is being conducted.
D) not acceptable to prepare the research participants for the experiment.

Question 19

A useful technique to minimize the effect of experimenter bias on the outcomes of an experiment is for the researcher to

A) fully inform the experimenters about the study's objectives and hypotheses.
B) conduct the experiment rather than train experimenters to do so.
C) train experimenters who are naive about the study to implement the treatments and collect the data.
D) make the control treatment as desirable as the experimental treatment.

Question 20

Treatment fidelity refers to the extent to which

A) the individuals who conduct the experimental and control interventions have been trained in the specifications of those interventions.
B) the research participants have received training in what the interventions require them to do.
C) the experimental and control conditions have been implemented according to the researchers' specifications.
D) the research participants are free to adjust the experimental and control conditions to their individual learning styles.

Question 21

Intact groups often pose a problem when designing experiments because

A) intact groups cannot be randomly selected from a defined population.
B) intact groups cannot be assigned to control treatments.
C) administrators are usually concerned that experiments may weaken the the groups' cohesiveness.
D) individuals within an intact group cannot be randomly assigned to the experimental and control groups.

Question 22

One good way to avoid the internal-validity threats that might result from withholding the experimental treatment from the control group is to

A) give the experimental treatment to a subsample of the control group chosen by public lottery.
B) identify volunteers who do not object to being in a no-treatment control group.
C) give the experimental treatment to the control group after the experiment is concluded.
D) work with the control group until all necessary data have been collected, and then begin working with the experimental group.

Question 23

The equivalence of the experimental and control groups in an experiment

A) is not guaranteed by the use of random assignment procedures.
B) is guaranteed by the use of random assignment procedures.
C) is not affected by the size of the total sample from which experimental and control groups will be formed.
D) does not affect the internal validity of an experiment.

Question 24

The one-group pretest-posttest design is best used when

A) the characteristic that constitutes the dependent variable is relatively stable.
B) the experimental treatment extends over a relatively long period.
C) history may function as an extraneous variable.
D) the control group has not been randomly selected.

Question 25

Which of the following experimental designs has the best internal validity?

A) The one-group pretest-posttest design.
B) The pretest-posttest control-group design.
C) The time series design.
D) The one-shot case study.

Question 26

A researcher designs an experiment in which student teachers are randomly assigned to three treatment groups: twice-weekly supervisory visits, weekly supervisory visits, and supervisory visits every two weeks. This is an example of a

A) pretest-posttest control-group design.
B) one-shot case study.
C) one-group pretest-posttest design.
D) one-variable multiple-condition design.

Question 27

(ob. 6) The posttest-only control-group design has an advantage over the pretest-posttest control-group design when

A) the total sample that is to be assigned to the experimental and control groups is small.
B) the researchers are interested in forming subgroups within the experimental and control groups.
C) there is a possibility that administration of a pretest will affect the experimental treatment.
D) there is likely to be differential attrition during the course of the experiment.

Question 28

Of the following experimental designs, the strongest in external validity is the

A) one-shot case study.
B) one-group pretest-posttest design.
C) posttest-only control-group design.
D) pretest-posttest control-group design.

Question 29

The main threat to the external validity of the pretest-posttest control-group design is the

A) Hawthorne effect.
B) interaction between history and the experimental treatment.
C) interaction between time of measurement and the experimental treatment.
D) interaction between the pretest and the experimental treatment.

Question 30

Of the following experimental designs, the one that is weakest in internal validity is the

A) one-group pretest-posttest design.
B) posttest-only control-group design.
C) pretest-posttest control-group design.
D) time-series design.

Question 31

Of the following procedures for testing statistical significance, the one most likely to be used in analyzing the data from a pretest-posttest control-group design is

A) path analysis.
B) analysis of covariance.
C) two t tests, one comparing the pretest and posttest means of the experimental group and the other comparing the pretest and posttest means of the control group.
D) a t test comparing the posttest means of the experimental and control groups.

Question 32

What is the preferred statistical technique for analyzing the data from an experiment that employs a posttest-only control-group design?

A) a t test comparing the posttest means of the experimental and control groups.
B) multiple regression.
C) analysis of covariance.
D) the correlation ratio.

Question 33

List and describe five types of extraneous variables that are threats to the internal validity of an experiment.

Question 34

What does it mean to say that an experiment has good internal validity?

Question 35

a. Describe pretest sensitization.
b. Is it a threat to the internal or external validity of an experiment?

Question 36

Representative design is an attempt to make experimental conditions more representative of the natural environment. What are two recommended approaches for making educational research more representative?

Question 37

What are two techniques that a researcher can use to avoid experimenter bias?

Question 38

How can randomization be achieved if individuals within intact groups cannot be randomly assigned to the experimental and control groups?

Question 39

a. What is one advantage of the pretest-posttest control-group design over the posttest-only control-group design?
b. What threat to external validity is present in the pretest-posttest control-group design?

Question 40

a. What is the most important limitation of the one-group pretest-posttest design?
b. What kinds of variables can be studied by using this design?

Question 41

A researcher conducts an experiment that employs a pretest-posttest control-group design. The pretest and the posttest are similar tests of American history. Describe the statistical technique that should be used to analyze the pretest and posttest scores.

Question 42

A researcher is planning to test the effectiveness of a new reading program. A sample of 40 teachers distributed evenly among five schools in one school district has volunteered to use the program for a semester. Using only this sample, the researcher wishes to form two equivalent groups, one of which will participate in the special program while the other receives the regular program. State methods for achieving equivalence between the groups through random assignment.

Question 43

A team of researchers want to test the effectiveness of providing high school students with a note-taking outline of each chapter that they are assigned to read in a history class. They have permission to conduct the experiment with two history classes, each containing 30 students. They randomly form two groups within each class: The experimental group receives a note-taking outline each time it is assigned a chapter, whereas the control group receives the same assignment but not a note-taking outline. On the basis of this information, what flaw have the researchers introduced into the experimental design?

Question 44

A researcher is planning an experiment to test the effectiveness of the discussion method at the high school level. One teacher has agreed to use the discussion method in her class several times a week for a semester. Another teacher has agreed to teach the same content but without using the discussion method. The researcher will collect the following data: student scores on an achievement test and on a scale measuring attitudes toward the instruction. What recommendations can you offer to improve the representativeness of this experimental design?

Question 45

Study the following research plan and identify any variables and/or conditions that may pose a threat to the internal validity of the experiment. Name the most likely threats to internal validity, and explain each briefly in terms of the research plan.Research plan: The researcher wants to determine the effect of a special reading program on the reading achievement of inner-city children. The study will be longitudinal, extending from the beginning of grade 2 to the end of grade 3. He identifies 50 children who are nonreaders at the end of grade 1; these children will receive the special program. He randomly selects 50 other children in the same school and grade level to serve as a control group. The researcher administers the same reading test to both groups at the start of grade 2 and at the end of grade 3. By the end of grade 3, there are 27 children who have remained in the special program and 36 children in the control group. The researcher compares the reading gains of the two groups and finds that each group has gained an average of 1.6 years in reading grade placement.

Question 46

A researcher is planning to test the effectiveness of a new reading program. A sample of 40 teachers distributed evenly among five schools in one school district has volunteered to use the program for a semester. Using only this sample, the researcher wishes to use random assignment to form groups, one of which will participate in the special program while the other receives the regular program. State methods for achieving equivalence between the groups through random assignment.

Question 47

A researcher wants to determine whether students learn more when they take unannounced pop quizzes or regularly scheduled, announced quizzes. The measure of learning will be a 50-item achievement test on the content covered in a semester-long course. Describe the steps involved in designing and conducting an experiment based on this information.