Deck 10: Experimental Research

Question 1

Which of the following is a threat to external validity?

A) Statistical regression
B) Testing effects
C) Specificity of variables
D) Instrumentation

Answer 1

C

Question 2

Gavin's study compared reading error rates and reading comprehension scores between two groups of 50 third-grade students that received different reading programs. The treatment variable in Gavin's study is

A) error rates.
B) third graders.
C) reading instruction.
D) reading comprehension.

Answer 2

C

Question 3

The threat to validity defined as the effects of the occurrence of events that are not part of the treatment but affect the dependent variable is

A) history.
B) testing.
C) statistical regression.
D) instrumentation.

Answer 3

A

Question 4

Treatment variable is another name for

A) independent variable.
B) dependent variable.
C) extraneous variable.
D) mediating variable.

Question 5

Consider the example of Roy, a researcher who studies the role of worked examples in engineering education. He measures students test performance and ratings of their instructors. He randomly assigns classes to condition. The independent variable in Roy's research is

A) engineering education.
B) worked example exposure.
C) test performance.
D) teacher ratings.

Question 6

A researcher's efforts to remove influence of variables other than the independent variable is referred to as

A) control.
B) assignment.
C) sorting.
D) determination.

Question 7

The final step in an experimental study is the

A) selection and definition of a problem.
B) execution of procedures.
C) formulation of conclusions.
D) analysis of data.

Question 8

Replications in experimental research are generally conducted to facilitate

A) internal validity.
B) external validity.
C) internal reliability.
D) external reliability.

Question 9

A laboratory study compared learning from text in two conditions, one with underlining of important terms and one without underlining, found that the underlining group did better on factual items. Marina then conducted a study with tenth-grade history students and compared one group that read a chapter with the important text underlined and one group that read a chapter without any underlining. Marina's study illustrates an example of a(n)

A) test for reliability.
B) internal validity study.
C) replication study.
D) regression study.

Question 10

In experimental studies, the sample may be derived from

A) one population.
B) two populations.
C) three populations.
D) more than three populations.

Question 11

The type of validity concerned with whether findings can be generalized to other groups or other settings is referred to as

A) internal validity.
B) external validity.
C) content validity.
D) construct validity.

Question 12

Which of the following is a threat to internal validity?

A) Differential selection of participants
B) Selection-treatment interaction
C) Posttest sensitization
D) Multiple-treatment interference

Question 13

The type of validity concerned with whether differences between groups are due to the manipulation of the independent variable is referred to as

A) internal validity.
B) external validity.
C) content validity.
D) construct validity.

Question 14

Gavin's study compared reading error rates and reading comprehension scores between two groups of 50 third-grade students that received different reading programs. His t test data analysis did not indicate any differences between the two types of instruction. The most likely reason for his findings is that

A) his measures did not measure reading.
B) his sample is too small.
C) he conducted the wrong analysis.
D) his groups were too similar.

Question 15

Consider the example of Roy, a researcher who studies the role of worked examples in engineering education. He measures students test performance and ratings of their instructors. He randomly assigns classes to condition. Roy argues that his research is experimental. However, you argue that it is quasi-experimental given the following reason.

A) He randomly assigned intact groups not individual students
B) Ratings of instructors are not measured in experiments.
C) He uses a correlational approach to the study.
D) There is no dependent variable in his research study.

Question 16

Patsy conducted an experiment that included two groups of junior high girls. One group received no training while the other received a 3-hour intervention about the benefits of careers in math and science domains. Patsy compared the willingness of girls to enroll in subsequent math courses as her outcome measure. Her t test analysis indicated no differences between her treatment group and the no-exposure control. This finding is likely due to

A) preexisting group differences.
B) limited treatment exposure.
C) sampling error.
D) too small a sample size.

Question 17

Marcy is concerned that her findings may be due to an extraneous uncontrolled variable and not her treatment. Marcy is most concerned with

A) internal consistency reliability.
B) inter-rater reliability.
C) external validity.
D) internal validity.

Question 18

Compared to all other types of research, experimental research

A) requires greater number of participants.
B) provides evidence concerning cause-effect relations.
C) measures dependent and independent variables.
D) collects samples to generalize to populations.

Question 19

Laura was conducting a study on two treatment interventions to increase patriotism. During her study, terrorist attacks on a federal building occurred. Laura should be concerned with which of the following validity threats?

A) Statistical regression
B) Instrumentation
C) History
D) Testing

Question 20

In experimental studies, but NOT in other studies,

A) the researcher manipulates the dependent variables.
B) the researcher manipulates the independent variables.
C) the researcher tries to eliminate the effects of extraneous variables.
D) the researcher examines a priori differences between groups.

Question 21

Max's study compared the differences in teachers' attitudes about inclusion before and after a Saturday in-service training. He assessed their post-intervention attitudes with a self-report questionnaire. With which of the following threats to validity should Max be particularly concerned?

A) Instrumentation
B) Mortality
C) Selection
D) History

Question 22

Anne's study compared pre-post differences in prekindergarten children's strategies for alternatives to violent displays when frustrated by others. She found that there were differences between pre and posttesting. With which validity threat should Anne be most concerned?

A) Differential selection
B) Statistical regression
C) Maturity
D) Instrumentation

Question 23

PJ conducts an experimental study on the effects of soft music during high stakes science testing. He randomly assigns students at the school. In one condition he does not provide music for testing while in the other group he does provide the music. One concern in his study is that during the year the music teacher at the school he administered the study shared with students the physiological benefits of soft music. The introduction of the benefits of music during the duration of his study can be considered which of the following threats?

A) History
B) Maturation
C) Mortality
D) Testing

Question 24

Linda is conducting a federally-funded research project that examines literacy skills in middle school learners. Bonnie recently started a new study that specifically focuses on reading strategies. Some of the students are participating in both studies.

A) This is of no concern because there are two different studies.
B) This is a mostly a threat to internal validity.
C) This is mostly a threat to external validity.
D) This is an example of the placebo effect.

Question 25

In an experimental study, the loss of participants in the experimental condition can be referred to as

A) a placebo effect.
B) attenuation.
C) mortality.
D) a novelty effect.

Question 26

Lisa's study assessed the effects of an intervention to facilitate the math skills of low-achieving second grade learners. She administered a treatment to the students in the form of structured computer lesson and found that students' skills increased. With which validity threat should Lisa be most concerned?

A) History
B) Statistical regression
C) Instrumentation
D) Mortality

Question 27

Given a choice of the following research designs, which design controls the most threats to internal validity?

A) One-group posttest-only design
B) One group pretest-posttest design
C) Posttest-only design with nonequivalent groups
D) Pretest-posttest control group

Question 28

PJ conducts an experimental study on the effects of soft music during high stakes science testing. He randomly assigns students at the school. In one condition he does not provide music for testing while in the other group he does provide the music. He administers a pretest at the beginning of the year and a posttest at the end of the year. PJ's design is best represented by which of the following?

A) Static-group comparison
B) Pretest-posttest control group design
C) One-shot case study
D) One-group pretest-posttest design

Question 29

Molly's study examines the benefits of exercise on weight loss. In one group she advocates exercise as well as a new diet program. In the comparison group she only presents the diet program. Those in the comparison group find out that some of Molly's clients are in an exercise program too, so they form an exercise group. This is an example of

A) compensatory rivalry.
B) placebo effect.
C) treatment diffusion.
D) experimenter effects.

Question 30

The most significant threat to internal validity of the time series design is

A) history.
B) maturation.
C) regression.
D) mortality.

Question 31

An experiment examined whether giving children recess prior to testing helped their test performance. For one of the semesters, the researcher sends half of the classes out for 10 minutes of recess prior to testing. For the other half, he provides 10 minutes of free time after the test. The students in the non recess condition decided they would try harder on the tests because they did not go to recess. This is an example of a(n)

A) Experimenter threat.
B) John Henry effect.
C) Placebo effect.
D) Hawthorne effect.

Question 32

Of the following, the design that best controls for internal validity threats is the

A) posttest only control group design.
B) pretest-posttest group design.
C) time series design.
D) Solomon four-group design.

Question 33

As a secondary mathematics teacher, Hernandez conducted a study that explored whether giving children recess prior to testing helped their test performance. For one of the semesters, he sends half of his classes out for 10 minutes of recess prior to testing for the other half, he provides 10 minutes of free time after the test. Which of the following best represents the design of Hernandez's study?

A) Posttest-only design with nonequivalent groups
B) One-group post-test only control group design
C) Solomon four group design
D) One-group pretest-posttest design

Question 34

In an experiment in a school the researcher is also the teacher administering the conditions. The situation introduces which potential threat to validity?

A) Experimenter
B) Placebo
C) John Henry
D) Instrumentation

Question 35

According to your text the research should report any difficulty obtaining samples because of the increased threat of

A) pretest-treatment interaction.
B) multiple-treatment interference.
C) selection-treatment interaction.
D) experimenter effects.

Question 36

Nadine conducted a pretest-posttest control group design. The comparison group received traditional lecture of course content while the treatment group received the lecture and instructor-provided summary notes. She found, however, no differences between conditions, due in part to students sharing the instructor summaries across conditions. This is an example of

A) compensatory rivalry.
B) treatment diffusion.
C) placebo effect.
D) experimenter effects.

Question 37

In Katia's remedial mathematics study, she collected pretest data from a group of participants. She divided the participants into three groups. One group received no treatment, one group received instruction by a teacher-delivered intervention, one group received peer tutoring on the same problems. After the intervention she tested their math skills. With which validity threat should Katia be most concerned?

A) Maturation
B) Testing
C) Instrumentation
D) Pretest-treatment interaction

Question 38

Of the following, the best way to decrease the threat of pretest-treatment interaction is

A) lengthen the duration of the study.
B) decrease the sample size.
C) increase the significance level.
D) use an attitude scale for the pretest.

Question 39

Hernandez conducted a study that explored whether giving children recess prior to testing helped their test performance. For one of the semesters, he sends half of his classes out for 10 minutes of recess prior to testing for the other half, he provides 10 minutes of free time after the test. Given the study was conducted in a school, students in the recess condition classes told students in the other classes that they were allowed to go to recess before the tests. This is an example of which of the following?

A) History effect
B) Instrumentation bias
C) Treatment diffusion
D) Placebo effect

Question 40

Russell's study compared GPA of those students who volunteered for academic study skills training and those who did not elect to take the training. He found that those who had the training also had higher GPA. With which validity threat should Russell be most concerned?

A) Maturation
B) Instrumentation
C) History
D) Selection

Question 41

Which of the following is a drawback of the Solomon Four group design?

A) Lack of control for threats to internal validity
B) The large number of participants needed
C) Systematic desensitization
D) Increased experimenter effects

Question 42

How many cells does a 2 x 4 factorial design have?

A) 2
B) 4
C) 6
D) 8

Question 43

A study that addressed the effects of gender and four types of practice (manipulated by the researcher) would be symbolized as

A) 2 x 2 factorial design.
B) 4 x 2 factorial design.
C) 2 x 4 factorial design.
D) Solomon four-group design.

Question 44

Which of the following is the primary drawback of the posttest-only control group design?

A) History
B) Selection-treatment interaction
C) Mortality
D) Maturation

Question 45

Which of the following threats to validity is the MOST concerning with the nonequivalent control group design?

A) Selection interactions
B) Instrumentation
C) Mortality
D) Testing

Question 46

Given the following notation, which design is illustrated?
$$\begin{array} { l l } \mathrm { X } _ { 1 } & \mathrm { O } \\\mathrm { X } _ { 2 } & \mathrm { O }\end{array}$$

A) One-group posttest-only design
B) One group pretest-posttest design
C) Posttest only control group design
D) Posttest-only design with nonequivalent groups

Question 47

Jack wants to test whether runners on his cross-country team improve with his new training regimen. He measures their run times at the beginning of the season and again at the end of the season. Runners in Jack's study improving their times due to running generally and not necessarily due to the new regimen is an example of which of the following threats?

A) Selection
B) Maturation
C) History
D) Instrumentation

Question 48

A factorial design includes:

A) One manipulated independent variable.
B) More than one manipulated independent variable.
C) Two or more independent variables, one of which is manipulated.
D) Multiple dependent variables.

Question 49

Jorge studies the effects of mnemonic training on learning vocabulary. He randomly assigns half of the students to a no treatment control group and assigns the others to a mnemonic training group. He assesses students at the end of the study and compares how many words they know. Which of the following designs is Jorge best illustrating?

A) Pretest-posttest control group design
B) Nonequivalent group control group design
C) Posttest-only control group design
D) Time series design

Question 50

Chin-Hsing conducted a study that examined motivation for sport and academics in a Taiwanese school for gifted athletes. One concern that Chin-Hsing has is that the results from the study may not generalize to another school or country. The validity threat most evident in his concern is

A) Internal validity
B) External validity
C) Construct validity
D) Predictive validity

Question 51

The third-grade teachers in one school are interested in the effects of seating assignments on self-esteem of students. Half of the teachers uses a new seating strategy for the year and half use the existing strategy. All students take a self-esteem measure at the beginning of the school year. At the end of the school year, students take a self-esteem measure. This study best illustrates which of the following designs?

A) Nonequivalent control group design
B) Solomon four-group design
C) Pretest-posttest control group design
D) One-group pretest-posttest design

Question 52

Given the following notation, which design is illustrated? $$\mathrm { X } \quad \mathrm { O }$$

A) One-group posttest-only design
B) One-group pretest-posttest design
C) Posttest only control group design
D) Posttest-only design with nonequivalent groups

Question 53

A study that investigates the effects of three different types of instruction (text only, text with pictures, text with animation), and spatial ability (high, low) would be symbolized as a

A) 2 x 2 factorial design.
B) 3 x 2 factorial design.
C) 2 x 3 factorial design.
D) Solomon four-group design.

Question 54

Of the following, which is an example of a pre-experimental design?

A) Time series design
B) Posttest-only design with nonequivalent groups
C) Posttest only control group design
D) Nonequivalent control group design

Question 55

Given the following notation, which design is illustrated? $$\begin{array} { l l l } \mathrm { O } & \mathrm { X } _ { 1 } & \mathrm { O } \\\mathrm { O } & \mathrm { X } _ { 2 } & \mathrm { O }\end{array}$$

A) Pretest-posttest control group design
B) Posttest only control group design
C) Nonequivalent control group design
D) Static group comparison design

Question 56

Kari wants to make sure that all students in her study receive both of the treatments she is testing. To ensure all subjects receive both interventions, which of the following designs should Kari consider?

A) Time-series design
B) Solomon four-group design
C) Counterbalanced design
D) Nonequivalent groups control group design

Question 57

Given the following notation, which design is illustrated? $$\begin{array} { l l l } \mathrm { R } & \mathrm { X } _ { 1 } & \mathrm { O } \\\mathrm { R } & \mathrm { X } _ { 2 } & \mathrm { O }\end{array}$$

A) Pretest-posttest control group design
B) Posttest only control group design
C) Nonequivalent control group design
D) Static group comparison design

Question 58

One caution with factorial designs is that

A) they present more validity threats than most designs.
B) they permit investigation of fewer variables than other designs.
C) with more factors interactions are difficult to interpret.
D) they are not used in educational studies.

Question 59

For which of the following designs is multiple-treatment interference a substantial threat?

A) Time series design
B) Counterbalanced design
C) Solomon four group design
D) Posttest-only control group design

Question 60

Nora is interested in differences in amount of content retained from a lecture based on time of class. She compares her two sections, one that meets at 8 AM and one that meets at 2 PM, by testing them, unannounced, on content that she taught two weeks ago. Which of the following designs is Nora best illustrating?

A) One-group posttest-only design
B) One group pretest-posttest design
C) Posttest only control group design
D) Posttest-only design with nonequivalent groups

Question 61

Grace randomly assigns college-bound African-American students to the intervention or to a control condition. Grace gives a mathematics achievement test in September to both groups. Through the academic year the treatment condition receives the intervention. She uses end of year achievement tests to see whether there are differences between conditions. Grace found that students studied together for the end of year achievement tests. She believes that some of the students in the treatment condition shared intervention information with the control group. Which of the following threats does Grace now have to be most concerned about?

A) Maturation by treatment
B) Treatment diffusion
C) Selection by treatment
D) Statistical regression

Question 62

Grace conducts research on race and achievement in mathematics. She designs a study to investigate whether a new intervention she designed increases African-American students' math achievement scores. She randomly assigns college-bound African-American students to the intervention or to a control condition. Grace gives a mathematics achievement test in September to both groups. Through the academic year the treatment condition receives the intervention. She uses end of year achievement tests to see whether there are differences between conditions. Which of the following best represents the design of Grace's study?

A) One-group posttest-only
B) One-group pretest-posttest
C) Pretest-posttest control group
D) Two by two factorial

Question 63

Audrey examines students' learning of science principles from interactive games. She randomly assigns the fifth grade students in her study to a control condition with no play time, a timed play condition, or an unlimited time play condition. She also groups her students into either high or low ability groups. She measures her students with a knowledge pretest, a post-test, and a delayed post-test. Which of the following is an independent variable in Audrey's study?

A) Ability
B) Fifth grade
C) Post-test exam
D) Science principles

Question 64

Audrey examines students' learning of science principles from interactive games. She randomly assigns the fifth grade students in her study to a control condition with no play time, a timed play condition, or an unlimited time play condition. She also groups her students into either high or low ability groups. She measures her students with a knowledge pretest, a post-test, and a delayed post-test. Which of the following best represents Audrey's study design?

A) Pretest-posttest control group design
B) A 3 x 3 factorial design
C) A 3 x 2 factorial design
D) A 3 x 2 x 3 factorial design

Question 65

Laura has developed a new treatment to increase spatial ability. She randomly assigns students to either her intervention or a control condition. In the treatment condition students engage in working puzzles for twenty minutes a day for 10 days. Unfortunately, students do not like her intervention and refuse to do puzzles for all of the days. When she examines post-test means on the spatial ability measure, there are no differences between her treatment and the control condition. Which of the following demonstrates a threat to validity demonstrated in this study?

A) Maturation
B) Testing
C) Instrumentation
D) Mortality

Question 66

Grace randomly assigns college-bound African-American students to an intervention to improve mathematics achievement or to a control condition. A participants in a control group told Grace, the researcher, that his group found out that 'the other kids were getting a program to help their math scores' and therefore they decided to get together in a study group once a week to help each other. Grace was very concerned. What threat does this student action represent?

A) John Henry effect
B) Multiple treatment interference
C) Experimenter effects
D) Treatment diffusion

Question 67

Khalad is conducting a study to determine if motivational posters inspire middle school students to be more pro-social and to complete more homework. He goes to a large school district to conduct his study. In one of the district's middle schools he puts the posters up and in the second school he does not. After the posters are up for two months he compares the GPA and number of disciplinary referrals between the two schools. Which of the following designs is most representative of Khalad's study?

A) 2 x 2 factorial design
B) Posttest-only design with nonequivalent groups
C) Posttest only control group design
D) Solomon four group design