Deck 15: Drawing Conclusions: the Search for the Elusive Bottom Line

Question 1

Evaluating the Experiment from the Inside: Internal Validity While several mistakes can threaten statistical conclusion validity, the two most common ones are

A)computing too many statistical tests and achieving insufficient statistical power.
B)confounding by extraneous variables and using too many subjects.
C)selecting too extreme a significance level and using the wrong test for experimental data.
D)using too many subjects and achieving insufficient statistical power.

Answer 1

computing too many statistical tests and achieving insufficient statistical power.

Question 2

Evaluating the Experiment from the Inside: Internal Validity ____ verifies how successfully the experimenter created values of the independent variable.

A)A manipulation check
B)Aggregation across subjects
C)Aggregation across trials
D)Pilot testing

Answer 2

A manipulation check

Question 3

Taking a Broader Perspective: The Problem of External Validity An experiment must be ____ to be externally valid.

A)internally valid
B)practically significant
C)replicable
D)internally valid and replicable

Answer 3

internally valid and replicable

Question 4

Evaluating the Experiment from the Inside: Internal Validity Statistical tests permit us to

A)directly test the experimental hypothesis.
B)make probability statements.
C)prove the experimental hypothesis.
D)rule out the possibility of Type 1 error.

Question 5

Taking a Broader Perspective: The Problem of External Validity When researchers replicate statistically significant research findings, this increases our confidence that these findings were not produced by

A)demand characteristics.
B)experimenter bias.
C)Type 1 error.
D)Type 2 error.

Question 6

Taking a Broader Perspective: The Problem of External Validity Amanda replicated a main effect of type of packaging on product recognition.This increases our confidence that her initial findings were not due to

A)demand characteristics.
B)the Rosenthal effect.
C)Type 1 error.
D)Type 2 error.

Question 7

Evaluating the Experiment from the Inside: Internal Validity ____ evaluates whether a sad film really made the subjects feel sad.

A)A manipulation check
B)Aggregation across subjects
C)Aggregation across trials
D)Pilot testing

Question 8

Evaluating the Experiment from the Inside: Internal Validity An experiment's internal validity depends on whether

A)probability sampling was used.
B)researchers have correctly interpreted statistical tests.
C)the experiment is free of confounding.
D)the research findings can be extended to other samples and situations.

Question 9

Taking a Broader Perspective: The Problem of External Validity ____ means that we can accurately extend experimental results to other situations.

A)External validity
B)Internal validity
C)Practical significance
D)Research significance

Question 10

Evaluating the Experiment from the Inside: Internal Validity Jack just completed a study of the effects of caffeine and rave music on recall.He is excited about finding a significant interaction and says, "This is so cool! I've proved my research hypothesis.Jolt cola with rave music is the way to study." Jack's experiment cannot prove his research hypothesis because his results could be due to

A)confounding.
B)Type 1 error.
C)Type 2 error.
D)confounding and/or Type 1 error.

Question 11

Evaluating the Experiment from the Inside: Internal Validity Which variable confounded the Ritchey (1982) study of the effects of age (third-grade, sixth-grade, and adult) and stimulus (words, outline drawings, and detailed drawings) on recall?

A)attention
B)detail
C)distinctiveness
D)visual memory

Question 12

Evaluating the Experiment from the Inside: Internal Validity Orne's (1969) "pact of ignorance" shows how validity issues can be hard to resolve because of

A)confounding.
B)demand characteristics.
C)experimenter bias.
D)reactive measurements.

Question 13

Evaluating the Experiment from the Inside: Internal Validity When a researcher computes too many a priori comparisons, this threatens

A)external validity.
B)internal validity.
C)statistical conclusion validity.
D)Type 2 error.

Question 14

Evaluating the Experiment from the Inside: Internal Validity If your experimental results were barely significant even though you tested 400 subjects, you should be concerned about

A)committing a Type 2 error.
B)confounding by extraneous variables.
C)insufficient statistical power.
D)statistical conclusion validity.

Question 15

Evaluating the Experiment from the Inside: Internal Validity Andrew designed an experiment that controlled extraneous variables so that the effects on the dependent variable could only be attributed to the independent variable.Based on this description, his experiment achieved satisfactory

A)construct validity.
B)ecological validity.
C)external validity.
D)internal validity.

Question 16

Evaluating the Experiment from the Inside: Internal Validity An experiment 's validity is threatened by ____ when subjects do not reveal all they know about the purpose of an experiment and researchers accept subject reports at face value.

A)a pact of ignorance
B)confounding
C)experimenter bias
D)history threat

Question 17

Evaluating the Experiment from the Inside: Internal Validity Experiments by Ritchey and Armstrong (1982) showed that subjects recalled ____ when the degree of ____ was controlled.

A)detailed drawings better than less elaborate drawings; attention
B)detailed drawings better than less elaborate drawings; distinctiveness
C)less elaborate drawings better than detailed drawings; attention
D)less elaborate drawings better than detailed drawings; distinctiveness

Question 18

Evaluating the Experiment from the Inside: Internal Validity Many researchers attempt to discover whether subjects understood and followed experimental instructions by

A)asking a panel of judges to rate instructions on clarity.
B)informally interviewing subjects at the end of an experiment.
C)performing a replication study.
D)statistically analyzing discarded data.

Question 19

Evaluating the Experiment from the Inside: Internal Validity Dr.Moss privately worries that his subjects may have guessed his research hypothesis, but doesn't press his subjects for information.Several of his subjects believe that they know the experiment's purpose, but do not reveal this to Dr.Moss.What problem does this situation describe?

A)a pact of ignorance
B)confounding
C)experimenter bias
D)history threat

Question 20

Evaluating the Experiment from the Inside: Internal Validity A well-designed experiment cannot prove that your research hypothesis is correct because there is always a chance of

A)insufficient statistical power.
B)Type 1 error.
C)Type 2 error.
D)all of these

Question 21

Taking a Broader Perspective: The Problem of External Validity Harris and Rosenthal (1985) combined the results of several studies of the Rosenthal effect and found that it was strong and consistent in experiments testing the effects of teacher expectancies.Which technique did they use?

A)meta-analysis
B)multivariate analysis
C)power analysis
D)regression analysis

Question 22

Taking a Broader Perspective: The Problem of External Validity Researchers pool data and base their conclusions on group averages in

A)aggregation over measures.
B)aggregation over stimuli.
C)aggregation over subjects.
D)aggregation over trials.

Question 23

Taking a Broader Perspective: The Problem of External Validity We can increase the external validity of our data and broaden the scope of experimental findings by grouping together and averaging data.Epstein (1980) called this process

A)aggregation.
B)field experimentation.
C)multivariate analysis.
D)unobtrusive measurement.

Question 24

Taking a Broader Perspective: The Problem of External Validity Moira studies helpfulness during holidays and other times of the year to control the effects of seasonal factors.This procedure represents

A)aggregation over measures.
B)aggregation over occasions.
C)aggregation over stimuli.
D)aggregation over subjects.

Question 25

Taking a Broader Perspective: The Problem of External Validity Researchers use induction in all these examples except

A)constructing theories from a series of experiments.
B)developing general principles from specific findings.
C)formulating research hypotheses.
D)predicting specific findings from a research hypothesis.

Question 26

Taking a Broader Perspective: The Problem of External Validity Which reasoning process do researchers require to extend experimental findings to broader principles and implications?

A)aggregation
B)deduction
C)induction
D)intuition

Question 27

Taking a Broader Perspective: The Problem of External Validity We must be cautious when drawing conclusions about concepts like "learning" from experimental results because

A)different operational definitions may lead to diverse outcomes.
B)experimenters can only study variables they can directly observe.
C)experimenters cannot operationally define concepts like learning.
D)statistically significant results may be due to Type 2 error.

Question 28

Taking a Broader Perspective: The Problem of External Validity All of these factors affect the degree to which we can generalize from the results of an experiment except

A)how we measured the dependent variable.
B)how we selected subjects.
C)subject demographics.
D)the significance level we chose to evaluate our findings.

Question 29

Taking a Broader Perspective: The Problem of External Validity Jeffrey administers a depression inventory and social activity journal to offset the errors that could result from using a single incomplete assessment instrument.This strategy is called

A)aggregation over measures.
B)aggregation over situations.
C)aggregation over stimuli.
D)aggregation over trials.

Question 30

Taking a Broader Perspective: The Problem of External Validity We might increase the external validity of an experiment on color matching that only uses 20 out of 7500 different colors by using

A)aggregation over measures.
B)aggregation over situations.
C)aggregation over stimuli.
D)aggregation over trials.

Question 31

Taking a Broader Perspective: The Problem of External Validity A researcher studying the use of condoms to prevent sexually transmitted disease (STD) combines data from heterosexual and homosexual couples to increase the study's external validity.This pooling of data illustrates

A)aggregation over measures.
B)aggregation over situations.
C)aggregation over subjects.
D)aggregation over trials.

Question 32

Taking a Broader Perspective: The Problem of External Validity A researcher studying public displays of affection by dating couples observes this behavior in shopping malls, airports, and supermarkets.This illustrates the technique of

A)aggregation over measures.
B)aggregation over situations.
C)aggregation over subjects.
D)aggregation over trials.

Question 33

Taking a Broader Perspective: The Problem of External Validity External validity means the extent to which experimental findings can be extended to different

A)procedures.
B)situations.
C)subjects.
D)all of these

Question 34

Taking a Broader Perspective: The Problem of External Validity Which of these is a concept as opposed to a specific operational definition?

A)learning
B)percentage of correct responses
C)response latency
D)total number of errors

Question 35

Taking a Broader Perspective: The Problem of External Validity Researchers can determine whether experimental findings extend to different groups of subjects by using ____ across several studies.

A)aggregation over situations
B)aggregation over subjects
C)block randomization
D)within-subject designs

Question 36

Taking a Broader Perspective: The Problem of External Validity Karen performed a large N experiment and based her conclusions on group averages.Which aggregation technique did she use?

A)aggregation over measures
B)aggregation over stimuli
C)aggregation over subjects
D)aggregation over trials

Question 37

Taking a Broader Perspective: The Problem of External Validity ____ describes the consistency of an effect across past studies and estimates average effect sizes.

A)Meta-analysis
B)Multivariate analysis
C)Power analysis
D)Regression analysis

Question 38

Taking a Broader Perspective: The Problem of External Validity Manufacturers often test new products like shampoo on laboratory animals and then generalize their results to humans because

A)human testing may be unethical due to the risk of injury.
B)it is easier to replicate animal studies than human studies.
C)it is often difficult to obtain human volunteers to test personal care products.
D)laboratory animals and humans often show identical allergic reactions.

Question 39

Taking a Broader Perspective: The Problem of External Validity When we evaluate research results for their consistency with prior studies, the extent to which they extend our knowledge, or their implications for broader theoretical issues, we are addressing their

A)external validity.
B)internal validity.
C)research significance.
D)statistical conclusion validity.

Question 40

Taking a Broader Perspective: The Problem of External Validity Researchers use aggregation over ____ to compensate for errors that may result from using an inadequate assessment instrument.

A)measures
B)situations
C)stimuli
D)trials

Question 41

Taking a Broader Perspective: The Problem of External Validity Which of the following effects can a MANOVA evaluate?

A)differences in trends among various dependent variables
B)main effects
C)interactions
D)all of these

Question 42

Taking a Broader Perspective: The Problem of External Validity Which is a major limitation of field experiments compared with studies conducted in the laboratory?

A)It is harder to employ unobtrusive measures.
B)They have less external validity.
C)We cannot manipulate antecedent conditions.
D)We often have little or no control over who participates in the experiment.

Question 43

Taking a Broader Perspective: The Problem of External Validity Whenever subjects in an experiment know they are being observed, they may behave differently than others who are not observed.This problem is called

A)attribution.
B)experimenter bias.
C)reactivity.
D)social facilitation.

Question 44

Taking a Broader Perspective: The Problem of External Validity Wally is a subject in an experiment.When he enters the lab, he behaves more politely than he generally would.This represents the phenomenon of

A)attribution.
B)experimenter bias.
C)reactivity.
D)social facilitation.

Question 45

Taking a Broader Perspective: The Problem of External Validity Jon measured beach litter at the end of the day after visitors departed.Which approach did he use?

A)field experiment
B)quasi-experiment
C)unobtrusive observation
D)unobtrusive measures

Question 46

Taking a Broader Perspective: The Problem of External Validity Researchers may slightly alter the words of the script across different sessions.Aggregation over ____ helps to demonstrate whether data obtained in specific testing sessions are replicable despite changes in language.

A)measures
B)occasions
C)stimuli
D)trials

Question 47

Taking a Broader Perspective: The Problem of External Validity Which technique manipulates antecedent conditions and observes outcomes in a natural setting?

A)correlational study
B)ex post facto
C)field experiment
D)quasi-experiment

Question 48

Taking a Broader Perspective: The Problem of External Validity Multivariate experiments have the potential to achieve ____ than univariate studies.

A)greater internal validity
B)greater external validity
C)higher risk of Type 2 error
D)lower risk of Type 1 error

Question 49

Taking a Broader Perspective: The Problem of External Validity When subjects react to being observed, they may behave differently than those who are not observed.This creates a problem involving

A)confounding.
B)experimenter bias.
C)external validity.
D)internal validity.

Question 50

Taking a Broader Perspective: The Problem of External Validity Judy varied the number of confederates waiting for a bus and their positions at the bus stop to study line-joining behavior.Which technique was this?

A)correlational study
B)ex post facto
C)field experiment
D)quasi-experiment

Question 51

Taking a Broader Perspective: The Problem of External Validity All of these are examples of unobtrusive measures except

A)comparing the condition of trails leading to natural park attractions to measure popularity.
B)counting the number of cans along a highway to measure littering.
C)randomly interviewing individuals at shopping mall to determine community attitudes towards pornography.
D)recording eating behavior of obese and thin diners without their knowledge.

Question 52

Taking a Broader Perspective: The Problem of External Validity The advantage of unobtrusively observing eating behavior in a restaurant (Marston, London, Cooper, & Cohen, 1977) instead of a laboratory is that the unobtrusive approach seems to have

A)fewer risks of confounding.
B)greater external validity.
C)greater internal validity.
D)fewer risks of confounding and greater external validity.

Question 53

Taking a Broader Perspective: The Problem of External Validity The primary advantage of multivariate designs is that they

A)allow us to study combinations of dependent variables that better represent reality.
B)allow us to narrow our focus to the single most important behavioral indicator.
C)eliminate confounding by extraneous variables.
D)increase the internal validity of an experiment.

Question 54

Taking a Broader Perspective: The Problem of External Validity Researchers can use ____ to effectively control reactivity in an experiment.

A)aggregation
B)control groups
C)cover stories
D)unobtrusive measures

Question 55

Taking a Broader Perspective: The Problem of External Validity A multivariate analysis of variance is used to evaluate experiments with

A)both between-subjects and within-subjects variable.
B)more than one independent variable.
C)multiple dependent variables.
D)two or more within-subjects variables.

Question 56

Taking a Broader Perspective: The Problem of External Validity Catelyn manipulates the placement of recycling containers in a campus building to determine the effect of container location on recycling behavior.Which procedure is this?

A)correlational study
B)ex post facto
C)field experiment
D)quasi-experiment

Question 57

Taking a Broader Perspective: The Problem of External Validity The advantage of unobtrusive measures is that they

A)have greater external validity.
B)reduce confounding.
C)yield more realistic data.
D)have greater external validity and yield more realistic data.

Question 58

Taking a Broader Perspective: The Problem of External Validity When independent variables produce stronger effects on some dependent variables than others, this represents a(n)

A)difference in trends.
B)higher-order interaction.
C)main effect.
D)interaction.

Question 59

Taking a Broader Perspective: The Problem of External Validity A design is multivariate when it studies

A)both between-subjects and within-subjects variables.
B)more than one independent variable.
C)multiple dependent variables in combination.
D)two or more within-subjects variables.

Question 60

Taking a Broader Perspective: The Problem of External Validity Which of these procedures is not multivariate?

A)factor analysis
B)multiple correlation
C)multivariate analysis of variance
D)two-way analysis of variance

Question 61

Handling a Nonsignificant Outcome All of the following are possible explanations for experiment results that do not support the researcher's predictions except

A)committing a Type 2 error.
B)inadequate manipulation of the independent variable.
C)masking of the treatment effects by extraneous variables.
D)using dependent variables with a low floor and high ceiling.

Question 62

Handling a Nonsignificant Outcome Bama conducted an experiment on rats to study the effects of prior experience on hunger motivation.He measured "hunger" by measuring how many food pellets out of 20 a rat consumed in two minutes.However, all the rats ate almost all 20 pellets in every condition.Bama's study was threatened by

A)a ceiling effect.
B)a floor effect.
C)confounding.
D)Type 1 error.

Question 63

Handling a Nonsignificant Outcome If you have designed a sound procedure for studying a hypothesis, have carefully executed the experiment, and still obtain statistically nonsignificant results, you should

A)attempt to replicate the nonsignificant findings with a different sample.
B)consider the possibility that the research hypothesis was faulty and rethink the problem.
C)reanalyze your data with a nonparametric statistical procedure in place of a parametric procedure.
D)redesign the study and test the hypothesis again.

Question 64

Handling a Nonsignificant Outcome Low statistical power reduces statistical conclusion validity and can produce

A)confounding.
B)faulty generalizations.
C)Type 1 errors.
D)Type 2 errors.

Question 65

Handling a Nonsignificant Outcome If Jennifer's results come close to reaching statistical significance, she should

A)be cautious in drawing any conclusions.
B)discuss the findings as though they would reach significance if she had run more subjects.
C)perform an a priori test.
D)raise the alpha level.

Question 66

Taking a Broader Perspective: The Problem of External Validity Which of the following would be the best way to check the external validity of results obtained in a laboratory experiment?

A)conduct a naturalistic observation study
B)conduct a survey
C)measure effect size
D)replicate the experiment