Deck 14: Replication, Generalization, and the Real World

Question 1

Which of the following types of replications investigate the same research question but use different procedures?

A)Statistical replication
B)Direct replication
C)Replication-plus-extension
D)Conceptual replication

Answer 1

Conceptual replication

Question 2

Which of the following is a reason that psychologists especially value meta-analyses?

A)Meta-analyses eliminate the need for replications.
B)Meta-analyses allow researchers to examine the strength of a relationship.
C)Meta-analyses are immune to threats to internal validity.
D)Meta-analyses always take less time to conduct.

Answer 2

Meta-analyses allow researchers to examine the strength of a relationship.

Question 3

RESEARCH STUDY 14.2: Cindy decides to conduct a meta-analysis examining the relationship between daily stress and cardiovascular health for her senior psychology research project.
Which of the following is NOT a step Cindy will do in conducting her meta-analysis?

A)Collect all the studies that have been conducted on daily stress and cardiovascular health
B)Measure daily stress and cardiovascular health in several new samples of participants
C)Calculate an effect size
D)Contact other researchers for unpublished studies

Answer 3

Measure daily stress and cardiovascular health in several new samples of participants

Question 4

Which of the following types of study support external validity?

A)A study using non-randomly selected participants
B)A study that adds a new independent variable
C)A study that is a direct replication
D)A failed replication study

Question 5

psychology is a subdiscipline of psychology that works primarily in the generalization mode.

A)Clinical
B)Experimental
C)Social
D)Cultural

Question 6

Responsible journalists do which of the following as it pertains to discussing replicability?

A)They report only on studies that have been directly replicated.
B)They report findings only from meta-analyses.
C)They provide readers with a sense of the entire literature as well as recent studies.
D)They provide readers with statistical values from each study.

Question 7

When generalizing the results of a sample to a population, which of the following is the most important question to ask?

A)Where were the participants tested?
B)How many participants were in the sample?
C)How were the participants sampled?
D)Do the participants come from the "real world"?

Question 8

The text states that the first step in establishing a study's importance is to establish its .

A)replicability
B)external validity
C)internal validity
D)acceptance by scientists

Question 9

Psychologists use which of the following strategies to determine a study's replicability?

A)Internal validity
B)Repeating the study
C)Generalizing
D)Study reframing

Question 10

Which of the following is true of a scientific literature?

A)It comprises studies conducted by a single researcher.
B)It comprises studies that have tested the exact same variable.
C)It comprises studies conducted all in the same year.
D)It comprises studies conducted with different methods.

Question 11

Which of the following is true of theory-testing mode?

A)It is related to the theory-data cycle.
B)The priority is external validity.
C)The focus is primarily on testing frequency claims.
D)Few psychological studies are of the theory-testing type.

Question 12

Which of the following is another term for ecological validity?

A)Mundane realism
B)Everyday realism
C)Experimental realism
D)Cultural realism

Question 13

In a previous correlational study, Dr. Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans. Which of the following would constitute a shift from theory-testing mode to generalization mode?

A)Conducting a study on the same sample of war veterans, this time using a different measure of daily stress
B)Conducting a study examining the stress reduction mechanism
C)Conducting an experimental study in which he assigns people to own a dog or not
D)Conducting a study on dog ownership using a sample of elderly participants from a local retirement home

Question 14

RESEARCH STUDY 14.1: Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand.
Mark decides to conduct the exact same study again to ensure that he can find the same difference between the groups a second time. This study is known as a:

A)direct replication.
B)replication-plus-extension.
C)conceptual replication.
D)secondary replication.

Question 15

Which of the following can direct replication studies change?

A)The dependent variable
B)The study procedures
C)The participants
D)The independent variable

Question 16

RESEARCH STUDY 14.1: Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand.
Mark decides to conduct a variation of the original study in which he measures participants' ability to solve verbal analogies as the cognitive task instead of measuring their ability to solve math problems. He finds that people who drink caffeine beforehand solve more verbal analogies than those who drink water beforehand, suggesting caffeine consumption causes improved cognitive performance. This study is known as a:

A)statistical replication.
B)replication-plus-extension.
C)conceptual replication.
D)secondary replication.

Question 17

RESEARCH STUDY 14.1: Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand.
Mark decides to conduct a conceptual replication. Which of the following could be in the study to make it this type of replication?

A)Measuring a different group of participants
B)Having participants drink a caffeinated energy drink rather than a caffeinated cola
C)Measuring the same participants 1 week later
D)Having participants complete 10 math problems instead of 15

Question 18

RESEARCH STUDY 14.2: Cindy decides to conduct a meta-analysis examining the relationship between daily stress and cardiovascular health for her senior psychology research project.
Cindy's advisor recommends that she contact several researchers in the field for articles that were not published and/or that found null effects. Doing this will address which of the following?

A)The file drawer problem
B)The need for overestimation
C)The lack of reporting significant findings
D)Ecological validity

Question 19

Which of the following studies is most likely to be conducted in generalization mode?

A)Examining whether dog ownership decreases stress levels
B)Exploring the relationship between weight and number of fast-food restaurants in one's neighborhood
C)Determining how frequently people report fighting with their bosses
D)Investigating whether there is a causal link between receiving cognitive therapy and increased coping skills

Question 20

Which of the following claims is always researched in generalization mode?

A)Association claims
B)Causal claims
C)Frequency claims
D)Experimental claims

Question 21

RESEARCH STUDY 14.3: Dr. Grayson, a health psychologist, conducts a study examining whether people eat more when they are exposed to pictures of food than when they are not exposed to pictures of food. She goes to a fast-food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast-food restaurant that does not have pictures on its menu board (about two blocks away from the first restaurant). She finds that the restaurant with pictures of food sells more food than the restaurant that does not have pictures.
Dr) Grayson would most be able to generalize her findings to which of the following populations?

A)People from Chicago
B)People who live in downtown Chicago
C)People who eat lunch at fast-food restaurants
D)People who eat out at restaurants

Question 22

Replicability helps interrogate which of the four big validities?

A)Construct validity
B)External validity
C)Internal validity
D)Statistical validity

Question 23

Another word for replicable is:

A)valid.
B)reproducible.
C)reliable.
D)scientific.

Question 24

Who determines the population to which a study's findings generalize?

A)The participants
B)Journalists
C)The IRB
D)The researcher

Question 25

RESEARCH STUDY 14.3: Dr. Grayson, a health psychologist, conducts a study examining whether people eat more when they are exposed to pictures of food than when they are not exposed to pictures of food. She goes to a fast-food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast-food restaurant that does not have pictures on its menu board (about two blocks away from the first restaurant). She finds that the restaurant with pictures of food sells more food than the restaurant that does not have pictures.
Dr) Grayson wants to conduct her study again, but this time she wants to conduct it in her laboratory. She wants people to feel and act the same way in her laboratory that they would when facing ordering choices in an actual fast-food restaurant. To this end, she must try to enhance which of the following?

A)Experimental realism
B)Generalizability
C)Statistical validity
D)Cultural relativity

Question 26

A cultural psychologist would be most interested in which of the following sets of participants?

A)A sample of community college students
B)A sample of 12-year-old children
C)A sample of homeless veterans
D)A sample of Taiwanese grandparents

Question 27

RESEARCH STUDY 14.3: Dr. Grayson, a health psychologist, conducts a study examining whether people eat more when they are exposed to pictures of food than when they are not exposed to pictures of food. She goes to a fast-food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast-food restaurant that does not have pictures on its menu board (about two blocks away from the first restaurant). She finds that the restaurant with pictures of food sells more food than the restaurant that does not have pictures.
Dr) Grayson decides to conduct her study again, but in the new study, she studies people at a sit-down restaurant, and she manipulates whether the menu they use at the table has pictures or not. This is known as which of the following?

A)Statistical replication
B)Conceptual replication
C)Replication-plus-extension
D)Direct replication

Question 28

Which of the following is true of cultural psychology?

A)Its researchers function only in theory-testing mode.
B)Its main focus is to disprove theories.
C)It typically finds support for a theory across different cultural contexts.
D)It starts with the assumption that theories apply to all humans, regardless of culture.

Question 29

According to the textbook, what is the problem with WEIRD samples?

A)They are not very representative of the world's population.
B)They are not good for theory-testing mode.
C)They are not used frequently enough.
D)They are not very easy to measure.

Question 30

In a conceptual replication, the are the same, but the is/are different from the original study.

A)variables; operationalization
B)independent variables; dependent variables
C)researchers; outcomes
D)methods; participants

Question 31

A study conducted in the "real world" is often said to be conducted in:

A)mundane surroundings.
B)a field setting.
C)an ecological situation.
D)authentic localities.

Question 32

Which of the following is true of experimental realism?

A)It exists only in field settings.
B)It is a synonymous term for mundane realism.
C)It supports external validity.
D)It is only important in generalization mode.

Question 33

Why do studies conducted in theory-testing mode de-emphasize ecological validity?

A)Ecological validity is never important.
B)Internal validity is the highest priority for theory testing.
C)Testing a theory never requires realism.
D)Theories do not need to explain the behavior of everyone.

Question 34

In a conceptual replication, which of the following is allowed to be dissimilar from the original study?

A)The dependent variable in the study
B)The independent variable in the study
C)The procedures of the study
D)The moderators used in the study

Question 35

What does it mean when a study cannot be replicated by an independent researcher?

A)The replication was done incorrectly.
B)The independent researcher's study is superior.
C)The importance of the finding should be interpreted cautiously.
D)The topic should no longer be researched.

Question 36

RESEARCH STUDY 14.3: Dr. Grayson, a health psychologist, conducts a study examining whether people eat more when they are exposed to pictures of food than when they are not exposed to pictures of food. She goes to a fast-food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30 a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast-food restaurant that does not have pictures on its menu board (about two blocks away from the first restaurant). She finds that the restaurant with pictures of food sells more food than the restaurant that does not have pictures.
Because Dr. Grayson conducted her study in a field setting, which of the following is an advantage of her study?

A)It automatically generalizes to other situations (e.g., fancy restaurants, cafeterias).
B)It automatically generalizes to other people.
C)It removes all threats to internal validity.
D)It has a high degree of ecological validity.

Question 37

A researcher in theory-testing mode focuses on validity, while a researcher in generalization mode focuses on validity.

A)internal; external
B)external; internal
C)external; statistical
D)statistical; external

Question 38

According to the textbook, which of the following is a component of WEIRD people?

A)Women
B)Educated
C)Interdependent
D)Republican

Question 39

Which of the following is an advantage of studies that are conducted in real-world settings?

A)They have a high degree of internal validity.
B)They are more important than studies conducted in laboratories.
C)They are high in ecological validity.
D)They automatically generalize to other situations.

Question 40

Which of the following claims is most likely to be made in generalization mode?

A)"A majority of women report considering infidelity at least once during their marriages."
B)"Men are more likely than women to report considering infidelity."
C)"Parental divorce is associated with greater likelihood of infidelity."
D)"Being exposed to attractive individuals increases thoughts of infidelity."

Question 41

Which of the following studies would be least likely to be influenced by cultural differences?

A)A study of the association between depression and cultural beliefs about mental health
B)A study examining rates of divorce across different cultures
C)A study examining the effect of parent education on child socialization
D)A study examining motor responses of newborns

Question 42

Which of the following participant samples would most likely generalize to the population of teenage American girls?

A)75 girls aged 13-18, randomly sampled from multiple sites across the country
B)All 500 of the girls at a single high school
C)200 girls recruited from Girl Scout troops across the country
D)50 after-school female volunteers who were available to participate in the research easily

Question 43

Explain what the file drawer problem is and how and why it needs to be addressed when conducting a meta-analysis.

Question 44

Preregistration occurs at which point in the scientific process?

A)Before data collection
B)Before developing hypotheses
C)After publication
D)After replication

Question 45

Explain what a meta-analysis is and how it is useful in understanding the importance of a scientific relationship.

Question 46

Explain why replication is important to science. Name a method that scientists use to determine whether a study is replicable.

Question 47

Explain why external validity is the priority in generalization mode but not in theory-testing mode.

Question 48

In open science, open data are more concerned with , while open materials are more interested in .

A)construct validity; statistical validity
B)replication; construct validity
C)generalizability; statistical validity
D)statistical validity; replication

Question 49

Some studies may not replicate because the original study used questionable scientific practices. Which of the following is a way open science practices attempt to deal with this problem?

A)Using larger sample sizes
B)Requiring scientists to make their data and materials available for review
C)Conducting direct replications
D)Encouraging preregistration

Question 50

RESEARCH STUDY 14.4: Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
Provide an example of each of the following for Dr. Tropez's original study: a direct replication, a conceptual replication, and a replication with extension.

Question 51

Which of the following MUST be done before meta-analyses can occur?

A)Only direct replications
B)Cultural psychology replications done on the topic
C)Collection of the scientific literature on a topic
D)Publication of the topic in the popular media

Question 52

Which types of claims are always made in generalization mode? Why is this the case?

Question 53

RESEARCH STUDY 14.4: Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
What is the difference between theory-testing mode and generalization mode? In which mode does Dr. Tropez seem to be operating?

Question 54

RESEARCH STUDY 14.4: Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
A colleague of Dr. Tropez questions how generalizable his study is to other participants by highlighting that he only studied 113 people. Dr. Tropez responds that generalizability comes not from the "how many" of the sample but the "how" of the sample. What does Dr. Tropez mean, and how generalizable is his study?

Question 55

A conceptual replication study, which supported a previous United States-based study on the association between self-associations and social behaviors, was just published. This study used a different method to assess social behavior and randomly sampled 50 people from the population in Canada, where the study was conducted. How might a cultural psychologist respond to the new article about this study?

A)With enthusiasm, since replication studies are rare
B)With skepticism, because this theory may not apply to more culturally diverse populations
C)With skepticism, since this study used a small sample size
D)With enthusiasm, because of the inclusion of one culturally diverse sample

Question 56

Dr. Fern recently collected data for a study examining whether an intervention targeting coping strategies can reduce depression. After analyzing the results, she finds no effect of her intervention on depression symptoms. Which of the following would you recommend she do next?

A)Preregister her study so she can publish the null findings.
B)Add more participants because her initial sample size may have been too small.
C)Contact journal editors to see if they will publish null results.
D)Try a different analytic approach.

Question 57

Which of the following is true of p-hacking?

A)It is considered a common and acceptable practice.
B)It is always done intentionally.
C)It is the same as HARKing.
D)It can involve running a different analysis than originally proposed.

Question 58

RESEARCH STUDY 14.4: Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
A colleague criticizes Dr. Tropez for using a WEIRD sample. Explain what this means and why it is a problem.

Question 59

Which of the following is a benefit of a meta-analysis compared to other types of publications?

A)Meta-analyses allow for calculation of effect size across multiple studies.
B)Only meta-analyses allow for examination of moderators.
C)Meta-analysis is the only type of publication that synthesizes an entire scientific literature.
D)Meta-analyses always avoid the file drawer problem.

Question 60

RESEARCH STUDY 14.4: Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
What are the two aspects of external validity? Pose a question for each aspect of external validity as it relates to Dr. Tropez's study.

Question 61

What does it mean that a sample is WEIRD? (In other words, what does WEIRD stand for?) Why can WEIRD samples be problematic for research?

Question 62

What is preregistration? Why is it important in the scientific process?

Question 63

What is experimental realism? What does this mean for studies conducted in artificial settings (e.g., a laboratory) and studies conducted in field settings?

Question 64

Name the subdiscipline of psychology that works exclusively in generalization mode, and explain why it works exclusively in this mode.

Question 65

Explain how a study can be important but lack a diverse, random sample and a real-world setting. What type of study is likely to have neither of these?

Question 66

Explain why using WEIRD samples tends not to be a problem for researchers operating in the theory-testing mode and why cultural psychologists question this practice.