Deck 13: Replicability,Generalization,and the Real World

Question 1

Which of the following is NOT a major type of replication?

A) Statistical replication
B) Direct replication
C) Replication-plus-extension
D) Conceptual replication
E) All of the above are major types of replications

Answer 1

Statistical replication

Question 2

The traditional value that determines whether a result is statistically significant is ___________.

A) .50
B) 1.00
C) .25
D) .10
E) None of the above

Answer 2

None of the above

Question 3

When using inferential statistics,researchers calculate a/an __________ to determine statistical significance.

A) Effect size
B) Coefficient of importance
C) Curve value
D) Probability estimate
E) Replication value

Answer 3

Probability estimate

Question 4

RESEARCH STUDY 13.1
Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola, and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand (p < 10.05).
Refer to Research Study 13.1 above to answer the following question. Mark decides to conduct a conceptual replication.Which of the following could be in the study to make it this type of replication?

A) Measuring a different group of participants
B) Having participants drink a caffeinated energy drink rather than a caffeinated cola
C) Having participants complete a 20-item math test
D) Having participants complete a 10-item math test
E) All of the above

Question 5

RESEARCH STUDY 13.1
Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola, and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand (p < 10.05).
Refer to Research Study 13.1 above to answer the following question. The notation "p < 0.05" indicates which of the following?

A) Mark conducted the study correctly
B) Mark's finding is statistically significant
C) Mark can conclude that it is unlikely he got his result by chance
D) Both b and c
E) All of the above

Question 6

Which of the following is another term for ecological validity?

A) Mundane realism
B) Ordinary application
C) Everyday extrapolation
D) Experimental realism
E) Cultural application

Question 7

Which of the following types of replications do NOT support external validity?

A) Conceptual replications
B) Direct replications
C) Replications-plus-extensions
D) Both a and c support external validity
E) All of the above support external validity

Question 8

When generalizing the results of a sample to a population,which of the following is the most important question to ask?

A) Where were the participants tested?
B) How many participants were in the sample?
C) How many times were the participants tested?
D) How were the participants sampled?
E) Do the participants come from the "real world"?

Question 9

RESEARCH STUDY 13.2
Cindy decides to conduct a meta-analysis examining the relationship between daily stress and cardiovascular health for her senior psychology research project.
Refer to Research Study 13.2 above to answer the following question. Cindy's advisor recommends that she contact several researchers in the field for articles that were not published and/or that found null effects.Doing this will address which of the following?

A) The file drawer problem
B) The need for overestimation
C) Effect size miscalculation
D) Ecological validity
E) None of the above

Question 10

If a statistical finding is significant,this means which of the following?

A) The finding is important
B) The statistical test was done properly
C) It was unlikely that the result happened by chance
D) Both a and b
E) All of the above

Question 11

Which of the following is a reason that psychologists especially value meta-analyses?

A) Meta-analyses can examine conceptual and direct replications.
B) Meta-analyses allow researchers to examine the strength of a relationship.
C) In Meta-analyses, moderators of relationships can be examined.
D) Both a and b are reasons psychologists especially value meta-analyses.
E) All of the above are reasons psychologists especially value meta-analyses.

Question 12

RESEARCH STUDY 13.2
Cindy decides to conduct a meta-analysis examining the relationship between daily stress and cardiovascular health for her senior psychology research project.
Refer to Research Study 13.2 above to answer the following question. Which of the following is NOT a step Cindy will do in conducting her meta-analysis?

A) Collect all the studies that have been conducted on daily stress and cardiovascular health
B) Measure daily stress and cardiovascular health in several new samples of participants
C) Calculate an effect size
D) Contact other researchers for unpublished studies
E) All of the above steps will need to be done by Cindy

Question 13

Which of the following is NOT true of theory-testing mode?

A) It is part of the theory-data cycle.
B) The priority is external validity.
C) The focus can be on testing association claims.
D) Most psychological studies are of the theory-testing type.
E) All of the above are true of theory-testing mode.

Question 14

Responsible journalists do which of the following as it pertains to discussing replicability?

A) They report only on studies that have been directly replicated.
B) They report findings only from meta-analyses.
C) They provide readers with a sense of the entire literature as well as recent studies.
D) They do both a and b.
E) They do all of the above.

Question 15

RESEARCH STUDY 13.1
Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola, and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand (p < 10.05).
Refer to Research Study 13.1 above to answer the following question. Mark decides to conduct a variation of the original study in which he measures participants' ability to solve verbal analogies as the cognitive task instead of measuring their ability to solve math problems.He finds that people who drink caffeine beforehand solve more verbal analogies than those who drink water beforehand,suggesting caffeine consumption causes improved cognitive performance.This study is known as a:

A) Statistical replication
B) Replication-plus-extension
C) Conceptual replication
D) Secondary replication
E) None of the above

Question 16

RESEARCH STUDY 13.1
Mark conducts a study for his research methods class to determine if consuming caffeine causes people to perform better on cognitive tasks. In his study, he gives half of his participants a glass of water and half of his participants a glass of cola, and then has them attempt to solve 15 math problems. He finds that people who drink caffeine beforehand solve more math problems than those who drink water beforehand (p < 10.05).
Refer to Research Study 13.1 above to answer the following question. Mark decides to conduct the exact same study again to ensure that he can find the same difference between the groups a second time.This study is known as a:

A) Statistical replication
B) Replication-plus-extension
C) Conceptual replication
D) Secondary replication
E) None of the above

Question 17

Which of the following is true of a scientific literature?

A) It comprises studies conducted by a single researcher.
B) It comprises studies that have tested the exact same variable.
C) It comprises studies conducted all in the same year.
D) None of the above are true of a scientific literature.
E) All of the above are true of a scientific literature.

Question 18

Replication-plus-extensions can add which of the following?

A) A participant variable
B) A situational variable
C) A conditional variable
D) Both a and b
E) All of the above

Question 19

The text states that the first step in establishing a study's importance is to establish its _______.

A) Replicability
B) External validity
C) Internal validity
D) Funding
E) Acceptance by scientists

Question 20

Psychologists use which two of the following strategies to determine a study's replicability?

A) Internal validity and external validity
B) Inferential statistics and repeating the study
C) Generalizing and theory testing
D) Study duplication and study reframing
E) None of the above

Question 21

RESEARCH STUDY 13.4
Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
Refer to Research Study 13.4 above to answer the following question.
What are the two aspects of external validity? Pose a question for each aspect of external validity as it relates to Dr.Tropez's study.

Question 22

RESEARCH STUDY 13.4
Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
Refer to Research Study 13.4 above to answer the following question.
Provide an example of a direct replication,conceptual replication,and a replication with extension for Dr.Tropez's original study.

Question 23

_________ is a subdiscipline of psychology that works primarily in the generalization mode.

A) Clinical psychology
B) Experimental psychology
C) Evolutionary psychology
D) Social psychology
E) Cultural psychology

Question 24

RESEARCH STUDY 13.3
Dr. Grayson, a health psychologist, conducts a study examining whether when exposed to pictures of food, people eat more than when they are not exposed to pictures of food. She goes to a fast food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30 a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast food restaurant that does not have pictures on their menu board (about two blocks away from the first restaurant). She finds that the restaurant that has pictures of food sells more food than the restaurant that does not have pictures.
Refer to Research Study 13.3 above to answer the following question. Dr.Grayson decides to conduct her study again,but in the new study,she studies people at a sit-down restaurant and,instead of looking at a menu board with or without pictures,she manipulates whether the menu they use has pictures or not.This is known as which of the following?

A) Statistical replication
B) Conceptual replication
C) Replication-plus-extension
D) Direct replication
E) None of the above

Question 25

Which of the following is true of experimental realism?

A) It exists only in field settings.
B) It is a synonymous term for mundane realism.
C) It interferes with internal validity.
D) It is only important in generalization mode.
E) None of the above are true.

Question 26

Why do studies conducted in theory-testing mode de-emphasize ecological validity?

A) Because ecological validity is not very important
B) Because internal validity often requires artificial control
C) Because testing a theory never requires realism
D) All of the above are true
E) None of the above are true

Question 27

Which of the following studies is most likely to be conducted in generalization mode?

A) Examining whether dog ownership decreases stress levels
B) Exploring the relationship between weight and number of fast food restaurants in one's neighborhood
C) Determining how frequently people report fighting with their bosses
D) Investigating whether there is a causal link between receiving cognitive therapy and increased coping skills
E) All of the above are likely to be conducted in generalization mode

Question 28

In a previous correlational study,Dr.Lafayette has found that owning dogs is associated with lower levels of daily stress in a sample of returning war veterans.Which of the following would constitute a shift from theory-testing mode to generalization mode?

A) Conducting a study on the same sample of war veterans, this time using a different measure of daily stress
B) Conducting a study examining the stress reduction in dogs
C) Conducting an experimental study in which he assigns people to own a dog or not
D) Conducting a study using a sample of elderly participants from a local retirement home
E) All of the above would be studies done in generalization mode

Question 29

RESEARCH STUDY 13.3
Dr. Grayson, a health psychologist, conducts a study examining whether when exposed to pictures of food, people eat more than when they are not exposed to pictures of food. She goes to a fast food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30 a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast food restaurant that does not have pictures on their menu board (about two blocks away from the first restaurant). She finds that the restaurant that has pictures of food sells more food than the restaurant that does not have pictures.
Refer to Research Study 13.3 above to answer the following question. Because Dr.Grayson conducted her study in a field setting,which of the following is an advantage of her study?

A) It automatically generalizes to other situations (e.g., fancy restaurants, cafeterias).
B) It automatically generalizes to other people.
C) It removes all threats to internal validity.
D) It has a high degree of ecological validity.
E) All of the above are advantages.

Question 30

According to the textbook,which of the following is NOT a component of WEIRD people?

A) Western
B) Educated
C) Interdependent
D) Rich
E) Democratic

Question 31

Explain why replication is important to science.Name two ways scientists can determine whether a study is replicable.

Question 32

RESEARCH STUDY 13.3
Dr. Grayson, a health psychologist, conducts a study examining whether when exposed to pictures of food, people eat more than when they are not exposed to pictures of food. She goes to a fast food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30 a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast food restaurant that does not have pictures on their menu board (about two blocks away from the first restaurant). She finds that the restaurant that has pictures of food sells more food than the restaurant that does not have pictures.
Refer to Research Study 13.3 above to answer the following question. Dr.Grayson wants to conduct her study again,but this time she wants to conduct it in her laboratory.She wants people to feel and act the same way in her laboratory that they would when facing ordering choices in an actual fast food restaurant.To this end,she must try to enhance which of the following?

A) Experimental realism
B) Generalizability
C) Construct validity
D) Cultural relativity
E) None of the above

Question 33

Which of the following is NOT true of cultural psychology?

A) It frequently challenges those who exist only in generalization mode.
B) It seeks to disprove theories.
C) It has never found support for a theory across different cultural contexts.
D) It starts with the assumption that theories apply to all humans, regardless of culture.
E) None of the above are true of cultural psychology.

Question 34

RESEARCH STUDY 13.3
Dr. Grayson, a health psychologist, conducts a study examining whether when exposed to pictures of food, people eat more than when they are not exposed to pictures of food. She goes to a fast food restaurant in downtown Chicago that has pictures of menu items on its menu board and measures the amount of food ordered during the lunch rush (11:30 a.m.-1:30 p.m.) on Tuesday. On Thursday, she does the same thing but at a fast food restaurant that does not have pictures on their menu board (about two blocks away from the first restaurant). She finds that the restaurant that has pictures of food sells more food than the restaurant that does not have pictures.
Refer to Research Study 13.3 above to answer the following question. Dr.Grayson would most be able to generalize her findings to which of the following populations?

A) People from Chicago
B) People who live in downtown Chicago
C) People who eat lunch at fast food restaurants
D) People who eat out at restaurants
E) All of the above

Question 35

A study conducted in the "real world" is often said to be conducted in:

A) Mundane surroundings
B) A field setting
C) An ecological situation
D) Authentic localities
E) None of the above

Question 36

Which of the following claims is always researched in generalization mode?

A) Association claims
B) Causal claims
C) Frequency claims
D) Both a and b
E) Both a and c

Question 37

Explain why finding a statistically significant result tells a reader about the study's replicability.

Question 38

Which of the following is an advantage of studies that are conducted in real-world settings?

A) They have a high degree of internal validity.
B) They are more important than studies conducted in laboratories.
C) They are high in ecological validity.
D) They automatically generalize to other situations.
E) All of the above are advantages.

Question 39

RESEARCH STUDY 13.4
Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
Refer to Research Study 13.4 above to answer the following question.
A colleague of Dr.Tropez questions how generalizable his study is to other participants by highlighting that he only studied 113 people.Dr.Tropez responds that generalizability comes not from the "how many" of the sample but the "how" of the sample.What does Dr.Tropez mean and how generalizable is his study?

Question 40

According to the textbook,what is the problem with WEIRD samples?

A) They are not very representative of the world's population.
B) They are not good for theory-testing mode.
C) They are not used frequently enough.
D) Both a and b
E) Both b and c

Question 41

Explain why external validity is the priority in generalization mode but not in theory-testing mode.

Question 42

What does it mean that a sample is WEIRD? (In other words,what does WEIRD stand for?)

Question 43

RESEARCH STUDY 13.4
Dr. Tropez conducts a study examining the relationship between exposure to religion in popular media and religiosity (defined as "the belief in and worship of a superhuman controlling power, especially a personal God or gods"). In his study, he randomly samples a group of 113 participants from his local community college. He generates a list of movies with religious themes (Year One, Dogma, Passion of the Christ) and asks participants to indicate how many they have seen. He then measures religiosity by asking participants how many times they have attended a house of worship (e.g., church, temple, synagogue, mosque) in the past year. He finds that having seen a higher number of religious-themed films is positively correlated with religious attendance.
Refer to Research Study 13.4 above to answer the following question.
What is the difference between theory-testing mode and generalization mode? In which mode does Dr.Tropez seem to be operating?

Question 44

Explain why using WEIRD samples tends not to be a problem for researchers operating in the theory-testing mode and why cultural psychologists question this practice.

Question 45

Explain what a meta-analysis is and how it is useful in understanding the importance of a scientific relationship.

Question 46

Name the subdiscipline of psychology that works exclusively in generalization mode and explain why it works exclusively in this mode.

Question 47

Explain how a study can be important but lack a diverse,random sample and a real-world setting.What type of study is likely to have neither of these?

Question 48

Explain what the file drawer problem is and how and why it needs to be addressed when conducting a meta-analysis.

Question 49

What is experimental realism? What does this mean for studies conducted in artificial settings (e.g.,a laboratory)and studies conducted in field settings?

Question 50

Which types of claims are always made in generalization mode? Why is this the case?