Deck 6: Evaluating Samples When Researchers Generalize

Question 1

All of the following statements about the relationship between a sample and population are true, EXCEPT:

A) A sample is a subset of a population.
B) Researchers study samples but not populations.
C) Researchers often study samples for the sake of efficiency.
D) When study results are obtained on a sample, they are then generalized to the relevant population.

Answer 1

B

Question 2

All of the following conditions help researchers to achieve an unbiased sample, EXCEPT:

A) Using random sampling.
B) Using a reasonably large sample.
C) Using a stratified random sample.
D) Using a convenience sample.

Answer 2

D

Question 3

Which one of the following scenarios illustrates a stratified random sampling procedure?

A) Putting all participants' names into a hat, and blindly drawing names from the hat.
B) Putting all female participants' names into one hat, male participants' names - into another, and blindly drawing names from each hat separately.
C) Seating all participants around square tables, and choosing participants who sit facing North (one side of each table).
D) Making a list of all potential participants, and choosing every third person on the list.

Answer 3

B

Question 4

What is the reason that geography is especially useful for sample stratification purposes?

A) People live in every type of geographic area.
B) People would not be offended if selected for a study based on their geographic location.
C) Many important variables in social sciences are clustered based on a geographic area.
D) Different geographic areas would allow researchers to enjoy the travel aspect of their study.

Question 5

There are several important reasons why researchers are using nonrandom samples in their studies. Which one of the following is NOT a valid reason?

A) Some people randomly selected for participation in a study refuse to participate.
B) Random sampling is often cumbersome, time-consuming, and expensive.
C) Nonrandom sampling produces a sample that is more representative of the population.
D) There are target populations for which there is no available list of all members.

Question 6

Which one of the following statements about random sampling is INCORRECT?

A) Most studies in social sciences are conducted on random samples.
B) Nonrandom samples can be used for generalizing.
C) If some participants randomly selected for a study have refused to participate, the resulting sample is not random.
D) When a professor invites all the students attending her classes to participate in her study, the sample is not random.

Question 7

When some participants refuse to participate in a study, what participation rate is considered to be reasonably high?

A) Above 50%.
B) Above 30%.
C) Higher than in the previous studies on a similar topic.
D) There is no clear rule for deciding which participation rate is reasonably high.

Question 8

All of the following considerations would be relevant for deciding whether the participation rate in a specific study is reasonably high, EXCEPT:

A) Were the study participants asked if they enjoyed their study experience?
B) Were potential participants contacted multiple times before the study?
C) Was the study invasive or uncomfortable?
D) Was the typical study participation session long?

Question 9

When comparing survey response rates for mail and online surveys, which one of the following statements is INCORRECT?

A) Multiple attempts to contact potential survey participants can be made using both methods.
B) It is generally preferable to contact potential study participants by mail first.
C) Any study comparing the mail vs web-based survey response rates would be limited to a specific target population and specific survey topic.
D) There is no clear answer whether the response rate is higher for mailed or online surveys.

Question 10

All of the following examples correctly reflect concerns with selection bias, EXCEPT:

A) Those who refused to participate in the study may be different in some important respects from those who agreed to participate.
B) In a longitudinal study, some participants who started participating, later withdraw from the study.
C) Additional participants have been selected for the study when it was already under way.
D) Researchers have found out that those from more affluent neighborhoods have a higher response rate to the study.

Question 11

What is the biggest threat from self-selection bias, according to the text?

A) Those who have refused to participate would have a negative view of the study.
B) Those who have agreed to participate enjoyed their study experience.
C) The results of the study may not be generalizable to the population.
D) The results of the study may not reach statistical significance.

Question 12

When a sample is nonrandom, there are some important ways to improve the researchers' ability to generalize their study results. Which one of the following sampling strategies does NOT improve the nonrandom sampling process?

A) A sample for a study of alcohol use among high school students was solicited from multiple high schools.
B) A sample for a study of mental health needs of homeless people was solicited from several homeless shelters.
C) A sample for a study of local-area parents was solicited at various school and community events for parents.
D) A sample for a study of young adults was solicited from several classes at a local college where the researcher works.

Question 13

There are several good reasons why it is important for researchers to discuss the limitations of their sampling in a research report. Which one of the following is the LEAST important reason?

A) The Discussion section needs to be filled with enough information.
B) Discussing study limitations helps the reader understand the flaws in the study.
C) Sampling limitations directly affect the generalizability of study results.
D) The readers of the journal will be expecting to see a discussion of study limitations.

Question 14

Researchers are supposed to describe the demographics of the sample in their article. Which one of the following variables is unlikely to be included among the demographics?

A) Age.
B) Gender.
C) Race.
D) Eye color.

Question 15

There are several considerations important to evaluating if the sample size is adequate. Which one of the following considerations is IRRELEVANT?

A) How accessible is the population that the researchers are interested in?
B) Could the study have been published if a smaller sample were used?
C) Is the number of participants so low that it would be hard to make sound generalizations?
D) Is the study supposed to provide important answers for informing policies and practice?

Question 16

Which one of the following statements about sample size is correct?

A) The size of the sample depends on how much error a researcher is willing to tolerate.
B) Studies using large nonrandom samples produce more generalizable results than small studies using random samples.
C) Random samples are generally larger than nonrandom ones.
D) Whether a sample is large enough does not have any bearing on the study results.

Question 17

When several subgroups are compared in a study, what is the rule of thumb about the sample or subsample size?

A) At least 100 participants in the study is needed.
B) At least 50 participants in the study is needed.
C) At least 30 participants in each subgroup is needed.
D) At least 15 participants in each subgroup is needed.

Question 18

There are several important rules about research ethics. Which one of the following rules is NOT among them?

A) Any study involving human subjects needs prior approval by an ethics review agency.
B) The goal of ethics review is to make sure that the study does not present undue harms to the participants.
C) Informed consent is required for any research study involving human subjects.
D) Informed consent notifies potential participants that they can withdraw from the study at any time without penalty.

Question 19

Which one of the following statements about sampling issues is INCORRECT?

A) One of the main goals of research is to generalize from samples to populations.
B) Generalizations cannot be drawn from studies with small or nonrandom samples.
C) A researcher may have serious flaws in the sampling and yet he has done a reasonably good job in the study.
D) When evaluating sampling procedures in a study, it is important to take into account real-world constraints.