Deck 10: Survey Instrumentation

Question 1

The cognitive processing perspective on survey interviewing

A) attends to the mental steps required to answer survey questions.
B) focuses on the mental tasks demanded of the interviewer.
C) focuses on the cognitive rules for successfully sustaining the respondent's attention.
D) suggests that interviews are more demanding cognitively than other forms of interaction.

Answer 1

A

Question 2

Theoretical analyses of the interviewer-respondent interaction suggest that respondents typically will

A) tell interviewers when they perceive a question as vague or ambiguous.
B) not answer opinion or attitude questions when they have no opinion.
C) assume automatically that more difficult questions require more time to answer.
D) feel pressure to respond promptly, even if their reply is a haphazard guess or hasty estimate.

Answer 2

D

Question 3

Suppose a campus survey includes the following two questions: "How would you rate your academic experience?" and "How would you rate your college experience?" According to Grice's maxim of nonredundancy, respondents should

A) be annoyed by having to answer questions that are nearly redundant.
B) interpret the questions as having a different meaning.
C) give highly similar responses to the two questions that will undermine measurement validity.
D) invariably ask interviewers to explain how the questions differ from one another.

Answer 3

B

Question 4

According to rational choice theory, __________ behavior is likely when questions are difficult and the respondent's motivation is low; __________ behavior is likely when questions are easily answered and the respondent's motivation is high.

A) satisficing; optimizing
B) optimizing; satisficing
C) antagonistic; cooperative
D) optimizing; cooperative
E) satisficing; antagonistic

Question 5

Open-ended questions work best

A) in questionnaires and in the later stages of research.
B) with less motivated respondents.
C) when respondents vary widely in their knowledge or prior thought on an issue.
D) with large samples that generate a wide variety of responses.

Question 6

Compared with closed-ended questions, open-ended questions

A) require skilled interviewers to elicit complete responses.
B) work best in mail questionnaire surveys where respondents have more time to respond.
C) are more difficult to develop.
D) tend to be used by researchers when they have a thorough knowledge of respondents.

Question 7

Compared with open-ended questions, closed-ended questions

A) are easier to develop.
B) require less prior knowledge of respondents.
C) require less effort from respondents and interviewers.
D) are more difficult to code and analyze.

Question 8

Experiments comparing responses to closed-ended and open-ended questions have found that

A) open-ended questions are better.
B) closed-ended questions are better.
C) the results are similar as long as the response options in the closed question represent the most frequent responses to the open question.
D) the results are similar as long as the questions are similarly worded.

Question 9

According to the textbook, indirect questions are

A) based on the notion of projection.
B) easier to code than direct questions.
C) frequently used in modern survey research.
D) relatively high in reliability and validity.

Question 10

Which of the following types of questions is rarely used in survey research today?

A) open-ended questions
B) closed-ended questions
C) contingency questions
D) filter questions
E) indirect questions

Question 11

Which of the following is consistent with Grice's relevance principle?

A) Respondents voluntarily answer "don't know" if this option is not explicitly offered.
B) Respondents see the numerical scale values of a question as pertinent to the interpretation of the question.
C) Respondents use the option "Other_____(please explain)" when the listed alternatives do not match their true feelings.
D) Respondents respond differently to a general question when it is preceded by a specific question than when the general question is asked first.

Question 12

Which of the following is true about response formats for survey questions?

A) It is always better to include an explicit "don't know" option for opinion questions.
B) For measuring attitudes, the optimum number of categories is four.
C) Because of their explicit meaning, the numerical values of rating-scale endpoints (e.g., 0 to 6 or -3 to +3) are arbitrary.
D) Adjectival labeling of each response category can clarify the meaning of rating scales.

Question 13

Which of the following questions would serve best as the opening question for a survey of campus alcohol use?

A) Do you ever drink alone?
B) Would your father approve of your present drinking habits?
C) What is your major?
D) Do you presently drink beer, liquor, or wine?
E) What is your year in school?

Question 14

In a survey study of campus alcohol and drug use, what is the best placement within a questionnaire or interview schedule for the following question: "Have you ever gotten into trouble with the law because of drinking?"

A) opening question
B) toward the beginning
C) middle
D) end

Question 15

What is the best placement in a survey for the following question: "What is your age?"?

A) opening question
B) toward the beginning
C) middle
D) end

Question 16

Which of the following questions should be placed toward the end of a questionnaire or interview in a survey study of campus alcohol use?

A) Do you ever drink alone?
B) What is your year in school?
C) Would your father approve of your present drinking habits?
D) When did you have your first drink of beer or wine?
E) How often do you have a beer, liquor, or wine?

Question 17

What is the major wording problem with the following question: "Do you approve of the irresponsible person who would drive after having several drinks?"?

A) inappropriate vocabulary
B) leading question
C) lack of precision
D) double-barreled

Question 18

What is the major wording problem with the following question (asked of respondents who say they drink beer): "When you drink beer, how much, on the average, do you usually drink at any one time? [quite a bit] [a moderate amount] [very little] "?

A) inappropriate vocabulary
B) leading question
C) lack of precision
D) double-barreled

Question 19

What is the major wording problem with the following question: "How satisfied are you with the number and fairness of the tests in this course?"

A) inappropriate vocabulary
B) leading question
C) insensitive wording
D) lack of precision
E) double-barreled

Question 20

What is the major wording problem with the following question: "On a typical weekday how many hours do you watch television? [ 0-2 ] [3-5] [6-10] [10] or more"?

A) inappropriate vocabulary
B) leading question
C) lack of precision
D) double-barreled

Question 21

What is the major wording problem with the following survey question: "Is the leadership in your family matriarchal, patriarchal, or egalitarian?"

A) inappropriate vocabulary
B) leading question
C) lack of precision
D) double-barreled
E) insensitive wording

Question 22

Survey questions that have been used in previous research

A) may be used without permission unless they have been copyrighted.
B) should be modified if used in subsequent research to provide evidence of construct validity.
C) often are so poorly written that they cannot be re-used.
D) are of little use because they seldom fit current research objectives.
E) Both c and d are true.

Question 23

Which of the following is not recommended as a way of resolving the frame of reference problem in survey instruments?

A) reason analysis
B) telescoping
C) funnel sequence of questions
D) inverted funnel sequence of questions

Question 24

In contrast to an inverted funnel sequence of questions, a funnel sequence

A) moves from more general to progressively more specific questions.
B) involves strictly open-ended questions.
C) moves from the most specific questions to the most general.
D) assures that respondents share the same frame of reference for answering general questions.

Question 25

The objective of reason analysis is to

A) generate a comprehensive set of questions for finding out why people do things.
B) get respondents to provide more rational responses to questions.
C) help respondents to avoid memory distortion.
D) prompt respondents to answer questions about which they have no firm opinions.

Question 26

Which of the following was not mentioned in the text as an effective way of helping respondents to remember better?

A) limit the reference period to six months or less
B) ask respondents to check their records
C) provide a helpful question sequence
D) ask respondents to consult their spouses or children when unsure

Question 27

Which of the following statements is true of survey instrument design?

A) It is unethical to use survey questions developed by other researchers.
B) Slight changes in question wording can produce great differences in responses.
C) The simple question "Why?" is usually the best way of finding out the reasons for people's behavior.
D) Survey researchers avoid trying to help respondents to remember because this invariably leads to memory distortion.

Question 28

Social desirability and acquiescence response tendencies draw attention to

A) difficulties respondents have in understanding open-ended questions.
B) biases produced by the form and/or content of survey questions.
C) problems with indirect questions.
D) the need for filter and contingency questions.

Question 29

The "social desirability effect" refers to

A) respondents' tendency to give answers that project a positive self-image.
B) true class differences in happiness and mental health.
C) the tendency to agree with a statement regardless of its content.
D) higher response rates obtained by pleasant than unpleasant interviewers.

Question 30

What kind of question is the following: "Do you ever drink beer?
Yes (Go to question 15) No (Go to question 20)"?

A) funnel question
B) inverted funnel question
C) filter question
D) contingency question

Question 31

Questions that apply to some respondents but not to others are called

A) funnel questions.
B) filter questions.
C) contingency questions.
D) matrix questions.

Question 32

A follow-up probe such as "What did you think I meant by 'stopping smoking'?" is most likely to be used during which phase of survey research?

A) cognitive interviewing pretesting
B) field pretesting
C) interviewer training
D) actual data collection

Question 33

Which of the following statements is true of cognitive interviewing and field pretesting?

A) Cognitive interviewing is one of several field pretesting techniques.
B) Both methods are likely to use unrepresentative samples of paid subjects.
C) Both methods focus exclusively on the respondent's perspective on the interviewing process.
D) Field pretesting generally precedes cognitive interviewing.
E) They use complementary techniques with different strengths and weaknesses.

Question 34

Which of the following questions is field pretesting least likely to address?

A) Is the language of questions appropriate for the target population?
B) Are transitions smooth and informative?
C) Are there questions that respondents resist answering?
D) Is a respondent's gender related to his or her responses to questions?

Question 35

Respondent and interviewer debriefings

A) inform respondents and interviewers of the true objectives of the survey, much like the debriefing session in an experiment.
B) provide different perspectives on question-and-answer problems with a survey instrument.
C) enable field supervisors to evaluate the effectiveness of interviewers.
D) prepare respondents and interviewers to perform their proper roles during an interview.

Question 36

Which of the following methods is most likely to be a part of field pretesting?

A) think-aloud interviews
B) probing questions
C) paraphrasing follow-ups
D) interviewer debriefings

Question 37

In deciphering their proper role in a survey interview, respondents may rely on the implied principles of ordinary conversation.

Question 38

Conversational analysis suggests that respondents assume that interviewers will ask only clear questions that are relevant to their knowledge or experiences.

Question 39

Respondents are likely to adopt a satisficing strategy when questions are easily answered and their motivation is high.

Question 40

The development of closed-ended questions requires considerable knowledge of respondents before the data are collected.

Question 41

Open-ended questions are best used in FTF surveys and in the early stages of research.

Question 42

Schuman and Scott found nearly identical responses to a closed-ended question and a similarly worded open-ended question.

Question 43

Indirect questions are rarely used in survey research today.

Question 44

It is unethical for researchers to use survey questions developed by other researchers.

Question 45

Slight changes in question wording can produce great differences in responses.

Question 46

It is good to use words such as "usually," "occasionally," and "sometimes" because respondents are uncomfortable in giving more precise answers.

Question 47

To economize on space and time, questions should be worded when possible so that they address two or more issues simultaneously.

Question 48

Leading questions are an effective way of getting honest, unbiased answers on sensitive issues.

Question 49

Routine background questions are best placed at the beginning of a questionnaire because that is where respondents expect to see them.

Question 50

Sensitive questions are best placed at the beginning of a survey when the respondent's interest is high.

Question 51

Arranging questions in a funnel sequence is one way to control for a respondent's frame of reference.

Question 52

The simple question "Why?" is usually the best way of finding out the reasons for people's behavior.

Question 53

Survey researchers avoid trying to help respondents to remember because this invariably leads to memory distortion.

Question 54

Some respondents tend to agree with attitude and opinion statements regardless of the statement's contents.

Question 55

The social desirability of survey responses tends to be associated with respondents' social class.

Question 56

The purpose of pretesting is to check for associations among variables to see if a larger study is worth doing.

Question 57

Unfortunately, the use of pretesting has little or no effect on the quality of survey data.

Question 58

In cognitive interviewing, interviewers interview themselves by imagining how they would interpret and answer the questions.

Question 59

The purpose of cognitive interviewing techniques is to identify potential wording, ordering, and formatting problems with survey questions.

Question 60

Cognitive interviewing procedures are better at diagnosing problems than providing solutions.

Question 61

The sample of persons used in field pretesting should be as heterogeneous as the target population.

Question 62

Field pretesting tests the survey instrument under realistic interview conditions.

Question 63

Explain what the textbook authors means when they state that a survey interview "is a social occasion subject to the influences of the social world." Provide at least two examples of how this perspective sheds light on the way in which respondents answer survey questions.

Question 64

Describe the conditions when it is best to use (a) open-ended questions and (b) closed-ended questions in survey research.

Question 65

A student planning a survey on altruism seeks your advice. She is interested in documenting the level of altruism at her college, plans to do a Web survey, and is trying to figure out what questions to ask.
a. Below are three of the many questionnaire items she is considering as possible measures of altruism. Focusing on the wording or language of the questions, carefully explain what is wrong with each item.
(1) Do you care about the homeless people in this country? [yes] [no]
(2) Do you regularly offer assistance to strangers? [yes] [no]
(3) Do you and your family contribute to United Way? [yes] [no]
b. How would you revise each item to make it acceptable?
c. Aside from your feedback on these items, what advice would you give her regarding types of question and question order?
d. What recommendations would you make regarding the administration of the survey?

Question 66

What is the "frame of reference" problem in survey instrumentation? Describe three ways in which it is resolved.

Question 67

What is the difference between cognitive laboratory interviewing and field pretesting? When is each set of methods used? What purposes and information does each form of pretesting provide?