Deck 15: Questionnaire Design and Scale Construction

Question 1

While sometimes they can appear more similar than different, the term _____(a)_____ broadly refers to a less formal, brief survey composed of single-item measures; _____(b)_____ broadly refers to a more formal, longer survey with many items representing each construct, and multiple constructs being measured.

A) scale; a questionnaire
B) questionnaire; a scale
C) survey; a questionnaire
D) scale; a survey

Answer 1

B

Question 2

Which of the following rules of thumb for constructing questionnaires or ratings scales are provided by Crano, Brewer, and Lac? (circle all that apply)

A) use similar numbers of positively- and negatively-worded items
B) avoid double-barreled questions
C) have your items be as directly-worded as possible
D) present the least threatening or sensitive questions first

Answer 2

B,C,D

Question 3

Zelda wants to construct a scale that is able to assess the extremity of people's attitudes toward great white sharks, but her available survey space is limited, so she decided to stick with dichotomous (yes/no) questions. Furthermore, she constructed the items so that each item in the survey was progressively more extreme than the previous (although these items would then be counterbalanced), and respondents would select the one item that best reflects their attitude toward great white sharks. The type of scale Zelda created is most likely a:

A) Likert scale
B) semantic differential scale
C) Guttman scale
D) Thurstone scale

Answer 3

D

Question 4

Nonmonotone items refer to _____(a)_____, and are used in _____(b)_____.

A) items that reflect different constructs; Likert scales
B) items that can be combined because they reflect the same construct; Guttman scales
C) items that should not be combined because agreement with one item does not increase the probability that other items will also be endorsed; Thurstone rating scales
D) items that can be combined even though they reflect different constructs; semantic differential scales

Question 5

Guttman scales and Thurstone scales differ in which of the following ways? (circle all that apply)

A) Thurstone scales uses dichotomous response options, while Guttman scales use ordinal response options
B) the items endorsed on Guttman scales are averaged while in Thurstone scales they are not
C) the items in Thurstone scales are averaged while in Guttman scales they are not
D) in Guttman scales, respondents with a moderately extreme attitude would be expected to endorse the "moderately extreme" item, as well as all less extreme items; in Thurstone scales, respondents with a moderately extreme attitude would be expected to endorse the predetermined "moderately extreme" item and none others

Question 6

Amos wanted to construct a scale that assessed people's attitudes toward a new electric car, but was having a hard time developing items that varied in content intensity. Instead, he decided to write questions that all reflected moderate liking or dislike of the electric car. This scale example would most likely be a:

A) Thurstone scale
B) Likert scale
C) semantic differential scale
D) Guttman scale

Question 7

Boris and Delilah were debating how many response options they should have in their Likert scale items. Boris suggested 21, Delilah suggested three. Based on what you know about response formats, how many response options is usually an optimal balance between depth of response and the amount of cognitive effort respondents have to put forth?

A) 3 response options
B) 7 response options
C) 11 response options
D) 21 response options

Question 8

Which of the following types of scales require more extensive time and effort commitments in the scale development stage? (circle all that apply)

A) Guttman scales
B) Thurstone scales
C) Likert scales
D) Osgood's semantic differential scales

Question 9

Erin wanted to avoid acquiescence bias in her scale, but was not sure if she had succeeded. Specifically, which of the following techniques would NOT be a way to address acquiescence bias?

A) reverse the response options for half of the items in a scale
B) use some negatively worded and some positively worded stems
C) use extreme response anchors to keep responses away from the ends of a scale
D) A and B
E) B and C
F) all of the above

Question 10

Questionnaires differ from rating scales in various ways, such as:

A) they allow for open-ended responses
B) sensitive information, such as demographics, is asked early on
C) care is taken to use short, simple questions
D) question ordering is not a large concern
E) none of the above

Question 11

Inclusion of a "no response" or "don't know" response option may:

A) threaten the generalizability of results
B) reduce refusal or drop-out rates
C) help respondents interpret the other response options
D) complicate interpretation of the results
E) none of the above

Question 12

In Guttman's scalogram scaling technique:

A) a high "coefficient of reproducibility" is desired
B) items are nonmonotone
C) a respondent's total score should provide an indication of which items were endorsed
D) items most often are tapping separate constructs
E) item responses options are equally balanced (i.e., "strongly agree - strongly disagree")

Question 13

Take the hypothetical scale question: "Legalization of marijuana would be a step in the wrong direction for the United States." For this question, the Likert-style response option "strongly agree" would be coded as ____, while "strongly disagree would be coded as ____.

A) +1, -1
B) -1, +1
C) +1, +5
D) -5, +5
E) +5, +1

Question 14

When seeking to determine which items may be removed from a scale to improve its reliability, a researcher should consult:

A) an exploratory factor analysis
B) a confirmatory factor analysis
C) coefficient alpha
D) item-total correlations
E) all of the above

Question 15

Which of the following can help capture a broader and wider range of responses from respondents?

A) a greater number of response options
B) reverse coding
C) ordinal, vs. rank ordered response options
D) larger coefficient alpha
E) less extreme anchor labels

Question 16

When designing questionnaire (or scale) items, some of the first decisions you make are in terms of the structure of the survey. What are three rules of thumb you should consider when wording specific items or ordering items in a questionnaire? For each rule of thumb, describe it, provide an example that demonstrates why it is an important rule of thumb, and explain how and why that example reflects the corresponding rule of thumb.

Question 17

There is ongoing debate about whether to use "don't know" or "no opinion" response options in questionnaire items. Why is this still a contentious issue? Describe two arguments for including a "no response" option in your response options, and two arguments against including such non-response options in a survey.

Question 18

What is the acquiescence bias, and why does it influence the way in which researchers format their items? Specifically, (a) describe the acquiescence bias, (b) describe how it impacts item construction and response formats, (c) offer one way to address this problem in scale construction, and provide an example that illustrates how it would address the problem of acquiescence bias.

Question 19

Describe two advantages of questionnaires, compared to scales, and describe two advantages of scales, compared to questionnaires.

Question 20

You are interested in constructing a new scale to tap people's attitudes toward social science methodology classes. However, since you cannot decide which type of scale would be best suited for this topic, you decide to write four different versions of items assessing attitudes toward methods classes. (a) Specifically, provide three example items that represent each of the four types of scales (i.e., Thurstone, Guttman, Likert, and semantic differentials). Be sure to explain why your example items reflect the qualities of their respective scales. (b) Which type of scale would be best suited for this topic, and why?

Question 21

What are some advantages and disadvantages of using open-ended vs. multiple choice items?
Compare and contrast Thurstone's method of equally-appearing intervals and Osgood's semantic differential scaling approach.

Question 22

What are the different types of factor analysis and how can they assist with scale construction?