Deck 10: Data Collection: Enhancing Response Rates while Limiting Errors

Question 1

Which of the following is NOT an example of a nonsampling error?

A) Using a telephone directory as a sampling frame.
B) Choosing a quota sample by strictly adhering to the response instructions.
C) Using a city map purchased in a gas station as the sampling frame.
D) Using state car title records as a sampling frame.
E) All of the above are examples of non-sampling errors.

Answer 1

B

Question 2

Nonsampling errors occur because of errors in:

A) conceptionalization of the response project.
B) reporting the results of a project.
C) arithmetic.
D) interpreting respondent replies.
E) All of the above are nonsampling errors.

Answer 2

E

Question 3

Increasing sample size ____ but may also ____.

A) decreases sampling error, increase nonsampling error
B) has no influence on sampling error, increase nonsampling error
C) increases nonsampling error, increase sampling error
D) increases sampling error, increase nonsampling error
E) decreases nonsampling error, decrease sampling error

Answer 3

A

Question 4

Which of the following is essentially a sampling frame problem?

A) Refusals
B) No one at home
C) Observation errors
D) Noncoverage errors
E) Designated respondent not at home when interviewer calls

Question 5

Which of the following is FALSE?

A) Nonresponse error is a potential problem in any project for which data are not collected from all respondents selected for the sample.
B) Nonresponse error only occurs when those who respond are systematically different from those who do not respond in some important way.
C) Refusal bias applies to projects using all forms of data collection.
D) The degree of nonresponse error is easy to assess.
E) All of the above statements are true.

Question 6

Which of the following is FALSE?

A) It is not difficult to estimate the size and effects of nonsampling errors.
B) Reduction of nonsampling error depends on improving method.
C) Nonsampling errors are not as manageable as sampling errors.
D) Nonsampling errors arise from weaknesses in research procedure. They are likely to increase with increases in the sample size.
E) Nonsampling errors contribute more than sampling errors to total error in most surveys.

Question 7

The best way to reduce nonsampling errors is to:

A) increase sample size.
B) improve data collection techniques.
C) offer incentives to respondents.
D) substitute the next house on the block for not-at-homes.
E) Nonsampling errors cannot be reduced.

Question 8

The difference between the observed values of a variable and the long-run average of the observed values in repetitions of the measurement is (the):

A) nonresponse error.
B) nonsampling error.
C) sampling error.
D) noncoverage error.
E) office error.

Question 9

Which of the following is typically estimated by comparing the sample survey results with some outside criterion?

A) Coding errors
B) Sampling errors
C) Noncoverage errors
D) Nonresponse errors
E) Refusal errors

Question 10

Which of the following statements is TRUE?

A) For most surveys, overcoverage is more common than noncoverage.
B) Noncoverage error is not a problem in every survey.
C) Noncoverage bias can be reduced by increasing the sample size.
D) Families with several phone listings have a lower probability of being included in the sample than do sampling units with one listing.
E) All of the above statements are false.

Question 11

The owner of Patty's Pizza Parlor wants to measure consumer awareness of the restaurant and uses the local telephone directory to draw a sample from which to survey the townspeople. However, some people do not have a telephone. This represents what type of problem?

A) Sampling error
B) Nonresponse error
C) Noncoverage error
D) Response error
E) Office error

Question 12

Failure to include some units, or entire sections, of the defined survey population in the actual operational sampling frame represents:

A) noncoverage error.
B) nonresponse error.
C) sampling error.
D) random error.
E) observation error.

Question 13

A random sample of 5,000 households are selected for interviews with the following results: 1,010 completed interviews
864 refusals by the respondent
3,126 no answer/not at home
The response rate is:

A) 19%.
B) 20%.
C) 36%.
D) 38%.
E) 51%.

Question 14

A researcher investigating the consumption of sugar takes a random sample from the population and computes the mean sugar consumption. He repeats the procedure a number of times using the exact same procedure, and finds the means obtained vary from sample to sample. This is an example of:

A) systematic error.
B) sampling error.
C) nonsampling error.
D) noncoverage error.
E) response error.

Question 15

Noncoverage error is not a problem for which of the following sampling frames?

A) Phone book
B) Mailing list
C) Map
D) None of the above have a potential noncoverage problem.
E) All of the above potentially have a noncoverage problem.

Question 16

Sampling errors can be decreased by:

A) decreasing the number of observations.
B) increasing the sample size.
C) providing better training for interviewers.
D) improving the data collection method.
E) using a quota sample.

Question 17

The probability of finding someone home is greater for:

A) high-income families.
B) low-income families.
C) rural families.
D) urban families.
E) Both b and c

Question 18

Which of the following is TRUE?

A) Noncoverage, nonsampling error is best treated by increasing the sample size that is selected.
B) Noncoverage, nonsampling error is essentially a sampling frame problem and if clear, complete, up-to-date sampling frames exist, noncoverage is not likely to be a problem.
C) While noncoverage can be a source of bias, overcoverage is not. In fact, overcoverage may improve estimates as it means that the sample size is larger than initially planned at no increase in cost.
D) Noncoverage bias is essentially eliminated with quota samples.
E) Noncoverage error can be statistically estimated.

Question 19

Which of these contributes to the nonresponse bias in a telephone survey?

A) 1, 2
B) 1, 2, 3
C) 3, 4
D) 2, 4
E) 1, 2, 3, 4

Question 20

Nonresponse error represents:

A) a measure of the difference between the size of the population and the size of the sample.
B) the number of sampling units not contacted on the first attempt.
C) a random distribution of unanswered questions.
D) a failure to obtain information from sampling units.
E) a uniform distribution of uncontacted respondents.

Question 21

During a recent telephone survey, the data collection effort produced the following results: completed interviews = 400; refusals = 200; not-at-home = 400 What is the response rate in this particular situation?

A) 67%
B) 44%
C) 100%
D) 40%
E) 50%

Question 22

Which of the following has NOT been shown to be effective in decreasing the number of refusals?

A) The use of incentives
B) Convincing the respondent of the value of the research
C) Use of experienced interviewers
D) Guarantees of anonymity
E) They have all been shown to be effective.

Question 23

While collecting data by conducting personal interviews in homes, Jake interviews the most accessible individual within the household, rather than using random selection. This increases the potential of:

A) sampling error.
B) response bias.
C) nonresponse error.
D) office error.
E) Both a and

Question 24

Low response rates may indicate:

A) lack of interest among respondents.
B) failure to gain the intended respondents' attention.
C) poor questionnaire design.
D) Both a and b
E) a, b, and c.

Question 25

To avoid low response rates, it is recommended that researchers:

A) conduct thorough exploratory research.
B) perform pre-testing of the questionnaire.
C) use a large sample size.
D) Both a and b
E) Both b and c.

Question 26

Which of the following is FALSE?

A) The main sources of nonresponse bias are not-at-homes and refusals.
B) The type of incentive used to reduce nonresponse may create another potential bias.
C) Technological advancements have helped researchers overcome the not-at-home problem.
D) Using incentives to reduce the nonresponse rate may inflate response errors.
E) Not-at-homes and refusals are both sources of error in telephone surveys.

Question 27

Student researchers collected data on name awareness of a local store. The researchers found that three-fourths of the people who refused to respond were members of the Greek system. If the Greek students who refused to participate in the survey were truly different on some key dimension from those who actually responded to the survey, ____ error could bias the research results.

A) observation
B) sampling
C) response
D) nonresponse
E) noncoverage

Question 28

The empirical evidence suggests which of the following does NOT affect the refusal rate?

A) Nature of the respondent
B) Sponsor of the research
C) Nature of the subject
D) Circumstances surrounding the contact
E) They all affect the refusal rate.

Question 29

Refusals are a more important source of nonresponse bias than the not-at-home problem in:

A) personal interviews.
B) mail surveys.
C) telephone surveys.
D) Both a and b
E) Not-at-homes are always a more important source of error than refusals.

Question 30

The response rate functions to:

A) assess the likely influence of nonresponse error on the study's results.
B) serve as an indicator of the overall quality of a data collection effort.
C) assess the degree of response error.
D) Both a and b
E) a, b, and c.

Question 31

The rate of refusals depends on the:

A) respondent attitude at the time of initial contact.
B) reason for the contact.
C) place where contact was initiated.
D) Both a and b
E) a, b, and c.

Question 32

An interviewer finds that a subject is not at home; to complete his sample, he substitutes the nearest available subject for the missing subject. This practice is:

A) reasonable and provides accurate results.
B) not recommended as it complicates the problems by increasing the proportion of at-homes in the sample.
C) recommended even though it increases the proportion of at-homes.
D) unreasonable although the results are generally accurate.
E) probably the best available technique for handling the not-at-home problem.

Question 33

Which of the following is typically NOT effective in reducing the incidence and problem of not-at-homes?

A) Making advance appointments by telephone
B) Employing callbacks
C) Substitution of neighboring dwelling units
D) Interviewer judgment in scheduling calls
E) They are all typically effective

Question 34

Which of the following statements concerning response errors is FALSE?

A) Respondents who understand a question may not always provide a truthful answer to the question.
B) It is possible that a respondent may understand a question correctly but not actually know the answer to the question.
C) Providing a "don't know" response category will often create difficulties in data analysis.
D) Pre-testing the questionnaire with members of the relevant population is generally not effective at eliminating problems due to respondents misunderstanding questionnaire items.
E) All of the above statements are true.

Question 35

Which of the following is TRUE?

A) The percentage of not-at-homes has been decreasing for a long time.
B) Answering machines help interviewers schedule callback times.
C) It is much easier to find a "responsible adult" at home than a specific respondent.
D) One to two calls are often needed to reach three-fourths of the sample of households.
E) To maintain consistency, callbacks should be made at the same time as the original call.

Question 36

Blame for response errors is usually assigned to:

A) the researcher.
B) the respondent.
C) Both a and b
D) nobody since it is no one's fault.
E) nobody since response errors do not cause significant problems.

Question 37

The refusal problem is the main source of nonresponse bias for a(n):

A) interview.
B) completely randomized experimental design.
C) mail survey.
D) field study.
E) panel.

Question 38

The respondent's cooperation will NOT be encouraged by:

A) convincing the respondent of the value of the research.
B) choosing an appropriate interviewer.
C) training the interviewer properly.
D) guaranteeing that the replies will be confidential.
E) asking sensitive questions.

Question 39

Which of the following is FALSE?

A) New sources of nonsampling error are being discovered all the time.
B) A large sample is much more likely to produce a "valid" result than a small sample is.
C) Total error, rather than any single type of error, is the key in a research investigation.
D) Sampling error can be estimated.
E) Nonsampling error is usually more troublesome than sampling error.

Question 40

Effective strategy(ies) to increase initial response rates is(are):

A) ask respondent to complete a simple task at the beginning.
B) offer respondents money for cooperation.
C) use a good quota sampling plan.
D) Both a and b
E) a, b, and c.

Question 41

In terms of error in research investigations, the key is:

A) total error.
B) sampling error.
C) nonresponse error.
D) systematic error.
E) observation error.

Question 42

Which method of data collection has traditionally produced the lowest response rates?

A) Personal interviews in homes
B) Telephone surveys
C) Mail surveys
D) Personal interviews in shopping malls
E) All of the above produce roughly equivalent response rates.

Question 43

Which of the following are office errors?

A) Not-at-homes
B) Refusals
C) Interviewer cheating
D) Coding
E) Response errors

Question 44

Which of the following statements concerning response incentives is FALSE?

A) Monetary incentives often have the greatest influence on response rates.
B) It probably isn't advisable to give more than $1 for completing a survey.
C) Large amounts of money are more effective than smaller amounts of money at increasing response rates.
D) A "gift" can be more psychologically motivating than "wages."
E) All of the above are true.

Question 45

Research evidence supports the contention that:

A) the gender of respondents and interviewers does NOT influence the answers in a survey.
B) black interviewers are likely to obtain more information on consumption expenditures in a predominantly white neighborhood.
C) it is better to have an old interviewer interview a young respondent and vice versa so as to keep the interview from wandering off the intended topic into areas of mutual interest.
D) the further the social distance between the interviewer and the respondent the greater the bias.
E) background factors of interviewer and interviewee do NOT generally influence the information obtained.

Question 46

To identify whether potential respondents meet eligibility requirements, an interviewer will typically use a(n):

A) eligibility percentage.
B) leading question.
C) loaded question.
D) screening question.
E) alternative questionnaire.

Question 47

Interviewers can make errors in asking any type of question; however, it is a particular problem with which of the following?

A) Multiple-choice questions
B) Dichotomous questions
C) Scale type of questions
D) Open-ended questions
E) Structured questions

Question 48

Errors introduced into the study during data processing are:

A) Type I errors.
B) nonobservation errors.
C) sampling errors.
D) total errors.
E) office errors.

Question 49

An accounting firm decided to conduct a survey among its past clients. A sample of 500 clients was randomly selected to receive a mail survey. A total of 175 clients responded to the survey; 30 of the mailing addresses were not valid. The response rate on the project would be correctly calculated as:

A) 85%.
B) 65%.
C) 37%.
D) 35%.
E) 17%.

Question 50

Which of the following is TRUE?

A) Short surveys are more likely to be completed than are long surveys.
B) Hand-addressed envelopes may increase response rates.
C) Respondents are more likely to open an envelope that has been stamped rather than one that has been sent through a postage meter.
D) Both a and b
E) a, b, and c.

Question 51

Which of the following does NOT lead to response bias?

A) Errors in asking questions
B) Errors in recording answers
C) Errors in probing when additional information is required
D) Errors due to cheating
E) A poorly specified sampling frame

Question 52

Commercial research firms typically validate ____ of completed interviews.

A) 100%
B) 80 to 100%
C) 60 to 70%
D) 10 to 20%
E) 20 to 30%

Question 53

The ____ is used to adjust the number of refusals and not-at-homes to reflect the fact that many of them would not have qualified to participate in the survey even had we successfully contacted them and gotten them to agree to participate.

A) eligibility percentage
B) screening question
C) contact rate
D) response rate
E) None of the above.

Question 54

The lower the response rate, the:

A) more likely it is that response error will affect research results.
B) less likely it is that response error will affect research results.
C) more likely it is that noncoverage error will affect research results.
D) less likely it is that nonresponse error will affect research results.
E) more likely it is that nonresponse error will affect research results.

Question 55

Exhibit 17.1
Student researchers randomly selected 1000 telephone numbers from the local telephone directory and attempted to contact each household. Here are the final results of the calls:

Refer to Exhibit 17.1. What is the response rate in this particular situation?

A) 80%
B) 73%
C) 70%
D) 62%
E) 41%

Question 56

The response rate formula for a telephone survey where there is no eligibility requirement excludes:

A) refusals.
B) wrong numbers.
C) not-at-homes.
D) completed interviews.
E) All of the above are included in the response rate formula for a telephone survey where there is no eligibility requirement.

Question 57

Exhibit 17.1
Student researchers randomly selected 1000 telephone numbers from the local telephone directory and attempted to contact each household. Here are the final results of the calls:

Refer to Exhibit 17.1. What is eligibility percentage in this particular situation?

A) 69%
B) 31%
C) 28%
D) 41%
E) 59%

Question 58

The primary way to control potential bias due to interviewer's attitudes, opinions, perceptions, etc., is through:

A) matching the backgrounds of the interviewer and interviewee.
B) training the interviewer.
C) recruiting interviewers with similar socioeconomic backgrounds.
D) conducting the survey using a rigid set of procedures which the interviewers must follow.
E) requiring the interviewer to complete the questionnaire so that it can be determined if there is a pattern between the interviewer's answers and the answers he or she secures when administering the questionnaire.

Question 59

In a mail survey, 500 questionnaires were sent out but only 200 were returned.

A) This survey has random sampling error.
B) This survey has a response bias.
C) This survey has nonresponse error.
D) This survey suffers from deliberate falsification.
E) None of the above.

Question 60

Which of the following leads to response bias?

A) Errors in asking questions
B) Errors in recording answers
C) Errors in probing when additional information is required
D) Errors due to cheating
E) All of the above.

Question 61

Lower response rates generally suggest fewer problems with nonresponse bias.

Question 62

The two types of nonresponse errors are refusals and item omission.

Question 63

Much nonsampling error occurs during "data collection" process in the field. These "data collection" errors may be divided into

A) Data error in; data error out.
B) Noncoverage error and nonrespondent error.
C) Intentional error and office error.
D) Response error and office error.
E) None of the above.

Question 64

Name the type of nonsampling error that occurs due to data editing, coding, or analysis errors: ____________________

Question 65

____________________ error arises because of a failure to include some part of the defined population in the sampling frame.

Question 66

The number of completed interviews with responding units divided by the number of eligible responding units in the sample is called ____________________.

Question 67

____________________ - this type of error occurs when an individual provides a response to an item, but the response is inaccurate for some reason.

Question 68

Which of the following methods used to reduce nonresponse error is best for reducing the number of not-at-homes, busy signals, and no answers in a telephone survey?

A) Advance notification
B) Monetary incentives
C) Callback
D) Contact them in person
E) None of the above.

Question 69

Nonresponse error represents a failure to obtain information from some elements of the population that were selected and designated for the sample.

Question 70

Noncoverage error arises because of a failure to include some part of the defined population in the sampling frame.

Question 71

Which of the following method(s) can be used to determine if nonresponse error is likely to be a problem in a particular study?

A) Contact a sample of nonrespondents.
B) Compare respondent demographics against known demographics of population.
C) Conduct an analysis of late respondents vs. early respondents.
D) All of the above.
E) None of the above.

Question 72

Not-at-home nonresponse bias is most affected by sample size.

Question 73

Blame for response errors can lay with the researcher, respondent or both.

Question 74

Discuss at least five (5) ways in which response rates might be improved.

Question 75

The lower the ____________________ rate, the more likely it is that nonresponse error will affect research results.

Question 76

If we define our population as all households in the city of Chicago, Illinois, and we use the Chicago telephone directory from which to draw our sample units, we would likely have a:

A) representative survey.
B) survey containing error.
C) survey with a poorly defined master list.
D) survey with sample frame error.
E) survey with no error.