Deck 8: Evaluating Measures

Question 1

Generally, it is desirable that researchers provide sample items from a test or questions from a questionnaire, to help the reader understand how the variable was measured. However, one valid reason for NOT providing such questionnaire items is:

A) The test is too long and would not fit into the space of a journal article.
B) The readers of the article may be confused by some questions from the questionnaire.
C) The test or questionnaire is copyrighted and cannot be reproduced in a journal article.
D) Journal editors prefer to publish articles without including any tests or questionnaires.

Answer 1

C

Question 2

All of the following statements about measurement are correct, EXCEPT:

A) More information should be provided on those measures that the researchers have developed compared to previously published ones.
B) Providing specific items or questions from a questionnaire is highly desirable.
C) For quantitative researcher, the settings and details of test administration should be provided.
D) Qualitative researchers do not need to provide information about the settings of their interviews.

Answer 2

D

Question 3

The following are valid reasons for using several sources to measure the same variable, EXCEPT:

A) A valid and reliable measure has already been developed in previous research.
B) Different measures have different sets of weaknesses.
C) Sometimes, it is not clear which measure would be most useful.
D) Consistent results from different measures help establish their validity.

Answer 3

A

Question 4

Which one of the following statements correctly describes triangulation?

A) People lying about their true age.
B) Using a questionnaire that has three-choice answer options for each question.
C) Using several sources of information about the same variable.
D) Measuring three different attributes in achievement tests.

Question 5

When a questionnaire asks about certain sensitive matters, people may become hesitant or uncomfortable answering it truthfully. Which one of the following topics is LEAST likely to elicit such reactions?

A) Gang involvement.
B) Shoplifting.
C) Sexual behavior.
D) Level of education.

Question 6

Below are some of the ways that researchers can use to ensure that their survey respondents answer honestly when asked about sensitive matters. Which one of the following is NOT a legitimate way of ensuring truthfulness?

A) Observe the respondents' behavior secretly.
B) Promise confidentiality to the respondents.
C) Let the respondents answer anonymously.
D) Ask what they think their peers do before asking what they themselves do.

Question 7

Sometimes, people change their behavior in response to being observed by researchers. What would be the best way to ensure that the study participants behave naturally?

A) Install a hidden camera or a one-way mirror to observe them.
B) Make the required observation a routine part of the setting.
C) Ask their friends to report on the respondents' behavior.
D) Make a surprise observation visit.

Question 8

What is the best way to reduce the subjectivity of observations in a study?

A) Have several observers rate the same behaviors.
B) Have unsuspecting study participants rate each other after the study.
C) Ask everybody present how they interpret the situation.
D) Ask the participant to rate his or her own behavior.

Question 9

All of the following statements about inter-rater reliability are correct, EXCEPT:

A) Inter-rater reliability is the rate of agreement among the observers rating the same observations.
B) Inter-rater reliability is the same as inter-observer reliability.
C) Inter-rater reliability refers to the correlation between two split-halves of a test.
D) Inter-rater reliability is calculated as a correlation coefficient.

Question 10

When do researchers need to calculate inter-rater reliability?

A) When two raters cannot agree on a test grade.
B) When the coding of observations involves subjectivity.
C) When aggressive behavior is measured.
D) When test scores vary widely among participants.

Question 11

Which one of the following statements about internal consistency is correct?

A) Internal consistency is especially important to compare qualitative observations.
B) Internal consistency is calculated to ensure all items measure the same trait.
C) Internal consistency can only be calculated by hand.
D) Internal consistency is calculated between the two raters of the same behaviors.

Question 12

All of the following statements about temporal stability are correct, EXCEPT:

A) Temporal stability is the same as test-retest reliability.
B) Temporal stability applies only to traits that do not change over short periods of time.
C) Temporal stability can only be calculated for measures that yield numerical scores.
D) Temporal stability refers to the correlation between two halves of the same test.

Question 13

Which one of the following statements about content validity is INCORRECT?

A) Content validity is most closely associated with measures of achievement.
B) Content validity can be established using experts' opinions.
C) Content validity is the same as test-retest reliability.
D) Content validity can be mentioned and then references to previous publications provided.

Question 14

Which one of the following statements about empirical validity is INCORRECT?

A) Empirical validity can only be established when enough data are gathered using the measure in question.
B) Empirical validity ensures that the measure can be used in open-ended interviews.
C) Empirical validity can be expressed as a correlation coefficient.
D) Information on validity is rarely discussed in detail in research reports.

Question 15

When researchers discuss the limitations of their measures in a research report, they do it because:

A) They want the readers to understand how the study was conducted.
B) They are confident that the measures used in the study are valid and reliable.
C) Journal editors like publishing research reports with discussions of measures.
D) Research reports are written for laymen who do not know much about research.