Question 1

Generally, it is desirable that researchers provide sample items from a test or questions from a questionnaire, to help the reader understand how the variable was measured. However, one valid reason for NOT providing such questionnaire items is:
A) The test is too long and would not fit into the space of a journal article.
B) The readers of the article may be confused by some questions from the questionnaire.
C) The test or questionnaire is copyrighted and cannot be reproduced in a journal article.
D) Journal editors prefer to publish articles without including any tests or questionnaires.

Accepted Answer

If a test or questionnaire is copyrighted, it cannot be reproduced without permission from the copyright holder. Therefore, the researcher may not be able to provide sample items from the test or questionnaire in the journal article.

Question 2

All of the following statements about measurement are correct, EXCEPT:
A) More information should be provided on those measures that the researchers have developed compared to previously published ones.
B) Providing specific items or questions from a questionnaire is highly desirable.
C) For quantitative researcher, the settings and details of test administration should be provided.
D) Qualitative researchers do not need to provide information about the settings of their interviews.

Accepted Answer

Qualitative researchers, like quantitative researchers, should provide information about their research settings, as this helps readers understand the context of the data collection and any potential sources of bias.

Question 3

The following are valid reasons for using several sources to measure the same variable, EXCEPT:
A) A valid and reliable measure has already been developed in previous research.
B) Different measures have different sets of weaknesses.
C) Sometimes, it is not clear which measure would be most useful.
D) Consistent results from different measures help establish their validity.

Accepted Answer

Option A states that a valid and reliable measure has already been developed, which would negate the need for using multiple sources to measure the same variable. Therefore, this option is not a valid reason for using several sources.

Question 4

Which one of the following statements correctly describes triangulation?
A) People lying about their true age.
B) Using a questionnaire that has three-choice answer options for each question.
C) Using several sources of information about the same variable.
D) Measuring three different attributes in achievement tests.

Accepted Answer

Triangulation refers to the use of multiple sources or methods to validate or confirm a finding. Option C, using several sources of information about the same variable, is the only option that describes this. Option A refers to lying, option B refers to a specific type of questionnaire, and option D refers to measuring multiple attributes rather than using multiple sources or methods.

Question 5

When a questionnaire asks about certain sensitive matters, people may become hesitant or uncomfortable answering it truthfully. Which one of the following topics is LEAST likely to elicit such reactions?
A) Gang involvement.
B) Shoplifting.
C) Sexual behavior.
D) Level of education.

Accepted Answer

Level of education is a factual matter that is unlikely to evoke embarrassment or defensiveness, whereas the other topics may be associated with shame, guilt, or stigma. Gang involvement and shoplifting may be perceived as criminal activities or deviant behavior, and disclosing sexual behavior could be embarrassing or taboo for some individuals.

Question 6

Below are some of the ways that researchers can use to ensure that their survey respondents answer honestly when asked about sensitive matters. Which one of the following is NOT a legitimate way of ensuring truthfulness?
A) Observe the respondents' behavior secretly.
B) Promise confidentiality to the respondents.
C) Let the respondents answer anonymously.
D) Ask what they think their peers do before asking what they themselves do.

Accepted Answer

Secretly observing respondents' behavior is not an ethical or legitimate way of ensuring truthfulness. It violates their privacy and autonomy and can lead to biased and inaccurate data. The other three options – promising confidentiality, allowing anonymous responses, and asking about peer behavior before self-behavior – are all legitimate methods for encouraging honest and accurate answers on sensitive topics.

Question 7

Sometimes, people change their behavior in response to being observed by researchers. What would be the best way to ensure that the study participants behave naturally?
A) Install a hidden camera or a one-way mirror to observe them.
B) Make the required observation a routine part of the setting.
C) Ask their friends to report on the respondents' behavior.
D) Make a surprise observation visit.

Accepted Answer

Making the required observation a routine part of the setting can help participants become used to being observed and therefore act more natural. A surprise observation visit may cause participants to feel anxious, which can alter their behavior. Installing a hidden camera or a one-way mirror may violate participants' privacy and consent. Asking their friends to report on their behavior may be biased or unreliable.

Question 8

What is the best way to reduce the subjectivity of observations in a study?
A) Have several observers rate the same behaviors.
B) Have unsuspecting study participants rate each other after the study.
C) Ask everybody present how they interpret the situation.
D) Ask the participant to rate his or her own behavior.

Accepted Answer

Having several observers rate the same behaviors can help reduce subjectivity as it allows for inter-rater reliability checks and can provide a more objective view of the behaviors being observed.

Question 9

All of the following statements about inter-rater reliability are correct, EXCEPT:
A) Inter-rater reliability is the rate of agreement among the observers rating the same observations.
B) Inter-rater reliability is the same as inter-observer reliability.
C) Inter-rater reliability refers to the correlation between two split-halves of a test.
D) Inter-rater reliability is calculated as a correlation coefficient.

Accepted Answer

Inter-rater reliability does not refer to the correlation between two split-halves of a test. This is known as split-half reliability.

Question 10

When do researchers need to calculate inter-rater reliability?
A) When two raters cannot agree on a test grade.
B) When the coding of observations involves subjectivity.
C) When aggressive behavior is measured.
D) When test scores vary widely among participants.

Accepted Answer

Inter-rater reliability is necessary when the coding or scoring of observations involves subjectivity, meaning that different raters could potentially interpret the same behavior differently. This could be true in a variety of research areas, such as psychology, sociology, or education. It is not necessary for situations where raters cannot agree on a test grade, as this can be resolved by discussing the criteria and reaching a consensus. It is also not directly related to the variability of test scores or the measurement of aggressive behavior.

Question 11

Which one of the following statements about internal consistency is correct?
A) Internal consistency is especially important to compare qualitative observations.
B) Internal consistency is calculated to ensure all items measure the same trait.
C) Internal consistency can only be calculated by hand.
D) Internal consistency is calculated between the two raters of the same behaviors.

Accepted Answer

Internal consistency is a measure of how well different items on a measure of a construct are correlated with each other. It is used to ensure that all items are measuring the same trait or concept. It can be calculated using a number of techniques, including Cronbach's alpha and split-half reliability. It is not specifically related to qualitative observations or the comparison of two raters.

Question 12

All of the following statements about temporal stability are correct, EXCEPT:
A) Temporal stability is the same as test-retest reliability.
B) Temporal stability applies only to traits that do not change over short periods of time.
C) Temporal stability can only be calculated for measures that yield numerical scores.
D) Temporal stability refers to the correlation between two halves of the same test.

Accepted Answer

Temporal stability, also known as test-retest reliability, refers to the consistency of a measure over time, not the correlation between two halves of the same test. The correlation between two halves of a test is related to internal consistency or split-half reliability, not temporal stability.

Question 13

Which one of the following statements about content validity is INCORRECT?
A) Content validity is most closely associated with measures of achievement.
B) Content validity can be established using experts' opinions.
C) Content validity is the same as test-retest reliability.
D) Content validity can be mentioned and then references to previous publications provided.

Accepted Answer

Content validity and test-retest reliability are two different concepts. Content validity refers to how well a test or assessment accurately measures the specific content or knowledge/skills it is intended to measure, while test-retest reliability is the consistency of scores over time. Therefore, Option C is incorrect.

Question 14

Which one of the following statements about empirical validity is INCORRECT?
A) Empirical validity can only be established when enough data are gathered using the measure in question.
B) Empirical validity ensures that the measure can be used in open-ended interviews.
C) Empirical validity can be expressed as a correlation coefficient.
D) Information on validity is rarely discussed in detail in research reports.

Accepted Answer

Empirical validity does not guarantee that a measure can be used in open-ended interviews. Empirical validity refers to the degree to which a measure correlates with other measures or with an outcome of interest, indicating that it is measuring what it is intended to measure. The appropriateness of a measure for use in open-ended interviews depends on other factors such as the type of data being collected and the research question being asked.

Question 15

When researchers discuss the limitations of their measures in a research report, they do it because:
A) They want the readers to understand how the study was conducted.
B) They are confident that the measures used in the study are valid and reliable.
C) Journal editors like publishing research reports with discussions of measures.
D) Research reports are written for laymen who do not know much about research.

Accepted Answer

Discussing the limitations of their measures helps readers understand the context and constraints under which the study was conducted, providing a clearer picture of the research's scope and applicability.

Quiz 8: Evaluating Measures

Generally, it is desirable that researchers provide sample items from a test or questions from a questionnaire, to help the reader understand how the variable was measured. However, one valid reason for NOT providing such questionnaire items is:

All of the following statements about measurement are correct, EXCEPT:

The following are valid reasons for using several sources to measure the same variable, EXCEPT:

Which one of the following statements correctly describes triangulation?

When a questionnaire asks about certain sensitive matters, people may become hesitant or uncomfortable answering it truthfully. Which one of the following topics is LEAST likely to elicit such reactions?

Below are some of the ways that researchers can use to ensure that their survey respondents answer honestly when asked about sensitive matters. Which one of the following is NOT a legitimate way of ensuring truthfulness?

Sometimes, people change their behavior in response to being observed by researchers. What would be the best way to ensure that the study participants behave naturally?

What is the best way to reduce the subjectivity of observations in a study?

All of the following statements about inter-rater reliability are correct, EXCEPT:

When do researchers need to calculate inter-rater reliability?

Which one of the following statements about internal consistency is correct?

All of the following statements about temporal stability are correct, EXCEPT:

Which one of the following statements about content validity is INCORRECT?

Which one of the following statements about empirical validity is INCORRECT?

When researchers discuss the limitations of their measures in a research report, they do it because: