Deck 23: Evaluating Research Validity: Part I

Question 1

What does DIAD stand for?

A) What works project design
B) Design interpretation analysis and discovery
C) Design and implementation device
D) Division analysis and discovery

Answer 1

Design and implementation device

Question 2

Overall research validity includes:

A) Internal validity and external validity
B) Measurement reliability and statistics
C) Measurement validity of the constructs
D) All of the above

Answer 2

All of the above

Question 3

The definition of ordinal includes:

A) Ordered levels that are normally distributed
B) Three or more ordered levels that are not normally distributed
C) Rankings are what is important, normal distribution is not a factor
D) Categorical variables
E) a and c

Answer 3

Three or more ordered levels that are not normally distributed

Question 4

Randomized experimental studies include:

A) Random selection from the population to the sample
B) Random comparison between variables
C) The use of associational statistics
D) The use of independent attribute variables
E) Random assignment of participants to groups

Question 5

Comparative studies include which of the following:

A) Random assignment to groups
B) The independent variable is active
C) The independent variable is controlled by the investigator
D) An attribute independent variable
E) The use of associational statistics

Question 6

Which of the following is not a type of reliability?

A) Test-retest
B) Parallel forms
C) Internal consistency
D) Convergence
E) a and d

Question 7

What question does measurement validity address?

A) Are you really measuring what you think you are measuring between the experimental and control group?
B) Are you able to generalize the results from the statistical sample to the population?
C) Do the scores on the instrument accurately reflect the construct?
D) Are the scores on the early part of the instrument similar to scores on the latter part of the instrument?
E) a and b.

Question 8

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-For the stress/humor scenario which of the following would be true?

A) Stress is an active independent variable
B) Calculus test score is an independent variable
C) Change over time is an independent variable
D) Both calculus score and stress are independent variables

Question 9

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-Which of the following would not be a research question for this scenario?

A) Is there an association between stress levels and calculus achievement scores?
B) Is there a difference between the classes who received the humor treatment and those that did not on their stress levels?
C) Is there a difference between the classes who received the humor treatment and those that did not on their calculus achievement scores?
D) Is there a change in stress levels from the baseline stress scores to the midterm stress scores?
E) Is there a combination of factors of stress, study habits, humor, and gender that can predict calculus achievement scores?

Question 10

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-For the stress/humor scenario what are two independent variables?

A) Calculus score and humor/no humor
B) Humor/no humor and stress levels
C) There is only one independent variable
D) Calculus scores and stress
E) Change over time and calculus scores

Question 11

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-Stress is most likely which of the following?

A) A variable with many levels and probably normally distributed
B) A variable with two levels and probably ordinal
C) A constant among students in college
D) A variable with three to four levels and probably ordinal
E) A nominal variable with many levels

Question 12

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-The dependent variable(s) for the quasi-experimental difference question in the stress/humor scenario is:

A) Humor/no humor and stress level
B) Calculus score and stress levels
C) Day of week (Monday, Wednesday, Friday) and time of day
D) Change over time

Question 13

A researcher was interested in studying stress as it related to academic performance incalculus. She also believed that humor reduced stress and might increase academicperformance. She had two classes one hour apart, one at 8 a.m. and one at 10 a.m. onMonday, Wednesday, and Friday. The students self selected into the classes. The researchercollected baseline stress data at the beginning of the semester. In one class she utilized humoron a daily bases and was sure to provide the class with at least one scripted joke or shorthumorous video per day. In the other class, she taught the students as normal with littlehumor, no scripted jokes, and no funny videos. On the day of the midterm, she measuredtheir stress levels before giving the calculus exam. She gave each class the same calculus testand recorded their achievements levels on the exams. Stress was measured using a 10question instrument which summated the 10 questions into one stress construct. Theinstrument had been around for decades and had high reliability and validity measures.

-In the stress/humor scenario day of the week (Monday, Wednesday, or Friday) is which of the following?

A) Change over time
B) An independent variable
C) A dependent variable
D) Constant

Question 14

Robert Rosenthal did a series of studies in which he carefully manipulated teachers' expectations of students; he did this by labeling children in different ways. The main concern in a controlled study of this sort is with:

A) External validity
B) Internal validity
C) Reliability
D) Effect sizes
E) b and c

Question 15

One key to the quasi-experimental approach is that it uses random assignment of participants to groups.

Question 16

One key to the comparative approach is that the independent variable is active.

Question 17

Associational questions usually have two variables with five or more levels.

Question 18

Typically, the number of levels of a randomized experimental approach or a comparative approach is five or more.

Question 19

Descriptive questions only have one independent variable.

Question 20

A form of measurement reliability which measures consistency or stability refers to participants' scores being the same or similar from one testing to another testing.

Question 21

Internal consistency reliability is the same as test-retest reliability.

Question 22

Internal consistency is similar to parallel forms reliability.

Question 23

Content validity indicates that all aspects of the construct are represented in the appropriate proportions.

Question 24

A parallel forms reliability of .55 is acceptable.

Question 25

An internal consistency reliability of .55 is acceptable.

Question 26

A convergent validity measure of .55 is acceptable

Question 27

It is acceptable to have a negative .55 reliability measure.

Question 28

It is acceptable to have a negative .55 discriminate validity measure.

Question 29

Associational approaches are usually rated low on the measures of internal validity.

Question 30

A comparison question which is comparing randomly assigned participants to groups is usually high on measures of internal validity.

Question 31

The ability to detect a statistically significant is most commonly referred to as reliability.

Question 32

Do coefficients for reliability need to be equal, lower, or higher than the coefficients for validity?

Question 33

What does internal validity tell us?

Question 34

Compare and contrast the following terms: nominal, dichotomous, ordinal, and scale.