Deck 5: Reliability and Validity

Question 1

In research, it is more important to be reliable than valid.

Answer 1

False

Question 2

From the standpoint of the researcher, unreliable tests lead to inaccurate conclusions.

Answer 2

True

Question 3

It is not possible for researchers to calculate reliability, they are only able to estimate it.

Answer 3

True

Question 4

Subject attrition refers to new individuals gained during a study.

Question 5

The best way to access generalizability in research is to repeat research in different populations.

Question 6

Threats to internal validity are more critical than threats to external validity.

Question 7

Systematic Errors occur by chance and are inconsistent across respondents.

Question 8

Systematic Errors can cause bias in data.

Question 9

Following the pilot testing and modification phase, researchers move forward with conducting the actual research.

Question 10

Internal consistency is also called __________.

Question 11

In Cronbach's Alpha, values closer to _______ indicate higher levels of internal consistency, than values closer to ________.

Question 12

When respondents perform well on pre-tests and poorly on post-tests or vice versa merely by chance, cancelling each other out when scores are averaged is called.

Question 13

__________ effects occurs when individuals perform extremely well on both pre and post-tests, and _________ effects occurs when individuals performance starts out low and remains low.

Question 14

___________ variables are fixed and not manipulated, whereas ___________ variables are those that researchers manipulate and control.

Question 15

As research study inclusion becomes more ____________, the results become less ______________.

Question 16

When a sample size is too ____________ significant differences among groups are statistically harder to determine.

Question 17

An important strategy that can be used to reduce measurement errors is __________ testing.

Question 18

If the results of a test are the same each time it is on the same individual over a period of time, the test is considered to be:

A) Valid
B) True
C) Reliable
D) Stable

Question 19

When you give an individual a test multiple times over a certain period of time in order to test stability, this technique is called:

A) Test-retest
B) Standardization
C) Validating
D) Consistency

Question 20

The extent to which each question on survey is related to the same topic is termed:

A) External Consistency
B) Internal Consistency
C) Stability
D) Qualitative research

Question 21

A measurement of internal consistency that allows researchers to determine how well different items measure different aspects of the same topic is:

A) Observational reliability
B) Test-retest method
C) Cronbach's Alpha
D) Checklists

Question 22

When scoring of a test no matter who is doing it, it is termed:

A) Internal validity
B) Scoring consistency
C) Observational Data
D) Inter-observational reliability

Question 23

Which of the following formulas do researchers use to calculate inter rater reliability?

A) Number of occurrences/number of opportunities for occurrence x 100
B) Number of opportunities for occurrence / number of occurrences x 100
C) Number of occurrences x number of opportunities for occurrences / 100
D) Number of occurrences / 100 x number of opportunities for occurrence

Question 24

Those things that confuse test and survey results and research findings serve as a:

A) Threat to external validity
B) Threat to internal validity
C) Threat to external reliability
D) Threat to internal reliability

Question 25

The threat to internal validity that is defined as the natural changes that occur over time with individuals is called:

A) Testing
B) History
C) Maturation
D) Instrumentation

Question 26

Which threat to internal validity measures changes in respondent performance, such as fatique, which cannot be credited to the treatment or intervention?

A) Instrumentation
B) Regression
C) Testing
D) Maturation

Question 27

The threat to internal validity that occurs when participating individuals have improved performance due to the knowledge that they are being watched is called:

A) Maturation
B) History
C) Regression
D) Hawthorne Effect

Question 28

One strategy that can be used to decrease the likelihood of random errors is to:

A) Collect data from a similar set of individuals
B) Collect data from a smaller sample size
C) Collect data from a larger sample size
D) It's impossible to reduce the chance of random error.

Question 29

The gold standard of research designs are:

A) Adequate sample sizes
B) Randomized controlled trials
C) Free of known bias
D) Adequate pilot testing

Question 30

Match the following types of validity with their corresponding definition:
-Face Validity

A)Measures the appearance of the test, does the test look reasonable and does the survey make logical sense?
B)Measures one topic in two different ways.
C)Measures a concept that is not actually observable, established by exploring relationships with similar measures, and comparing scores among defined groups.
D)How well the test measures the specific items that it is intended to measure.

Question 31

Match the following types of validity with their corresponding definition:
-Criterion-related Validity

A)Measures the appearance of the test, does the test look reasonable and does the survey make logical sense?
B)Measures one topic in two different ways.
C)Measures a concept that is not actually observable, established by exploring relationships with similar measures, and comparing scores among defined groups.
D)How well the test measures the specific items that it is intended to measure.

Question 32

Match the following types of validity with their corresponding definition:
-Construct Validity

A)Measures the appearance of the test, does the test look reasonable and does the survey make logical sense?
B)Measures one topic in two different ways.
C)Measures a concept that is not actually observable, established by exploring relationships with similar measures, and comparing scores among defined groups.
D)How well the test measures the specific items that it is intended to measure.

Question 33

Match the following types of validity with their corresponding definition:
-Content Validity

A)Measures the appearance of the test, does the test look reasonable and does the survey make logical sense?
B)Measures one topic in two different ways.
C)Measures a concept that is not actually observable, established by exploring relationships with similar measures, and comparing scores among defined groups.
D)How well the test measures the specific items that it is intended to measure.