Deck 10: Errors in Hypothesis Testing, Statistical Power, and Effect Size

Question 1

Within a college course, a ______ error occurs if an instructor decides a student ______ when he/she in fact ______.

A) type I; has cheated; did cheat
B) type II; has not cheated; did cheat
C) type II; has cheated; did not cheat
D) type I; has not cheated; did not cheat

Answer 1

B

Question 2

A type II error occurs when researchers make the decision to ______ when they ______.

A) not reject the null hypothesis; should
B) reject the null hypothesis; should not
C) reject the alternative hypothesis; should not
D) not reject the null hypothesis; should not

Answer 2

A

Question 3

The concept of statistical power relates most directly to ______.

A) type I error
B) type II error
C) effect size
D) t-tests

Answer 3

B

Question 4

An example of a Type I error is when a researcher concludes ______.

A) an effect does not exist when it actually does
B) an effect exists when it actually doesn't
C) groups do not differ when they actually do
D) groups are the same when they're actually different

Question 5

Statistical power is the probability of ______.

A) detecting an effect when it exists
B) detecting an effect when it does not exist
C) not detecting an effect when it exists
D) not detecting an effect when it does not exist

Question 6

A type I error occurs when researchers conclude an effect ______ when it ______.

A) exists; does
B) does not exist; does not
C) exists; does not
D) does not exist; does

Question 7

A type I error means that a researcher has ______.

A) correctly concluded that an effect does not exist
B) correctly concluded that an effect exists
C) falsely concluded that an effect does not exist
D) falsely concluded that an effect exists

Question 8

In airport security screening, ______ errors occur when the screening device ______ when in fact ______.

A) type I; beeps; it should not
B) type I; does not beep; it should
C) type II; beeps; it should not
D) type II; beeps; it should

Question 9

A type I error occurs when a researcher ______.

A) does not reject the null hypothesis when he should
B) rejects the alternative hypothesis
C) rejects a true null hypothesis
D) concludes an effect does not exists when it does

Question 10

A type II error means that a researcher has ______.

A) correctly concluded that an effect does not exist
B) correctly concluded that an effect exists
C) falsely concluded that an effect does not exist
D) falsely concluded that an effect exists

Question 11

There exists the possibility of a ______ error whenever a researcher makes the decision to ______.

A) type I; not reject the null hypothesis
B) type I; reject the alternative hypothesis
C) type II; not reject the null hypothesis
D) type II; reject the null hypothesis

Question 12

A type II error occurs when a researcher says an effect ______ when it in fact ______.

A) exists; does
B) exists; does not
C) does not exist; does
D) does not exist; does not

Question 13

A type I error occurs when researchers make the decision to ______ when they ______.

A) reject the null hypothesis; should
B) reject the null hypothesis; should not
C) reject the alternative hypothesis; should not
D) not reject the null hypothesis; should not

Question 14

A type II error occurs when a researcher makes the decision to ______.

A) reject the null hypothesis
B) not reject the null hypothesis
C) accept the alternative hypothesis
D) prove the null hypothesis

Question 15

A type II error occurs when a researcher ______.

A) correctly rejects the null hypothesis
B) says an effect does not exist when it in fact does
C) falsely concludes that an effect exists
D) rejects the null hypothesis when he should

Question 16

Within a marriage, a ______ error occurs if someone decides their spouse ______ when in fact he/she ______.

A) type I; has not been unfaithful; has been unfaithful
B) type II; has not been unfaithful; has not been unfaithful
C) type I; has been unfaithful; has not been unfaithful
D) type II; has been unfaithful; has been unfaithful

Question 17

If major league baseball players were to be tested for steroids, a ______ error would occur if the test states a player ______ steroids when in reality he ______.

A) type I; uses; does not
B) type I; does not use; does not
C) type II; uses; does not
D) type II; uses; does

Question 18

A type I error occurs when the researcher ______, while a type II error occurs when the researcher ______.

A) says an effect does not exists when it does; says an effect exists when it does not
B) reject the null when they should not; does not reject the null when the alternative hypothesis is true
C) does not reject the null; does reject the null
D) does not reject the null when the alternative hypothesis is true; rejects the null when they should not have

Question 19

A type I error may occur when a researcher makes the decision to ______.

A) reject the null hypothesis
B) fail to reject the null hypothesis
C) support the alternative hypothesis
D) prove the research hypothesis

Question 20

A ______ error may be characterized as ______.

A) type I; saying an effect exists when it does not
B) type II; rejecting a true null hypothesis
C) type I; saying an effect does not exist when it does
D) type II; rejecting a false null hypothesis

Question 21

One way to reduce the probability of making a type I error is to ______.

A) make it easier to reject the null hypothesis
B) increase sample size
C) increase alpha
D) lower alpha

Question 22

The probability of making a type I error is typically equal to ______.

A) α
B) β
C) .95
D) μ

Question 23

The probability of making a type II error is ______.

A) .05
B) 1 - α
C) .95
D) none of these

Question 24

One way to reduce the probability of making a type I error is to ______.

A) make it harder to reject the null hypothesis
B) make it easier to reject the null hypothesis
C) use a one-tailed test instead of a two-tailed test
D) make the region of rejection larger

Question 25

The probability of making a type II error is typically equal to ______.

A) 1 - β
B) 1 - α
C) μ
D) none of these

Question 26

The probability of NOT making a ______ error is equal to ______.

A) type I; 1 - α
B) type II; 1 - α
C) type I; 1 - β
D) type II; 0

Question 27

The MOST commonly used strategy to decrease type I error is to ______.

A) make alpha bigger
B) collect more data
C) always reject the null hypothesis
D) make it harder to reject the null hypothesis

Question 28

The chances of making a type I error is typically equal to ______.

A) the probability of rejecting the null hypothesis
B) .95
C) the region of non-rejection
D) μ

Question 29

The probability of making a type I error may be represented by ______.

A) α
B) 1 - t
C) β
D) 1 - β

Question 30

By never rejecting the null hypothesis one has completely eliminated the possibility of ______.

A) a type II error
B) a type I error
C) increasing the statistical power
D) decreasing the statistical power

Question 31

The probability of making a ______ error is equal to ______.

A) type I; 1 - β
B) type II; 0
C) type II; α
D) type II; β

Question 32

______ is defined as the probability of rejecting the null hypothesis when the alternative hypothesis is true.

A) Type I error
B) Error variance
C) Statistical Power
D) Effect size

Question 33

Which of these strategies is most commonly used to reduce the probability of type II error?

A) use a non-directional (two-tailed) instead of a directional (one-tailed) alternative hypothesis
B) never conclude an effect exists regardless of the evidence
C) increase sample size
D) lower alpha

Question 34

Reducing the likelihood of making a type II error involves ______.

A) changing the research hypothesis
B) increasing the likelihood of rejecting the null hypothesis
C) decreasing the likelihood of rejecting the null hypothesis
D) never rejecting the null hypothesis

Question 35

The only way to completely eliminate the possibility of type I error is to ______.

A) never reject the null hypothesis
B) always reject the null hypothesis
C) collect data from the entire population
D) always use non-directional hypotheses

Question 36

The probability of making a type I error is typically equal to ______.

A) .01
B) .05
C) .95
D) .99

Question 37

The probability of making a ______ error may be represented by ______.

A) type I; p = α
B) type II; 1 - α
C) type I; p = β
D) type II; 1 - β

Question 38

α is the probability of ______.

A) making a type I error
B) making a type II error
C) not making a type I error
D) not making a type II error

Question 39

Which of these strategies is MOST commonly used to decrease type I error?

A) using a directional alternative hypothesis
B) lowering alpha
C) increasing sample size
D) increasing alpha

Question 40

The probability of making a type II error is typically ______.

A) .01
B) .05
C) .10
D) none of these

Question 41

Why is the term "highly significant" inappropriate to use when interpreting the probability?

A) because statistical significance is a dichotomy
B) because statistical significance is on a continuum
C) because p < .001 is less significant than p < .01
D) because p < .001 is 'better' than p < .01

Question 42

If there is no relationship between two variables, how much of the variance in one variable is explained by the other variable?

A) 0%
B) 25%
C) 50%
D) 100%

Question 43

Which of the following is a reason the phrase "highly significant" is improper to use?

A) The level of significance is a direct function of sample size.
B) Statistical significance is a dichotomy.
C) p < .01 is not necessarily 'better' than a finding at the p < .05 level.
D) all of these

Question 44

When testing the difference between two sample means, one measure of effect size that may be calculated is the ______ statistic.

A) α
B) t²
C) r²
D) df

Question 45

The main concern with lowering the probability of making a type II error is that it ______.

A) increases the probability of making a type I error
B) decreases the probability of making a type II error
C) changes the effect size
D) decreases the likelihood of finding effects when they in fact do not exist

Question 46

Never making a ______ can be accomplished by always rejecting the null hypothesis.

A) type I error
B) type II error
C) error in sampling
D) decision on effect size

Question 47

Changing alpha (α) from .05 to .01 ______ the probability of making a type I error and ______ the probability of making a type II error.

A) increases; increases
B) increases; decreases
C) decreases; increases
D) decreases; decreases

Question 48

The ______ statistic is a measure of effect size that may be calculated when testing the difference between two sample means.

A) r²
B) α
C) t²
D) df

Question 49

If there is a "perfect" relationship between two variables, how much of the variance in one variable is explained by the other variable?

A) 0%
B) 25%
C) 50%
D) 100%

Question 50

When the within-group variability increases the value of the t-statistics increases, increasing the likelihood of ______.

A) rejecting the null hypothesis
B) not rejecting the null hypothesis
C) rejecting the alternative hypothesis
D) increasing the between group variability

Question 51

For which of these strategies used to reduce the probability of type II error are pragmatic concerns such as time and cost most relevant?

A) using a directional (one-tailed) rather than non-directional (two-tailed) alternative hypothesis
B) increasing sample size
C) making it harder to reject the null hypothesis
D) changing alpha

Question 52

Which of these strategies is MOST commonly used to decrease the probability of making a type II error?

A) never rejecting the null hypothesis
B) increasing the sample size
C) lowering alpha
D) increasing the probability of making a type I error

Question 53

Which of these strategies is MOST commonly used to decrease type II error?

A) using a non-directional (two-tailed) alternative hypothesis (H₁)
B) using a directional (one-tailed) null hypothesis (H₀)
C) increasing sample size
D) lowering alpha

Question 54

The main concern with lowering the probability of making a type I error is that it ______

A) increases the probability of making a type II error
B) decreases the probability of making a type II error
C) requires collecting more data
D) increases the likelihood of finding effects when they in fact do not exist

Question 55

A measure of effect size is a statistic that measures the ______.

A) direction of the relationship between variables
B) nature of the relationship between variables
C) magnitude of the relationship between the sample and population
D) magnitude of the relationship between variables

Question 56

Which statement would be inappropriate to use when interpreting the significance level of a variable?

A) The difference was found to be significant at the p < .001 level.
B) The difference was found to be significant at the p < .001 level, making it highly significant.
C) The difference was found to not be significant with a p value of p > .05.
D) The difference was found to be significant at the p <.05 level.

Question 57

When the within-group variability in a study is small, the likelihood of rejecting the null hypothesis is ______.

A) smaller
B) not changed
C) larger
D) the same

Question 58

Holding all else constant, increasing sample size ______ the probability of making a type I error and ______ the probability of making a type II error.

A) increases; decreases
B) decreases; increases
C) does not affect; decreases
D) decreases; does not affect

Question 59

Using a more homogeneous sample is one strategy used by researchers to ______.

A) increase between-group variability
B) decrease between-group variability
C) increase within-group variability
D) decrease within-group variability

Question 60

Which of the following is a range of possible values for a measure of effect size?

A) 50%-75%
B) 25%-75%
C) 10%-90%
D) 0%-100%

Question 61

A(n) ______ is a statistic that measures the magnitude of the relationship between variables.

A) statistical power
B) error variance
C) measure of effect size
D) within-group variability

Question 62

Calculate the r² statistic for t = 4.64 with N = 50 for each of the two groups.

A) .05
B) .18
C) .30
D) .31

Question 63

How could a researcher interpret an r² value of .35?

A) .35% of the variance in one variable is explained by another variable.
B) 3.5% of the variance in one variable is explained by another variable.
C) 35% of the variance in one variable is explained by another variable.
D) 65% of the variance in one variable is explained by another variable.

Question 64

Calculate the r² statistic for t = 3.02 with N = 100 for each of the two groups.

A) .02
B) .04
C) .08
D) .09

Question 65

Calculate the r² statistic for t = 2.56 and two groups of 20 individuals each.

A) .06
B) .15
C) .25
D) .27

Question 66

Which statistic is an estimate of the magnitude of the difference between the means of two groups measured in standard deviation units?

A) r²
B) t²
C) Cohen's d
D) β

Question 67

Calculate Cohen's d for t = 2.45 with N = 40 for each of the two groups.

A) .06
B) .28
C) .55
D) .79

Question 68

Calculate Cohen's d for t = 2.12 with N = 25 for each of the two groups.

A) .09
B) .31
C) .61
D) .88

Question 69

Calculate the r² statistic for t = 2.98 and two groups of 15 individuals each.

A) .10
B) .24
C) .37
D) .41

Question 70

How would a researcher interpret an r² value of .05?

A) 95% of the variance in one variable is explained by another variable.
B) 5% of the variance in one variable is explained by another variable.
C) .5% of the variance in one variable is explained by another variable.
D) .05% of the variance in one variable is explained by another variable.

Question 71

Calculate the r² statistic for t = 3.11 and two groups of 10 individuals each.

A) .15
B) .35
C) .49
D) .55

Question 72

Calculate Cohen's d for t = 2.35 with N = 30 for each of the two groups.

A) .08
B) .31
C) .62
D) .89

Question 73

Calculate Cohen's d for t = 2.04 with N = 40 for each of the two groups.

A) .05
B) .23
C) .46
D) .66

Question 74

______ is an estimate of the magnitude of the difference between the means of two groups measured in standard deviation units.

A) r²
B) t²
C) β
D) Cohen's d

Question 75

Calculate Cohen's d for t = 4.22 with N = 60 for each of the two groups.

A) .07
B) .39
C) .78
D) 1.11

Question 76

Calculate the r² statistic for t = 3.41 and two groups of 20 individuals each.

A) .08
B) .23
C) .37
D) .39

Question 77

The percentage of variance in one variable that is accounted for by another variable is ______.

A) r²
B) t²
C) d
D) α

Question 78

The r squared (r²) statistic may be defined as ______.

A) the variation in one variable
B) the percentage of the standard deviation in one variable that is accounted for by another
C) the percentage of the mean difference in one variable that is accounted for by another variable
D) the percentage of variance in one variable that is accounted for by another variable

Question 79

Calculate Cohen's d for t = 3.12 with N = 30 for each of the two groups.

A) .11
B) .41
C) .82
D) 1.18

Question 80

Cohen's d is an estimate of ______.

A) the difference between the mean and the population in standard deviation units
B) the magnitude of the difference between the means of two groups measured in standard deviation units
C) the magnitude of the difference between the means of two groups measured in standard error units
D) the magnitude of the difference between the means of two groups measured in z-score units