Question 1

Just because a difference is statistically significant does not mean that it is a large difference.

Accepted Answer

Statistical significance indicates that a difference is unlikely to be due to chance alone, but it does not necessarily imply that the difference is large or practically important.

Question 2

Which of the following is not included in the three-step process of evaluating a difference?&#10;A) Evaluate if a statistically significant difference has practical significance&#10;B) Use a significance to test to determine if a difference is statistically significant&#10;C) If there is no significance, one should not assert that the experimental treatment is superior to the control condition and assess for possible practical implications&#10;D) Assume that statistically significant difference always has practical significance

Accepted Answer

The three-step process of evaluating a difference typically includes determining if a difference is statistically significant, evaluating the practical significance of that difference, and considering the implications if no significant difference is found. Assuming that a statistically significant difference always has practical significance (D) is incorrect, as statistical significance does not necessarily imply practical or real-world importance.

Question 3

the fundamental limitations of significance testing are: (1) It fails to indicate the size of a difference, and (2) it does not assess the practical significance of a difference.

Accepted Answer

Significance testing only tells us if a result is likely not due to chance, but it does not provide information on the magnitude of the effect (how large or small the difference is) nor does it address whether the difference is meaningful or relevant in a practical or real-world context.

Question 4

all statistically significant differences are inherently large and therefore of practical importance.

Accepted Answer

Statistical significance does not necessarily imply practical significance; a result can be statistically significant but still be of small magnitude or practical importance.

Question 5

Which of the following is NOT true?&#10;A) Small difference can be statistically significant&#10;B) Large difference is always practically significant&#10;C) Large difference can be practically significant&#10;D) Small difference can be practically significant

Accepted Answer

Large differences between groups or variables in a study can be statistically significant, indicating a low probability that the observed differences occurred by chance. However, statistical significance does not always equate to practical significance. Practical significance refers to the real-world relevance or importance of the difference. A large difference might not be practically significant if it does not have a meaningful impact on the field of study, does not affect decision-making processes, or is not considered important by practitioners or stakeholders. Therefore, the statement that a large difference is always practically significant is not true, as practical significance depends on context, relevance, and application rather than just the size of the difference.

Deck 29: Limitations of Significance Testing