A research group conducted a placebo-controlled clinical trial to assess whether a new drug to treat acute migraine with or without aura in adults is more effective than standard therapy. A total of 3,500 patients with acute migraine were enrolled in the study and randomly assigned to either the new drug or standard treatment. During the data analysis phase, the researchers decide to set alpha at 0.01 rather than 0.05. Which of the following is the most likely result of this change?

Question

A research group conducted a placebo-controlled clinical trial to assess whether a new drug to treat acute migraine with or without aura in adults is more effective than standard therapy.  A total of 3,500 patients with acute migraine were enrolled in the study and randomly assigned to either the new drug or standard treatment.  During the data analysis phase, the researchers decide to set alpha at 0.01 rather than 0.05.  Which of the following is the most likely result of this change?

Quizplus · Accepted Answer

11ec160b_3705_0586_9249_379251e364be_MD0001_00 Alpha (α) is the significance level used to establish the statistical significance of study results. It is usually predetermined by investigators as the threshold to reject a null hypothesis (H₀). The results of a study are considered statistically significant when the p-value (ie, probability of obtaining results as extreme as the observed results when H₀ is true) is less than the significance level (ie, p-value < α). Alpha is also the probability of making a type I error, which is rejecting a true H₀ (ie, a false positive: finding a statistically significant difference when one does not truly exist). The complement of alpha/significance level is the confidence level (1 − α), which represents the probability of not rejecting a true H₀ (ie, a true negative, or not finding a statistically significant difference when one does not truly exist). The lower the significance level, the lower the probability of a type I error. However, this means that the threshold to attain statistical significance is more stringent. Therefore, reducing the significance level (eg, from α = 5% to α = 1%) decreases the probability of a type I error along with the probability of finding statistical significance (eg, p-value < 0.05 to p-value < 0.01) (Choices B and C). A decreased significance level also means an increased confidence level (1 − α); therefore, any statistically significant findings can be reported with a greater level of confidence. (Choices D and E) A type II error occurs when a false H₀ is not rejected (ie, a false negative: not finding a statistically significant difference when one truly exist). The probability of a type II error (known as beta [β]) is the complement of power (1 - β). Statistical power is the probability of rejecting a false H₀. That is, the power to find a statistically significant difference when one truly exists (ie, true positive). Reducing the significance level from 0.05 to 0.01 makes the threshold to attain statistical significance more stringent. Consequently, the study will have less statistical power to detect a statistically significant difference when one truly exists and will have a higher probability of a type II error (the complement of power). Educational objective: Reducing the significance level alpha (α) in a study allows researchers to report any significant findings with greater confidence.

A Research Group Conducted a Placebo-Controlled Clinical Trial to Assess