Researchers studying the effects of hormone replacement therapy (HRT) on the risk of myocardial infarction (MI) among postmenopausal women calculate the relative risk (RR) of MI to be 1.30 (p = 0.07) among women who are taking HRT compared to those who are not. The researchers conclude that there is no statistically significant increased risk of MI with HRT (based on a cutoff of α = 0.05). Subsequently, the results of a meta-analysis determine that there actually is an increased risk of MI, with an overall RR = 1.32 (p = 0.03) among postmenopausal women who are taking HRT compared to those who are not. Which of the following was the most likely problem in the first study?

Question

Researchers studying the effects of hormone replacement therapy (HRT) on the risk of myocardial infarction (MI) among postmenopausal women calculate the relative risk (RR) of MI to be 1.30 (p = 0.07) among women who are taking HRT compared to those who are not.  The researchers conclude that there is no statistically significant increased risk of MI with HRT (based on a cutoff of α = 0.05).  Subsequently, the results of a meta-analysis determine that there actually is an increased risk of MI, with an overall RR = 1.32 (p = 0.03) among postmenopausal women who are taking HRT compared to those who are not.  Which of the following was the most likely problem in the first study?

Quizplus · Accepted Answer

11ec160b_3711_afca_9249_0974dff7e2e6_MD0001_00 The first study and the meta-analysis (combined results of several studies) had similar outcomes in terms of relative risk (RR); however, they arrived at different conclusions because the p-value reached statistical significance (p <α = 0.05) in the meta-analysis but not in the first study. The most probable reason behind this is that the larger meta-analysis reflects the true status (increased risk of myocardial infarction [MI] with hormone replacement therapy [HRT]), whereas the first study result represented a type II (β) error (falsely concluded there was no increased risk of MI with HRT). The probability of a type II (β) error is related to how much power a study has to detect a difference when a difference actually exists (power = 1 - β). Sample size and power are related in that studies with a larger sample size have greater power to detect differences if these exist. The small sample size of the first study made it underpowered to detect a difference in outcome between patients who were treated with HRT and those who were not. The sample size did not change the value of α (typically set at 0.05). However, because the first study was underpowered, the p-value obtained was not statistically significant (p = 0.07 >α = 0.05), leading to a false conclusion of no increased risk of MI with HRT. (Choice A) Berkson's bias refers to selection bias that can be created by selecting hospitalized patients as the control group. (Choice B) Placebo effect refers to patients' expectations affecting an outcome. However, in this case, the control group is taking neither HRT nor a placebo medication. (Choices C and D) Given that the RR obtained in the first study was similar to the one obtained in the meta-analysis (1.32 versus 1.30), it is unlikely that design flaws (eg, poor blinding, researcher expectancy) were present in the study. Educational objective: A study's power increases as its sample size increases. Therefore, the larger the sample, the greater the ability of a study to detect a difference when one truly exists.

Researchers Studying the Effects of Hormone Replacement Therapy (HRT) on the Risk