Question 1

The difference in means of the treated and control groups is not a good measure of the treatment effect when randomization fails to create balanced treatment and control groups.

Accepted Answer

When randomization fails to create balanced treatment and control groups, the difference in means may be influenced by the initial imbalances between the groups, rather than the treatment effect alone. Therefore, the difference in means alone is not a reliable measure of treatment effect in such cases.

Question 2

In order to check for balance we assess whether the dependent variable differs for the treatment and control groups.

Accepted Answer

To check for balance, we assess whether the covariates or characteristics of the participants differ between the treatment and control groups before the intervention, not the dependent variable.

Question 3

One way to check for balance between the treatment and control groups is to look for a statistically significant coefficient on the treatment dummy variable in a model in which variable X1 is the dependent variable.

Accepted Answer

If the treatment and control groups are balanced, the treatment dummy should not have a statistically significant effect on any covariate like X1, indicating that the groups are similar in terms of that covariate.

Question 4

A compliance problem can lead to endogeneity because non-compliance is generally not-random.

Accepted Answer

Non-compliance is usually influenced by factors such as attitudes, beliefs, and external pressures, which can in turn affect the outcome being observed. This means that non-compliance is not independent of the outcome and can lead to endogeneity issues in statistical analysis.

Question 5

Attrition generally will not lead to bias because OLS simply deletes these observations from the sample.

Accepted Answer

Attrition can lead to bias in the sample if the missing data are systematically different from the non-missing data on the outcome or predictor variables. Deleting observations simply ignores this bias rather than correcting for it, potentially leading to invalid conclusions. Alternative approaches such as imputation or weighting can help address this issue.

Question 6

A natural experiment occurs when:&#10;A) The values of an independent variable have been determined by an endogenous process.&#10;B) The values of an independent variable have been determined by an exogenous process.&#10;C) The values of the dependent variable have been determined by an endogenous process.&#10;D) The values of the dependent variable have been determined by an exogenous process.

Accepted Answer

A natural experiment occurs when the values of an independent variable have been determined by an exogenous process, such as a natural disaster, change in policy, or random assignment. This allows for a comparison of the dependent variable to a control group or pre-intervention levels, without interference from endogenous factors.

Question 7

Which of the following is not one of the problems that leads to endogeneity in randomized experiments.&#10;A) Random attrition&#10;B) Lack of balance for covariates&#10;C) Non-compliance&#10;D) Natural experiments

Accepted Answer

The answer of Which of the following is not one...

Question 8

When faced with issues of non-compliance, the coefficient on an intention-to-treat treatment variable will be&#10;A) Larger than if there was perfect compliance&#10;B) Smaller than if there was perfect compliance&#10;C) The same as if there was no non-compliance&#10;D) Potentially larger in magnitude than the actual treatment effect.

Accepted Answer

Non-compliance will lead to dilution of treatment effect and make the coefficient on the intention-to-treat treatment variable smaller compared to the case of perfect compliance where the coefficient would capture the full effect of the treatment.

Question 9

Which of the following explains why a 2SLS model can be used to analyze randomized experiments with imperfect compliance?&#10;A) The random assignment (Z) is uncorrelated with the independent variable.&#10;B) The random assignment variable (Z) is correlated with the error term in the outcome equation.&#10;C) The random assignment (Z) is correlated with the independent variable but not with the error term in the outcome equation.&#10;D) The random assignment (Z) causes multicollinearity.

Accepted Answer

The 2SLS (Two-Stage Least Squares) model is used in the context of instrumental variables estimation, where the random assignment (Z) serves as an instrument for the independent variable. In randomized experiments with imperfect compliance, not all subjects adhere to their assigned treatment, making direct analysis problematic. The key for an instrument like Z to be valid is that it must be correlated with the independent variable (ensuring it has a causal pathway to affect the dependent variable through the independent variable) but not with the error term in the outcome equation (ensuring that the instrument is not directly related to the outcome except through its effect on the independent variable). This ensures that the instrument induces variation in the independent variable that is exogenous, allowing for consistent estimation of the causal effect of the independent variable on the outcome.

Question 10

Which of the following must be undertaken when the treatment and control groups differ with regard to some independent variable, X1.&#10;A) Run an intention-to-treat analysis&#10;B) Run a 2SLS model where the instrument is the random assignment&#10;C) Use multivariate OLS and control for X1.&#10;D) Restart the experiment and pick a new randomized sample.

Accepted Answer

When the treatment and control groups differ with regard to some independent variable, X1, the best approach is to use multivariate OLS and control for X1. This allows for the statistical control of confounding variables, which can help to isolate the effect of the treatment on the outcome variable. Running an intention-to-treat analysis is not useful in this situation as it does not address the issue of confounding variables. A 2SLS model with random assignment as the instrument is not appropriate here since the groups are already randomized. Restarting the experiment and picking a new randomized sample is not practical or necessary if statistical control can be achieved through multivariate OLS.

Question 11

Which of the following is a reason on as to why non-compliance can be a problem?&#10;A) Non-compliance can lead to a correlation between treatment delivered and the error term.&#10;B) Non-compliance can lead to a correlation between treatment and other independent variables.&#10;C) Even completely random causes biased estimates of the treatment effects.&#10;D) Non-compliance is not a problem if compliance is not random.

Accepted Answer

The answer of Which of the following is a reason...

Question 12

When using the intention-to-treat method to deal with non-compliance:&#10;A) The higher the compliance rate, the more likely the ITT estimate overstates the true coefficient.&#10;B) The higher the compliance rate, the further away (lower) the coefficient estimate is from the true coefficient.&#10;C) The lower the compliance rate, the closer the coefficient estimate is to the true coefficient.&#10;D) The lower the compliance rate, the further away (lower) the coefficient estimate is from to the true coefficient.

Accepted Answer

The answer of When using the intention-to-treat method to deal...

Question 13

Which of the following is a method for dealing with attrition in a randomized experiment?&#10;A) Control for the variables associated with attrition&#10;B) A Heckmann selection model.&#10;C) Use a trimmed data set&#10;D) All of the above

Accepted Answer

The answer of Which of the following is a method...

Question 14

Suppose we run an experiment and measure several possible independent variables after treatment. Which of the following best characterizes why this is a problem?&#10;A) Independent variables measured after treatment may be affected by the treatment and may therefore soak up some of the treatment effect.&#10;B) Including post-treatment variables will induce multicollinearity.&#10;C) Post-treatment variables will not satisfy the inclusion condition.&#10;D) Including post-treatment variables will cause imbalance in the treated and control groups.

Accepted Answer

The answer of Suppose we run an experiment and measure...

Deck 10: Experiments: Dealing With Real-World Challenges