Deck 10: Experiments: Dealing With Real-World Challenges

Question 1

The difference in means of the treated and control groups is not a good measure of the treatment effect when randomization fails to create balanced treatment and control groups.

Answer 1

True

Question 2

In order to check for balance we assess whether the dependent variable differs for the treatment and control groups.

Answer 2

False

Question 3

One way to check for balance between the treatment and control groups is to look for a statistically significant coefficient on the treatment dummy variable in a model in which variable X1 is the dependent variable.

Answer 3

True

Question 4

A compliance problem can lead to endogeneity because non-compliance is generally not-random.

Question 5

Attrition generally will not lead to bias because OLS simply deletes these observations from the sample.

Question 6

A natural experiment occurs when:

A) The values of an independent variable have been determined by an endogenous process.
B) The values of an independent variable have been determined by an exogenous process.
C) The values of the dependent variable have been determined by an endogenous process.
D) The values of the dependent variable have been determined by an exogenous process.

Question 7

Which of the following is not one of the problems that leads to endogeneity in randomized experiments.

A) Random attrition
B) Lack of balance for covariates
C) Non-compliance
D) Natural experiments

Question 8

When faced with issues of non-compliance, the coefficient on an intention-to-treat treatment variable will be

A) Larger than if there was perfect compliance
B) Smaller than if there was perfect compliance
C) The same as if there was no non-compliance
D) Potentially larger in magnitude than the actual treatment effect.

Question 9

Which of the following explains why a 2SLS model can be used to analyze randomized experiments with imperfect compliance?

A) The random assignment (Z) is uncorrelated with the independent variable.
B) The random assignment variable (Z) is correlated with the error term in the outcome equation.
C) The random assignment (Z) is correlated with the independent variable but not with the error term in the outcome equation.
D) The random assignment (Z) causes multicollinearity.

Question 10

Which of the following must be undertaken when the treatment and control groups differ with regard to some independent variable, X1.

A) Run an intention-to-treat analysis
B) Run a 2SLS model where the instrument is the random assignment
C) Use multivariate OLS and control for X1.
D) Restart the experiment and pick a new randomized sample.

Question 11

Which of the following is a reason on as to why non-compliance can be a problem?

A) Non-compliance can lead to a correlation between treatment delivered and the error term.
B) Non-compliance can lead to a correlation between treatment and other independent variables.
C) Even completely random causes biased estimates of the treatment effects.
D) Non-compliance is not a problem if compliance is not random.

Question 12

When using the intention-to-treat method to deal with non-compliance:

A) The higher the compliance rate, the more likely the ITT estimate overstates the true coefficient.
B) The higher the compliance rate, the further away (lower) the coefficient estimate is from the true coefficient.
C) The lower the compliance rate, the closer the coefficient estimate is to the true coefficient.
D) The lower the compliance rate, the further away (lower) the coefficient estimate is from to the true coefficient.

Question 13

Which of the following is a method for dealing with attrition in a randomized experiment?

A) Control for the variables associated with attrition
B) A Heckmann selection model.
C) Use a trimmed data set
D) All of the above

Question 14

Suppose we run an experiment and measure several possible independent variables after treatment. Which of the following best characterizes why this is a problem?

A) Independent variables measured after treatment may be affected by the treatment and may therefore soak up some of the treatment effect.
B) Including post-treatment variables will induce multicollinearity.
C) Post-treatment variables will not satisfy the inclusion condition.
D) Including post-treatment variables will cause imbalance in the treated and control groups.