How? Potential Outcomes Framework

What? Causal Estimands

Why? Causal Questions and Study Design

What do you think are the biggest issues here?

Before we start...

Be clear about your language
Be clear about your data
Be clear about your assumptions

What is Causal Inference?

Inferring the effect of one thing on another thing

• "My headache went away because I took an aspirin".

• "The new marketing campaign increased our sales by 20%"

• "Providing students support when filling out FAFSA forms improves college access and completion."

A world of potential (outcomes)

• Under a binary treatment or intervention, there are two potential worlds:

A world of potential (outcomes)

• A potential outcome is the outcome under each of these scenarios or "worlds".

  • There will be one for each path!

• A priori, each of these scenarios has a potential outcome

• A posteriori, I can only observe at most one of the potential outcomes

Fundamental Problem of Causal Inference

Potential Outcomes Examples

• "My headache went away because I took an aspirin".

Headache status if I take an aspirin/ Headache status if I don't take an aspirin

• "The new marketing campaign increased our sales by 20%"

• "Providing students support when filling out FAFSA forms improves college access and completion."

Let's see a specific example

• You work at a retail company and you are debating on whether to send out an email campaign to boost your sales:

• You are interested in two specific outcomes:

Potential Outcomes Framework

Let's introduce some notation:

• Let $Y_i$ be the observed outcome for unit $i$ (e.g. whether a person makes a purchase or not).
• Let $Z_i$ be the treatment or intervention (e.g. receiving a promotional email (1) or not (0)).
• Let $Y_i\left(z\right)$ be the potential outcome under treatment $Z=z$. (e.g. whether the person would make a purchase or not if they received treatment z).

Then, if a person is treated, $Z_i=1$, then their observed outcome $Y_i$ will be the same as their potential outcome under treatment, $Y_i\left(1\right)$

$$Y_i|(Z_i=1)\stackrel{\Delta }{=}{Y}_i\left(1\right)$$ In the same fashion, if a person is not treated, $Z_i=0$, then their observed outcome $Y_i$ will be the same as their potential outcome under control, $Y_i\left(0\right)$

$$Y_i|(Z_i=0)\stackrel{\Delta }{=}{Y}_i\left(0\right)$$

Potential Outcomes Framework

This means that we can write the observed outcome as a function of the potential outcomes:

$$\to Y_i=Z_i\cdot Y_i\left(1\right)+(1-Z_i)\cdot Y_i\left(0\right)$$

• This definition will be useful because we can see this as a missing data problem.

Causal Effects

Individual Causal Effect

$$IC{E}_i=Y_i\left(1\right)-Y_i\left(0\right)$$

Causal Effects

Individual Causal Effect

$$IC{E}_i=Y_i\left(1\right)-Y_i\left(0\right)$$
Can we ever observe individual causal effects?

Causal Effects

Individual Causal Effect

$$IC{E}_i=Y_i\left(1\right)-Y_i\left(0\right)$$

Can we ever observe individual causal effects?

No!*

Only one realization

Estimands vs Estimates vs Estimators

Estimands vs Estimates vs Estimators

Estimands vs Estimates vs Estimators

Source: Deng, 2022

Estimands vs Estimates vs Estimators

• Some important estimands that we need to keep in mind:

Average Treatment Effect (ATE)

Average Treatment Effect on the Treated (ATT)

Conditional Average Treatment Effect (CATE)

Estimands vs Estimates vs Estimators

• Some important estimands that we need to keep in mind:

ATE: E.g. Average Treatment Effect for all customers

ATT: E.g. Average Treatment Effect for customers that received the email

CATE: E.g. Average Treatmenf Effect for customer under 25 years old

Estimands vs Estimates vs Estimators

• Some important estimands that we need to keep in mind:

$$ATE=E\left[Y\right(1)-Y(0\left)\right]$$

$$ATT=E\left[Y\right(1)-Y(0)|Z=1]$$

$$CATE=E\left[Y\right(1)-Y(0\left)|X\right]$$

Getting around the fundamental problem of causal inference

• Let's go back to our original example: Does an email campaign increase sales?

Getting around the fundamental problem of causal inference

• We have a missing data problem

Getting around the fundamental problem of causal inference

• Compare those who received the email to the ones did not received the email.

Getting around the fundamental problem of causal inference

• Compare those who received the email to the ones did not received the email.

Getting around the fundamental problem of causal inference

• Compare those who received the email to the ones did not received the email.

$$\hat{\tau }=\frac{1}{3}\sum _{i\in Z=1}{Y}_i-\frac{1}{3}\sum _{i\in Z=0}{Y}_i=0.333$$

Getting around the fundamental problem of causal inference

I we had more data, we could do the same with a simple regression:

$$Purchase={\beta }_0+{\beta }_{1}Email+\epsilon$$

Imagine you get the following results:

$$Purchase=0.4+0.33Email+\epsilon$$

• Interpret the coefficient for Email:

What could be the problem with comparing the sample means?

Look at your green piece of paper and go to the following website

Would you go to a physician/urgent care?

Under what assumptions is our estimate causal?

We are using: $$\hat{\tau }=\frac{1}{3}\sum _{i\in Z=1}{Y}_i-\frac{1}{3}\sum _{i\in Z=0}{Y}_i)$$ to estimate:

$$\tau =E\left[Y_i\right(1)-Y_i(0\left)\right]$$

Under what assumptions is our estimate causal?

We are using: $$\hat{\tau }=\frac{1}{3}\sum _{i\in Z=1}{Y}_i-\frac{1}{3}\sum _{i\in Z=0}{Y}_i)$$

to estimate:

$$\tau =E\left[Y_i\right(1)-Y_i(0\left)\right]$$

Let's do some math

Under what assumptions is our estimate causal?

$$\tau =E\left[Y_i\right(1)-Y_i(0\left)\right]$$ $$=E\left[Y_i\right(1\left)\right]-E\left[Y_i\right(0\left)\right]$$Key assumption:

Ignorability

Ignorability means that the potential outcomes $Y\left(0\right)$ and $Y\left(1\right)$ are independent of the treatment, e.g. $\left(Y\right(0),Y(1\left)\right)\perp \perp Z$.

$$E\left[Y_i\right(1)|Z=0]=E\left[Y_i\right(1)|Z=1]=E\left[Y_i\right(1\left)\right]$$ and

$$E\left[Y_i\right(0)|Z=0]=E\left[Y_i\right(0)|Z=1]=E\left[Y_i\right(0\left)\right]$$

Under what assumptions is our estimate causal?

$$\tau =E\left[Y_i\right(1)-Y_i(0\left)\right]$$ $$=E\left[Y_i\right(1\left)\right]-E\left[Y_i\right(0\left)\right]$$

• Under ignorability (see previous slide), $E\left[Y_i\right(1\left)\right]=E\left[Y_i\right(1)|Z=1]=E[Y_i|Z=1]$ and $E\left[Y_i\right(0\left)\right]=E\left[Y_i\right(0)|Z=0]=E[Y_i|Z=0]$, then:

$$\tau =E\left[Y_i\right(1\left)\right]-E\left[Y_i\right(0\left)\right]=\underset{\text{Obs. Outcome for T}}{\underset{}{E\left[Y_i\right(1)|Z=1]}}-\stackrel{\text{Obs. Outcome for C}}{\stackrel{}{E\left[Y_i\right(0)|Z=0]}}$$

Ignorability Assumption

We can just "ignore" the missing data problem:

Ignorability Assumption

We can just "ignore" the missing data problem:

Ignorability Assumption

We can just "ignore" the missing data problem:

Main takeaway points

Causal Inference is hard

• Think about the causal problem

• Check validity of assumptions (Is ignorability plausible? Am I controlling for the right covariates?)

• Most of this chapter will be spent on looking for exogeneous variation to make the ignorability assumption happen.

Next week

References

• Angrist, J. & S. Pischke. (2015). "Mastering Metrics". Chapter 1.

• Cunningham, S. (2021). "Causal Inference: The Mixtape". Chapter 4: Potential Outcomes Causal Model.

• Neil, B. (2020). "Introduction to Causal Inference". Fall 2020 Course