STA 235H - Ignorability Assumption and Randomized Controlled Trials

class: center, middle, inverse, title-slide

.title[
# STA 235H - Ignorability Assumption and Randomized Controlled Trials
]
.subtitle[
## Fall 2023
]
.author[
### McCombs School of Business, UT Austin
]

---

.small .remark-code { /*Change made here*/
  font-size: 80% !important;
}

.tiny .remark-code { /*Change made here*/
  font-size: 90% !important;
}

.large .remark-code { /*Change made here*/
  font-size: 120% !important;
}
</style>

# Housekeeping

- **.darkorange[Homework 2]** is due this Friday.

- Remember to ask questions **.darkorange[in advance]**! (Discussion board for general q's/clarifications; email/OH if it shows your work)
  - **.darkorange[Check your full code]** before submission: Instructions on the course website.
  
--

- About **.darkorange[JITT feedback]**:

- Thanks for questions/suggestions!
  - Currently ~90% ok with pace.
  - Additional support is available.
  
--

- **.darkorange[No OH next Thursday (09/28)]** (changed to Tuesday; check OH calendar).

---
# Last week

.pull-left[
- Finished our chapter on **.darkorange[multiple regression]**.
  
  - **.darkorange[How to add flexibility to our model]**: Regressions with polynomial terms.
  - For small changes in `$X$` (e.g. one-unit increase), we can approximate `$\Delta Y$` with the derivative!
  
- Introduced **.darkorange[Causal Inference]**
]

.pull-right[
.center[
![:scale 100%](https://media.giphy.com/media/JUGrMkg1JQlMnr9qLE/giphy.gif)]
]

---
# Today

.pull-left[
.center[
![](https://media.giphy.com/media/xlVvWxyeUJsDcHPMsO/giphy.gif)]
]
.pull-right[

- Continue with **.darkorange[causal inference]**:

- Ignorability assumption

- **.darkorange[Introduction to Randomized Controlled Trials]**:

- Why do we randomize?
  - How to analyze RCTs?
  - Are there any limitations?
  
]

---
.center2[
.box-2LA[Similar to last week: Let's do a little exercise]
]

---
<br>

.box-5Trans[Look at your **.darkorange[green]** piece of paper and go to the following website]

.center[
[![:scale 20%](https://github.com/maibennett/sta235/raw/main/exampleSite/content/Classes/Week5/1_RCT/images/qr_code_randomization.png)](https://utexas.qualtrics.com/jfe/form/SV_cZIPHGWcO1onOuy)
]
.center[https://sta235h.rocks/week5]

.box-5trans[I will now decide whether you go to the hospital or not!]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
Causal Inference: Things we can "ignore"
]

---
# Potential Outcomes

- Last week we discussed potential outcomes, (e.g. `$Y_i(1)$` and `$Y_i(0)$`):
  
  .center["The outcome that *we would have observed* under different scenarios"]
  
--

- Potential outcomes are related to your choices/possible conditions:
  - One for each path!
  - Do not confuse them with the **.darkorange[values]** that your outcome variable can take.

- Definition of **.darkorange[Individual Causal Effect]**:

`$$ICE_i = Y_i(1) - Y_i(0)$$`

---
<br>
<br>
<br>
<br>
<br>
<br>
<br>

.box-7Trans[What was the problem with comparing the sample means to get a causal effect?]

---
# Remember our exercise last week!

.pull-left[
![](f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-3-1.png)
]

.pull-right[
![](f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-4-1.png)
]

---
# Under what assumptions is our estimate causal?

We are using a difference in means:
`$$\hat{\tau} = \frac{1}{n_1}\sum_{i \in Z=1}Y_i - \frac{1}{n_0}\sum_{i \in Z=0}Y_i)$$`
to estimate:

`$$\tau = E[Y_i(1) - Y_i(0)]$$`

---
# Under what assumptions is our estimate causal?

We are using a difference in means:
`$$\hat{\tau} = \frac{1}{n_1}\sum_{i \in Z=1}Y_i - \frac{1}{n_0}\sum_{i \in Z=0}Y_i)$$`

to estimate:

`$$\tau = E[Y_i(1) - Y_i(0)]$$`

.box-7LA[Let's do some math]

---
# Under what assumptions is our estimate causal?

`$$\tau = E[Y_i(1) - Y_i(0)] = E[Y_i(1)] - E[Y_i(0)]$$`
--

- Can we observe `$E[Y_i(1)]$`? and `$E[Y_i(0)]$`?

**.darkorange[Key assumption]**:

.box-3[Ignorability]

Ignorability means that the potential outcomes `$Y(0)$` and `$Y(1)$` are independent of the treatment, e.g. `$(Y(0), Y(1)) \perp\!\!\!\perp Z$`.

`$$E[Y_i(1)| Z = 0] = E[Y_i(1)| Z = 1] = E[Y_i(1)]$$`
.center[and]

`$$E[Y_i(0)| Z = 0] = E[Y_i(0)| Z = 1] = E[Y_i(0)]$$`
---
# Under what assumptions is our estimate causal?

`$$\tau = E[Y_i(1) - Y_i(0)]= E[Y_i(1)] - E[Y_i(0)]$$`

**.darkorange[Key assumption]**:

.box-3[Ignorability]

Ignorability means that the potential outcomes `$Y(0)$` and `$Y(1)$` are independent of the treatment, e.g. `$(Y(0), Y(1)) \perp\!\!\!\perp Z$`.

`$$E[Y_i(1)| Z = 0] = \color{#900DA4}{\overbrace{E[Y_i(1)| Z = 1]}^\text{Obs. Outcome for T}} = E[Y_i(1)]$$`
.center[and]

`$$\color{#F89441}{\underbrace{E[Y_i(0)| Z = 0]}_\text{Obs. Outcome for C}} = E[Y_i(0)| Z = 1] = E[Y_i(0)]$$`

---
# Under what assumptions is our estimate causal?

`$$\tau = E[Y_i(1) - Y_i(0)]= E[Y_i(1)] - E[Y_i(0)]$$`
- Under ignorability (see previous slide), `$E[Y_i(1)] = E[Y_i(1) | Z = 1] = E[Y_i | Z = 1]$` and `$E[Y_i(0)] = E[Y_i(0) | Z = 0] = E[Y_i | Z = 0]$`, then:

`$$\tau = E[Y_i(1)] - E[Y_i(0)] = \color{#900DA4}{\underbrace{E[Y_i(1)| Z=1]}_\text{Obs. Outcome for T}} - \color{#F89441}{\overbrace{E[Y_i(0)| Z=0]}^\text{Obs. Outcome for C}} = E[Y_i|Z=1] - E[Y_i|Z=0]$$`
--

- If the **.darkorange[ignorability assumption holds]**, we can use the difference in means between two groups to estimate the **.darkorange[average treatment effect]**.

---
<br>
<br>
<br>
<br>
<br>
<br>
.box-7Trans[Let's see an example: Why did you enroll in the Honors program?]

---
# Let's see the distributions of potential outcomes

---
# Let's see the distributions of potential outcomes

<img src="f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-6-1.svg" style="display: block; margin: auto;" />
---
# We can only observe one distribution per group!

.pull-left[
<img src="f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-7-1.svg" style="display: block; margin: auto;" />
]

.pull-right[
<img src="f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-8-1.svg" style="display: block; margin: auto;" />
]
---
# Under Ignorability Assumption

.pull-left-little_r[
`$$Y(0),Y(1)\perp\!\!\!\perp Z$$`

`$$Income(0),Income(1)\perp\!\!\!\perp Honors$$`]

.pull-right-little_r[

<img src="f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-9-1.svg" style="display: block; margin: auto;" />
]

---
# What about if the ignorability assumption doesn't hold?

.pull-left-little_r[
`$$Y(0),Y(1)\not\perp Z$$`

E.g. Individuals that can take **.darkorange[more advantage from honors program (in terms of income) are more likely to go]**.]

.pull-right-little_r[

<img src="f2023_sta235h_7_RCT_files/figure-html/unnamed-chunk-10-1.svg" style="display: block; margin: auto;" />
]

---
<br>
<br>
<br>
<br>
<br>
<br>
.box-4Trans[What can we do to make the ignorability assumption hold?]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
The Magic of Randomization
]

---
# The problem with self-selection

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-2c8e477f7a1aabea9e62" style="width:504px;height:504px;"></div>
<script type="application/json" data-for="htmlwidget-2c8e477f7a1aabea9e62">{"x":{"visdat":{"97447bf97277":["function () ","plotlyVisDat"]},"cur_data":"97447bf97277","attrs":{"97447bf97277":{"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"x":{},"y":{},"frame":{},"type":"scatter","mode":"markers","symbol":{},"marker":{"color":{},"size":{},"line":{"color":{},"width":3}},"inherit":true}},"layout":{"width":1000,"height":500,"margin":{"b":10,"l":50,"t":50,"r":30,"pad":4},"xaxis":{"domain":[0,1],"automargin":true,"title":"","showgrid":false,"zeroline":false,"showline":false,"showticklabels":false,"range":[-0.04,0.53]},"yaxis":{"domain":[0,1],"automargin":true,"title":"","showgrid":false,"zeroline":false,"showline":false,"showticklabels":false,"range":[-0.05,0.54]},"plot_bgcolor":"rgba(0, 0, 0, 0)","paper_bgcolor":"rgba(0, 0, 0, 0)","autosize":false,"hovermode":"closest","showlegend":false,"sliders":[],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":1300,"easing":"linear"},"frame":{"duration":1500,"redraw":false}}]}]}]},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false,"displayModeBar":false},"data":[{"x":[0.258,-0.009,0.498,0.129,0.36,0.222,0.342,0.489,0.138,-0.018,0.138,0.369,0,0.489,0.258,0.018,0.351,0.489,0.12,0.222],"y":[0.025,0.04,0.075,0.025,0.04,0.21,0.2,0.19,0.2,0.2,0.325,0.34,0.36,0.375,0.375,0.49,0.5,0.5,0.5,0.525],"frame":"1","type":"scatter","mode":"markers","marker":{"color":["#9565C2","#C2679A","#EDE266","#C2679A","#9565C2","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#C2679A","#9565C2","#C2679A"],"symbol":["triangle-up","circle","square","circle","square","square","triangle-up","circle","square","square","square","triangle-up","circle","triangle-up","triangle-up","circle","triangle-up","square","circle","square"],"size":[35,45,25,45,45,25,25,25,35,45,45,45,45,35,35,45,25,35,25,45],"line":{"color":["#5601A4","#BF3984","#FCCE25","#BF3984","#5601A4","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#BF3984","#5601A4","#BF3984"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"1","data":[{"x":[0.258,-0.009,0.498,0.129,0.36,0.222,0.342,0.489,0.138,-0.018,0.138,0.369,0,0.489,0.258,0.018,0.351,0.489,0.12,0.222],"y":[0.025,0.04,0.075,0.025,0.04,0.21,0.2,0.19,0.2,0.2,0.325,0.34,0.36,0.375,0.375,0.49,0.5,0.5,0.5,0.525],"frame":"1","type":"scatter","mode":"markers","marker":{"color":["#9565C2","#C2679A","#EDE266","#C2679A","#9565C2","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#C2679A","#9565C2","#C2679A"],"symbol":["triangle-up","circle","square","circle","square","square","triangle-up","circle","square","square","square","triangle-up","circle","triangle-up","triangle-up","circle","triangle-up","square","circle","square"],"size":[35,45,25,45,45,25,25,25,35,45,45,45,45,35,35,45,25,35,25,45],"line":{"color":["#5601A4","#BF3984","#FCCE25","#BF3984","#5601A4","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#BF3984","#5601A4","#BF3984"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"2","data":[{"x":[0.258,-0.009,0.498,0.129,0.36,0.222,0.342,0.489,0.138,-0.018,0.138,0.369,0,0.489,0.258,0.018,0.351,0.489,0.12,0.222],"y":[0.025,0.04,0.075,0.025,0.04,0.21,0.2,0.19,0.2,0.2,0.325,0.34,0.36,0.375,0.375,0.49,0.5,0.5,0.5,0.525],"frame":"2","type":"scatter","mode":"markers","marker":{"color":["#BCBABA","#FF5582","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA"],"symbol":["triangle-up","circle","square","circle","square","square","triangle-up","circle","square","square","square","triangle-up","circle","triangle-up","triangle-up","circle","triangle-up","square","circle","square"],"size":[35,45,25,45,45,25,25,25,35,45,45,45,45,35,35,45,25,35,25,45],"line":{"color":["#9E9D9D","#FF1654","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"3","data":[{"x":[0.258,-0.009,0.498,0.129,0.36,0.222,0.342,0.489,0.138,-0.018,0.138,0.369,0,0.489,0.258,0.018,0.351,0.489,0.12,0.222],"y":[0.025,0.04,0.075,0.025,0.04,0.21,0.2,0.19,0.2,0.2,0.325,0.34,0.36,0.375,0.375,0.49,0.5,0.5,0.5,0.525],"frame":"3","type":"scatter","mode":"markers","marker":{"color":["#BCBABA","#FF5582","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA"],"symbol":["triangle-up","circle","square","circle","square","square","triangle-up","circle","square","square","square","triangle-up","circle","triangle-up","triangle-up","circle","triangle-up","square","circle","square"],"size":[35,45,25,45,45,25,25,25,35,45,45,45,45,35,35,45,25,35,25,45],"line":{"color":["#9E9D9D","#FF1654","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"4","data":[{"x":[0.106,0.5,0.002,0.34,0.163,0.051,0.22,0.43,0.161,-0.004,0.106,0.222,0.5,0.051,0.07,0.35,0,0.14,0.42,0.21],"y":[0.165,0.266,0.165,0.28,0.135,0.15,0.15,0.28,0.27,0.27,0.27,0.27,0.38,0.285,0.385,0.38,0.415,0.385,0.366,0.385],"frame":"4","type":"scatter","mode":"markers","marker":{"color":["#BCBABA","#FF5582","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA","#BCBABA","#FF5582","#BCBABA"],"symbol":["triangle-up","circle","square","circle","square","square","triangle-up","circle","square","square","square","triangle-up","circle","triangle-up","triangle-up","circle","triangle-up","square","circle","square"],"size":[35,45,25,45,45,25,25,25,35,45,45,45,45,35,35,45,25,35,25,45],"line":{"color":["#9E9D9D","#FF1654","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D","#9E9D9D","#FF1654","#9E9D9D"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>

---
# The power of randomization

- One way to make sure the ignorability assumption holds is to do it by design:

.box-6Trans[Randomize the assignment of Z]

i.e. Some units will **.darkorange[randomly]** be chosen to be in the treatment group and others to be in the control group.

.box-4trans[What does randomization buy us?]

---
# The power of randomization

- One way to make sure the ignorability assumption holds is to do it by design:

.box-6Trans[Randomize the assignment of Z]

i.e. Some units will **.darkorange[randomly]** be chosen to be in the treatment group and others to be in the control group.

.box-4trans[What does randomization buy us?]

.box-3trans[No (systematic) selection on observables OR unobservables]

---
# Randomization of z

<div class="plotly html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-4933365e249c7a5dce50" style="width:504px;height:504px;"></div>
<script type="application/json" data-for="htmlwidget-4933365e249c7a5dce50">{"x":{"visdat":{"974432a4407a":["function () ","plotlyVisDat"]},"cur_data":"974432a4407a","attrs":{"974432a4407a":{"alpha_stroke":1,"sizes":[10,100],"spans":[1,20],"x":{},"y":{},"frame":{},"type":"scatter","mode":"markers","symbol":{},"marker":{"color":{},"size":{},"line":{"color":{},"width":3}},"inherit":true}},"layout":{"width":1000,"height":500,"margin":{"b":10,"l":50,"t":50,"r":30,"pad":4},"xaxis":{"domain":[0,1],"automargin":true,"title":"","showgrid":false,"zeroline":false,"showline":false,"showticklabels":false,"range":[-0.04,0.53]},"yaxis":{"domain":[0,1],"automargin":true,"title":"","showgrid":false,"zeroline":false,"showline":false,"showticklabels":false,"range":[-0.01,0.56]},"plot_bgcolor":"rgba(0, 0, 0, 0)","paper_bgcolor":"rgba(0, 0, 0, 0)","autosize":false,"hovermode":"closest","showlegend":false,"sliders":[],"updatemenus":[{"type":"buttons","direction":"right","showactive":false,"y":0,"x":0,"yanchor":"top","xanchor":"right","pad":{"t":60,"r":5},"buttons":[{"label":"Play","method":"animate","args":[null,{"fromcurrent":true,"mode":"immediate","transition":{"duration":1300,"easing":"linear"},"frame":{"duration":1500,"redraw":false}}]}],"visible":true}]},"source":"A","config":{"modeBarButtonsToAdd":["hoverclosest","hovercompare"],"showSendToCloud":false,"displayModeBar":false},"data":[{"x":[0.378,0,0.24,0.48,0.111,0.129,0.342,-0.009,0.489,0.249,0.378,0.471,-0.018,0.249,0.102,0.369,0.018,0.231,0.48,0.102],"y":[0.04,0.05,0.04,0.05,0.05,0.19,0.2,0.175,0.19,0.225,0.34,0.325,0.325,0.325,0.325,0.525,0.49,0.525,0.49,0.5],"frame":"1","type":"scatter","mode":"markers","marker":{"color":["#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266"],"symbol":["circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up","circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up"],"size":[35,45,25,45,45,25,25,25,35,45,35,45,25,45,45,25,25,25,35,45],"line":{"color":["#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"frames":[{"name":"1","data":[{"x":[0.378,0,0.24,0.48,0.111,0.129,0.342,-0.009,0.489,0.249,0.378,0.471,-0.018,0.249,0.102,0.369,0.018,0.231,0.48,0.102],"y":[0.04,0.05,0.04,0.05,0.05,0.19,0.2,0.175,0.19,0.225,0.34,0.325,0.325,0.325,0.325,0.525,0.49,0.525,0.49,0.5],"frame":"1","type":"scatter","mode":"markers","marker":{"color":["#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266"],"symbol":["circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up","circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up"],"size":[35,45,25,45,45,25,25,25,35,45,35,45,25,45,45,25,25,25,35,45],"line":{"color":["#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"2","data":[{"x":[0.006,0.076,0.134,0.006,0.146,0.064,0,0.186,0.12,0.066,0.405,0.445,0.5,0.345,0.47,0.35,0.415,0.35,0.41,0.475],"y":[0.2,0.18,0.19,0.28,0.29,0.29,0.39,0.4,0.4,0.4,0.22,0.2,0.2,0.2,0.3,0.28,0.3,0.4,0.38,0.41],"frame":"2","type":"scatter","mode":"markers","marker":{"color":["#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266"],"symbol":["circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up","circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up"],"size":[35,45,25,45,45,25,25,25,35,45,35,45,25,45,45,25,25,25,35,45],"line":{"color":["#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"3","data":[{"x":[0.006,0.076,0.134,0.006,0.146,0.064,0,0.186,0.12,0.066,0.405,0.445,0.5,0.345,0.47,0.35,0.415,0.35,0.41,0.475],"y":[0.2,0.18,0.19,0.28,0.29,0.29,0.39,0.4,0.4,0.4,0.22,0.2,0.2,0.2,0.3,0.28,0.3,0.4,0.38,0.41],"frame":"3","type":"scatter","mode":"markers","marker":{"color":["#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266","#EDE266","#EDE266","#EDE266","#C2679A","#C2679A","#EDE266","#9565C2","#C2679A","#9565C2","#EDE266"],"symbol":["circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up","circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up"],"size":[35,45,25,45,45,25,25,25,35,45,35,45,25,45,45,25,25,25,35,45],"line":{"color":["#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25","#FCCE25","#FCCE25","#FCCE25","#BF3984","#BF3984","#FCCE25","#5601A4","#BF3984","#5601A4","#FCCE25"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]},{"name":"4","data":[{"x":[0.006,0.076,0.134,0.006,0.146,0.064,0,0.186,0.12,0.066,0.405,0.445,0.5,0.345,0.47,0.35,0.415,0.35,0.41,0.475],"y":[0.2,0.18,0.19,0.28,0.29,0.29,0.39,0.4,0.4,0.4,0.22,0.2,0.2,0.2,0.3,0.28,0.3,0.4,0.38,0.41],"frame":"4","type":"scatter","mode":"markers","marker":{"color":["#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#BCBABA","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582","#FF5582"],"symbol":["circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up","circle","triangle-up","square","triangle-up","circle","triangle-up","triangle-up","square","circle","triangle-up"],"size":[35,45,25,45,45,25,25,25,35,45,35,45,25,45,45,25,25,25,35,45],"line":{"color":["#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#9E9D9D","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654","#FF1654"],"width":3}},"error_y":{"color":"rgba(31,119,180,1)"},"error_x":{"color":"rgba(31,119,180,1)"},"line":{"color":"rgba(31,119,180,1)"},"xaxis":"x","yaxis":"y","visible":true}],"traces":[0]}],"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
---
# Non-Experimental Causal Graph

- Confounder is a variable that **.darkorange[affects both the treatment AND the outcome]**

.center[
![:scale 70%](https://raw.githubusercontent.com/maibennett/sta235/main/exampleSite/content/Classes/Week5/1_RCT/images/obs_dag.png)
]

---
# Let's identify some confounders

- Estimate the effect of <u>insurance vs no insurance</u> on <u>number of accidents</u> `$\rightarrow$` Compare people with insurance vs people without insurance.

<br>
- Estimate the effect of <u>attending office hours vs not attending</u> on your <u>grade</u> `$\rightarrow$` Compare people who attend OH vs people who don't.

---
# Experimental Causal Graph

- Due to randomization, we know that **.darkorange[the treatment is not affected\* by a confounder]**

.center[
![:scale 70%](https://raw.githubusercontent.com/maibennett/sta235/main/exampleSite/content/Classes/Week5/1_RCT/images/experimental_dag.png)
]

---
# If I randomize treatment allocation...

.center2[
.box-3LA[Can the treatment be potentially correlated with a confounder?]]

---
# Just by chance!

.center[
![:scale 100%](https://raw.githubusercontent.com/maibennett/sta235/main/exampleSite/content/Classes/Week5/1_RCT/images/nicholas_cage.png)]

---
# RCTs: The Gold Standard

.pull-left[
.center[
![:scale 80%](https://raw.githubusercontent.com/maibennett/sta235/main/exampleSite/content/Classes/Week5/1_RCT/images/nobel.png)
]
]

.pull-right[
.center[
![:scale 90%](https://raw.githubusercontent.com/maibennett/sta235/main/exampleSite/content/Classes/Week5/1_RCT/images/nobel2.png)
]
]

---
# How to analyze RCTs?

.box-7LA[Easy! (Statistically speaking)]

---
# How to analyze RCTs?

.box-7LA[Easy! (Statistically speaking)]
<br>
<br>
.box-6Trans[1) Check for balance]
<br>

---
# How to analyze RCTs?

.box-7LA[Easy! (Statistically speaking)]
<br>
<br>
.box-6Trans[1) Check for balance]
<br>
.box-6Trans[2) Calculate difference in sample means between treatment and control group]

---

background-position: 50% 50%
class: center, middle

.box-6LA[Let's see an example]

---
# Are Emily and Greg More Employable Than Lakisha and Jamal?

- Actual **.darkorange[field experiment]** conducted in Boston and Chicago.

- Send out resumes with **.darkorange[randomly assigned names]**:

- Female- and male-sounding names.
  
  - White- and African American-sounding names
  
- Measure whether **.darkorange[applicant was called back]**

---
# Are Emily and Greg More Employable Than Lakisha and Jamal?

.small[
| Variable      | Description                                                    |
|---------------|----------------------------------------------------------------|
| education     | 0 = not reported; 1 = High school dropout (HSD); 2 = High school graduate (HSG); 3 = Some college; 4 = college + |
| ofjobs        | Number of jobs listed on resume                                |
| yearsexp      | Years of experience                                            |
| computerskills| Applicant lists computer skills                                |
| sex           | gender of the applicant (according to name)                    |
| race          | race-sounding name                                             |
| h             | high quality resume                                            |
| l             | low quality resume                                             |
| city          | c = chicago, b = boston                                        |
| call          | applicant was called back                                      |
]

---

background-position: 50% 50%
class: center, middle

.box-3LA[Let's go to R]

---
background-position: 50% 50%
class: left, bottom, inverse
.big[
When we assume...
]
---
# Other potential issues to have in mind

.box-3trans[Generalizability of our estimated effects]

- Where did we get our sample for our study from? Is it representative of a larger population?

.box-5trans[Spillover effects]

- Can an individual in the control group be affected by the treatment?

.box-7trans[General equilibrium effects]

- What happens if we scale up an intervention? Will the effect be the same?

---
# Next class

.pull-left[

- **.darkorange[Limitations]** of RCTs

- Selection on **.darkorange[observables]**

- The wonderful world of **.darkorange[matching!]**
]

.pull-right[
![:scale 120%](https://media.giphy.com/media/oF5oUYTOhvFnO/giphy.gif?cid=ecf05e47gav69lw422g7irxfhq04y3z6yi1zid1oeh7a2glh&rid=giphy.gif&ct=g)
]

---
# References

- Angrist, J. and S. Pischke. (2015). "Mastering Metrics". *Chapter 1*.

- Heiss, A. (2020). "Program Evaluation for Public Policy". *Class 7: Randomization and Matching, Course at BYU*

- Imbens, G. and D. Rubin. (2015). "Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction". *Chapter 1*