From Natural Variation To Optimal Policy? The Lucas Critique Meets Peer Effects
The authors take cohorts of entering freshmen at the United States Air Force Academy and assign half to peer groups with the goal of maximizing the academic performance of the lowest ability students. The assignment algorithm uses peer effects estimates from the observational data. They find a negative and significant treatment effect for the students they intended to help. They show that within the "Optimal" peer groups, students self-selected into bifurcated sub-groups with social dynamics entirely different from those in the observational data. The results suggest that using reduced-form estimates to make out-of-sample policy predictions can lead to unanticipated outcomes.