Replication Code for "Supporting Teacher Autonomy to Improve Education Outcomes: Experimental Evidence from Brazil"

🇧🇷 🏫 🎒 📖 💡 6️⃣ ↗️ ✔️ 🎓 💸

 

This repository contains the codes that replicate the figures and tables presented in the World Bank Policy Research Working Paper "Supporting Teacher Autonomy to Improve Education Outcomes: Experimental Evidence from Brazil" (2020) by Rafael Dantas, Andre Loureiro, Caio Piza, Matteo Ruzzante, and Astrid Zwager. [PDF] [BibTeX]

It is also featured in the World Bank's Reproducible Research Repository.

Read First

The whole analysis in the paper can be rerun by using the master script PIP-Master.do. It is only necessary to add your computer's username and path to the downloaded replication folder in line 131-133 of such do-file in PART 1. You can select which sections to run by editing the locals in the preamble of the do-file. Make sure to run the packages section – PART 0 to install all necessary packages before running the other sections.

The master script will take up to one week on a reasonable cluster. Without considering the do-files using randomization inference procedures (see section Code Process below), it would take around 6 minutes.

The individual do-files with their respective inputs and outputs are explained below. The do-files employ finalized datasets, which are constructed from various data sources, listed and described below. The project data have not been posted yet for proprietary reasons.

Computational reproducibility was verified by DIME Analytics. Details of the reproducibility checklist can be found in the online appendix of the paper.

 

Abstract

What is the impact of greater teacher autonomy on student learning? This paper provides experimental evidence from a program in Brazil. The program supported teachers, through a combination of technical assistance and a small grant, to autonomously develop and implement an innovative project aimed at engaging their students. The findings show that the program improved student learning by 0.15 standard deviation and grade passing by 13 percent in sixth grade, a critical year of transition from primary to lower-secondary education. The paper explores two mechanisms: teacher turnover and student socio-emotional skills. Teacher turnover is reduced by 20.7 percent, and the impacts on student outcomes are concentrated in the schools with the largest reductions. The findings also indicate positive impacts on conscientiousness and extroversion among the students. The results suggest that increasing the autonomy of public servants can improve service delivery, even in a low-capacity context.

World Bank Policy Research Working Paper 9371

Latest Version of the Paper

Online Appendix

Final Dataset Description

Datasets used for analysis are aggregated at different levels, namely at the student, teacher and school level, and described below. Master datasets contain records of all students, teachers and schools, in the experimental sample. Other administrative datasets refer to the universe of schools or students in Rio Grande do Norte or Brazil. [NOTE: Data have not been posted yet in this repository for proprietary reasons.]

The datasets used in the do-files are the following:

1. original_sample.dta contains the list of students and schools in the experimental sample. Source: project.
2. master_studentlevel.dta contains all the information at the student level. Sources: project, State Secretariat of Education (SEE) of Rio Grande do Norte (RN), Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (INEP) school census, Sistema Integrado de Gestão da Educação (SIGEduc) portal.
3. master_schoollevel.dta contains all the information at the school level and the averages of numeric variables at the student level. Sources: project, SEE of RN, INEP school census, SIGEduc portal.
4. master_teacherlevel.dta contains all the information at the teacher level. Turnover dummies are in wide format at the teacher level. Sources: 2016 and 2017 INEP teacher censuses.
5. scores_rescaled_ProvaBrasil.dta contains test scores data rescaled by Sistema de Avaliação da Educação Básica (SAEB) or Prova Brasil. Source: SEE of RN.
6. rates_panel.dta is a panel of progression rates of schools in the experimental sample, containing also grades which were not targeted by the project, from 2015 to 2017. Source: SIGEduc.
7. RN_students_panel.dta is a panel of all students from RN created with census data from 2011 to 2017. Sources: 2011 to 2017 INEP school censuses. Raw data can be downloaded from http://portal.inep.gov.br/microdados.
8. RN_salaries_2016.dta contains data on 2016 salaries for RN. Source: Relação Anual de Informações Sociais (RAIS) from Ministry of Labour and Employment.
9. Brazil_school_indicators.dta contains school indicators, such as progression rates, age-grade distorsion and teacher permanence index, for all schools in Brazil from 2015 to 2017. Source: INEP 2015-2017 school indicators. Raw data can be downloaded from http://portal.inep.gov.br/indicadores-educacionais.
10. Brazil_rates.dta contains progress rates by grade and state in Brazil. Source: INEP 2015 state indicators.
11. Brazil_ProvaBrasil.dta contains average SAEB scores for Brazil and RN by grade in 2013 and 2017. Source: INEP.
12. Brazil_IDEB.dta contains average state-school IDEBs by state in Brazil. Source: INEP. Raw data can be downloaded from http://ideb.inep.gov.br/.

Datasets (1)-(6) are specific to the project evaluated in this paper, while (7)-(12) are information general to the school system and job market of RN and Brazil.

Code Process

The name of the do-files corresponds to the .tex or .png files to be created in the output folder. All tables and figures were included – without further editing – in the TeX document containing the current version of the paper and its online appendix.

The PIP-Master.do file executes the following codes:

Figures

1. fig1-retention_grade6.do uses RN_students_panel.dta and plots Figure 1 – Panels (a) and (b).
2. fig2-implementation_byGrade.do uses master_schoollevel.dta and plots Figure 2.
3. figA1-IDEB_byState.do uses Brazil_IDEB.dta and plots Figure A1.
4. figA2-grade_comparison.do uses RN_rates.dta and plots Figure A2.
5. figA3-grant.do uses master_schoollevel.dta and plots Figure A3.
6. Figure A4 is produced by the R-script figA4-treat_map.R. This code uses identified data on schools (which is not part of the final datasets) and requires you to have a Google API key to retrieve the base map. [NOTE: The figure produced by R was then manually cropped and the clarity of the image adapted.].
7. figA5-predict_participation.do uses master_studentlevel.dta and plots Figure A5.
8. figA6-qreg_media_grade6.do uses master_studentlevel.dta, estimates and plots Figure A6.
9. figA7a-kdensity_grade6_byGender.do uses master_studentlevel.dta, estimates and plots Figure A7 – Panel (a).
10. figA7b-qreg_media_grade6_byGender.do uses master_studentlevel.dta, estimates and plots Figure A7 – Panel (b).
11. figA8-scatter_test_socio.do uses master_studentelevel.dta, estimates and plots Figure A8 – Panels (a) and (b).
12. figA9-itt_ProvaBrasil.do uses Brazil_ProvaBrasil.dta and scores_rescaled_ProvaBrasil.dta, estimates and plots Figures A9 – Panels (a) and (b).
13. figA10-itt_IDEB.do uses master_schoollevel.dta and Brazil_IDEB.dta, estimates and plots Figure A10.
14. figB1-kdensity_wage.do uses RN_salaries_2016.dta and estimates and plots Figure B1.
15. figC1-qreg_bySubject_grade6.do uses master_studentlevel.dta estimates and plots Figure C1 – Panels (a), (b), (c), and (d).

Programs

1. blockdim.ado defines a command to estimate block difference-in-means regressions. This is then employed in tabC2-test_studentlevel_DIM.do and tabC10-socio_studentlevel_DIM.do.

Tables

1. tab1-sample.do uses original_sample.dta and produces Table 1.
2. tab2-baltab.do uses master_schoollevel.dta, master_teacherlevel.dta and master_studentlevel.dta, estimates and produces Table 2 [NOTE: This code may take a long time as it employs randomization inference techniques with 10,000 replications.].
3. tab3-test_studentlevel.do uses master_studentlevel.dta, estimates and produces Table 3.
4. tab4-promotion.do uses master_schoollevel.dta, estimates and produces Table 4.
5. tab5-turnover_teacherlevel.do uses master_teacherlevel.dta, estimates and produces Table 5.
6. tab6-turnover.do uses master_studentlevel.dta, RN_students_panel.dta and master_teacherlevel.dta, estimates and produces Table 6.
7. tab7-spillover_other_grades.do uses RN_students_panel.dta and master_teacherlevel.dta, estimates and produces Table 7.
8. tab8-socio_studentlevel.do uses master_studentlevel.dta, estimates and produces Table 8.
9. tabA1-correlates_turnover.do uses Brazil_school_indicators.dta, estimates and produces Table A1.
10. tabA2-baltab_participation.do uses master_studentlevel.dta, estimates and produces Table A2. [NOTE: This code may take a long time as it employs randomization inference techniques with 10,000 replications.].
11. tabA3,4-baltab_test_takers_schoollevel.do uses uses master_schoollevel.dta, master_teacherlevel.dta and master_studentlevel.dta, estimates and produces Tables A3 and A4. [NOTE: This code may take a long time as it employs randomization inference techniques with 10,000 replications.].
12. tabA5-promotion_het_gender.do uses RN_students_panel.dta, estimates and produces Table A5.
13. tabA6-promotion_het.do uses master_schoollevel.dta, estimates and produces Table A6.
14. tabA7-retention_grade6_regs.do estimates and produces Table A7.
15. tabA8-predict_implementation.do uses master_schoollevel.dta, estimates and produces Table A8.
16. tabA9-clearance_certificate.do uses master_schoollevel.dta, estimates and produces Table A9.
17. tabB1-IDEB_schoollevel.do uses master_schoollevel.dta, estimates and produces Table B1.
18. tabC1-test_studentlevel_ctrl.do uses master_studentlevel.dta, estimates and produces Table C1.
19. tabC2-test_studentlevel_DIM.do uses master_studentlevel.dta, estimates and produces Table C2.
20. tabC3-test_studentlevel_IWE.do uses master_studentlevel.dta, estimates and produces Table C3.
21. tabC4-test_studentlevel_RWE.do uses master_studentlevel.dta, estimates and produces Table C4.
22. tabC5-test_schoollevel.do uses master_schoollevel.dta, estimates and produces Table C5.
23. tabC6-test_rescaled_studentlevel.do uses scores_rescaled_ProvaBrasil.dta, estimates and produces Table C6.
24. tabC7-promotion_other_grades.do uses rates_panel.dta, master_schoollevel.dta and RN_students_panel.dta, estimates and produces Table C7.
25. tabC8-het_spillover_other_grades.do uses master_teacherlevel.dta, RN_students_panel.dta and master_schoollevel.dta, estimates and produces Table C8.
26. tabC9-socio_studentlevel_ctrl.do uses master_studentlevel.dta, estimates and produces Table C9.
27. tabC10-socio_studentlevel_DIM.do uses master_studentlevel.dta, estimates and produces Table C10.
28. tabC11-socio_studentlevel_IWE.do uses master_studentlevel.dta, estimates and produces Table C11.
29. tabC12-socio_studentlevel_RWE.do uses master_studentlevel.dta, estimates and produces Table C12.
30. tabC13-socio_schoollevel.do uses master_schoollevel.dta, estimates and produces Table C13.

Divulgation

• Twitter thread (in Portuguese)
• Some of Development Impact's favorite development papers of 2020

Contact

If you have any comment, suggestion or request for clarifications, you can contact Matteo Ruzzante at matteo.ruzzante@u.northwestern.edu or directly open an issue or pull request in this GitHub repository.

 

This work is licensed under a Creative Commons Attribution 4.0 International License.