Skip to content

johneparker/af-edfi-shared

 
 

Repository files navigation

af-edfi-shared

Description

This repository contains the starter code to transfer data from the Ed-Fi ODS to a reporting database and Tableau Desktop templates. In our ETL framework, source data is first loaded into the Ed-Fi ODS v3.3 and then transferred to a staging database. This staging database also contains the ODS v3.3 data schema. Additional datasets including custom datasets and ad-hoc data trackers are loaded directly into the staging database. Finally, this data is transferred to a central reporting database that powers data visualizations in Tableau.

The data pipeline code to transfer student and enrollment data from the ODS to the staging & reporting db can be found in the following directories:

  • ODS – Staging – IC
  • Staging – Reporting – IC

The data pipeline code to transfer assessment data from the ODS to the staging & reporting db can be found in the following directories:

  • ODS – Staging – Illuminate
  • Staging – Reporting – Illuminate

The data structures for the reporting database can be found in the Reporting Data Structures directory. The Tableau Desktop templates and the underlying SQL queries can be found in the Tableau Desktop Templates and SQL Queries directory respectively.

Prerequisites

  1. Data pipelines – Python version 3.6 or higher
  2. Tableau Templates – Tableau Desktop version 2019.2 or higher
  3. Staging database - Amazon Aurora
  4. Reporting database - Amazon Redshift

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 66.9%
  • TSQL 30.1%
  • PLpgSQL 3.0%