Skip to content

This is a demo repo to demonstrate how to use Golang to parse JSON files into CSV file. I also used Golang to generate the volume of JSON files (10,000).

Notifications You must be signed in to change notification settings

easonlai/go_json_to_csv

Repository files navigation

Parsing JSON to CSV in Golang

This is a demo repo to demonstrate how to use Golang to parse JSON files into CSV file. I also used Golang to generate the volume of JSON files (10,000). It read 10,000 of rows data entries from a CSV file and created corresponding JSON files. I also prepared a Python version to parse JSON files into CSV file for performance comparison.

Content:

  • /data/ <-- Folder to store input JSON files.
  • to_json.csv <-- Source CSV files for JSON files generation.
  • csv_to_json.go <-- Golang progrm to read 10,000 of rows data entries from a CSV file and created corresponding JSON files. It can directly run by "go run .\csv_to_json.go". Or build this Golang program by "go build .\csv_to_json.go" as exe to run.
  • json_to_csv.go <-- Golang program to parse JSON files into CSV file. It can directly run by "go run .\json_to_csv.go". Or build this Golang program by "go build .\json_to_csv.go" as exe to run.
  • json_to_csv.py <-- Python script to parse JSON files into CSV file.
  • json_to_csv_parallel.py <-- Optimized Python script to parse JSON file into CSV file with multiprocessing pool.

Performance comparison with Golang and Python

On average, compiled Golang program is 4 times faster than the Python script. alt text

Performance reference of optimized Python with multiprocessing pool

On average, optimized Python with a multiprocessing pool is 4 times faster than script without optimization. It is almost on par with compiled Golang program. alt text

Enjoy!

About

This is a demo repo to demonstrate how to use Golang to parse JSON files into CSV file. I also used Golang to generate the volume of JSON files (10,000).

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published