This is a demo repo to demonstrate how to use Golang to parse JSON files into CSV file. I also used Golang to generate the volume of JSON files (10,000). It read 10,000 of rows data entries from a CSV file and created corresponding JSON files. I also prepared a Python version to parse JSON files into CSV file for performance comparison.
Content:
- /data/ <-- Folder to store input JSON files.
- to_json.csv <-- Source CSV files for JSON files generation.
- csv_to_json.go <-- Golang progrm to read 10,000 of rows data entries from a CSV file and created corresponding JSON files. It can directly run by "go run .\csv_to_json.go". Or build this Golang program by "go build .\csv_to_json.go" as exe to run.
- json_to_csv.go <-- Golang program to parse JSON files into CSV file. It can directly run by "go run .\json_to_csv.go". Or build this Golang program by "go build .\json_to_csv.go" as exe to run.
- json_to_csv.py <-- Python script to parse JSON files into CSV file.
- json_to_csv_parallel.py <-- Optimized Python script to parse JSON file into CSV file with multiprocessing pool.
Performance comparison with Golang and Python
On average, compiled Golang program is 4 times faster than the Python script.
Performance reference of optimized Python with multiprocessing pool
On average, optimized Python with a multiprocessing pool is 4 times faster than script without optimization. It is almost on par with compiled Golang program.
Enjoy!