Project Description

Large File Splitting with Python Threading for Improved Performance. I run a 50 GB txt file(.log) through this Python script that splits it into 51 smaller files successfully.

parameter

modify these parameters in main.py

input_file_path = 'C:/Users/user/Downloads/test.csv'
output_dir_path = 'output'
num_threads = 50

run

python main.py

Reference

Large File Splitting with Python Threading for Improved Performance