This project aims to provide an efficient data processing framework that prepares large datasets for a deep learning algorithm focused on lateral movement detection in cybersecurity. (for details of algorithm implementation, see Bai et al. 2020.
- Batch processing of large files (over 35GB compressed)
- Rust-based performance for time-efficient data processing
- Application in real-world big data scenarios and cybersecurity
- Rust Programming Language
- Various Rust Libraries (
std::env,std::fs,bzip2,serde_json,rayon)
- Clone the repository
- Navigate to the project directory
- Build the project with
cargo build
To run the project, execute the following command:
cargo run <path_to_bz2_files>If you're interested in contributing to this data processing tool aimed at detecting lateral movement in cybersecurity, particularly involving RDP logs, you're welcome to reach out. I am are interested in expanding this tool to accommodate Python users and other real-world scenarios.
For more information, questions, or collaborations, please contact me.