batchgo is a Golang library for batch processing data. It provides a simple way to process large amounts of data in batches.
- Asynchronous batch processing
- concurrency-safe mechanism to add item to batch
Slicer
interface gives more power to user on how to merge/append the data
Documentation at pkg.go.dev
go get github.com/WheeskyJack/batchgo
user need to implement the Slicer intface on the batching object for this pkg to work. It has Append
, Len
Export
and OnFailure
methods.
Append
method is called to merge the item into existing batch.
Len
method is called to get the length of current batch.
Export
method is called once batch is full and ready for processing.
OnFailure
method is called if Export method fails.
see example dir for the working example. It deomonstartes how Add() can be called concurrently for batching.
Contributions are welcome! Please feel free to create an issue and submit a pull request.
This project is licensed under the MIT License - see the LICENSE file for details.