Botasaurus perfomance

[Halone228]

I would like to know the performance of the framework. When parsing a large amount of data, even through queries, the program's memory usage exceeded 9GB, the program did not cache anything, writing everything to the database on its own. And I also got the error "[Errno 24] Too many open files", I tried to explore on my own, but I didn't find anything. There is an assumption that when running in parallel, the program retains the startup context, even after completion, which can lead to such problems. Maybe I'll find something else and I'll definitely add it!

[surmatech]

ulimit -n alargenumber should solve for too many open files

[Chetan11-dev]

We have released v4, which solves many bugs and adds awesome feautres like UI Run. I suggest using it and let me know if this issue reoccurs, to do so, please run the following commands:

python -m pip install bota botasaurus_api botasaurus_driver bota botasaurus-proxy-authentication botasaurus_server --upgrade

Then read the documentation at https://github.com/omkarcloud/botasaurus.