Background generation of data

[datamgmt]

Hi

I have an unstable internet connection, joys of country living.
If I am generating a large data set it via the website it has a nice UI showing progress but if the connection is lost then I have to re-start
I would like to suggest that there is a 'background' generate function that does not require the UI to remain available and sends an email when the generation is complete - giving the user 8 or 24 hours to collect it before it is purged - this would consume minimal extra resources and in my case stop me from re-generating data due to lost connection

Thans for all the hard work

David

[benkeen]

Hey @datamgmt, thanks for posting! Agreed, this is exactly where I want to take it. Now the new website is up the next task is to convert the script to be available as a headless npm package, to allow scripting the data generation (the website will offer a schema generation so you can copy & paste what you've configured via the UI & pass it into a script). Once that's done, the plan was to offer a service via the site where you could offload the data generation to be done elsewhere - and get notified when it's done - just like you described.

I've been taking a bit of a hiatus from the project for a bit, but I should be returning in a few weeks.

Ben