You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ability to use sub run to execute a Notebook definition. This is helpful for long-running Notebooks that aren't a Dataset nor a Model.
Use case: Running a Notebook that might take 10+ hours, for example, I wanted to filter out valid K8s YAML manifests from the-stack dataset which downloads 100+GB of data and in the notebook I push the result back to HuggingFace datasets.
Expected behavior:
If inside the directory there is a Notebook resource YAML file then also allow running that
The run or view command should show the HTML or rendered ipynb file and refresh that every 10 seconds
Why?
Provide more flexibility to folks that just want to run a notebook and watch the results remotely whenever they want to. The notebook command launches jupyter lab but once you exit the browser session you have no way of getting back the results of the run.
Alternatives
Utilize the Dataset or Model resource for Notebooks that aren't technically either
Re-design of how Substratus core resources and shared filesystem be namespace scoped instead of resource scoped
The text was updated successfully, but these errors were encountered:
Ability to use
sub run
to execute a Notebook definition. This is helpful for long-running Notebooks that aren't a Dataset nor a Model.Use case: Running a Notebook that might take 10+ hours, for example, I wanted to filter out valid K8s YAML manifests from
the-stack
dataset which downloads 100+GB of data and in the notebook I push the result back to HuggingFace datasets.Expected behavior:
Why?
notebook
command launches jupyter lab but once you exit the browser session you have no way of getting back the results of the run.Alternatives
The text was updated successfully, but these errors were encountered: