Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native Cached Results #1306

Open
dmadisetti opened this issue May 3, 2024 · 1 comment
Open

Native Cached Results #1306

dmadisetti opened this issue May 3, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@dmadisetti
Copy link
Contributor

Description

I find myself using this pattern a lot

if not os.path.isfile("sample_selection_test_all.npy"):
    np.save(
        "sample_selection_test_all", data_sample_export(test_data, test_sample)
    )
    np.save(
        "sample_selection_train_all",
        data_sample_export(train_data, train_sample),
    )

exported_train = np.load("sample_selection_train_all")
exported_test = np.load("sample_selection_test_all")

I wonder if saving computations + saving outputs is on the roadmap.
(docs recommend functools memo, but I like to restart notebooks sometime)

Suggested solution

result = mo.saved(x)

execution unwraps result, saves it to file, or loads it.
or after cell execution could unwrap result

for stateless, could hash ref dag as file name. (hash concated cell contents)


mo.save(plt.gca()) # Last line

allows cell to have a default output. State ensured by the ref dag hash

Alternative

Tons of ways doing this, just provided a suggestion

Additional context

No response

@akshayka
Copy link
Contributor

akshayka commented May 3, 2024

It's something Myles and I have talked about, yes! We even considered using the same API name. So yes, we'll likely support something like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants