Replies: 1 comment
-
This is on the roadmap and currently being worked on. We'll store the intermediaries as files as opposed to SQLLite and be able to re-use them. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
It would be awesome with some way to pre-compute / materialize values in local mode, such as using an SQLite database.
We are huge fans of the local mode, as it seems like a great way to let individual data scientists in different collaborations (for example between us consultants and data scientists at our customers) where a server setup might not be practical, to work efficiently, using featureform as a single place to query out data for experimentation.
Some feature transformations we are adding can be quite heavy though, for example if running CNN-based computer vision, to extract feature from images.
We could of course do that as a preprocessing step, but it would just be very practical to include even those transformations inside featureform, especially for being able to keep track of the full dependency network of how features were originally created.
We have so far solved this with some manual disk caching code inside the transformation, but it would be nice to not need to think about those aspects in the transformations.
Beta Was this translation helpful? Give feedback.
All reactions