Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEEDBACK] LabSDK: Use a similar bucketing mechanism for windowed-features #77

Open
AlmogBaku opened this issue Jun 14, 2022 · 2 comments
Labels
component/labsdk Related to the LabSDK enhancement New feature or request roadmap This issue is part of the roadmap

Comments

@AlmogBaku
Copy link
Member

AlmogBaku commented Jun 14, 2022

Background

Currently, feature values of a "windowed features" are calculated in a different mechanism than the way they're processed in the Natun operator:

  • In Natun - we're calculating "on-the-fly" buckets of these aggregations and then performing a final aggregation when building the FeatureSet
  • In LabSDK - we're calculating the rolling window based on the specified time we got,

Although the LabSDK has a much simpler approach and yields the same results, this is preventing us from pouring the historical data back to the Natun Operator and might yield inconsistencies.

What do you propose to do?

Calculate rolling features similar to the Natun implementation:

  1. First, calculate the rolling data into "buckets" (retrieve the bucket name from go)
  2. Then, calculate the "finalized" feature value only in the materialized view that being generated in fs.historical_get() is

What have you already tried?

No response

What else should we know?

We should set/get the bucket names via the go binding to prevent inconsistency.

@AlmogBaku AlmogBaku added the enhancement New feature or request label Jun 14, 2022
@AlmogBaku AlmogBaku added the component/labsdk Related to the LabSDK label Jun 22, 2022
@github-actions
Copy link
Contributor

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

@github-actions github-actions bot added the Stale label Aug 22, 2022
@AlmogBaku AlmogBaku removed the Stale label Aug 28, 2022
@github-actions
Copy link
Contributor

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

@github-actions github-actions bot added the stale This issue is stale label Oct 28, 2022
@AlmogBaku AlmogBaku removed the stale This issue is stale label Nov 13, 2022
@AlmogBaku AlmogBaku reopened this Nov 13, 2022
@AlmogBaku AlmogBaku added the roadmap This issue is part of the roadmap label Jan 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/labsdk Related to the LabSDK enhancement New feature or request roadmap This issue is part of the roadmap
Projects
None yet
Development

No branches or pull requests

1 participant