Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IKV integration - new online store for feature serving - docs.inlined.io #1369

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

pushkarmoi
Copy link
Contributor

@pushkarmoi pushkarmoi commented Mar 3, 2024

Description

This PR adds https://github.com/inlinedio/ikv-store as a new online store for feature form.
IKV client library: go get github.com/inlinedio/ikv-store@latest

OnlineStore and OnlineStoreTable interfaces have been implemented ie. Create/Get/DeleteTable and then Set/Get operations on individual tables.

Pending (next PR): docs/markdown changes to add IKV here: https://docs.featureform.com/inference-online-stores

New Online Store plugin

No open issue

make test_online provider=ikvonline

--- PASS: TestOnlineStoreIKV (35.32s)
    --- PASS: TestOnlineStoreIKV/CreateGetTable_IKV_ONLINE (0.00s)
    --- PASS: TestOnlineStoreIKV/TableAlreadyExists_IKV_ONLINE (0.00s)
    --- PASS: TestOnlineStoreIKV/TableNotFound_IKV_ONLINE (0.00s)
    --- PASS: TestOnlineStoreIKV/SetGetEntity_IKV_ONLINE (10.07s)
    --- PASS: TestOnlineStoreIKV/EntityNotFound_IKV_ONLINE (0.00s)
    --- PASS: TestOnlineStoreIKV/MassTableWrite_IKV_ONLINE (13.29s)
    --- PASS: TestOnlineStoreIKV/TypeCasting_IKV_ONLINE (10.20s)

Type of change

New infrastructure provider integration (online store)

Does this correspond to an open issue?

No open issue

Select type(s) of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have fixed any merge conflicts

@anthonylasso
Copy link
Contributor

This is awesome @pushkarmoi, the dev team will give this a look soon and give you any feedback!

client/src/featureform/register.py Outdated Show resolved Hide resolved
provider/provider_config/ikv_config.go Outdated Show resolved Hide resolved
provider/ikv.go Outdated Show resolved Hide resolved
@anthonylasso
Copy link
Contributor

anthonylasso commented Mar 19, 2024

Solid stuff, giving the initial approve and I'll get with the team for the next merge step.

(shouldn't take long, just logistical stuff)

@pushkarmoi
Copy link
Contributor Author

Solid stuff, giving the initial approve and I'll get with the team for the next merge step.

(shouldn't take long, just logistical stuff)

No worries, thanks a lot for the review!

@pushkarmoi
Copy link
Contributor Author

pushkarmoi commented Mar 20, 2024

Solid stuff, giving the initial approve and I'll get with the team for the next merge step.

(shouldn't take long, just logistical stuff)

Some questions, @anthonylasso when you get a chance -
Also cc @sdreyer

  1. I noticed there is a hook to update configs of online-stores (?at runtime?)

    func (resource *providerResource) isValidConfigUpdate(configUpdate pc.SerializedConfig) (bool, error) {

    I didn't understand it fully - and have not allowed any config changes for IKV. What are the implications?

  2. The register method for IKV has the following signature. The user needs to specify a "mount point" on the local container/host where they spin up the provider - where all IKV data will be stored (its an embedded database). I'm unsure about which directory the end user can use for this purpose (considering this runs inside featureform docker)

ikv = ff.register_ikv(
            name="ikv-quickstart",
            store_name="ikv-store-name",
            account_id="ikv-account-id",
            account_passkey="ikv-account-passkey",
            mount_dir="/path/to/mount_dir",
        )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants