Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scientific Metadata Search Engine client service using TypeSense #697

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Kezzsim
Copy link
Contributor

@Kezzsim Kezzsim commented Mar 22, 2024

We can successfully instantiate and connect to a new Typesense instance with this .yml file and the command tiled serve config config.yml

authentication:
  # The default is false. Set to true to enable any HTTP client that can
  # connect to _read_. An API key is still required to write.
  allow_anonymous_access: false
  single_user_api_key: "secret"  # for dev
trees:
  - path: /
    tree: catalog
    args:
      uri: "sqlite+aiosqlite:///:memory:"
      # or, uri: "sqlite+aiosqlite:////catalog.db"
      # or, "postgresql+asyncpg://..."
      writable_storage: "tmp/"
      init_if_not_exists: true
      typesense_client:
        api_key: "secret"
        schemas:
        nodes:
          - host: "http://localhost"
            port: 8108
            protocol: "http"

However there is still a lot of specific work to be done, namely items mentioned in #605 :

  • Set up Typesense client dependencies
  • Instantiate a client and have it connect with a typesense server
  • Create a bluesky event model reader that generates a standard base schema
  • Save schemas to files that are extensible and include-able by name so the entire schema does not have to be explicitly set each time.
    • (e.g. you could have a schema file for each beamline at NSLS-II with some of their metadata shape imparted)
  • Hook into Context.__init__
    • Initialize Typesense instance by iterating over all data in the tiled instance
    • Upsert search engine data on database insertion by registering [after_insert] sqlalchemy hook

@Kezzsim Kezzsim added the smse Scientific Metadata Search Engine, everything pertaining to natural language search label Mar 22, 2024
@Kezzsim Kezzsim self-assigned this Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
smse Scientific Metadata Search Engine, everything pertaining to natural language search
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant