Create a HTTP server on top of struct_db

[elliot14A]

Similar to SurrealDB and IndraDB create a HTTP server. If you need this feature please let me know I'll start working on it.

[vincent-herlemont]

Hi @elliot14A,

Thank you so much for your interest in the project and for proposing this feature. Initially, I hadn't considered creating an HTTP server as a primary feature, but upon reflection, I believe it could add significant value and versatility to struct_db!

I'm totally on board about the potential this brings to the project and for you to start working on this.

Before you start, could you share some more details? Do you have any specific server functionalities in mind? Are there any particular Rust libraries that you'd like to use? Also, do you have a specific design and code organization structure that you're thinking about?

Any insight would be incredibly beneficial as it will help to align our visions and ensure a seamless integration of the feature with the existing codebase.

Looking forward to your response, and thanks again for your initiative!

[elliot14A]

Hi @vincent-herlemont,

Thanks for the positive response.

Regarding your questions, I think it would be great to have basic CRUD (Create, Read, Update, Delete) operations through HTTP endpoints, as well as the ability to query the database through URL parameters for the start, just as the implementation of SurrealDB. I think IndraDB which is a GraphDB uses gRPC. I'm open to both implementations.

Coming to the architecture I really like ports and adapters architecture I think we should follow this design pattern and code organization structure. I think axum is great choice for http server. Would love to listen to your opinion on these approach I have mentioned.

[vincent-herlemont]

@elliot14A I agree, starting with a simple HTTP server allowing CREATE/READ operations would be a good first iteration to validate the prototype.

My main concern would be the separation of the server and database code. I suggest creating a sub-separate crate which could be optionally enabled as a feature and added as a member of the workspace members = ["struct_db_macro", "<the HTTP server crate>"], what is your opinion about that?

Another aspect to consider is that SurrealDB and IndraDB are not embedded databases by default; they are designed to be used as separate servers. Here, we are dealing with an entirely different paradigm. As struct_db uses Rust types as its schema, it can only be embedded within a Rust program. It will be impossible to dynamically change the shape of the data as on SurrealDB and IndraDB. The purpose of adding a server is merely to help the Rust program using struct_db to communicate with the outside world. I'd like to hear your thoughts on this.

Regarding the HTTP server library, I also agree on using axum.

[elliot14A]

@vincent-herlemont I understand now, even I thought of creating a separate crate for the http server implementation as well. So, on a higher level struct_db is imported into a Rust program then they create structs implementing struct_db macro. Then we create a http server on top this structs/tables .

POST api/tables/{table_name} to insert into the table
GET api/tables/{table_name} to fetch from the table
we can also have an endpoint to fetch schemas of the table
GET api/schemas/{table_name}`

is this fine?

[vincent-herlemont]

@elliot14A Yes, your approach looks sound to me!

You're welcome to create the PR when you're ready. In the meantime, I'm setting up the continuous integration (CI) to ensure that we maintain a high level of code quality throughout the project.

[elliot14A]

Hey @vincent-herlemont, I have started working the implementation. But, I have faced an issue where I need your advise.
So when get a json body from post request to insert it into one of the defined tables/structs how should I deserialise the json data based on table_name which I can extract from path parameter(api/tables/{table_name}). I'm working on this but I want to listen to the approach you might follow.

[elliot14A]

here is an more elaborate explanation:

#[derive(Serialize, Deserialize, Debug, Clone)]
#[struct_db::struct_db(fn_primary_key(p_key))]
struct Item {
    id: i32,
    list: Vec<String>,
}

#[derive(Serialize, Deserialize, Debug, Clone)]
#[struct_db::struct_db(fn_primary_key(p_key))]
struct Collection {
    id: i32,
    items: Vec<Item>,
}

impl Collection {
    pub fn p_key(&self) -> Vec<u8> {
        self.id.to_be_bytes().to_vec()
    }
}

here we have two structs collection and item.
let's say we get a json data of body:

{
  "id": 2,
  "items": [{
    "id": 2,
    "name": ["Item 2"]
  }]
}

so how should I serialise this json data from the request just by knowing the name of the table. Is there by which I can create the struct instance from the name of the struct.

[vincent-herlemont]

@elliot14A Unfortunately, the approach you're trying to take with Collection containing nested Items isn't quite right in the context of struct_db. It's important to remember that struct_db is essentially an overlay that provides some extra functionalities on top of a key-value database. It is not, in any case, a relational database.

If you want to establish a relationship between Item and Collection, you would have to do this manually. Instead of nesting Items in Collection, you can maintain a list of item IDs within Collection like so:

#[derive(Serialize, Deserialize, Debug, Clone)]
#[struct_db::struct_db(fn_primary_key(p_key))]
struct Collection {
    id: i32,
    item_ids: Vec<i32>,
}

This will allow you to create a relationship between Item and Collection, but this has to be managed manually.

As a tip, you can use the fn struct_db_schema() -> struct_db::Schema function to retrieve the schema. This may provide some of the information you need. If you run into any issues, feel free to open a PR and I'll be glad to assist.