Database DDL & Migration Support for plugins

[surapuramakhil]

Is your feature request related to a problem? Please describe

Apache Answer should support a way for plugin devs to have DDL and migration specific to their plugins / code.

There are plenty of use cases for this.
Most basic use cases is their tables would act as meta for their plugins.

Describe the solution you'd like

Apache Answer should support a way for plugin devs to have DDL and migration specific to their plugins / code.

Describe alternatives you've considered

Alternative is plugin having its own database. This idea sucks when you have use cases of joins between answer db to plugin db.

[LinkinStars]

@surapuramakhil Currently, the plugin does not have access to the database. All data is sourced from the answer itself, including the plugin's configuration information. Therefore, when upgrading, the answer will automatically handle any database changes. If there is a situation where the plugin needs to access the database in the future, it may be necessary for the plugin itself to handle the changes brought about by the version upgrade. Of course, how to design a convenient upgrade functionality for the plugin can be considered in the future.

[surapuramakhil]

@LinkinStars share ETA of future. You can at least share minimum time frame answer will be strict NO.

Why future is not today?

[LinkinStars]

@LinkinStars share ETA of future.

Why future is not today?

@surapuramakhil Currently, the plugin does not have access to the database. All data is sourced from the answer itself, including the plugin's configuration information.

[surapuramakhil]

@LinkinStars I think my question was conveyed wrongly. My question is "why are planning this for future but not today".
What I mean by this is "Plugin having DDL and Migration support / capabilities"

Understand, Even converse is true. There is no plugin today because answer doesn't support that.

Here are list of plugins that can be created, Just as Examples

• very basic Plugin is for having some additional content. AI copilot for moderators, AI Copilot for users etc
• A plugin for exposing some additional stats etc. where you store aggregated results for faster fetch

Existing search plugins also come under this category, but as of today the unique value prop of those is underlying search Databases.

Without need, is request would haven't been raised in first place.

[surapuramakhil]

All data is sourced from the answer itself, including the plugin's configuration information.

This causes additional work on answer wrt to plugin needs. That's completely unwanted and avoidable.

[LinkinStars]

Sorry, maybe I misunderstood your requirements...

My previous understanding is that you want the plugin to have the ability to upgrade database fields. In the context of Answer, migration specifically refers to the steps involved in upgrading from version A to version B.

In the context of SQL, data definition or data description language (DDL) is a syntax for creating and modifying database objects such as tables, indices, ...

Firstly, you want the plugin to have the ability to manipulate database table structures, such as modifying existing table structures, right?

Secondly, you want the plugin to have the ability to migrate data in order to obtain data from the Answer itself, right?

May continue providing further details on the capabilities that should be included. For example, specify the specific abilities the plugin should have and the type of data it needs to access. For example, want the plugin to be able to access data related to question-and-answer content.

[surapuramakhil]

@LinkinStars Not exactly true. I will update Suggestion solution in feature card.

Firstly, you want the plugin to have the ability to manipulate database table structures, such as modifying existing table structures, right?

Right words is plugin should have support creating its own tables, extending existing tables (joins are expensive), Adding Indexes (for performance optimization wrt plugin use cases). I support you wrt modifying existing tables
There is a difference between extending tables (adding new column) and modification (rename and dropping column).

When it comes to data Access - (Migration tool ideally should take care of assigning ownership, like who created that able), ownership of columns (who added column)
Rights for owners module would be different to others. Modules which has ownership - has full control and other modules will only by having read-only access.

Examples of modules - incubator-answer, google-connector, basic-connector, Slack Notification ..... etc

Building everything in one go would be tough, complicated and uncertainly do exits. For now, plugins having their migration sequence https://github.com/apache/incubator-answer/blob/main/internal/migrations/migrations.go and DB access would be a starting point.