Plugins architecture proposal

[ashish173]

With the basic e-commerce platform released we are set to start working on supporting plugins for aviacommerce.

Plugins for aviacommerce would work similar to how magento and shopify plugins work. A seller can install these apps in their online store and would be able to use the features of this plugin. For eg. If a seller installs MailChimp plugin/application then they would get following functionalities with it broadly categorised in 2 categoreis:-

1. A new tab/view in the admin dashboard with a custom option for MailChimp like a list of all abandoned carts by customers.
2. Ability to push data to the MailChimp servers for eg. when a new user signs up for registering the user at mailchimp's end.

The platform would be powered by at least 200 to 300 plugins. Every seller will integrate an almost unique combination of plugins. From this we can infer a few things about the behaviour of plugins:-

1. We can't have all the plugins already installed the application and activate them based on the sellers choice by including the package name in mix.exs file.
2. Even if we inject code using the approach mentioned here we still have to include the plugin name in the project.

So, the need is to have the plugins(and their associated code) included individually for each seller. Aviacommerce is a platform supporting multi-tenancy.

The problem ahead of us is to come up with an approach to achieve this dynamic nature of code injection for each individual seller using the plugins(service specific).

Approach 1

One of the approach we discussed initially was a way to highjack the control flow. For the sake of discussing this approach lets take a basic requirement of sending an event trigger for product view to a marketing service. A naive implementation would be triggering an async job/worker to send the event. Code for this would go in the products controller show method if we don’t have support for plugins in place. However, If we have plugins support then we’d have a plugin with code(service specific). On activation this plugin code would highjack the product controller show method , trigger the async code and return control to the platform products controller show method.

This is a very basic use case and does not provide solution for exposing views from the plugins.

Approach 2

...

Plugins usage

Sellers should be able to use the user interface to activate plugins to integrate third party services. These activations happen in realtime.

For the sake of POC we can just implement the plugins for services which just send a trigger to a third party service. For eg. sending orders list to sendgrid, sending events to segment.com and similar user cases.

Ejabbered(Erlang project) has implemented a plugin approach similar to what we want in the system.

[ramansah]

Principle

The plugin architecture can be divided into two logical problem statements.

1. How to dynamically install packages in the main app
2. How to interface the plugin module with the main app

I have a proof of concept for the second problem as of now. I am currently researching a solution for problem 1 and update accordingly.

Theory

A phoenix application is basically a combination of plugs. If you inspect endpoint.ex in a Phoenix app, you will notice that on hitting any API, a bunch of plugs are executed and at the last, MyApp.Router is called. At that point, the Router has rendered the view, set response metadata and performed all the required activities with the conn object.

Therefore, on accessing conn object at that time we have access to:

1. Request path + method
2. Request + Response body
3. Request + Response headers
4. Conn.assigns map etc.

On intercepting this conn object all "hook" based actions can be called independently.

Plugin Requirements

Every plugin is required to have a method matching this signature in the root level module for supporting all possible action based hooks.

MyPlugin.run(conn, repo)

To show a new layout in Admin dashboard, every plugin must also contain controller module with index action to supply data and supporting view module.

MyPluginWeb.DashboardController.index(conn, params, repo)
MyPluginWeb.DashboardView

A sample plugin is uploaded here
https://github.com/ramansah/plugin_sample

Implementation

endpoint.ex

 plug(AdminAppWeb.Router)
 # Add this line
 plug(Snitch.Core.Tools.PluginHandler.Postprocessor)

postprocessor.ex

  def call(conn, opts) do
    # Assume a method returns list of plugins' module names from DB for the tenant
    enabled_plugins = ["Mailchimp"]
    Enum.map(enabled_plugins, fn x ->
      apply(String.to_existing_atom("Elixir." <> x), :run, [conn, Repo])
    end)
  end

mailchimp.ex (external plugin)

  def run(%{ request_path: "/session",
              method: "POST",
              params: params,
              status: 302,
              resp_headers: resp_headers } = conn, repo) do
    %{ "session" => %{ "email" => email } } = params
    conn
    |> Plug.Conn.get_resp_header("location")
    |> handle_login(email, repo)
    conn
  end
    
  # Bypass other actions
  def run(conn, _), do: conn

  # Failed login attempt
  def handle_login(["/session/new"], email, repo) do
    insert_record(email, false, repo)
  end

  # Successful login attempt
  def handle_login(_, email, repo) do
    insert_record(email, true, repo)
  end

  def insert_record(email, is_successful, repo) do
    %{email: email, is_successful: is_successful}
    |> LoginAttempt.changeset()
    |> repo.insert()
  end

New Section in Admin

An additional plug will be called before calling Router which will set the active plugins for the view.

 # Add this line
 plug(Snitch.Core.Tools.PluginHandler.Preprocessor)
 plug(AdminAppWeb.Router)
 plug(Snitch.Core.Tools.PluginHandler.Postprocessor)

preprocessor.ex

  def call(conn, opts) do
    # This will be fetched from DB
    links = [
      {"Mailchimp", "/plugin/mailchimp"}
    ]
    assign(conn, :plugin_links, links)
  end

This will show the links for plugins in dashboard

On clicking that link, this controller action will be invoked

router.ex

get("/plugin/:plugin_name", DashboardController, :plugin_index)

dashboard_controller.ex

  def plugin_index(conn, params) do
    # The plugin name wil be taken from url params and converted into module
    conn
    |> MailchimpWeb.DashboardController.index(params, Repo)
    |> put_view(String.to_existing_atom("Elixir." <> "Mailchimp" <> "Web.DashboardView"))
    |> render("index.html", %{})
  end

The plugin is responsible for sending data & providing layout.

mailchimp_controller.ex

def index(conn, params, repo) do
    data = %{
      attempts: repo.all(LoginAttempt)
    }
    assign(conn, :mailchimp_data, data)
  end

index.html.eex

<tbody>
  <%= for attempt <- @conn.assigns.mailchimp_data.attempts do %>
    <tr>
      <td><%= attempt.email %></td>
      <td><%= attempt.is_successful %></td>
      <td><%= attempt.inserted_at %></td>
    </tr>
  <% end %>
</tbody>