Make Feathers server framework independent to work with Express, Koa and Hapi

[daffl]

Now that Feathers works library independent on the client already (see #193), we don't really need Express as hard dependency for the server anymore either. Instead, we can create separate modules for Express, Koa and potentially Hapi that can be configured just like any other plugin:

Express

The only thing to change upgrading from a Feathers 2 application would be to app.configure(express()):

const feathers = require('feathers');
const express = require('feathers-express');

const app = feathers()
  // Make this app Express compatible
  .configure(express())
  // Configure REST API that uses Express
  .configure(express.rest());

// Use any Express middleware
app.use(bodyParser.json());
// Use Feathers services normally
app.use('/todos', {
  get(id) {
    return Promise.resolve({ id, description: `You have to do ${id}!` });
  }
});

Koa

Koa support came up several times (see #83 and #58). This can now be done very similar:

const feathers = require('feathers');
const koa = require('feathers-koa');

const app = feathers()
  // Make this app Koa compatible
  .configure(koa())
  // Configure Koa REST handler
  .configure(koa.rest());

// Use normal Koa middleware
app.use(function *(){
  this.body = 'Hello World';
});

// Use a Feathers service through app.service
app.service('/todos', {
  get(id) {
    return Promise.resolve({ id, description: `You have to do ${id}!` });
  }
});

[ekryski]

The biggest barrier to this I think will be authentication, specifically how we get access to the request and response object and how the passport middleware works with Koa, etc.

At a quick glance it doesn't look too bad with Koa. We'd just need to use https://github.com/rkusa/koa-passport-example and ctx instead of req and res. With Hapi I'm not so sure but I'm not convinced there is a lot of value in supporting Hapi, it doesn't really provide anything different than Express. At least with Koa you have generator support.

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

[ekryski]

For socket support with Koa we use this for inspiration: https://github.com/koajs/koa.io.

[ahdinosaur]

IMHO, if we are looking to be more flexible and not tied to a framework we're probably better just using standalone modules for routing, content negotiation, etc.

http-framework! 😉

[ekryski]

@ahdinosaur ya that looks like more the right way to go IMHO.

[jeffijoe]

I think we should make Feathers a library instead of a framework - that would make it more independent on the transport as well.

Examples:

Common code

const feathersApp = feathers().configure(rest());
feathersApp.service('todos', new NeDB('todos'));
export default feathersApp;

Framework-specific

Koa

import feathersApp from './feathersApp';
import Koa from 'koa';
import adapter from 'feathers-koa';

const app = new Koa();
app.use(adapter(feathersApp));

Express

import feathersApp from './feathersApp';
import express from 'express';
import adapter from 'feathers-express';

const app = express();
app.use(adapter(feathersApp));

Basically, the adapters will create the middleware for their particular framework.

[daffl]

Note: https://github.com/spirit-js/spirit

[rickmed]

@daffl I just checked spirit and it looks very neat. It would be amazing to decouple feathers from express. With the up rise of so many new implementations different from express it would make feathers more robust in the future. Any decisions on this?

[ekryski]

This Authentication PR feathersjs-ecosystem/authentication#336 should be the only major piece we need in order to be able to support to different frameworks underneath.

[ekryski]

I've been giving this a lot more thought over the last few days and now that we are getting close to winding up Auk this is going to be the major goal for the Buzzard release. As nice as it is to be modular looking at usage numbers Express isn't going anywhere. It has 70 million downloads this last year, Koa is right up there with 1.4 million and Hapi with ~2.4 million.

I think these numbers can be artificially inflated by build systems, deploy frequency, size of deployments, etc. but this gives a general idea of how popular things are.

Being completely honest, looking at the numbers there really isn't a lot of incentive to support anything but express. The major reasons I see would be:

• http2 support (which looks to be coming in express5 eventually....)
• reducing dependencies. Being able to use raw node http(s) or something more minimal when you are building a stateless API or microservice (primary feathers use case)
• being future proof 👍
• becoming more modular, so that you can swap your router, template engines, etc. Feathers really is just an architectural pattern + utility lib on top of core tech.

In my mind the first "engine" to support other than Express would be Koa. It's the most similar in design to Express and provides nice support for future ES6/ES7 language functions. It also seems to be our most requested. Personally I'd rather have support for raw node libs but that might be a lot of work.

What needs to happen

• Identify the common top level Feathers API. I don't think we can identify this fully until we try at least one other engine underneath. However I'd ❤️ for it to be as easy as:

  const feathers = require('feathers');
  const app = feathers();
  
  // use by string name
  app.engine('express');
  
  // or pass the engine with string
  const koa = require('koa');
  app.engine('koa', koa);
  
  // or simply pass the engine. I like this best
  const koa = require('koa');
  app.engine(koa);

  This is conceptually similar to what @jeffijoe was suggesting however, I'd prefer to have people feel that they are interacting with a Feathers app directly. I think it will make for a cleaner API. However, that being said it's a bit early to tell as we might then have to shim a ton of methods. Further investigation will be required before we can determine the top level API.

• Make sure all express methods that are used are either polyfilled, aliased or we remove dependency on them. There may be more but the ones I can think of off the top of my head are:

  • app.get
  • app.set
  • app.render
  • app.send
  • app.json
  • req.accepts
  • req.xhr

• Make auth engine/transport agnostic (1.0 Pre-release feathersjs-ecosystem/authentication#336)

• Alias/alter how we register REST routes.

• Alias/alter how we set up sockets

There might be more things. @daffl would have a better idea, specifically around socket/rest setup.

Other Considerations

• Are we going to support all of Express's HTTP verb methods? There are a lot. Wouldn't that mean implementing a lot of Express?
• What Express specific things are we currently relying on?
• What helper methods on the app object do you want to keep as part of the core Feathers API.
• What is Express 5 looking like for proposal? Been a while since I looked at it. Would be good to connect with @dougwilson and see if we can lend a hand (if it makes sense, there might already be enough cooks in that kitchen).
• Are there modules like what @dougwilson was working on or spiritjs, http-framework, etc. that would give us the modularity without having to rewrite those abstractions over top of core node functionality.

[Download]

// or simply pass the engine. I like this best
const koa = require('koa');
app.engine(koa);

Agreed! This way, people can learn Feathers once and deploy on any server that has an adapter. Great idea!

[jeffijoe]

The challenge lies in converting libraries that rely on connection-specific things like headers.

About the popularity contest, my primary reason for wanting to use Koa is not because it's popular (not as much as express), but because it's more stable in terms of handling middleware errors.

[kristianmandrup]

Please let's move Feathers to an architecture more suitable to a thin API gateway (classic web server) and stupid/simple web services that can be deployed stand-alone and are protocol independant, just listening to messages of interest (ie. best practice Micro Service pattern). Then we can start to integrate smoothly with Seneca and other popular Node.js Micro Services frameworks.

And yes, FeathersJS should be agnostic to Express, Koa, Hapi, whatever...
I'd love to see it on Nginx with HTTP2/Push as well :)

Happy days!

[andreafalzetti]

Have you guys seen this https://github.com/fastify/fastify?

I'd love to use it with FeathersJS, what's the status of this issue?

[marshallswain]

@andreafalzetti still moving forward. You can see some progress going on here: feathersjs-ecosystem/express#3

[kristianmandrup]

Yeah, would be super sweet to integrate feathers with fastify! Let's do it :)

[daffl]

The basic integration should be fairly straightforward, however, it's authentication (specifically passport and oAuth) where things get hairy.

Our plan was to remove the hard dependency on Express and after v3 investigate what integrations make sense. I saw a talk on Fastify last week and while it was interesting, it might make even more sense for Feathers to just use Nodes HTTP (and HTTP2!) with the router Fastify is using as the main integration.

[kristianmandrup]

FYI, I started work on feathers-koa REST integration in feathers-rest-koa

I think it would make sense to extract the REST client into a separate module/package and repo ;)

[daffl]

The clients already are at https://github.com/feathersjs/feathers-rest-client also related: feathersjs-ecosystem/express#3 by @christopherjbaker

[bitflower]

As a newbie to Feathers: By 2018 is Feathers completely independent from Express?

EDIT: Or in other words: Which other frameworks are supported. Is KOA fully supported?

Thanks! Love the framework and thanks for the hard work!

[kristianmandrup]

Ask @daffl, he's been working on it... Not sure about the current state of affairs though.

[daffl]

Feathers is framework independent (as in you can e.g. only use it with @feathersjs/socketio or as a standalone client to talk to other services) but it only has HTTP API bindings for Express (in @feathersjs/express).

Since the whole point of Feathers is abstracting the protocol specific things, the HTTP framework it is using ultimately shouldn't really matter that much and abstracting things like authentication away from Express is a pretty big task (all of Passport is built for Express and even the current Koa integrations just hack around that fact by mucking around with its request object to make it look like Express). Highest priority for new framework bindings for me would be plain Node HTTP which with a new service lookup mechanism would yield similar performance to Fastify and make websocket connections even faster.

[lock]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue with a link to this issue for related bugs.