how to configure to use the standard ES6 Promise?

[c0b]

when running with node v6, is there a way to configure to use the standard ES6 Promise only (get rid of bluebird dependency)?

for example in promise-queue package, it defaults to use whatever is globally available Promise,

https://www.npmjs.com/package/promise-queue

or user can explicitly configure it by Queue.configure(require('vow').Promise);

So can this package implement a similar strategy?

[johanneslumpe]

Curious: you want to replace a faster library with a slower built-in? On the client it wouldn't matter but in node speed is a factor. What's your reasoning for this?

[elhigu]

@johanneslumpe It is factor in some applications, with knex I highly doubt that used Promise library has any significant effect to performance. It has been discussed that we should write all Promise code to use only A+ APIs so that bluebird wouldn't be needed.

After that it should be easy to override, which Promise library is used.

[wubzz]

I agree it feels pointless in a library such as knex. Given that knex's codebase already imports its own promise.js file internally, it would technically still be very easy to implement, provided the API is the same (which I'm not sure it is?). In this file one could default to global promises, else require configured library, or something like that.

[c0b]

it's all about giving users a choice; for Node V6+ users, give a choice of managing 3rd party dependencies as minimum as possible.

Curious: you want to replace a faster library with a slower built-in?

what you're saying might be true in the past, but how about now or 6 months later? (I did some search and can only find out some 2015 benchmarks, if you have more recent comparison, please post some links)
I believe the whole point of making Promise the ES6 standard is for people to easily use Promise, not having to rely on a 3rd party library, and Node or V8's core team can't be blind of the performance difference for ever, as long as the two projects' open source licenses are compatible, they can even borrow some code; or just give them some time I believe the builtin Promise can be faster and better.

[c0b]

see Amazon's AWS-SDK: it also default to use whatever Promise is globally available; while give user a choice to configure the favorite Promise library as well

http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-making-requests.html#Support_for_Promises

it's all about choices

[wubzz]

On second thought, it won't be as easy as changing a single file. Knex currently relies a lot on Bluebird utility functions such as .timeout, .tap, .reduce, .map and so on, which im assuming and expecting not to exist in ES6 Promises.

[rhys-vdw]

I am interested in supporting ES6 promises. Ideally we'd require a second argument to the Knex constructor that takes a Promise constructor. Back compatibility would be achieved like so:

const Promise = require('bluebird');
const knex = Knex(myKnexConfig, Promise);

Perhaps we could conditionally alias map, filter etc. based on their presence on supplied Promise.prototype?

[tgriesser]

I think this should be pretty low on the priority list, and it'd require quite a bit of internal changes, not to mention the decrease in performance (though admittedly haven't seen benchmarks in awhile) and the fact that folks might be relying on the fact that the returned promises is a bluebird (for conditional catch, etc).

I'd be more inclined to wait until async/await lands in a stable node to look at addressing this.

[c0b]

for people want the minimum 3rd party dependencies, the native Promise can be better,

like @google-cloud and many other libraries, you can make it default to use native Promise and accept a promise parameter for whom want to use 3rd party libraries

https://googlecloudplatform.github.io/google-cloud-node/#/docs/google-cloud/

var gcloud = require('google-cloud')({
  promise: require('bluebird')
});

[malexdev]

@tgriesser I know this is an old issue, but we've now got async/await in both stable and Carbon.

Given that it is preferred for async/await to work with native promises (per the spec, async functions must return a native promise) is this any higher priority?

If native promise support is something that Knex would like to have happen, but it's not currently on the radar, would a PR be welcome?

Thanks for your time.

[elhigu]

@malexdev async functions do return native promises it doesn't have much to do with knex currently. It doesn't mean that await needs native promises to work properly. Could you clarify in which case this is a problem / benefit (except of dropping one dependency)?

That being said I'm not against of dropping bluebird, but it really does need quite a bit internal changes. We might need to implement some methods that are currently exposed from bluebird to knex APIs, unless old functionality is hidden under some configuration switch and by default native promises would be used. This way migrating wouldn't be impossible to people who has been relaying on fact that knex returns bluebird promises.

[malexdev]

@elhigu The main benefit for me personally is that I'm using TypeScript with a TSLint rule enforcing native promises being used for await, so I have to wrap all Knex calls inside a Promise.resolve(). I realize this isn't anything to do with Knex specifically though, and is likely an issue unique to me.

Other than that in my opinion having less 3rd party dependencies is better, and as @c0b mentioned more options is never a bad thing.

I realize that it would be a lot of work, which is one reason I'm more than happy to spend time on this, if it's something that Knex is interested in moving towards.

[ericeslinger]

Yeah, I landed here coming from a typescript issue - I'm using Knex as the SQL engine for my multi-storage datastore library, and while i'm ambivalent vis-a-vis native vs bluebird promises, I can't easily use typescript on knex for this reason. I treat knex thenables as following the native spec (I don't use any of the bluebird extensions), but typescript bugs me about returning a Bluebird when the method declaration is a Promise.

This is kind of two levels deep here, as we're dealing with both the promise implementation and the typings for knex (which are handled by different devs), but I'm basically stuck here - I'm technically breaking the type contract by returning a Bluebird when I've declared a Promise (I guess there's stuff in the Promise api that Bluebird doesn't support?) but I really don't feel like putting a bunch of return Promise.resolve(bluebirdthing) everywhere either.

I've spent enough time digging around in knex guts for the past year and working with promises in general that I'd be willing to take something up here and work on a PR to modularize out the Promise implementation if people want - would you be open to a PR? It'd end up being something like what @elhigu mentioned - re-implementing some of the utility functions to use whatever Promise constructor was passed in at instantiation so to avoid code-rewriting needs. Not sure about performance, of course, but that's something that can be benchmarked.

Having it all done up fancy with async / await would be cool too, and I'm not in a hurry to have this fixed (ultimately for my use case I just end up flagging things as any and dealing with those codebranches as if they were javascript instead of typescript).

[elhigu]

@ericeslinger I don't see why real Promise implementation that is being used would cause problems with TypeScript, I've been mixing bluebird and native promises a year and half without any problems...

I just haven't been using any typings that would introduce types for bluebird, I just tell typescript typings that they are normal Promises and it doesn't see any difference (ofc. it will complain if I try to use bluebird's special methods).

Which typescript version and is there any example project to reproduce... e.g. github repo with npm start script.

This is kind of two levels deep here, as we're dealing with both the promise implementation and the typings for knex (which are handled by different devs), but I'm basically stuck here - I'm technically breaking the type contract by returning a Bluebird when I've declared a Promise (I guess there's stuff in the Promise api that Bluebird doesn't support?) but I really don't feel like putting a bunch of return Promise.resolve(bluebirdthing) everywhere either.

Are those knex typings from npm? Are bluebird typings from npm? Which packages? I'm having hard time to understand why that would happen. If there is separate Bluebird type it should be inherited from native promise and be ok to return from APIs that tells that they will return Promise. From the description it sounds that the problem is very broken typing implementations. And that is not relevant to this issue (typescript doesn't mind about real types, so it won't know what knex returns).

[ericeslinger]

I'm getting my typings from the DefinitelyTyped repo via installing @types/knex. That definition pulls with it @types/bluebird, and all the knex methods are typed as returning Bluebird objects.

As a specific thing, I cannot do this:

function mungeData(v: any): DataItem {} 
function foo(): Promise<DataItem[]> {
  return knex('data').select()
  .then((rows) => rows.map(row => mungeData(row)))
} // error, cannot return Bluebird<DataItem[]> as Promise<DataItem[]>

using these typings. This is because knex.select().then() is typed to return a Bluebird in the DefinitelyTyped repo, and those chain together to make more Bluebirds, and saying something like return foo as Promise<any>() when foo is a Bluebird will fail (at least it fails in typescript 2.4), because Bluebirds aren't assignable to Promises (they lack [Symbol.toStringTag] according to this, so coercing one to the other would be in error, albeit a small error).

Instead I can change to

function bar(): Promise<DataItem[]> {
  return Promise.resolve<DataItem[]>(foo())
}

or do other tricks to wrap all calls to knex inside a native Promise.resolve() call. This will cause typescript to stop complaining downstream of my library functions, while still allowing me to use knex typings inside my library functions.

Prior to yesterday, I hadn't used @types/knex at all - I was just typing knex as an any. The code works fine either way at runtime (for my use case at least), it's just

[malexdev]

@elhigu: The issue is not broken typing implementations.
TypeScript sets the type for async functions as Promise<[type]>, which is correct per JS spec.
Knex returns Bluebird<[type]>, which the typings accurately reflect.

I just haven't been using any typings that would introduce types for bluebird, I just tell typescript typings that they are normal Promises and it doesn't see any difference

This is lying to the compiler, as Knex functions do actually return Bluebirds. Not interested.
You're correct that Bluebirds are compatible with Promises, but part of the deal with TypeScript is that you actually return what you say you're returning.

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.
There are various tricks we can do (such as what @ericeslinger mentioned about using any, or wrapping in Promise.resolve()) but at the end of the day tricks like that make us lose out on much of what TypeScript provides.

At the end of the day, the reality is there are at least two users now who are saying "Using native promises is important to us, and we're willing to put in the work to make the promise functionality more generic".

I realize you're just trying to help, but frankly instead of hearing "you could do it this way" I'd like to hear whether the promise changes proposed by myself / @ericeslinger / @c0b are acceptable so that I can either start on a PR or what.

[elhigu]

@malexdev @ericeslinger Thanks for the more info! Looks like it actually is not possible inherit your own class from Promise so that might be the reason why returning Bluebirds from function that is typed to be Promise<> fails :(

@ericeslinger Anyways this is not a problem when you create asyncfunctions, since they automatically wraps results to native promises internally. Following complies without a problem, with typings from @types/bluebird and compiled to either ES2015 or ESNEXT.

import * as Bluebird from 'bluebird';

// declaring function async converts bluebird implicitly to native Promise
async function asyncReturningPromise(): Promise<string> {
    const blueBirdPromise = new Bluebird<string>((resolve, reject) => { 
        resolve('yay asyncReturningPromise');    
    });
    return blueBirdPromise;
}

// main func to run the code using async / await
Bluebird.resolve().then(async () => {
    console.log("await function returning promise (bluebird)", await asyncReturningPromise());

    const blueBird = new Bluebird((resolve, reject) => { resolve(); });
    const returnedFromAsync = asyncReturningPromise();

    console.log("Bluebird instanceof Promise:", blueBird instanceof Promise);
    console.log("async retval instanceof Promise:", returnedFromAsync instanceof Promise);
});

output:

await function returning promise (bluebird) yay asyncReturningPromise
Bluebird instanceof Promise: false
async retval instanceof Promise: true

So for now when you are using knex APIs for now you actually need to tell that you are returning Bluebird unless you are using async functions / methods, which wraps bluebird automatically to native Promises.

@malexdev

This is lying to the compiler, as Knex functions do actually return Bluebirds. Not interested.
You're correct that Bluebirds are compatible with Promises, but part of the deal with TypeScript is that you actually return what you say you're returning.

Actually deal with typescript is that it is enough that returned object implements the interface correctly, for example this is perfectly fine:

class FakePromise<T> implements Promise<T>  {
    [Symbol.toStringTag]: "Promise";
    then<TResult1, TResult2>(onfulfilled?: (value: T) => TResult1 | PromiseLike<TResult1> | null | undefined, onrejected?: (reason: any) => TResult2 | PromiseLike<TResult2> | null | undefined): Promise<TResult1 | TResult2> {
        return new Promise((resolve, reject) => { resolve('Im totally broken and fake!); });
    }
    catch<TResult>(onrejected?: (reason: any) => TResult | PromiseLike<TResult> | null | undefined): Promise<T | TResult> {
        throw new Error("Method not implemented.");
    }
}

// this works and  fake promise instance has nothing to do with native promise
function returningPromiseInterface(): Promise<string> {
    const fakePromise = new FakePromise<string>();
    return fakePromise;
}

// compiling this fails, because looks like Bluebird actually doesn't implement Promise interface correctly
function asyncReturningPromise(): Promise<string> {
    const blueBirdPromise = new Bluebird<string>((resolve, reject) => { 
        resolve('yay asyncReturningPromise');    
    });
    return blueBirdPromise;
}

It messes up instanceof, but typescript actually hasn't even promised that you returned native Promise instance, only interface.

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.
There are various tricks we can do (such as what @ericeslinger mentioned about using any, or wrapping in Promise.resolve()) but at the end of the day tricks like that make us lose out on much of what TypeScript provides.

I would hate to see people having to do that kind of tricks / changing JS implementations just to satisfy bad typings.

At the end of the day, the reality is there are at least two users now who are saying "Using native promises is important to us, and we're willing to put in the work to make the promise functionality more generic".

I realize you're just trying to help, but frankly instead of hearing "you could do it this way" I'd like to hear whether the promise changes proposed by myself / @ericeslinger / @c0b are acceptable so that I can either start on a PR or what.

Thanks for understanding :) Changing knex to use native promises was already started and implemented to some point last year then it was changed back by @tgriesser so I would say that for now it is better not to start this change.

Also I still consider these typescript problems mentioned in this thread as a problems in typings declaration (why bluebird doesn't implement Promise correctly... I need to dig in more deep on that?), than problems in knex implementation.

That being said, I'm not opposed of getting rid of bluebird in some timespan just seeing two separate issues here.

[sustained]

👍 for native Promises.

[malexdev]

@elhigu:

Actually deal with typescript is that it is enough that returned object implements the interface correctly

Fair enough. I still stand by my opinion that less dependencies and more choice is better, but I now see what you meant about broken typings.

So still 👍 for native Promises (which I'm still willing to help with), but I see now my immediate issue can be resolved by fixing Bluebird typings. Thanks for the info.

[tgriesser]

So I've started to use TypeScript quite a bit and I both love it and now realize the issues here are a real pain point. As async/await gains more of a foothold in recent Node land the Bluebird utility fns (map, reduce, tap, bind, return) become less useful. I'd be cool with continuing to use Bluebird internally but officially "deprecating" the public api of knex query builder returning all of the additional utility methods.

With that merged, we could then upgrade the TypeScript definitions to drop the Bluebird typings (except for toBluebird), and change the Bluebird typings to Promise typings.

If anyone has bandwidth wants to tackle this, here's what I'm thinking for a plan of action:

• Add a deprecation warning for all of the Bluebird proxied methods (tap, map, reduce, bind, return).
• Add a .toBluebird() method which will be a migration path for those who want to continue using Bluebird (they can do so with a simple find/replace of all of the calls of the above methods and just add it before those are called)
• Once this is merged/new release cut (0.15) we can update the Typescript definitions
• Eventually we can drop these methods entirely. This simpler API paves the way for eventually using native Promises & async/await when it makes sense.

Let me know if this makes sense and if anyone is interested in taking a shot at this.

[malexdev]

Would definitely be interested to help with this.

Does this mean Knex would start maintaining its own TypeScript definitions? Would allow us to do some cool stuff with generics that autogenerated typings will never support.

[danielgatis]

I started this fork as a first attempt to add support for native Promises:
https://github.com/tgriesser/knex/pull/2523/files

[insidewhy]

Like this comment from 2016:

Curious: you want to replace a faster library with a slower built-in

Amazing how much can change in 2 years.

[insidewhy]

When returning a Bluebird from a function that has been typed to return Promise, TypeScript complains because type Bluebird is not the same as type Promise.

@malexdev Actually typescript uses structural typing (flow uses nominal typing and would work the way you describe) so as long as your type fulfills the Promise interface it is compatible with Promise whether it explicitly extends/implements it or not.

[qubyte]

How's this progressing? I reckon a good first step would be to factor out Bluebird specific method calls within knex (i.e. don't actually remove it yet). Removing bluebird and providing an option for a custom Promise constructor would follow (and give folks using Bluebird methods an upgrade path).

I don't starting work on the first step if there are no objections. Existing work appears to have died down.

[kibertoad]

@qubyte I don't think there is an active effort to do the change, incremental changes were made here and there, but that's about it.

[qubyte]

Ok. In my next chunk of free time I'll make some small-as-possible changes to factor out each method.

[kibertoad]

@Bessonov Context? How bumping min node to 10 affects this issue? Note that we dropped node 6 support already.

[Bessonov]

I'm not familliar with knex code base, but maybe there are some features, which can help you to get bluebird away. For example, node 10 has a support for Promise.finally.

But anywa, I'm happy to see the progress on this topic 👍

[maximelkin]

About disposer pattern - could we just add optional callback for things, which returning disposable promise?
(Just as with transactions)

getDisposableConnection(config, cb) {
    const connection = await getConnection(config)

   // user want autodisposable connection
    if (cb) 
      Promise.resolve(cb(connection)).then(() => connection.dispose())
   // user will dispose by himself
   return connection
}

[maximelkin]

Which level of promise library independence we need?

1. all use native promises
2. internals native promises, user can set own promise lib for interface
3. user can set promise lib for internals and interface

[wtrocki]

What is the current state of this issue. Generally knex work now with async await but typescript will report warning that we awaiting method that is not native promise.

So to answer the original issue question. Current workaround is to simply await and add something like // tslint:disable-next-line: await-promise

[Bessonov]

@maximelkin I vote for the option 1. In long term I hope every promise library would be obsolete.

[chaffeqa]

id second that, at this point we are beyond promise polyfills even for the majority of browsers

[maximelkin]

@Bessonov currently on knex depends libraries (and maybe projects), which requires exactly bluebird

we should give some fallback solution for them

[elhigu]

@Bessonov currently on knex depends libraries (and maybe projects), which requires exactly bluebird we should give some fallback solution for them

It doesn't matter if knex's users are dependent on bluebird. Knex can still use native promises and they will interoperate just fine with bluebird promises. We should absolutely not give any fallbacks.

[jehy]

So this issue started with request for feature with choosing promise implementation.
Out of nowhere, it mutated in to removing bluebird for no reason, and breaking all dependents. Without any warning, changelog, option for fallback and major release.

But I suppose that all 1.5 typescript users are happy now.

[elhigu]

So this issue started with request for feature with choosing promise implementation.
Out of nowhere, it mutated in to removing bluebird for no reason, and breaking all dependents. Without any warning, changelog, option for fallback and major release.

At least earlier with knex 0.x versions has been considered as major releases with potentially breaking changes, so only updating to 0.20.x should have been considered safe upgrade (semver is really loose when version number < 1).

Removing bluebird has been on the table for a long time, it is not only about this issue.

removing bluebird for no reason

Removing bluebird has not been for no reason. You can still externally use bluebird with knex promises. One big reason to drop bluebird has been that async functions implicitly creates native promises, so in future keeping on using Bluebird would have required extra bluebird wrapping code to be added in knex API for no reason what so ever.

Without any warning, changelog,

Agreed. I hauled through latest changelogs... Sadly it really looks like we have failed to list breaking changes between versions. We need to be more careful when writing changelogs to really point out the changes, which breaks the old APIs. For example many of the typings changes actually will break old TS code.

[jehy]

There was the same issue on ioredis project redis/ioredis@da60b8b. They wanted to support native promises - and guys made a really good solution - they added an option to support any custom promise library and they use native promise by default. Why not? Setting a custom promise library is fast and doesn't require patching all application code.

keeping on using Bluebird would have required extra bluebird wrapping code to be added in knex API for no reason what so ever.

Yup. But why not wrap module calls in bluebird (or any other promise library) if it was explicitly specified? That's one simple wrapper, zero overhead, and it would allow users to use whatever promise library they want. If no one needs bluebird, no one would use this options, and you can safely deprecate it in time.

Also, I saw an opinion that

In long term I hope every promise library would be obsolete.

But IMHO there are two wrong assumptions:

• Bluebird is used because it's faster.
• Bluebird is used as a pollyfill.

I think that's not the case for really complex applications, which go beyond async-await one liners.
Bluebird has many features that are absolutely necessary for complex async flow - like timeouts, custom error handling, mapping with concurrency, cancellation, reducing and so on. All those features can be implemented in native promises, but that's a lot of useless boilerplate. In 2020, we are still using bluebird in Node 12 because we don't want all this boilerplate.

[qubyte]

Why not? Setting a custom promise library is fast and doesn't require patching all application code.

Anything which uses async-await internally is going to coerce promises to native promises, so your options become either wrapping the output of each method in the custom promise or banning async-await in internal code. It's not as small an undertaking as it might seem on first inspection.

[jehy]

@qubyte

It's not as small an undertaking as it might seem on first inspection.

No, that's just as simple as I already told. You make a wrapper for exported external functions and that's all. About 10 lines of code. And write all internal code in any way you want.

[briandamaged]

@jehy : Feel free to submit a PR for those 10 lines of code if you see a straightforward way to implement them.

I'll also spend some time today trying to come up w/ a work-around.

[qubyte]

For what it's worth, much of the API of bluebird is duplicated with the same or close API using native promises by these packages: https://github.com/sindresorhus/promise-fun

[jehy]

For what it's worth, much of the API of bluebird is duplicated with the same or close API using native promises by these packages

~50 packages instead of 1? Seriously?

[qubyte]

Yes, though most of the time only a few are required (p-map for example). Your mileage may vary of course. It's offered only as one potential route to what you want.

[briandamaged]

@jehy : Here is something you can try as a temporary work-around within your application code:

const Bluebird = require('bluebird');


const prototypesNeedingDecoration = [
  require('knex/lib/query/builder').prototype,
  require('knex/lib/schema/builder').prototype,
  require('knex/lib/transaction').prototype,
  require('knex/lib/raw').prototype,
];

const corePromiseMethods = ["then", "catch", "finally"];


function decoratePromiseMethods(target) {
  for(const m of corePromiseMethods) {
    const original = target[m];

    target[m] = function(...args) {
      return Bluebird.resolve(original.apply(this, args))
    }
  }  
}

function hackBluebird() {
  for(const target of prototypesNeedingDecoration) {
    decoratePromiseMethods(target);
  }
}


hackBluebird();

This is not really an adequate solution to the overall problem. There are other temporary objects created within knex that would need to be decorated in a similar manner.

[briandamaged]

Also, disclaimer: the work-around ☝️ has had very little testing. So, you should re-run your application's tests to ensure that nothing has broken.

[chaffeqa]

Just wanted to add my 2 cents here: i really appreciate all the work put into this migration, regardless of the negative feedback.

From our app's perspective, knex was the last library forcing us to require Bluebird, and conforming to full native promise support means that:

1. we no longer have sullied stack traces
2. we reduced our SSR weight by a decent amount
3. we improved perf since native async await is now more performant than bluebird (and growing more and more!)

its such a huge win to continue to run towards the es standard... and I know that isnt easy for library maintainers so i wanted to shout out to you guys and thank you for taking on such a burden!

for those suffering from the change: I'd love to help since we have benefitted, so please reach out if you need help debugging or migrating!

[kibertoad]

@chaffeqa Thank you for this feedback, it means a lot!

[briandamaged]

@jehy : Have you had a chance to try to work-around that was proposed? If so, did it resolve your immediate issues?

[jpike88]

Is this effort dead? I'm struggling trying to make sense of our server's errors and it's a real problem for us.

[kibertoad]

@jpike88 Latest knex version does not use Blubird anymore, native promises are used for everything.