Should load MySQL, tries to load mssql

[NicoAiko]

Environment

Knex version: 0.15.2
Database + version: MariaDB v10.3
OS: macOS Mojave (v10.14)
Typescript: v3.1.1

Bug

Expected behaviour:
Should load the MySQL dialect.

Actual behaviour
Tries to load the mssql dialect.

Error Message:
"Module not found: Can't resolve 'mssql/package.json' in 'XYZ/node_modules/knex/lib/dialects/mssql'

My code:

import * as Knex from 'knex';

const knex = Knex({
  client: 'mysql',
  connection: {
    database: process.env.DB_NAME,
    host: process.env.DB_HOST,
    password: process.env.DB_PASS,
    user: process.env.DB_USER,
  },
});


export default knex;

[kibertoad]

That's highly improbable, given that code for resolving dialect is

Dialect = require(`./dialects/${CLIENT_ALIASES[clientName] ||
      clientName}/index.js`);
  }

and there is no alias for mssql.
Please double-check your code, e. g. do mass-search of a word "mssql", I'm pretty sure it is referenced somewhere, e. g. inside a knexfile.

[NicoAiko]

No "mssql" found, sorry. I also didn't alter anything. The project is pretty small and the only database I define is in this very file.

[kibertoad]

@NicoAiko Could you upload entire project in a zipfile somewhere? I'm pretty sure this has to work.

[NicoAiko]

I can make a repository for it, one sec

[NicoAiko]

Here's the repo: https://github.com/NicoAiko/typescript-knex-error

EDIT:
Just do
npm i
and then
npm start

[elhigu]

does it try to load mssql also if you try to use postgresql or sqlite? This could be related to a fact that knex monkeypatches mssql internally... maybe those requires are leaked by accident somewhere.

[NicoAiko]

Changed it to

const knex = Knex({
  client: 'sqlite3',
  connection: { filename: './db.sqlite' },
});

Same as before: Tries to load mssql.

[elhigu]

Does CRA bundle all the code somehow? That could be one reason why it fails (there would be other errors too from other dialects though)

[kibertoad]

It appears that transpilation is to blame, as error is not thrown from inside the knex. I think what happens is that you attempt to resolve all imports from within all dependencies, and since dialects attempt to import their drivers, that fails.

[NicoAiko]

So, what to do?

[NicoAiko]

The same also happens with sequelize, it seems. I try to load the mysql dialect and it wants to require pg-hstore which is for postgres :/

Is that only on my machine? What am I doing wrong?

[kibertoad]

@NicoAiko Considering that it seems to be failing during bundling phase, can you try disabling bundling and see if that helps?

[kibertoad]

I see that node_modules are excluded in TS configuration, so it's likely to be a Webpack thing.

[NicoAiko]

Oof... I have to look into that actually. I have no custom webpack config.

[kibertoad]

Yeah, so probably easiest way would be to eject config and see what's inside. Could be that default webpack config bundles way too aggressively, you may need additional excludes there.

[NicoAiko]

Would you be able to tell me what might be wrong? Because I don't really know.

[NicoAiko]

I have the webpack config on the master branch in my repo that I linked above.

[kibertoad]

@nickwhiteley Why are you bundling a backend dependency inside of a frontend application anyway, though?

[NicoAiko]

It's a test project on what I can do with typescript. It doesn't matter that much if there is the database stuff in the react-project. I just want to try things out.

[NicoAiko]

And I'm not Nick Whiteley :D

[kibertoad]

Sorry :D.
I would recommend you to start two separate projects, one for frontend React/Webpack, and another one for backend with Node.js/Knex.
Since currently you are trying to solve a problem that you are unlikely to ever have in real world.

[NicoAiko]

Seems about right ^^'
Though still unfortunate that this happens.

[kibertoad]

It is, but I don't think it could be resolved from knex side gracefully, unfortunately.

[NicoAiko]

I close this issue as the transfer to a backend project really solved it.
Stupid webpack xD

[burningTyger]

I'm having the same issue. There's some recent changes going on with webpack that breaks things. Can't pinpoint it though. Mine's in an electron project so I can't really separate the two.

[kibertoad]

@burningTyger Why do you need webpack on backend application, though?

[burningTyger]

@kibertoad I edited my comment. It's an electron app.

[kibertoad]

@burningTyger OK, that's actually a legitimate use-case. Please let me know if you will find either a configuration solution from your side or figure out what knex could be doing differently to avoid breaking stuff - so that we could improve either our documentation or implementation.

[burningTyger]

Can we move this to gitter? I posted an error log some time ago to which you answered

[burningTyger]

moving stuff to the main thread helped a lot

[dkcamargox]

I'm also doing a electron-react app

my knex is having the same problem.

TELL ME WHAT YOU DISCOVERED

[burningTyger]

Put your knex queries into the main thread. Don't use the renderer for knex stuff. Only communicate via IPC from main to renderer. Then you should be good. I noticed using the dB drivers directly also works from the renderer and is a lot more lightweight.

[dkcamargox]

you're awesome!!

[aprilcoskun]

I have this issue with angular universal.

[michaelavila]

@kibertoad I have a one off application https://github.com/michaelavila/knex-querylab that I found useful when I was writing a lot of knex, and I've tried to keep it online over the years in case others find it useful. This site is made to let you quickly tinker with the knex dsl. All of the work is done in the browser.

I could pass the input off to a server to translate and send back to me, but that would mean I'd need to put together and deploy some server. Right now, this is able to all run in browser and I can host it cheaply and easily on github pages. So, I don't want to go down that other route.

Here are a few thoughts I have:

1. I exclude the mssql dialect from the querylab, because I don't have the drivers. Ideally, I shouldn't ever see these driver errors because I'm not using them. But I am seeing these errors. Can we move the point at which these errors are surfaced to the first time they are used?
2. I suspect that it's uncommon to need all of the knex dialects available in the library all of the time. Would it be better to make new dialects available as a dependency, e.g. npm install -D knex-dialects-mssql? I think this would also give a better place to surface these missing driver errors, when trying to install the dependency.

I'm sure I'm missing something and there are likely better alternatives than mine, but there you have it.

[kibertoad]

@michaelavila But knex is already doing that, all drivers are lazy-loaded when first used, which is why most applications work just fine with just a subset of drivers. The only case where all drivers are loaded eagerly is when using a misconfigured webpack.

Note that react-scripts which you are using are relying on webpack under-the-hood. See this thread for advice on how to configure webpack externals so that knex works correctly: #1128

Please let me know if that helps.

[michaelavila]

@kibertoad I'll dig into that now, thanks!