Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call? #2820

Closed
umaar opened this issue Sep 26, 2018 · 56 comments

Comments

@umaar
Copy link

umaar commented Sep 26, 2018

Environment

Knex version: 0.15.2
Database: sqlite3
OS: Ubuntu 18

I'm getting lots of errors like TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

Any ideas? All my code lives in this one file: https://github.com/umaar/wiki-globe/blob/master/index.js - that code powers this globe which I posted in a Hacker News comment and in turn led to the discovery of such error.

An example query looks like this:

await knex('edits').insert([{
	field1: JSON.stringify(data),
	field2: 'field2'
}]);

Does sqlite3 only allow a single connection to it because it's file-based? I'll try adding .transacting() calls to see if that makes a difference.

I guess it comes down to: some technical limitation with sqlite3 (maybe lack of pooling?) or some badly written code DB/queries on my part! Any insight you have would be much appreciated, thanks!

@wjohnsto
Copy link

wjohnsto commented Sep 26, 2018

From the documentation:

The client created by the configuration initializes a connection pool, using the generic-pool library.
This connection pool has a default setting of a min: 2, max: 10 for the MySQL and PG libraries, and a
single connection for sqlite3 (due to issues with utilizing multiple connections on a single file). To
change the config settings for the pool, pass a pool option as one of the keys in the initialize block.

So if you're using sqlite3 you will run into issues with more than 1 connection.

@umaar
Copy link
Author

umaar commented Sep 26, 2018

Oh got it, so the issue is: many clients are backed up in a queue waiting for a single available connection to sqlite3, but the timeout passes and that error is thrown? As a quick fix for a hobby project, I wonder if it's acceptable to simply increase the timeout

@umaar umaar closed this as completed Sep 26, 2018
@Miteshdv
Copy link

Miteshdv commented Sep 28, 2018

I am facing this issue though i am just trying to create a table in sample project , only one client is connected still this error , Any fix for this ? This is related to #2824

@tahv0
Copy link

tahv0 commented Nov 19, 2018

+1 having this issue too

@elhigu
Copy link
Member

elhigu commented Nov 19, 2018

@tahv0 there are 100 different reasons why that might happen, so you need to find out how to reproduce the problem. Probably by then you will also see why that is happening.

@JSEA
Copy link

JSEA commented Nov 21, 2018

Knex version: 0.15.2
Database: MySQL 8
OS: MacOSX

I'm having the same connection issue when I run "knex migrate:latest", and have scoured google with no luck. I've tested connections between Node and MySQL with a simple SQL script and that's connecting fine. Initially there was an authentication issue with the connection which I was able to resolve by altering the user privileges with 'mysql_native_password' which has to be done in MySQL 8 to be able to support legacy connections I believe. However I was unable to get Knex the connection going.

Does Knex support MySQL 8? The examples in the documentation show MySQL 5.6 and there is no explicit mention of MySQL 8 support.

I know this can be anything, so here's all the code I have for this project.

knexfile.js

require('dotenv').config();

module.exports = {

  development: {
    client: 'mysql2',
    connection: {
      port: process.env.db_port,
      host: process.env.db_host,
      database: process.env.db_name,
      user: process.env.db_user,
      password: process.env.db_key,
      connectTimeout: 90000
    },
    debug: true,
    pool: {
      min: 1,
      max: 20,
    },
    migrations: {
      directory: __dirname + '/knex/migrations',
    },
    seeds: {
      directory: __dirname + '/knex/seeds',
    },
  }
};

knex.js

require('dotenv').config();
const env = process.env.environment || 'development'
const config = require('../knexfile.js')[env];
const knex = require('knex')(config);

module.exports = knex;

server.js

require('dotenv').config();
const express = require('express');
const bodyParser = require("body-parser");
const path = require('path');
const PORT = process.env.db_port || 3002;
const knex = require('../knex/knex.js');

const app = express();

app.use((req, res, next) => {
  res.header("Access-Control-Allow-Origin", "*");
  res.header(
    "Access-Control-Allow-Headers",
    "Origin, X-Requested-With, Content-Type, Accept"
  );
  res.header("Access-Control-Allow-Methods", "GET,PUT,POST,DELETE");
  next();
});

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.static(`${__dirname}/../client/dist`, { maxAge: "365d" }));



app.listen(PORT, () => {
  console.log(`listening on port: ${PORT}`);
});

migration file

exports.up = function(knex, Promise) {
  return knex.schema.createTable('sdcapstone', (t) => {
    t.increments('_id').primary().unique();
    t.string('name').notNullable();
    t.integer('rating');
    t.integer('reviewCount');
    t.integer('itemNum').notNullable();
    t.integer('price').notNullable();
    t.string('mainImage');
    t.json('images');
  })
};

exports.down = function(knex, Promise) {
  return knex.schema.dropTable('sdcapstone');
};

Here is the result of a console log of "knex" from knex.js

{ [Function: knex]
  Promise: 
   { [Function: Promise]
     TypeError: [Function: TypeError],
     RangeError: [Function: RangeError],
     CancellationError: [Function: SubError],
     TimeoutError: [Function: SubError],
     OperationalError: [Function: OperationalError],
     RejectionError: [Function: OperationalError],
     AggregateError: [Function: SubError],
     _peekContext: [Function],
     onPossiblyUnhandledRejection: [Function],
     onUnhandledRejectionHandled: [Function],
     longStackTraces: [Function],
     hasLongStackTraces: [Function],
     config: [Function],
     getNewLibraryCopy: [Function],
     is: [Function],
     fromCallback: [Function],
     fromNode: [Function],
     all: [Function],
     cast: [Function],
     fulfilled: [Function],
     resolve: [Function],
     rejected: [Function],
     reject: [Function],
     setScheduler: [Function],
     pending: [Function],
     defer: [Function],
     method: [Function],
     try: [Function],
     attempt: [Function],
     bind: [Function],
     PromiseInspection: [Function: PromiseInspection],
     join: [Function],
     Promise: [Circular],
     version: '3.5.2',
     map: [Function],
     using: [Function],
     delay: [Function],
     coroutine: { [Function] addYieldHandler: [Function] },
     spawn: [Function],
     promisify: [Function],
     promisifyAll: [Function],
     props: [Function],
     race: [Function],
     reduce: [Function],
     settle: [Function],
     some: [Function],
     _SomePromiseArray: [Function: SomePromiseArray],
     filter: [Function],
     each: [Function],
     mapSeries: [Function: PromiseMapSeries],
     any: [Function],
     noConflict: [Function: noConflict] },
  queryBuilder: [Function: queryBuilder],
  raw: [Function: raw],
  batchInsert: [Function: batchInsert],
  transaction: [Function: transaction],
  initialize: [Function: initialize],
  destroy: [Function: destroy],
  ref: [Function: ref],
  domain: null,
  _events: {},
  _eventsCount: 0,
  _maxListeners: undefined,
  setMaxListeners: [Function: setMaxListeners],
  getMaxListeners: [Function: getMaxListeners],
  emit: [Function: emit],
  addListener: [Function: addListener],
  on: [Function: addListener],
  prependListener: [Function: prependListener],
  once: [Function: once],
  prependOnceListener: [Function: prependOnceListener],
  removeListener: [Function: removeListener],
  removeAllListeners: [Function: removeAllListeners],
  listeners: [Function: listeners],
  listenerCount: [Function: listenerCount],
  eventNames: [Function: eventNames],
  with: [Function],
  select: [Function],
  as: [Function],
  columns: [Function],
  column: [Function],
  from: [Function],
  fromJS: [Function],
  into: [Function],
  withSchema: [Function],
  table: [Function],
  distinct: [Function],
  join: [Function],
  joinRaw: [Function],
  innerJoin: [Function],
  leftJoin: [Function],
  leftOuterJoin: [Function],
  rightJoin: [Function],
  rightOuterJoin: [Function],
  outerJoin: [Function],
  fullOuterJoin: [Function],
  crossJoin: [Function],
  where: [Function],
  andWhere: [Function],
  orWhere: [Function],
  whereNot: [Function],
  orWhereNot: [Function],
  whereRaw: [Function],
  whereWrapped: [Function],
  havingWrapped: [Function],
  orWhereRaw: [Function],
  whereExists: [Function],
  orWhereExists: [Function],
  whereNotExists: [Function],
  orWhereNotExists: [Function],
  whereIn: [Function],
  orWhereIn: [Function],
  whereNotIn: [Function],
  orWhereNotIn: [Function],
  whereNull: [Function],
  orWhereNull: [Function],
  whereNotNull: [Function],
  orWhereNotNull: [Function],
  whereBetween: [Function],
  whereNotBetween: [Function],
  andWhereBetween: [Function],
  andWhereNotBetween: [Function],
  orWhereBetween: [Function],
  orWhereNotBetween: [Function],
  groupBy: [Function],
  groupByRaw: [Function],
  orderBy: [Function],
  orderByRaw: [Function],
  union: [Function],
  unionAll: [Function],
  having: [Function],
  havingRaw: [Function],
  orHaving: [Function],
  orHavingRaw: [Function],
  offset: [Function],
  limit: [Function],
  count: [Function],
  countDistinct: [Function],
  min: [Function],
  max: [Function],
  sum: [Function],
  sumDistinct: [Function],
  avg: [Function],
  avgDistinct: [Function],
  increment: [Function],
  decrement: [Function],
  first: [Function],
  debug: [Function],
  pluck: [Function],
  clearSelect: [Function],
  clearWhere: [Function],
  clearOrder: [Function],
  insert: [Function],
  update: [Function],
  returning: [Function],
  del: [Function],
  delete: [Function],
  truncate: [Function],
  transacting: [Function],
  connection: [Function],
  client: 
   Client_MySQL {
     config: 
      { client: 'mysql',
        connection: [Object],
        debug: true,
        pool: [Object],
        migrations: [Object],
        seeds: [Object],
        log: [Object] },
     logger: 
      Logger {
        _debug: [Function: debug],
        _warn: [Function: warn],
        _error: [Function: error],
        _deprecate: [Function: deprecate] },
     connectionSettings: 
      { port: '3001',
        host: 'localhost',
        database: redacted,
        user: redacted,
        password: redacted },
     driver: 
      { createConnection: [Function: createConnection],
        createPool: [Function: createPool],
        createPoolCluster: [Function: createPoolCluster],
        createQuery: [Function: createQuery],
        escape: [Function: escape],
        escapeId: [Function: escapeId],
        format: [Function: format],
        raw: [Function: raw] },
     pool: 
      Pool {
        creator: [Function: create],
        destroyer: [Function: destroy],
        validate: [Function: validate],
        log: [Function],
        acquireTimeoutMillis: 60000,
        createTimeoutMillis: 30000,
        idleTimeoutMillis: 30000,
        reapIntervalMillis: 1000,
        createRetryIntervalMillis: 200,
        propagateCreateError: true,
        min: 0,
        max: 20,
        used: [],
        free: [],
        pendingCreates: [],
        pendingAcquires: [],
        destroyed: false,
        interval: null },
     valueForUndefined: 
      Raw {
        client: [Circular],
        sql: 'DEFAULT',
        bindings: undefined,
        _wrappedBefore: undefined,
        _wrappedAfter: undefined,
        _debug: true },
     _events: 
      { start: [Function],
        query: [Function],
        'query-error': [Function],
        'query-response': [Function] },
     _eventsCount: 4,
     makeKnex: [Function: makeKnex] } }

@JSEA
Copy link

JSEA commented Nov 21, 2018

Update, I've reverted back to MySQL version 5.7.24 and I'm still getting the same timeout error

@tahv0
Copy link

tahv0 commented Nov 21, 2018

@tahv0 there are 100 different reasons why that might happen, so you need to find out how to reproduce the problem. Probably by then you will also see why that is happening.

For me this was Google Cloud specific problem, because one can not take connection from GAE to Google SQL with public IP. I fixed problem by reading some more docs.

@ashanker2
Copy link

@tahv0 there are 100 different reasons why that might happen, so you need to find out how to reproduce the problem. Probably by then you will also see why that is happening.

For me this was Google Cloud specific problem, because one can not take connection from GAE to Google SQL with public IP. I fixed problem by reading some more docs.

@tahv0 I am having the same issue with Google Cloud. Can you share the docs that helped you fix this problem?

@tahv0
Copy link

tahv0 commented Aug 20, 2019

@ashanker2
Copy link

@ashanker2
this doc is for Postgres https://cloud.google.com/appengine/docs/flexible/nodejs/using-cloud-sql-postgres

@tahv0 Thank you for the doc. Seems like we have incorporated these steps in our setup but will verify it again. Is there anything specific you had to do to fix this problem?

@elhigu
Copy link
Member

elhigu commented Aug 21, 2019

@ashanker2 Does your code work with locally installed postgresql server?

@tahv0
Copy link

tahv0 commented Aug 21, 2019

@ashanker2
I fixed problem by using
config.host = /cloudsql/${process.env.INSTANCE_CONNECTION_NAME};

and not ip&port. GAE mounts tcp/unix socket to a container when you have configured Cloud SQL instance. There may be a problem if db is in a different region than service that uses it and you may need to use this beta_settings-flag in your .yaml like in the docs.

@ashanker2
Copy link

@ashanker2
I fixed problem by using
config.host = /cloudsql/${process.env.INSTANCE_CONNECTION_NAME};

and not ip&port. GAE mounts tcp/unix socket to a container when you have configured Cloud SQL instance. There may be a problem if db is in a different region than service that uses it and you may need to use this beta_settings-flag in your .yaml like in the docs.

@tahv0 Thank you. It worked with INSTANCE_CONNECTION_NAME instead of Host & Port.

@batadamnjanovic
Copy link

I solved this problem with these versions:

"knex": "^0.21.1",
"objection": "^2.1.3",
"pg": "^8.0.3"

@demisx
Copy link

demisx commented Apr 30, 2020

@batadamnjanovic Thank you so much! I've been pulling my hair trying to understand what the hell happened. Upgrading to the latest pg did the trick.

@joaotanaca
Copy link

joaotanaca commented Jun 4, 2020

for me it was a silly error, I ended up not putting the await trx.commit() before the return

@elhigu
Copy link
Member

elhigu commented Jun 8, 2020

@joaotanaca Good way to avoid that error is to use implicit commit/rollback style of transactions

await knex.transaction(async trx => {
  // use trx here... if handler rejects, transaction will be rolled back if it returns, transaction will be automatically committed
});

@joaotanaca
Copy link

joaotanaca commented Jun 8, 2020

@joaotanaca Good way to avoid that error is to use implicit commit/rollback style of transactions

await knex.transaction(async trx => {
  // use trx here... if handler rejects, transaction will be rolled back if it returns, transaction will be automatically committed
});

I didn't end up checking the transaction method, thank you very much

@rossPatton
Copy link

i am getting this on a small not-yet-public site with no more than 1-2 connections at a time and im not using transactions, on postgres

nothing about my setup is particularly complicated, and the db isn't very big

@elhigu
Copy link
Member

elhigu commented Jun 8, 2020

@rossPatton try to isolate the exact case where it is happening and then you might find what is wrong in your app. Maybe your queries are really slow. Anyways that error is pretty hard to create without flooding huge amount of queries or using transactions.

@jj0b
Copy link

jj0b commented Jun 25, 2020

This issue popped up for me today on a a very simple and small project that has been working fine for weeks and whose code hasn't changed in weeks. I am not doing any transactions and there was never more than 2 connections, and those only had to handle less than 10 very small queries a day. The only thing that I could find to fix this was to upgrade to the latest version of Knex. As soon as I did that this issue went away, but I'm left feeling a little nervous as I don't understand what could have caused the issue for me in the first place.

@elhigu
Copy link
Member

elhigu commented Jun 26, 2020

@jj0b What was the old version of knex you were using? For example one old bug which caused connections closed by the server, because of timeout to be not rejected from pool, could have caused that kind of behavior.

To be sure for your use case is probably good to set pool min: 0 and connection idle timeout to be smaller than connection timeout of the server (few minutes).

@jj0b
Copy link

jj0b commented Jun 26, 2020

@elhigu I had been using knex 0.20.15.

My Express server had been running fine for weeks, and this issue only appeared yesterday on my local development version, which was only being queried a few times a day by only two clients. My staging and production versions which are similarly very low traffic with only two connections max have not so far had this issue.

I'll take a look at the settings you recommend. Thank you. :)

@elhigu
Copy link
Member

elhigu commented Jun 26, 2020

@jj0b hmm.. are you sure it doesn't have something to do with thise #3831

@jj0b
Copy link

jj0b commented Jun 26, 2020

@jj0b hmm.. are you sure it doesn't have something to do with thise #3831

Not really sure. All I know is that I hadn't made any changes recently and everything had been working when this error popped up yesterday when querying the database. All I did to fix it was update the latest version of Knex and the error stopped happening. I didn't update anything else, including Node as it seems some had to in that thread you shared. I also haven't been doing any migrations which seemed to have triggered this issue for some.

@vdegenne
Copy link

This feeling when you think you'll spend the entire day trying to fix such a damn issue, but you find the right spot and the right people that ease the pain in a flash 😃

@himharsh1997
Copy link

himharsh1997 commented Mar 25, 2021

Hi! guys in my application I was facing (sometime in between)
TimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

My config of knex is

 const config =    client: process.env.CLIENT,
    connection: {
      host: process.env.DBHOST,
      user: process.env.DBUSER,
      password: process.env.DBPASS,
      database: process.env.DATABASE
    },
    pool: { min: 0, max: 30, acquireTimeoutMillis: 60 * 1000 },
    seeds: {
      directory: './db/sds'
    },
    migrations: {
      directory: './db/mg'
    }
}
import knexLib from 'knex';
export const con = knexLib(config);

and I'm using it something like

import { con } from './con';
import { FormatError } from '../err'


const handler = (req)=>{
const trx = con.transaction();
try{
  const result = await con('insert-table').transacting(trx).insert(req.list).returning('*');
  const resultOfLog = await Promise.all(
  result.map((o)=>{
 return con('log-table').insert({event_id: 1, resource: o.id});
})
);
trx.commit();
return result;
} catch(error){
   trx.rollback();
  return new FormatError(error);
}
}

@minaairsupport
Copy link

I got the same error but the reason was VPC

@orlovcs
Copy link

orlovcs commented Jul 22, 2021

Issue for me were the inbound rules as well, helps to do a quick psql from the EC2 instance to make sure it has access.

@wvvinicius
Copy link

wvvinicius commented Aug 8, 2021

Hi devs, i have the same problem with my application.

Today it has a service that crashes the application and I need to restart.

I've tried a lot of things but I wasn't successful.

Here is my error and current configuration.

import knex from 'knex'

const connection = knex({
    client: 'mysql',
    connection: {
        host: '*****',
        port: *****,
        user: '***',
        password: '*****',
        database: '******',
        timezone: 'UTC',
        dateStrings: true,
        

    },
    pool: {
        min: 2,
        max: 6,
        createTimeoutMillis: 3000,
        acquireTimeoutMillis: 10000,
        idleTimeoutMillis: 30000,
        reapIntervalMillis: 1000,
        createRetryIntervalMillis: 100,
        propagateCreateError: false // NEVER USE THIS WITH KNEX, KNEX IS NOT DESIGNED TO WORK WITH IT
    },
    useNullAsDefault: true,
    debug: true,

})


export default connection


----------------


`    public async listarCompetencias() {
        let qry = "";
        let result = false;

        qry = `
            SELECT *
            FROM competencias
            ORDER BY id DESC
            `

        try {
            const trx = await knex.transaction()
            await knex.raw(qry).then((row) => {
                trx.commit()
                result = row[0]
            })
                      
            
            return result;
               

        } catch (error) {

            throw Error(error)
        }
        
    }

Version: "knex": "^0.21.1"

@elhigu
Copy link
Member

elhigu commented Aug 9, 2021

Maybe this code works better, which implicitly closes transactions also in case of an error...

import knex from 'knex'

const connection = knex({
    client: 'mysql',
    connection: {
        host: '*****',
        port: *****,
        user: '***',
        password: '*****',
        database: '******',
        timezone: 'UTC',
        dateStrings: true,
        

    },
    pool: {
        min: 0,
        max: 6,
        idleTimeoutMillis: 10000
    },
    debug: true,
})


export default connection


----------------


    public async listarCompetencias() {
        const qry = `
            SELECT *
            FROM competencias
            ORDER BY id DESC
            `

        try {
            return await knex.transaction(async (trx) => {
               const res = await knex.raw(qry);
               return res[0];
            });
        } catch (error) {
            throw Error(error)
        }        
    }
    

@vvo
Copy link
Contributor

vvo commented Sep 16, 2021

Hey there, I too had this error in development, and I wasn't explicitly using transactions. Maybe one of the objection.js codes I was using was, but I doubt it given logs.

It turns out I was running docker-compose with postgres:13.2-alpine on my MacBook m1 computer. The alpine version is built for amd64 architectures, while m1 MacBooks are running on arm64 architectures.

While this works with Docker, it's not recommended and can lead to errors like "stuck spinlock detected at LWLockWaitListLock" and "core dumped" in your Postgres logs (see docker/for-mac#5122) which is the root cause of Knex/Tarn not being able to get more connections (tl;dr; the database goes into crazy mode)

So I switched back to just using the image postgres:13.2, and it works 👍, no more errors :)

@knex knex deleted a comment from guerrerocarlos Sep 26, 2021
@knex knex deleted a comment from mjurincic Sep 26, 2021
@jtara1
Copy link

jtara1 commented Jun 18, 2022

It sounds like this error is really ambiguous. Anyway to make it more clear? Do you expect it to be thrown from any knex ORM query like await knex('my_table').select('*'); or await knex.raw('select * from my_table;') or others?

I'm finding that this error is consistently reproducable by running a function of mine that makes additional queries in parallel with lots of input.

@jtara1
Copy link

jtara1 commented Jun 18, 2022

In all the error logs for this, I'm seeing there is no additional info on where the error was thrown from

    error KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
        at Client_PG.acquireConnection (/home/j/projects/my-project/node_modules/knex/lib/client.js:348:26)
        at runNextTicks (internal/process/task_queues.js:60:5)
        at listOnTimeout (internal/timers.js:526:9)
        at processTimers (internal/timers.js:500:7) {
      sql: undefined,
      bindings: undefined
    }

There's no file or line number to trace back to your module. sql and bindings is undefined

I think reproducing the error locally is acceptable, but before that, it would help to actually understand which calls or queries are throwing. I would recommend adding your own stack trace in your catch blocks, .catch calls, or .on('error', err => callback.

using node.js 14

  const stackTrace = new Error().stack.split('\n').slice(1, 3).map(s => s.trim()).join('\n');
  log.error({ stackTrace }, 'error', err);

@Abdelatief
Copy link

for me increasing the aquireTimeout was the solution

@mohan-ganesh
Copy link

run into the same problem with the strapi and had to modify the following to get the connection reliably.

export default ({ env }) => ({ connection: { client: 'postgres', connection: { host: env('DATABASE_HOST', '10.100.80.5'), port: env.int('DATABASE_PORT', 5432), database: env('DATABASE_NAME', 'XXXXXXX'), user: env('DATABASE_USERNAME', 'XXXXXX'), password: env('DATABASE_PASSWORD', 'XXXXXX'), ssl: env.bool('DATABASE_SSL', false), }, debug: true, acquireConnectionTimeout: 1000000, options: { pool: { min: 1, max: 10, acquireTimeoutMillis: 900000, createTimeoutMillis: 900000, destroyTimeoutMillis: 900000, } }, },});

@karltaylor
Copy link

run into the same problem with the strapi and had to modify the following to get the connection reliably.

export default ({ env }) => ({ connection: { client: 'postgres', connection: { host: env('DATABASE_HOST', '10.100.80.5'), port: env.int('DATABASE_PORT', 5432), database: env('DATABASE_NAME', 'XXXXXXX'), user: env('DATABASE_USERNAME', 'XXXXXX'), password: env('DATABASE_PASSWORD', 'XXXXXX'), ssl: env.bool('DATABASE_SSL', false), }, debug: true, acquireConnectionTimeout: 1000000, options: { pool: { min: 1, max: 10, acquireTimeoutMillis: 900000, createTimeoutMillis: 900000, destroyTimeoutMillis: 900000, } }, },});

Can you format your reply better?

@mohan-ganesh
Copy link

here you go for Strapi version 3.5

module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME'),
username: env('DATABASE_USERNAME'),
password: env('DATABASE_PASSWORD'),
ssl: false,
},
acquireConnectionTimeout: 1000000,
debug:true,
options: {
pool: {
min: 0,
max: 1,
acquireTimeoutMillis: 900000,
createTimeoutMillis: 900000,
destroyTimeoutMillis: 900000,
}
},
},
},
});

Strapi version 4.5.4

If you would any one would like to try self hosting one can try these steps https://medium.com/@mohanganesh/strapi-headless-cms-google-cloud-run-and-postgresql-6126b597b10c

@matsko
Copy link

matsko commented Mar 28, 2023

For anyone running into this issue, let me explain what learned here.

I was getting this same error with my database connection because the config.connection setting was async due to the database password being loaded from a "secrets" service that GCP was offering. The secrets service would randomly timeout, just take too long for Knex or throw an error randomly. I don't think it was scaled to handle this usecase.

The end result was that Knex would consider all instances of this error its own timeout error. Knex would also timeout itself if the config.connection async resolution took to long. What this amounted to was a very difficult to pin down error that Knex gave zero context on.

I was able to completely fix the problem by converting all runtime secret loading to ENV variables that are filled during deploy (you can do this natively via most hosting providers ... in my case it was Cloud Run on GCP).

The nightmare is finally over.

@evanrosa
Copy link

I'm unfortunately getting this error still after reading this thread. I'm following @mohan-ganesh GCP article and haven't a clue on how to resolve this error. Any additional suggestions?

[2023-04-26 15:41:06.732] debug: ⛔️ Server wasn't able to start properly.
[2023-04-26 15:41:06.737] error: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?
KnexTimeoutError: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

@evanrosa
Copy link

Following back up, Mohan was able to help me solve the issue on my end. I was confused on GCP cloud run ports where I tried setting the port to 1337 rather than 8080. Since cloud run only runs on 8080 it was throwing the error. On top of that, I had to add my variables into my cloud run env. Once completed, it worked like a charm. Again, thanks @mohan-ganesh for helping me debug.

@hmatsuda
Copy link

hmatsuda commented May 6, 2023

@evanrosa We have been running our web application on port 8080 with CloudRun for several years and have the same problem. Could you please tell me why changing the port to 1337 would stop this problem?

@tcherokee
Copy link

@evanrosa Could you explain exactly what the fix is? I tried running the cloud application port on 8080 and 1337, all with no luck. I even tried changing the web app port (strapi) from 1337 to 8080, also with no luck.

@mohan-ganesh
Copy link

@evanrosa Could you explain exactly what the fix is? I tried running the cloud application port on 8080 and 1337, all with no luck. I even tried changing the web app port (strapi) from 1337 to 8080, also with no luck.

hey Evan, the actual changes are documented at #2820 (comment)
Regarding CloudRun, when we deploy the app the port that it expected should be 8080.

@ematthewsBW
Copy link

I also forgot to close out a transaction resulting in a connection leak. I figured out that I can add an assertion to my tests that assures I've not leaked any connections by checking the following

knex.client.pool.numUsed() === 0

I'm on knex 2.4.2 which uses tarn 3.0.2 (the pooling library that has the numUsed function)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests