Sometimes database isn't created on running

[PunkChameleon]

Sometimes, when I start the goblin without a database, it will say the database is created but will not create it.

To reproduce:

1. Delete any existing database
2. Run 'node .'
3. Observe that the database is reported created, but is not actually created
4. Stop and start again.
5. Observer that 'goblin' database is created.

This issue is a bit elusive. Please comment if you can reproduce it as well!

[PunkChameleon]

Running into this again. Anyone else have similar experiences?

[PunkChameleon]

Just in case anyone else runs into this, a fix is:

1. Go to your CouchDB
2. Create a database named 'goblin'
3. Then run goblin normally

The issue seems to have something to do with actually creating the DB itself.

[PunkChameleon]

Okay, I'm still looking into this.

Right now, my line of thinking is that it may have to do in 'lib/couchdb.js' when the db module is exported. If cradle has to create the database, asynchronously, maybe the 'db' variable is exporting before the creation is done. It still gets done of course, which is why it works on the second try.

Any thoughts on this?

[PunkChameleon]

Okay, so I'm more sure it is this issue.

The problem is that db.create() is being executed after the module is exported.

One workaround is to use callbacks, but that would mean moving most of goblin.js, utils.js and gb-admin.js into an enormous callback, which I want to avoid if possible. I've experimented a bit with using node-sync and such but to no avail.

Does anyone have an insight or possible solutions?

[joshuatrii]

I've had this problem as well, what do you think about not using cradle for just the db creation part?

[PunkChameleon]

Good to know it's happening to more than just me! That is a possibility, though I do wonder if that will fix the async part.

For reference, I also posted this question in a SO post : http://stackoverflow.com/questions/20666150/asynchrously-load-npm-module

[SotiriosVrachas]

i don't have this problem, but it is possible to happen since if the process just exits. It would be more appropriate especially for remote databases if it would loop until it connects. Now the closure just makes sure that the db exists before exporting the module (modules load synchronously) but if the db is just slow it will fail again.

[PunkChameleon]

Also for clarification, this bug only happens when you don't have a database already set up. It basically runs, creates it, then the application fails when you try to visit a page / use the DB. When you restart it works.

[SotiriosVrachas]

ok i did it... but it is not elegant at all , their must be a better way that looping, ive started studying cradles source, i am no expert on nosql (and/or) drivers but i don't like it

lib/couchdb.js

var cradle = require('cradle'),
    config = require('./config.js');

var db = new(cradle.Connection)(config.couchURL, config.couchPort, {
    auth: {
        username: config.couchUsername,
        password: config.couchPassword
    }
}).database('goblin'); // change this for quick debuging


db.exists(function (err, exists) {
    eeexists (err, exists)
})

function eeexists (err, exists) {
    if (err) {
      console.log('error', err);
      db.exists(function (err, exists) {
            eeexists (err, exists)
      })
    } else if (exists) {
      console.log('db goblin exists');
    } else {
      console.log('database gobin does not exist. Creating...');
      db.create();
      console.log('database created');
    }
}

module.exports = db

[PunkChameleon]

I agree cradle is being a bit difficult with this one. @joshuatrii do you know the couchAPI call that would do this?

@SotiriosVrachas I agree some looping has to be done. I tested the code you posted against a new db name (change 'goblin' to 'goblin1' as long as 'goblin1' doesn't exist, and this will test db creation) but am getting the same error. I think it's because the db.exists call is not syncrhous and the module is exporting before it's created.

I put an issue up on the cradle github (flatiron/cradle/issues/237) to see if we can make a synchronous option to this call. I agree that we may have to dive deeper into the cradle source to make this happen like we need it to.

[SotiriosVrachas]

of course you are getting the same error... but isn't the database "goblin1" created eventually?

[PunkChameleon]

It is, but this is the initial behavior of it before the fix. The async call is completed and the db is made, but it is after the db variable is exported. Maybe we can export a self-invoking function?

[SotiriosVrachas]

I am confused isn't the title "Sometimes database isn't created on running" ? This make sure that the the database is created on running. The db function is not exported before this is done.

Modules are loading synchronously in node, this is done because they are needed only in the initiation of an instance. .exists in cradle is a .prototype internal property of the db function (for how .prototype works: http://stackoverflow.com/questions/572897/how-does-javascript-prototype-work). By setting our own .exists to anything we change the default callback. what i am doing above is setting the callback to a loop if the database dose not exist. Until the database begins to exist the initial loop will keep blocking the whole instance

[PunkChameleon]

I see what your method is and how you're doing it -- if you delete the goblin database, then run goblin, are you successfully getting a running instance with the 'goblin' db created and filled with all the necessary defaults?

For clarity, here are the steps to reproduce:

1. Make sure you don't have a 'goblin' db set up (either delete 'goblin' from couch, or rename it in lib/couchdb.js to something else for ease of testing).
2. Run the application ('node .')
3. Visit localhost:8000/ -- observe that the application crashes.
4. Visit the DB and see it was created, but not filled with all default values necessary to run.
5. Re-run the application and see it work, and all defaults are filled in.

My hypothesis is that the 'db' variable is being exported but the callback inside of db.exist are being executed afterwards. Unsure of the specific reason though.

[SotiriosVrachas]

Oh my source code... this is a new issue, #63 . It is a database population issue, located in the lib/goblin.js, and to me this have happened only once so far, every time i try to recreate it works fine, it might gets cached.

[PunkChameleon]

@SotiriosVrachas that makes sense, thanks for creating the new issue. I agree it's a hard issue to catch (caching must be involved some how) but the steps above were the only way I can consistently reproduce.

Thanks again for your help with this issue and the effort you've been putting into it. This one is a zinger!

[PunkChameleon]

Hey all,

As this thread discussed, it may have something to do with cradle itself. I'm therefore experimenting with the possibility of moving from cradle to a more minimal couchDB driver like nano.

If you want to follow the progress (or even more helpfully, help out and contribute to it!) you can follow the nano branch on my fork here. I'll keep this thread updated, and see if it fixes problems like this.

[SotiriosVrachas]

have you considered NOT having bugs in our code?

[PunkChameleon]

That's brilliant! We should try that instead :)

Also, on the note about nano, it looks promising but I don't think it's going to solve this problem, i just think it's the async nature of couch. Maybe we can just leave the db creation to an installing script and assume it's there while running?

[SotiriosVrachas]

Making the whole think more modular would be awesome.