Decimal128 being read as new Decimal128("#") instead of the number

[basieboots]

Prerequisites

• I have written a descriptive issue title

Mongoose version

7.4.2

Node.js version

20.2.0

MongoDB version

1.42.1

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

Sonoma 14.2.1

Issue

I'm trying to use a double/float data type for example (rev: 1.2), when I was using Number, mongoose would write it as an int so I've switched to using Decimal128 which would be better for our long term goals but I can't get rid of "new Decimal128". I'm open to whatever will let us use decimals and really would love to keep it simple.

For instance rev 1.2 was being written as 1, and now that I've switched to Decimal128 it's reading it from the database (console.log on read) as:
rev: new Decimal128("1.2"),

Schema:

const { Types, model, Schema } = require('mongoose');

const InfoSchema = new Schema({
    id: String,
    rev:  {type: Types.Decimal128, 
        get:  v => Types.Decimal128((+v.fromString()).toFixed(4)), 
        set: v =>Types.Decimal128.fromString(v.toFixed(4)),
    },
    starRating:{type: Types.Decimal128, 
        get: v => Types.Decimal128((+v.fromString()).toFixed(4)), 
        set: v => Types.Decimal128.fromString(v.toFixed(4)),
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}});
 
 module.exports = model('exInfo', ExInfoSchema);   

Database stores it as:

rev : 1.2

I've been trying all sorts of different things but I can't manage to get mongoose to read from the MongoDB without the "new Decimal128" part of it. I've tried parseFloat, v.getDecimals and not using getters and setters.

Thanks in advance for any help.

MongoDB version for MongoDBCompass

[FaizBShah]

The reason you are seeing this is because you are trying to fetch a Decimal128 object, not the actual decimal value inside it. To fix this, I think you can remove the set property, and change your get property to:

get: v => (+v.toString()).toFixed(4)

For reference: https://stackoverflow.com/questions/65722311/mongoose-insert-decimal128-data-in-db-saved-as-string-for-required-nested-docume

[basieboots]

The reason you are seeing this is because you are trying to fetch a Decimal128 object, not the actual decimal value inside it. To fix this, I think you can remove the set property, and change your get property to:

get: v => (+v.toString()).toFixed(4)

For reference: https://stackoverflow.com/questions/65722311/mongoose-insert-decimal128-data-in-db-saved-as-string-for-required-nested-docume

I tried to do that v =>(+v.toString().toFixed(4)) and still have the same issue.

I would also really like to avoid converting it to a String at any point since my front end (Flutter) uses a double data type

[FaizBShah]

Did you try removing the set, and only use the above get method? Also, toFixed() will convert your string back to a decimal number

[basieboots]

Did you try removing the set, and only use the above get method? Also, toFixed() will convert your string back to a decimal number

removing the set? I dropped the collection, restarted everything and created a new record each time I've made changes. I can try it again a little later, maybe I made a typo the first time

[FaizBShah]

Ohh ok, I understood, you are trying to use the set to directly convert string to decimal128, so removing it might not be a solution

[FaizBShah]

Just for debugging purpose, can you console.log what does v and v.toString() prints? Also, check if this answer is of any help to you (although this is a bit old) - https://stackoverflow.com/questions/61380443/mongodb-returning-with-numberdecimal-in-the-response-of-a-query

[FaizBShah]

I tried to do that v =>(+v.toString().toFixed(4)) and still have the same issue.

It should be v => (+v.toString()).toFixed(4) not v =>(+v.toString().toFixed(4)) (the parentheses are wrong), otherwise it will throw an error internally

[basieboots]

I tried to do that v =>(+v.toString().toFixed(4)) and still have the same issue.

It should be v => (+v.toString()).toFixed(4) not v =>(+v.toString().toFixed(4)) (the parentheses are wrong), otherwise it will throw an error internally

v => (+v.toString()).toFixed(4) had the same result of: rev: new Decimal128("1.2"),
I tried it again with the same result.

Where can I put the console log to access v? If i do it in my controller it doesn't see it and I didn't think I could do that in the getter or schema?

[FaizBShah]

I think it can be done in the get method probably

[basieboots]

I think it can be done in the get method probably

I'm not having any luck getting it to console.log in either the

rev:  {type: Types.Decimal128, 
        get: (v) => {
            console.log(`${v} print`);
            console.log(v.toString());
        }
        // get:   v => (+v.toString()).toFixed(4), 
        // set: v => (+v.toString()).toFixed(4),
    },

when I do a console.log((data.rev).toString) in the controller I get this error:
TypeError: Cannot read properties of undefined (reading 'toString')

Why is the Decimal128 being read as an object? shouldn't Mongoose be set up in a way where you don't have to do extra steps to read in an approved data type?

[FaizBShah]

when I do a console.log((data.rev).toString) in the controller I get this error:
TypeError: Cannot read properties of undefined (reading 'toString')

Why are you modifying the set method? The set doesn't need to be modified. Like, the return type for the set method should be an Decimal128 object, but you are returning a number. I think the issue is happening because the value is being set wrong, due to which when you are trying to get it, its returning undefined, because I tested it in my local, and for me the get is working fine. I'll suggest you can try removing the set method and only use the get method.

Why is the Decimal128 being read as an object? shouldn't Mongoose be set up in a way where you don't have to do extra steps to read in an approved data type?

Mongoose uses bson library's Decimal128 internally

[basieboots]

the set method is commented out. I modified it because I was trying all options before asking for help even if it didn't seem like it would work.

Where else could it possibly be set? If I remove the set method we still have the issue so is there possibly somewhere else in the program it could be being set wrong? I'm picking up from someone else and I did look first to see if there could be something happening elsewhere but I'm out of ideas as to where it could be happening

[FaizBShah]

I'll try to look into the cause later on, but for now you can try setting the type of the field to Schema.Types.Decimal128 instead of Types.Decimal128 as that's what is recommended in the doc

[basieboots]

I'll try to look into the cause later on, but for now you can try setting the type of the field to Schema.Types.Decimal128 instead of Types.Decimal128 as that's what is recommended in the doc

I did the Schema.Types.Decimal128 first, then added Types in the declaration in the top so that it could be abbreviated.

[FaizBShah]

#6268 You can call the .toJson() method on the retrieved document to get the transformed data

[vkarpov15]

Why is the Decimal128 being read as an object? shouldn't Mongoose be set up in a way where you don't have to do extra steps to read in an approved data type? Because JavaScript doesn't support decimal arithmetic numbers natively, so we need an object wrapper for that.

The reason why you're getting the new Decimal() output is because your getter is returning a Decimal128 instance, not a number. If you want to transform the data that Mongoose loads from MongoDB into a different type, a getter is the place to do that, you shouldn't use the getter to just transform into a Decimal128. Try the following instead:

const { Types, model, Schema } = require('mongoose');

const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128, 
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}});
 
 const Info = model('exInfo', InfoSchema);      

const doc = new Info({ rev: '123.456789' });
console.log(doc.toJSON()); // Includes `rev: 123.4568`

[StaffordInnovations]

Why is the Decimal128 being read as an object? shouldn't Mongoose be set up in a way where you don't have to do extra steps to read in an approved data type? Because JavaScript doesn't support decimal arithmetic numbers natively, so we need an object wrapper for that.

The reason why you're getting the new Decimal() output is because your getter is returning a Decimal128 instance, not a number. If you want to transform the data that Mongoose loads from MongoDB into a different type, a getter is the place to do that, you shouldn't use the getter to just transform into a Decimal128. Try the following instead:

const { Types, model, Schema } = require('mongoose');

const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128, 
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}});
 
 const Info = model('exInfo', InfoSchema);      

const doc = new Info({ rev: '123.456789' });
console.log(doc.toJSON()); // Includes `rev: 123.4568`

I'm still getting the object in the console.log() when trying to pull it from the database. The other items are put in as Number

creatorDescrip: '',
title: 'test',
description: 'test',
rev: new Decimal128("1.2"),
starRating: 3,
dateCreated: 1708958613180,
dateLastAccessed: 1708958613180

[vkarpov15]

"I'm still getting the object in the console.log() when trying to pull it from the database." <-- can you please provide a code sample that demonstrates what you mean here?

[StaffordInnovations]

"I'm still getting the object in the console.log() when trying to pull it from the database." <-- can you please provide a code sample that demonstrates what you mean here?

The model:

const { model, Schema } = require('mongoose');
const permissionSchema = require('./permission');

const ExInfoSchema = new Schema({
    id: String,
    creatorId: String,
    title: String,
    description: String,
    rev:  {
        type: Schema.Types.Decimal128, 
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    starRating: Number,
    dateCreated: Number,
    }, {'toJSON': {getters: true}});
 
 module.exports = model('exInfo', ExInfoSchema);   

The controller call:

exports.getExerSessByUserId = asyncHandler(async (req,res,next) => {
   const { creatorId} = req.body;
   console.log('get all exercise sessions by user ID');
   

   const exerSession = await ExerciseSession.find({creatorId: creatorId});
 
   console.log(exerSession);


   if(!exerSession){
      return (next(new ErrorResponse(`exercise session read failed `, 404)));
   }

   res.status(200).json({ success: true, data: exerSession});
});

the console.log output with some of the information hashed for privacy:

get all exercise sessions by user ID
[
  {
    _id: new ObjectId("*******"),
    id: '**************',
    creatorId: '********', (matches)
    creatorDescrip: '',
    title: 'test',
    description: 'test',
    rev: new Decimal128("1.3"),
    starRating: 3,
    dateCreated: 1709202400475,
    __v: 0
  }
]

I'm sending the information from our Flutter UI and it is writing to the database as:

_id : *********
id : "*********"
creatorId : "***********"
creatorDescrip : ""
title : "test"
description : "test"
rev : 1.3 
starRating : 3
dateCreated :  1709202400475
```

The only problem is rev is still being read as and sent back to our flutter UI as an object and it's not really a conversion that I have something in Flutter to handle (that I know of).  I did try doing the toString as fixed as well as splitting the result and trying to get out just the number but since it's expecting JSON it's not accepting that work around.

[vkarpov15]

@StaffordInnovations add the toObject.getters option as well and then console.log() output will show the number form as well:

  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

toJSON is specifically for JSON serialization.

[StaffordInnovations]

@StaffordInnovations add the toObject.getters option as well and then console.log() output will show the number form as well:

  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

toJSON is specifically for JSON serialization.

adding the toObject:{getters: true} isn't changing the console.log results

I'm still getting

rev: new Decimal128("1.2"),

[vkarpov15]

Can you please provide a complete script that demonstrates the issue you're seeing? I'm running the following script:

const { Types, model, Schema } = require('mongoose');
  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

 const Info = model('exInfo', InfoSchema);

const doc = new Info({ rev: '123.456789' });
console.log(doc);

Get the following output:

$ node gh-14360.js 
{ rev: 123.4568, _id: new ObjectId("65eb83baa6410ae49da506d4") }

[StaffordInnovations]

Can you please provide a complete script that demonstrates the issue you're seeing? I'm running the following script:

const { Types, model, Schema } = require('mongoose');
  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

 const Info = model('exInfo', InfoSchema);

const doc = new Info({ rev: '123.456789' });
console.log(doc);

Get the following output:

$ node gh-14360.js 
{ rev: 123.4568, _id: new ObjectId("65eb83baa6410ae49da506d4") }

I explained above that the controller is calling the model using the following code:

exports.getExerSessByUserId = asyncHandler(async (req,res,next) => {
   const { creatorId} = req.body;
   console.log('get all exercise sessions by user ID');
   

   const exerSession = await ExerciseSession.find({creatorId: creatorId});
 
   console.log(exerSession);


   if(!exerSession){
      return (next(new ErrorResponse(`exercise session read failed `, 404)));
   }

   res.status(200).json({ success: true, data: exerSession});
});

The information is saved in mongoDB as a double/float/decimal ie 1.2 instead of 1 but then when the controller fetches it, it's putting on the object so that the console.log says new Decimal128("1.2"). There doesn't appear to be an issue in saving it to the database but in reading it back out from MongoDB.

No change by adding the toObject getters

Your code isn't interfacing with a mongoDB database which is where there's a difference and appears to be the problem.

[vkarpov15]

Same behavior when reading from MongoDB:

const { connect, Types, model, Schema } = require('mongoose');
  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

const Info = model('exInfo', InfoSchema); 

run().catch(err => {
  console.error(err);
  process.exit(-1);
});

async function run() {
  await connect('mongodb://127.0.0.1:27017/mongoose_test');

  const doc = new Info({ rev: '123.456789' });
  await doc.save();

  const fromDb = await Info.findById(doc._id).orFail();
  console.log(fromDb);
}

Output:

$ node gh-14360.js 
{
  _id: new ObjectId('6602f0d48ba70bf6dea983b3'),
  rev: 123.4568,
  __v: 0
}

Can you please modify this script to demonstrate the issue you're seeing?

[github-actions]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 5 days

[StaffordInnovations]

Same behavior when reading from MongoDB:

const { connect, Types, model, Schema } = require('mongoose');
  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

const Info = model('exInfo', InfoSchema); 

run().catch(err => {
  console.error(err);
  process.exit(-1);
});

async function run() {
  await connect('mongodb://127.0.0.1:27017/mongoose_test');

  const doc = new Info({ rev: '123.456789' });
  await doc.save();

  const fromDb = await Info.findById(doc._id).orFail();
  console.log(fromDb);
}

Output:

$ node gh-14360.js 
{
  _id: new ObjectId('6602f0d48ba70bf6dea983b3'),
  rev: 123.4568,
  __v: 0
}

Can you please modify this script to demonstrate the issue you're seeing?

I can't really see what you want me to modify. The major difference is that I'm sending the information to and from my flutter front end and that I'm using .find{} with a different reference object. It seems like there must be an issue pulling from the DB but the whole point of the function I'm calling is to find all of the items from a specific user.

here's the code to fetch the data again:

exports.getExerSessByUserId = asyncHandler(async (req,res,next) => {
   const { creatorId} = req.body;
   console.log('get all exercise sessions by user ID');
   

   const exerSession = await ExerciseSession.find({creatorId: creatorId});
 
   console.log(exerSession);


   if(!exerSession){
      return (next(new ErrorResponse(`exercise session read failed `, 404)));
   }

   res.status(200).json({ success: true, data: exerSession});
});

If you could be more specific as to what you want me to modify I can try again. So far there have been no changes with the issue

[vkarpov15]

The output from the script I pasted shows that rev is a JavaScript number when printing the object. If I'm understanding your issue correctly, your issue is that 'the console.log says new Decimal128("1.2")'. The script shows this is not the case.

[StaffordInnovations]

The output from the script I pasted shows that rev is a JavaScript number when printing the object. If I'm understanding your issue correctly, your issue is that 'the console.log says new Decimal128("1.2")'. The script shows this is not the case.

As I mentioned you're not following the same processes I am and it's not just the console.log but the front end that's reporting an object being sent instead of the decimal128. You can't say that it's not the case when I'm very clearly showing you that it is the case that I'm getting an object instead of the correct data type

[vkarpov15]

If you run the following script, you'll see that JSON.stringify(), which is what Express' res.json() uses to serialize data to JSON, also applies getters correctly:

const { connect, Types, model, Schema } = require('mongoose');
  
const InfoSchema = new Schema({
    id: String,
    rev:  {
        type: Schema.Types.Decimal128,
        get:  v => v == null ? null : +(+v.toString()).toFixed(4)
    },
    dateCreated: Number,
    }, {'toJSON': {getters: true}, toObject:{getters:true}});

const Info = model('exInfo', InfoSchema);

run().catch(err => {
  console.error(err);
  process.exit(-1);
});

async function run() {
  await connect('mongodb://127.0.0.1:27017/mongoose_test');

  const doc = new Info({ rev: '123.456789' });
  await doc.save();

  const fromDb = await Info.findById(doc._id).orFail();
  console.log(fromDb);
  console.log(JSON.stringify(fromDb, null, '  ')); // Prints doc with `rev` as a number
}

Please provide a complete repro script that demonstrates the issue you're seeing.