[BUG] "RangeError: Maximum call stack size exceeded" occurs when a schema includes type: Buffer

[k1350]

Summary:

When a schema includes type: Buffer and the byte length of the buffer is very long, the following error occurs when executing Model.get.

2023-06-02T02:33:35.358Z	36a19b32-d48b-4a57-a22d-9262ab32e0e5	ERROR	RangeError: Maximum call stack size exceeded
    at /var/task/index.js:8644:23
    at Array.reduce (<anonymous>)
    at Object.main2 [as entries] (/var/task/index.js:8639:37)
    at Item.fromDynamo (/var/task/index.js:9443:32)

In my environment, an error occurs when the result of Buffer.byteLength is greater than 105730.

Code sample:

export interface TestSchema extends Item, TestSchemaAttributes {}
export interface TestSchemaAttributes {
  id: string;
  sk: string;
  content: Buffer;
}
export const TestSchemaDefinition: SchemaDefinition = {
  uid: {
    type: String,
    required: true,
    hashKey: true,
  },
  sk: {
    type: String,
    required: true,
    rangeKey: true,
  },
  content: {
    type: Buffer,
    required: true,
  },
};

const model = dynamoose.model<TestSchema>(
    "TestSchemaTable",
    [new dynamoose.Schema(TestSchemaDefinition)],
    {
      create: false,
      waitForActive: { enabled: false },
      throughput: "ON_DEMAND",
    }
  );

await model.get({id: "id", sk: "sk}); // `RangeError: Maximum call stack size exceeded` occurs when "content" is very large.

Current output and behavior (including stack trace):

RangeError: Maximum call stack size exceeded occurs.

2023-06-02T02:33:35.358Z	36a19b32-d48b-4a57-a22d-9262ab32e0e5	ERROR	RangeError: Maximum call stack size exceeded
    at /var/task/index.js:8644:23
    at Array.reduce (<anonymous>)
    at Object.main2 [as entries] (/var/task/index.js:8639:37)
    at Item.fromDynamo (/var/task/index.js:9443:32)
    at new Item (/var/task/index.js:9425:63)
    at new TestSchema (/var/task/index.js:12852:13)
    at itemify (/var/task/index.js:13359:35)

Expected output and behavior:

No error occurs.

Environment:

Operating System: Amazon Linux 2 (AWS Lambda)
Node.js version (node -v): 18x
Dynamoose version: 3.2.0

Other information (if applicable):

Other:

• I have read through the Dynamoose documentation before posting this issue
• I have searched through the GitHub issues (including closed issues) and pull requests to ensure this issue has not already been raised before
• I have searched the internet and Stack Overflow to ensure this issue hasn't been raised or answered before
• I have tested the code provided and am confident it doesn't work as intended
• I have filled out all fields above
• I am running the latest version of Dynamoose

[gatapia]

Did you manage to find a workaround for this?

[k1350]

@gatapia No, unfortunately we did not find a workaround.

[TheKnightCoder]

Looks like this is the commit that introduced this bug, trying to convert Uint8Array to Buffer
06912e2
packages/dynamoose/lib/Item.ts lines 74-80

hopefully someone can fix it, makes dynamoose unusable as you can't get items with large Uint8Arrays

[TheKnightCoder]

Workaround I am using for now is to use patch-package: edit node_modules/js-object-utilities/dist/entries.js and add in && !(value instanceof Uint8Array) into line 8, then run npx patch-package js-object-utilities

see here for the code change required

[Edited comment for a working fix]

@fishcharlie can we merge this fix in
rrainn/js-object-utilities#2