Is There a Recommended Way to Create a Generic Repository in DynamoDB Toolbox?

[hdrhmd]

Hello,

I've been using DynamoDB Toolbox extensively for various projects and find it to be a robust solution for working with AWS DynamoDB. In my projects, I generally define a Model and a Repository class for each entity in my database schema. While this approach works well, I notice that there's a lot of repetitive boilerplate code in each repository class that can be shared among all repositories.

To streamline this, I've been attempting to create a generic repository that contains common methods, which can be extended or aggregated in each individual repository. While I've managed to make it work to some extent using TypeScript's ts-ignore and redefining the Query object, I wonder if there is a more elegant or recommended approach to achieve this goal.

Here's a stripped-down example of my current implementation in TypeScript:

interface UserModel {
  // User attributes
}

interface PostModel {
  // Post attributes
}

export const UserEntity = new Entity({
  name: 'User',
  attributes: {
    // ...
  },
  table: Table,
} as const);

export const PostEntity = new Entity({
  name: 'Post',
  attributes: {
    // ...
  },
  table: Table,
} as const);

class GenericRespository<M> {
  constructor(
    private readonly entity: Entity
  ) {}

  async queryOne(pk: string, query?: Query): Promise<M | undefined> {
    const result = await this.entity.query(pk, query);
    if (result?.Items?.length > 0) {
      return result.Items[0] as M;
    }
    return undefined;
  }

  // Other generic methods
}

class UserRepository {
  private readonly genericRepo: GenericRespository<UserModel>;

  constructor() {
    this.genericRepo = new GenericRespository<UserModel>(UserEntity);
  }

  async getByUserId(userId: string): Promise<UserModel | undefined> {
    return await this.genericRepo.queryOne(`userId:${userId}`);
  }

  // ...
}

class PostRepository {
  private readonly genericRepo: GenericRespository<PostModel>;

  constructor() {
    this.genericRepo = new GenericRespository<PostModel>(PostEntity);
  }

  async getByPostId(postId: string): Promise<PostModel | undefined> {
    return await this.genericRepo.queryOne(`postId:${postId}`);
  }

  // ...
}

Would love to get the community's thoughts on this, especially if there's a more idiomatic way to achieve what I am aiming for.

Thank you in advance for your guidance!

[naorpeled]

Hey @hdrhmd,
the following should improve the autocompletions and type safety:

class GenericRepository<E extends Entity> {
  constructor(
    private readonly entity: E
  ) {}

  async queryOne(pk: string, options?: QueryOptions<E>) {
    const result = await this.entity.query(pk, options);
    if (result?.Items?.length??0 > 0) {
      return result.Items![0]
    }
    return undefined;
  }
}

  class UsersRepository {
    private readonly genericRepository;
    private readonly userEntity;

    constructor() {
      this.userEntity = new Entity({
        name: 'User',
        attributes: {
          userId: { type: 'string', partitionKey: true },
          firstName: { type: 'string' },
          lastName: { type: 'string' },
        },
        table: myTable,
      } as const);
      this.genericRepository = new GenericRepository(this.userEntity);
    }

    async getById(userId: string, options?: QueryOptions<typeof this.userEntity>) {
      return this.genericRepository.queryOne(`userId:${userId}`, options);
    }
  }

Marking this issue as resolved but feel free to ping me if you need anything else 😎

[nevolgograd]

@naorpeled

Thanks for your help with the query method.
I'm now looking to get a similar setup for the put method, and I'm hitting some type issues. Any chance you could share some pointers on that as well?
With this setup I'm getting Expression produces a union type that is too complex to represent..

Sorry to jump back into a closed issue, but your advice would be super helpful.
Appreciate it.

Generic repo

import type { Entity, EntityItem, PutOptions, QueryOptions } from 'dynamodb-toolbox';

abstract class DynamoDbRepository<E extends Entity> {
  constructor(private readonly entity: E) {}

  put<O extends PutOptions<E>>(item: EntityItem<E>, options: O) {
    return this.entity.put(item, options);
  }

  query<O extends QueryOptions<E>>(pk: string, options: O) {
    return this.entity.query(pk, options);
  }

Entity repo:

class AuditTrailRepository extends DynamoDbRepository<typeof auditTrailEventEntity> {
  constructor() {
    super(auditTrailEventEntity);
  }
}

Entity and a table:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
import { Entity, Table } from 'dynamodb-toolbox';
import { ulid } from 'ulid';

const DocumentClient = DynamoDBDocumentClient.from(new DynamoDBClient({ region: REGION }), {
  marshallOptions: {
    convertEmptyValues: false,
  },
});

const auditTrailTable = new Table({
  name: process.env.AUDIT_TRAIL_TABLE_NAME,
  partitionKey: 'pk',
  sortKey: 'sk',
  DocumentClient,
});

type AuditEvent = {
  userId: string;
  eventId: string;
  entityId: string;
  entityName: string;
};

type CompositeKey = {
  pk: string;
  sk: string;
};

const auditTrailEventEntity = new Entity<'AuditTrailEvent', AuditEvent, CompositeKey, typeof auditTrailTable>({
  name: 'AuditTrailEvent',
  table: auditTrailTable,

  attributes: {
    pk: {
      partitionKey: true,
      default: ({ userId }) => userId,
      dependsOn: ['accountId'],
      hidden: true,
    },
    sk: {
      sortKey: true,
      default: ({ eventId, recordType }) => `audit#${eventId}`,
      dependsOn: ['eventId'],
      hidden: true,
    },
   
    userId: {
      type: 'string',
      required: true,
    },
    eventId: {
      type: 'string',
      default: () => ulid(),
    },
    entityId: {
      type: 'string',
      required: true,
    },
    entityName: {
      type: 'string',
      required: true,
    },
  },
} as const);

[naorpeled]

@naorpeled

Thanks for your help with the query method. I'm now looking to get a similar setup for the put method, and I'm hitting some type issues. Any chance you could share some pointers on that as well? With this setup I'm getting Expression produces a union type that is too complex to represent..

Sorry to jump back into a closed issue, but your advice would be super helpful. Appreciate it.

Generic repo

import type { Entity, EntityItem, PutOptions, QueryOptions } from 'dynamodb-toolbox';

abstract class DynamoDbRepository<E extends Entity> {
  constructor(private readonly entity: E) {}

  put<O extends PutOptions<E>>(item: EntityItem<E>, options: O) {
    return this.entity.put(item, options);
  }

  query<O extends QueryOptions<E>>(pk: string, options: O) {
    return this.entity.query(pk, options);
  }

Entity repo:

export class AuditTrailRepository extends DynamoDbRepository<typeof auditTrailEventEntity> {
  constructor() {
    super(auditTrailEventEntity);
  }
}

Entity and a table:

import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient } from '@aws-sdk/lib-dynamodb';
import { Entity, Table } from 'dynamodb-toolbox';
import { ulid } from 'ulid';

const DocumentClient = DynamoDBDocumentClient.from(new DynamoDBClient({ region: REGION }), {
  marshallOptions: {
    convertEmptyValues: false,
  },
});

const auditTrailTable = new Table({
  name: process.env.AUDIT_TRAIL_TABLE_NAME,
  partitionKey: 'pk',
  sortKey: 'sk',
  DocumentClient,
});

export type AuditEvent = {
  userId: string;
  eventId: string;
  entityId: string;
  entityName: string;
};

type CompositeKey = {
  pk: string;
  sk: string;
};

export const auditTrailEventEntity = new Entity<'AuditTrailEvent', AuditEvent, CompositeKey, typeof auditTrailTable>({
  name: 'AuditTrailEvent',
  table: auditTrailTable,

  attributes: {
    pk: {
      partitionKey: true,
      default: ({ userId }) => userId,
      dependsOn: ['accountId'],
      hidden: true,
    },
    sk: {
      sortKey: true,
      default: ({ eventId, recordType }) => `audit#${eventId}`,
      dependsOn: ['eventId'],
      hidden: true,
    },
   
    userId: {
      type: 'string',
      required: true,
    },
    eventId: {
      type: 'string',
      default: () => ulid(),
    },
    entityId: {
      type: 'string',
      required: true,
    },
    entityName: {
      type: 'string',
      required: true,
    },
  },
} as const);

will try to take a look a bit later/tomorrow and ping you when I have some thoughts 🙏

[nevolgograd]

@naorpeled Cheers! Sorry for being so pushy, but perhaps you had time to look at the problem?
The more I look into it, the more it appears to be a dynamodb-toolbox issue; is there anything I can do about generic typesafety?

[naorpeled]

@naorpeled Cheers! Sorry for being so pushy, but perhaps you had time to look at the problem? The more I look into it, the more it appears to be a dynamodb-toolbox issue; is there anything I can do about generic typesafety?

@nevolgograd Feel free to ping me, all good!
Sorry that it's taking me longer, super busy with work and life related stuff.
The pre v1 versions are not perfect type safety wise, we have some issues and I try to fix as many as I can in each version so it's good to have this kind of feedback :)

I'll hopefully get to digging into it tonight, there might be some missing utility type defs that I need to add 🙏