Improved typescript experience

[DanRibbens]

Payload has been a typescript project since it launched and there have been large improvements over time. Now we want to address more typescript sticking points to make working with Payload even better.

Below are some of the items we are interested in:

• create and update local operations do not validate incoming data according to the passed generic type
• collection slug is not typed
• The Payload base class does not have types for what collections are available, etc.
• Can we remove the need for generating types and maybe write out a declaration file dynamically that includes all of those types? If not:
• Improve generated types - Right now all of our local APIs are generics, but what if we could assume the types from the passed collection string?
• Generated relationship field string | RelationshipType is not ideal. There are cases where you know that the doc is populated, so we should enforce RelationshipType and disregard string

@echocrow offered up some great detail for some of these in #1319.

Related:

• Hooks values `originalDoc` and `previousDoc` are badly typed #1851

[jmikrut]

Update - progress is going well! New patterns are being defined in the WIP PR below:

#1854

One of the next things on the radar (if anyone in the community is up for helping) is to create a set of TypeScript utilities that can take in an auto-generated collection / global type and "flatten" relationships into only an ID.

Right now, our exported types are as safe as possible when it comes to relationship fields.

For example, let's take the following configs:

const categories: CollectionConfig = {
  slug: 'categories',
  fields: [
    {
      name: 'title',
      type: 'text',
      required: true,
    }
  ]
}

const posts: CollectionConfig = {
  slug: 'posts',
  fields: [
    {
      name: 'title',
      type: 'text',
      required: 'true',
    },
    {
      name: 'category',
      type: 'relationship',
      relationTo: 'categories',
      required: true,
    }
  ]
}

Auto-generating types for these collections will result in the following:

export interface Category {
  title: string
}

export interface Post {
  title: string
  category: string | Category // could be populated, could not be
}

You'll see that the category property of Post is set up to be typed for being populated OR not being populated.

The task at hand

It would be great to have a TS utility that takes an auto-generated interface and flatten any unions into only the interfaces, or only the "basic" types of string or number.

This would be handy for typing the data argument of local API functions like payload.update, payload.create, etc.

Here's the intended API:

type PostData = CreateData<Post>

// PostData would be transformed into:

// {
//   title: string
//   category: string
// }

Additionally, we could create a utility that would dynamically specify population depth, as such:

type PostData = WithDepth<Post, 2>

// PostData would be transformed into:

// {
//   title: string
//   category: WithDepth<Category, 1>
// }

We are going to be looking into building this stuff over the next few weeks but would love a hand from anyone willing to contribute!

[jmikrut]

Ummmm this is AMAZING.

I took a brief pass at what you've got in the Stackblitz post above and it's epic.

As for adjusting our generated interfaces, that could be a tricky one, as we are pretty much at the mercy of what [json-schema-to-typescript](https://github.com/bcherny/json-schema-to-typescript) supports there. It's worth a look though.

I will follow up here shortly - got some big things on the radar and this will be awesome to implement!

[echocrow]

Glad to hear this is useful!

(I made a quick edit, simplifying the DropString<...> to a built-in Exclude<..., string>; couldn't let that slip once I realized TS already got this covered.)

Happy to further help out with this, could be a fun TS Type Challenge.

Re json-schema-to-typescript: I've hit that limitation there too; IIRC it does not support generics. Back then I worked around that via some basic (albeit hacky) string manipulation - at the end of the day, it's just text.

Another workaround might be to switch to a more specialized library: Instead of building an abstract JSON schema just to then generate TS types, a library focused specifically on generating declarations could offer wider feature support, e.g. dts-dom.

[Snailedlt]

Does depth here refer to the depth parameter in the REST-API?

If so, here's a related discord thread on what kinds of problems having depth as a parameter to the type would solve:
https://discord.com/channels/967097582721572934/1169683227477282906

[echocrow]

Does depth here refer to the depth parameter in the REST-API?

@Snailedlt correct, the Depth type here is referring to the depth param used in the REST API or the Local API SDK.

That said, from personal experience at my company over the last few months: Internally, we have actually moved away from worrying about accurately resolving relationship types at the query level. IIRC, we felt the type definitions and previews, while technically more accurate when implemented correctly, provided a worse DX. Instead of previewing a cleaner string | MyThing | undefined, we'd see a wildly less friendly conglomerate of types. I believe there were other reasons, but it's been a while.
Instead, we've resorted to just wrapping loaded references in a very basic deref(...) function, which returns a clean TS type of MyThing | undefined, and provides logs when the item passed was not correctly de-references. This way we still get type safety, and don't need to as-cast anything.

[Snailedlt]

@jmikrut I noticed that this issue got labeled as roadmap-complete, but I don't see any documentation on the withDepth util types (Post<Depth> or WithDepth<Post, 2>) that were discussed here.

Is that being tracked in a different discussion or issue, or has it been lost in the sea of other discussions?

[Elliot67]

Don't know if it's useful for you, but since my project is using building blocks, I achieved some sort of type safety with the followings :

Getting the type of a single block

// Where Homepage is a generated payload interface from a global page containing a block
// field called `blocks` where every possible blocks are defined
export type BlockTypePicker<T> = Extract<Homepage['blocks'][number], { blockType: T }>;

Force the relation of a field on that block

export type RemoveStringType<T> = T extends string ? never : T;

// Remove type string of a relation property (MK) in an object (T)
export type ForceRelationObj<T extends Record<any, any>, MK extends keyof T> = {
  [K in keyof T]: K extends MK
    ? T[MK] extends Array<unknown>
      ? RemoveStringType<T[MK][number]>[]
      : RemoveStringType<T[MK]>
    : T[K];
};

Which can be used like that

type MyObject = {
  media: Media | string;
  title: string;
};

type BetterTypedObject = ForceRelationObj<MyObject , 'media'>;
// {
//   media: Media;
//   title: string;
// }

And when combining the utilities together, you get a fully typed block

type FullyTypedBlock = ForceRelationObj<BlockTypePicker<'hero-banner'>, 'thumbnail'>

In my case I have to specify the field name thumbnail, but it should be possible to change the K extends MK condition to an array of fields, or even something that detects payload relations.

[MoSattler]

Super excited about this! thank you all so much!

[xHomu]

Local API type inferences are GOAT!

[alamit]

Hello, just reposting here what I was posting on Discord:

At the moment I am using the following workaround (it's kinda Regex ninja, but works in my case):

1. Defined a scripts/gen-generic-types.js

const fs = require('fs');
const path = require('path');

const file = fs.readFileSync(path.join(__dirname, '../src/payload-types.ts'), 'utf8');

const types = Array.from(file.matchAll(/export interface (.*) \{/g)).map((match) => match[1]);

const start = file.indexOf('export');

const withDecrement =
  file.substring(0, start) +
  'type Decrement = [never, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10];\n' +
  file.substring(start);

const regex1 = RegExp(`(string|string\\[\\]) \\| (${types.join('|')})(\\[\\])?`, 'g');
const withDepth = withDecrement.replace(
  regex1,
  'D extends Decrement[number] ? D extends 0 ? $1 : $2<Decrement[D]>$3 : $1 | $2$3'
);
const regex2 = RegExp(`(export interface )(${types.join('|')})`, 'g');
const withTypes = withDepth.replace(regex2, '$1$2<D extends Decrement[number] | void = void>');

const regex3 = RegExp(`(: )(${types.join('|')})\;`, 'g');
const withConfig = withTypes.replace(regex3, '$1$2<D>;');

try {
  console.log('Creating generic depth types...');
  fs.writeFileSync(path.join(__dirname, '../src/payload-types.ts'), withConfig);
  console.log('Done.');
} catch (err) {
  console.error(err);
}

2. Create the following package.json script:

{
  "scripts": "cros-env PAYLOAD_CONFIG_PATH=src/payload.config.js payload generate:types && node scripts/gen-generic-types.js"
}

NB:

• It has the advantage to be backward compatible with previous types, as the default type stays the same for every type.
• I haven't tested it yet with polymorphic relation types, but it probably won't work yet.
• Full Regex is hardly maintainable.
• Works in my case though.

I would happily contribute to enhance this as a plugin or something and I am thinking of the following roadmap:

• Add support for polymorphic types.
• Add some utility functions, such as:

function decreaseDepth(object: Collection<D extends number>): Collection<Decrement<D>>

What do you think?