Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simple use of serialize<T>(data) is not working - "No type information received. Circular import or no runtime type available." #562

Open
juanda147 opened this issue Mar 28, 2024 · 33 comments

Comments

@juanda147
Copy link

Hi, I'm working on POCs to migrate from @marcj/marshal to @deepkit/type . Following the steps here I should get a serialized object. I'm not doing anything complex, and I removed the @f annotation from the class just in case. Using the serialize function like below, I'm always getting:

"No type information received. Circular import or no runtime type available."

image

@marcj
Copy link
Member

marcj commented Mar 28, 2024

did you enable reflection in tsconfig and installed @deepkit/type-compiler?

@juanda147
Copy link
Author

Yes I have installed @deepkit/type-compiler. Do I need an additional library to enable reflection in tsconfig?

@marcj
Copy link
Member

marcj commented Mar 28, 2024

you have to enable reflection in tsconfig, see https://deepkit.io/documentation/runtime-types/getting-started

@juanda147
Copy link
Author

I missed the "reflection": true. But after adding it I still get the error

@marcus-sa
Copy link
Contributor

marcus-sa commented Mar 28, 2024

@juanda147 you also have to run deepkit-type-install. I suggest adding it to postinstall script in package.json.

@juanda147
Copy link
Author

juanda147 commented Apr 1, 2024

I have both "@deepkit/type": "^1.0.1-alpha.143", and "@deepkit/type-compiler": "^1.0.1-alpha.143", installed.

The error I got is happening here:

image

@marcj
Copy link
Member

marcj commented Apr 1, 2024

and how do you build and execute it?

@juanda147
Copy link
Author

it is failing at running the tests

@marcj
Copy link
Member

marcj commented Apr 1, 2024

what test framework

@marcj
Copy link
Member

marcj commented Apr 1, 2024

is the test framework using tsc? Use DEBUG=deepkit env variable to get more information

@juanda147
Copy link
Author

jest

@marcj
Copy link
Member

marcj commented Apr 1, 2024

can't help like that must be an user error since we use jest as well and it works very well. I would make sure that ts-jest is used correctly, the correct tsconfig is loaded (see my DEBUG note), deepkit was correctly installed as seen in the docs. then it will work

@juanda147
Copy link
Author

I created an isolated project with some basic configuration and basically the same as the main project, and I got the same error, please find attached zip file.
test_deepkit.zip

@marcus-sa
Copy link
Contributor

marcus-sa commented Apr 1, 2024

@juanda147 your repro is broken

marcus-sa@pop-os:~/Downloads/test_deepkit/playground$ npx jest
 FAIL  DeepKit/Playground.test.ts
  ● given a serialized PhoneNumber › when the PhoneNumber class is deserialized › Then the class get methods functions as expected

    TypeError: Cannot read properties of undefined (reading 'getNationalNumber')

      49 |
      50 |   public get formatPhoneNumberAsE164(): string {
    > 51 |     return this.phoneUtil.format(this.parsedNumber, libphonenumber.PhoneNumberFormat.E164);
         |                           ^
      52 |   }
      53 |
      54 |   private static parsePhoneNumber = function (

      at i18n.phonenumbers.PhoneNumberUtil.format (node_modules/google-libphonenumber/dist/libphonenumber.js:935:74)
      at PhoneNumber.get formatPhoneNumberAsE164 [as formatPhoneNumberAsE164] (DeepKit/entities/PhoneNumber.ts:51:27)
      at Object.<anonymous> (DeepKit/Playground.test.ts:18:48)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        0.888 s

Did you do what I mentioned here #562 (comment) ?

@juanda147
Copy link
Author

juanda147 commented Apr 1, 2024

This is really odd. I removed all external dependencies and left a very simple class, and still seeing the error. Just run it using: npx ts-node .\DeepKit\Run.ts

Thanks a lot for your help so far

playground_poco_class.zip

image

@marcj
Copy link
Member

marcj commented Apr 1, 2024

Can you please provide a proper Git repository. We are here on Github, there is no need to share source-code as Zip files.

@juanda147
Copy link
Author

@marcj
Copy link
Member

marcj commented Apr 1, 2024

Works fine

docker run -ti --rm node:20-alpine /bin/sh
/ # apk add git openssh
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/aarch64/APKINDEX.tar.gz
(1/20) Installing ca-certificates (20240226-r0)
(2/20) Installing brotli-libs (1.1.0-r1)
(3/20) Installing c-ares (1.27.0-r0)
(4/20) Installing libunistring (1.1-r2)
(5/20) Installing libidn2 (2.3.4-r4)
(6/20) Installing nghttp2-libs (1.58.0-r0)
(7/20) Installing libcurl (8.5.0-r0)
(8/20) Installing libexpat (2.6.2-r0)
(9/20) Installing pcre2 (10.42-r2)
(10/20) Installing git (2.43.0-r0)
(11/20) Installing openssh-keygen (9.6_p1-r0)
(12/20) Installing ncurses-terminfo-base (6.4_p20231125-r0)
(13/20) Installing libncursesw (6.4_p20231125-r0)
(14/20) Installing libedit (20230828.3.1-r3)
(15/20) Installing openssh-client-common (9.6_p1-r0)
(16/20) Installing openssh-client-default (9.6_p1-r0)
(17/20) Installing openssh-sftp-server (9.6_p1-r0)
(18/20) Installing openssh-server-common (9.6_p1-r0)
(19/20) Installing openssh-server (9.6_p1-r0)
(20/20) Installing openssh (9.6_p1-r0)
Executing busybox-1.36.1-r15.trigger
Executing ca-certificates-20240226-r0.trigger
OK: 30 MiB in 37 packages

/ # git clone https://github.com/juanda147/deepkit-test.git
Cloning into 'deepkit-test'...
remote: Enumerating objects: 14, done.
remote: Counting objects: 100% (14/14), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 14 (delta 1), reused 10 (delta 0), pack-reused 0
Receiving objects: 100% (14/14), 9.36 KiB | 2.34 MiB/s, done.
Resolving deltas: 100% (1/1), done.

/ # cd deepkit-test/playground

/deepkit-test/playground # npm i

added 52 packages, and audited 53 packages in 2s

9 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities

/deepkit-test/playground # ./node_modules/.bin/deepkit-type-install
Deepkit Type: Injected TypeScript transformer at /deepkit-test/playground/node_modules/typescript/lib

/deepkit-test/playground # npx ts-node ./DeepKit/Run.ts
{ empCode: 1, empName: 'John Doe' }
1

@juanda147
Copy link
Author

I don't understand at this point what could be different on my side

image

@marcus-sa
Copy link
Contributor

marcus-sa commented Apr 1, 2024

@juanda147 because you still haven't done what I've asked you to do twice already.
#562 (comment)

@vany0114
Copy link

vany0114 commented Apr 2, 2024

Hey @marcj @marcus-sa first things first, thanks for the help here! really appreciate it since we're trying to run some quick POCs before migrating from @marcj/marshal to @deepkit/type.

One of them is a wrapper that we have for an external library (google-libphonenumber). It has a class member called parsedNumber which type comes from the external library.

So for some reason when you serialize the wrapper it returns nothing which causes an issue when afterwards it's deserialized as the parse Number member is undefined:
https://github.com/juanda147/deepkit-test/blob/9c2344944e097e12bd9775e0377b48fdf67f440c/playground/DeepKit/Run.ts#L17

By using @marcj/marshal we just annotate the member using the @f.any() and it works just fine, so not sure what's the right way to handle those kind of scenarios with this new package.

Do we need to create the "mapping" using the annotateClass? if so that would be kinda convoluted since we would need kinda duplicate the type definition that lives in the given package, also not sure whether for the google-libphonenumber PhoneNumber class it would work as it only expose methods.

Thanks in advance!

@marcj
Copy link
Member

marcj commented Apr 2, 2024

@vany0114 can you show me how you annotated the member of PhoneNumber in Marshal? I was looking at its definition at https://www.npmjs.com/package/@types/google-libphonenumber?activeTab=code but it seems PhoneNumber doesn't have any class property members, so I wonder how you did that.

You can use annotateClass if PhoneNumber is really a class, but this only works if there are actually data members, which I did not find as mentioned.

Alternatively you can register your own class serializer handler, see https://deepkit.io/documentation/runtime-types/custom-serializer

import * as libphonenumber from 'google-libphonenumber';
import { serializer } from '@deepkit/type'; //default serializer used in cast/is/serialize/etc functions

serializer.serializeRegistry.registerClass(libphonenumber.PhoneNumber, (type, state) => {
    // ${state.accessor} is the reference to the runtime PhoneNumber instance. do whatever to convert it to JSON serialisable structure like numbers/string/objects.
    state.setContext({ phoneUtil: libphonenumber.PhoneNumberUtil.getInstance() });
    state.addSetter(`phoneUtil.format(${state.accessor}, libphonenumber.PhoneNumberFormat.NATIONAL);`);
});

serializer.deserializeRegistry.registerClass(libphonenumber.PhoneNumber, (type, state) => {
    // ${state.accessor} is the value set by the serialize code above (e.g. numbers/string/objects, whatever you used)
    state.setContext({ phoneUtil: libphonenumber.PhoneNumberUtil.getInstance() });
    state.addSetter(`phoneUtil.parse(${state.accessor}, region)`);
});

this way you can just use

import { PhoneNumber }  from 'google-libphonenumber';

class User {
   phone?: PhoneNumber;
}

and everything just works when doing cast<User>({phone: 1234566}).

if you also want to make is<User>({phone: 'invalidnumber'}) to work, you might want to add type guards:

serializer.typeGuards.getRegistry(1).registerClass(libphonenumber.PhoneNumber, (type, state) => {
    // ${state.accessor} is the reference to the runtime PhoneNumber instance, do whatever to check if is valid by setting false/true
    state.setContext({ phoneUtil: libphonenumber.PhoneNumberUtil.getInstance() });
    state.addSetter(`phoneUtil.isPossibleNumber(${state.accessor}) === true && phoneUtil.isValidNumber(${state.accessor})`);
});

if you go with the way to make the default serializer aware of your PhoneNumber class, then you can literally use this type everywhere and get proper serialization/validation out of the box, including with cast, is, assert, serialize, deserialize, validate plus if you use the deepkit framework as whole, all http requests are validated automatically with the same mechanisms:

router.get('/user/phone-number/:number', (number: PhoneNumberUtil) {
    number instanceof PhoneNumberUtil; // always true
});

# curl http://localhost/user/phone-number/12345456
# curl http://localhost/user/phone-number/invalid-number -> fails

@marcj
Copy link
Member

marcj commented Apr 2, 2024

regarding the custom serializer, we might have to check if it works with packages without runtimes types like google-libphonenumber though. it might be you have to define you own type alias in your own code where runtime types are enabled and then use this type everywhere, e.g.

# my-types.ts
import { PhoneNumber }  from 'google-libphonenumber';

export class Phone extends PhoneNumber {}

serializer.serializeRegistry.registerClass(Phone, (type, state) => {
   //...
});
import { Phone } from './my-types.js';

export class User {
   phone?: Phone;
}

@vany0114
Copy link

vany0114 commented Apr 2, 2024

@marcj this is how we annotate it using @marcj/marshal

image

And thanks for the detailed explanation, we'll give it a try!

BTW has @deepkit/type a delegate similar to @marcj/marshal @OnLoad()? we also use in some classes to initialize stuff after deserialization and I was wondering what's the right way to do so with this package.

@marcj
Copy link
Member

marcj commented Apr 2, 2024

No, @OnLooad() doesn't exist anymore. Maybe in the constructor is good enough? Or you can add additional calls to the serializer templates

serializer.deserializeRegistry.registerClass(libphonenumber.PhoneNumber, (type, state) => {
   // as described set the value via state.addSetter(`xy`);
});
serializer.deserializeRegistry.appendClass(libphonenumber.PhoneNumber, (type, state) => {
   //state.accessor is the value xy now from previous step
   state.addCode(`${state.accessor}.additionalCalls()`);
});

@vany0114
Copy link

vany0114 commented Apr 2, 2024

@marcj this is how it looks like after your suggestion about using a custom serializer, in this case, I just added the setter w/o doing anything else but just passing the current state. And you were right, I had to define the alias as well for it to work.

serializer.serializeRegistry.registerClass(GooglePhone, (_, state) => {
    state.addSetter(`${state.accessor}`);
});

serializer.deserializeRegistry.registerClass(GooglePhone, (_, state) => {
    state.addSetter(state.accessor);
});

@marcj Thanks again, we'll probably keep bugging you as we continue with the migration to @deepkit/type

@juanda147
Copy link
Author

Hi, thanks for all your help so far. I was wondering if there is a replace for @f.type(MyClass) and @f.asName('')

@marcus-sa
Copy link
Contributor

marcus-sa commented Apr 6, 2024

@juanda147

  1. You just reference classes directly as a type
  2. There's a MapName type annotation. See https://deepkit.io/documentation/runtime-types/types

@juanda147
Copy link
Author

Hi Again, we've been facing an issue trying to deserialize a generic type. I added a sample here https://github.com/juanda147/deepkit-test/blob/main/playground/Marshal/DeserializeGeneric.ts . Our question is if we can get rid off the type parameter type: new (...args: any ) => S and let deepkit to resolve the generic type using reflection?

@marcj
Copy link
Member

marcj commented Apr 17, 2024

@juanda147

    import { ClassType } from '@deepkit/core';
    import { deserialize, resolveReceiveType } from '@deepkit/type';

    const testDeserialized = <T>(payload: Record<string, unknown>, type: ClassType<T>): T => {
        return deserialize(payload, undefined, undefined, undefined, resolveReceiveType(type));
    };

    class TestSource {
        name: string = '';
    }
    const payload = { name: 23 };
    const res = testDeserialized(payload, TestSource);
    console.log(res);

@marcj
Copy link
Member

marcj commented Apr 17, 2024

@juanda147 alternatively

    import { deserialize, resolveReceiveType, ReceiveType } from '@deepkit/type';

    const testDeserialized = <T>(payload: Record<string, unknown>, type?: ReceiveType<T>): T => {
        return deserialize(payload, undefined, undefined, undefined, resolveReceiveType(type));
    };

    class TestSource {
        name: string = '';
    }
    const payload = { name: 23 };
    const res = testDeserialized<TestSource>(payload);
    console.log(res);

@juanda147
Copy link
Author

Hi, we still have some issues with the serializing/deserializing objects. We've been trying all of your suggestions but on every iteration some different fails. We may get the object with a few properties, the entire object with all properties as undefined, below are some screenshots describing our situations:

With the below implementation it looks like this, ignoring all properties from Order class and leaving only the props from the AggregateRoot:
image

Above object should look something like this:
image

using serializer.serializeRegistry. If I'm serializing, why am I getting the object typed as the class? Here some properties are undefined for some odd reason.

const serializedOrder = serialize<Order>(testOrder, undefined, undefined, undefined, resolveReceiveType(Order));
   
 serializer.serializeRegistry.registerClass(Order, (_, state) => {
      state.addSetter(`${state.accessor}`);
    });

image

using the @marcj/marshal library, returns the object as needed, plus, no typed associated to it:
const serializedOrder = classToPlain(Order, testOrder);
image

@marcj
Copy link
Member

marcj commented May 3, 2024

I assume this code is wrong:

serializer.serializeRegistry.registerClass(Order, (_, state) => {
  state.addSetter(`${state.accessor}`);
});

it does basically nothing. It tells the serializer to use a custom serializer template for Order, but the given template just forwards the value as-is, without doing any serialization. So this code explains why you get Order instance as serialization result. The code is wrong and needs to be removed.

It's always better to provide fully functioning code examples/reproduction. This way I can precisely see what is going on, what is the expectation, and where the error is. With screenshots and snippets alone it's a guessing game.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants