Manually generating metadata.lib.js

[progrium]

I'm using Deno and esbuild, so I'm excited to hear you'll be supporting esbuild soon. However, I'm guessing your approach will be similar to what you can already do to use transformers in esbuild, which comes at the cost of speed ... the whole point of using esbuild and avoiding tsc. So I'm wondering if I can manually create the metadata.lib.js file, without using ttsc to also compile TS.

The thought is during development I can let esbuild create bundles quickly on file changes, but if an API changes that will impact my use of reflection, I can manually generate a new metadata.lib.js file.

[progrium]

I guess the way this is designed, it creates metadata based on usage of getType and does transform the call, so this would be pointless yea?

[Hookyns]

Hello,
approach of the new system, that should support all the build tools (including esbuild), is completely different from how it works now, even how it works in current closed Alpha.

I split it into two parts. Generation of the "metadata library" (one big bundle with metadata of all the modules and all the types inside project) is one separated thing; the second thing is code transformation that solves the getType call expressions.

I created tool typegen. It is standalone cmd tool that generates the metadata library. typegen is based on typescript and its typechecker but it is parallel and it use custom caching, so it's quite fast so far. It has watch mode ofc.

The second part are thin plugins for each build system that will only replace getType calls by logic that will return the type from the metadata library. So those plugins will be fast, without typescript.

I'll complete a prototype of typegen in a week IMHO.

Runtime part match current Alpha; it is quite different from current version of tst-reflect. The principle is the same but interface/API is much better.

[Hookyns]

Good to mention... projects using something else than typescript compiler will not be able to solve inference; maybe the simple ones though.

In this example I cannot know what the type of foo variable is because the type is not specified; it have to be infered from the usage.

import { Foo } from "./foo";
export const foo = new Foo();

That was quite simple example so it can be implemented, but something like the next example will not be possible without type checker:

import { doSomething } from "./foo";
export const foo = doSomething();

EDIT: This can be solved when Donny completes his stc.

[progrium]

This is great to hear. I'm wondering if it would be possible to use the system without getType transformations?

I've been working on an alternative to this project for a while now (a subset of functionality) to satisfy my requirements. Until now, it's also written around the Typescript compiler so it has the same issue. Since the "schema" data needed for reflection is only declarations, I think it's possible to write a tool that can generate this without the need of a full Typescript parser. I'm going to investigate a little further as this would result in a tool written in Go not dependent on tsc. It's possible this could be adopted here in the place of typegen, or at least maybe be a compatible alternative.

Meanwhile, I've been working on a way to associate at least classes and functions with their metadata with as little transformation as possible. Maybe it ends up being equivalent to your transformation process, but I'm trying an approach that doesn't need type information.

As an example, the general idea is to append a fully qualified name to symbols. So in your cases above:

import { Foo } from "./foo";
export const foo = new Foo();

foo.constructor.__fqn would provide a name like "path/to/foo/Foo", which could be looked up in metadata. Similar with the doSomething function, appending a key to lookup its type information on the function itself. This would happen during bundling, but can be done without the AST in many cases.

And yes, stc would help hugely, but no idea when it will be usable.

[Hookyns]

This is great to hear. I'm wondering if it would be possible to use the system without getType transformations?

Yes, it is possible. You can test it in a week or so. 👍
You can import the library and you'll have access to every module and every type in your project.

import { Metadata } from "./metadata.typelib";

Metadata.getTypes() // get all the types in project
Metadata.getModules() // get all the modules in project

const module = Metadata.getModules()[0];
module.getTypes(); // All types in that module

const importedModule: { [exportName: string]: any } = await module.import(); // import of the module

or at least maybe be a compatible alternative.
My idea behind the new version is something like runtime reflection standard. I think it would be great if people can change generators without changing the runtime. So if it will be compatible it will be great. 🙂

It sounds you rly do a lot of thing the same way.

Alpha is based on generated type identifiers which are generated from package name, module path and type names; even nested types are supported.
Alpha's version of that id generator is based on TS's type checker but I rewrite the algorithm for the typegen version that is based just on the AST so the same algorithm can be writen in Rust, Go etc. so plugin for eg. SWC can pair types with those in metadata library generated by typegen.

BTW
You can take a look at a playground with the Alpha https://stackblitz.com/edit/rttist-playground?file=src%2Findex.ts

[progrium]

I'm very excited to try this!

[Hookyns]

Just to make it clear (or more clear) about the projects/version,

tst-reflect (runtime) and the tst-reflect-transformer are the oldest ugly things that I didn't toucher for about a year. It works somehow but it has a lof of issues caused by the design.

So I started with the Alpha. I created whole new runtime package that replace the tst-reflect and I took tst-reflect-transformer and I rewrote huge part of it so it worked quite nice until,.. everything started changing; updates of TS, updates of Webpack, ts-loader etc.. I have decided to discontinue the development of the Alpha, because it's tight coupled to everything and it's broken majority of time because of changes in tooling.

So I created the typegen. Whole new system how the metadata are generated. Separated from project's tooling. Running on its own.