Code debugging and database migrations

[martin-braun]

Hi there. :)

I like to support directus since I discovered and used it in a project around one year ago. A very small project (licensing backend server with administrative capabilities). I think directus is brilliant, especially since it is open-source and free. I'm facing a larger project soon and want to consider directus as backend again to bypass all the necessary work for building an administrative backend once again, my customer is very happy with it and I would have more time for the precious front-end.

However, in my experience directus was rather hard to develop for:

• I use Visual Studio Code, but despite directus running on node, I was unable to debug my extensions, because I think directus injects the extension scripts on runtime and runs them somewhere else
• I'm sure it is possible to write TypeScript, but it was not worth to bother, I would like to change that, but there is no sample project and no typings
• The inability to step into the code with debugger and investigate variables and possible types made it very hard to learn how to do simple things in such extensions
• directus' database structure is setup in the UI, so there is no database migration (for own models) which makes it hard to incorporate it into a repository with growing models, although I could imagine that directus could support creating snapshots from those models which allow to upgrade/downgrade migrations

Please note: With "database migrations" I'm not talking about the system tables of directus. I'm rather talking about own tables that might get changed, added or removed in the process lifetime of a product that uses directus as database handler. This on the equivalent of Artisans or EntityFrameworks "create migration" process to alter or change the layout of the database. Something that is done in directus by simply editing the database on the UI, but I want to tie the current layout of the database to a commit on the repository and need the ability to rollback to test older versions with compatible database layouts. Strapi has a plugin for that, but I really want directus to be a successor here.

Did any of those things changed in the time? Can I get happy with directus? What are my options to address those problems?

[benhaynes]

I just wanted to say think you for the kind words — we'll get you a response tomorrow!

[rijkvanzanten]

• Debugging

Directus is a node process that uses regular require() to inject the custom logic. Therefore, if you run Directus itself from the VS Code debugger, you should be able to debug the extensions as well 🤔

• Write typescript where? In custom extensions? The extensions-sdk can also scaffold extensions in typescript iirc

• See pt 1 :)

• There's a schema snapshot and schema apply command you can use to generate and apply schema files that can be tracked 🙂

[martin-braun]

Thanks for the heads up, I will look into this and return to this discussion.

[kevincam3]

I use WebStorm any help with how to setup it up in Webstorm. I've tried as well with no sucess.

[martin-braun]

@rijkvanzanten It's been a while, I wish to pickup my old discussion as I work on a new project in which I picked Directus once more eventually. I want to share my discoveries to give feedback and help others finding this thread.

I found https://github.com/bcc-code/directus-schema-sync which seems to leverage schema snapshot and schema apply internally, but it also automate the process via environment variables and it also exports and imports data, so I can specify default users, roles, etc. I think it would be awesome to have something like this without the need of an extension.

In regards of TypeScript and debugging in general, I figured loading extensions the old fashioned way doesn't seem to work anymore. The docs are horrible loose on that detail. The docs however are rich on building new rich extensions, but Directus behavior of loading those extensions other than through the package.json is undocumented. I wanted to have an endpoint extension as part of my project (not as separate extension) and it took a lot of trial and error to figure out that I can simply add an extension (created with npx create-directus-extension@latest) into the extensions directory and when I build it, Directus will see the dist folder and load my extension (no more /custom/<name>/ route, route is now /<extension-name>).

I think this use case is clunky, because I have to hack a bit within the root package.json to auto-build my endpoint extension. The process is flawless for extern extensions that I install via npm, but not very straight-forward for embedded extensions.

Anyways, I felt I need to share my experience in that regard.

Very cool to have this working eventually.

[flevi29]

Yeah, the docs aren't very detailed, I too struggled way more than I could've with the new extensions loading format. I was thinking about using the bundle extension type to simplify things, but my toilet paper has better explanations on its packaging than Directus docs on how to set up a bundle extension.

Also I hoped the new way of loading extensions would make things simpler, but no, they just made it harder, because now I need a package.json for each of my extensions, so in my build script I also need to write the package.json -s. Previously I have proposed to have an option of a JS file describing each extension and their path, as that would make bundling extensions a lot simpler, but that was ignored.

[flevi29]

UPDATE: Better do it like in the comment below, so you don't have to rely on unexported node_modules file: #13743 (reply in thread)

For anyone else stumbling here in search for a satisfactory answer on how to debug Directus API extensions:
The methods of doing so suggested by the maintainers and others aren't ideal, and are awkward at best in my opinion.

Instead you can just:
scripts/start.js:

// your setup code here, for example environment setup ....

import "../node_modules/directus/cli.js"; /* (".." or whatever path leads you to where node_modules is located) */

Make any necessary setup smoother, more readable and dynamic.

Ideally this cli.js should be exported for this reason, but it's not.

[flevi29]

I do get some weird breaks in two initiated threads, this might be because of how cli.js works, or tsx and lastly perhaps WebStorm. But hey, it works and I can set up my env however I want this way.

[MichaelFBA]

Can you provide an example of your startup script?

[flevi29]

Apparently the cli version is only needed if you also want Directus to check if it's out of date. That's not necessary, so we import @directus/api/server.

Now I'm doing it like so:
start.ts

import { env as processEnv, default as process } from "node:process";
import { fileURLToPath, URL } from "node:url";
import { access } from "node:fs/promises";
import { env } from "./directus.config.js";

const extensionsDir = fileURLToPath(new URL("extensions", import.meta.url));

try {
  await access(extensionsDir);
} catch (error: unknown) {
  process.exitCode = 1;
  throw error;
}

env.EXTENSIONS_PATH = extensionsDir;

Object.assign(processEnv, env);

// Everything Directus must be imported after resolving env, because as soon
// as anything Directus related is imported, Directus reads env
const { default: logger } = await import("@directus/api/logger");
(await import("@directus/api/server"))
  .startServer()
  .catch((reason: unknown) => {
    logger.error(reason);
    process.exitCode = 1;
  });

Checking extensions dir is unnecessary, I just do it to make sure things are in place, or at least partially so.

[flevi29]

I'm dynamically importing Directus, because otherwise esbuild is hell bent on putting every import at the top, nullifying my env changes. You might not have to do this, if your bundler or transpiler doesn't do that, or you're writing plain javascript.