Proposal: Support for multiple tsconfig.json per project

[SetTrend]

Referencing #3645:


I agree with @Eisenspalter: Multiple tsconfig files are indispensable for large enterprise apps.

@mhegazy : As you wrote you don't want to support tsconfig.json file inheritance, I'd like to propose the following:

My proposal

TSC should come with a new command line parameter, enabling the following behaviour:

1. The TSC process will browse the project directory for tsconfig.json files.
2. If no tsconfig.json files will be found, TSC proceeds with its normal behaviour: Compiling all .ts files found in the projects sub directories, using the default configuration.
3. If, instead, any tsconfig.json file will be found, the TSC process refrains from compiling. Instead it spawns a new TSC child process for the tsconfig.json file and continues browsing the project's sub directories for tsconfig.json files. It does this for each tsconfig.json file it reaches.
4. With each new TSC process it creates it sets the -project parameter and current working directory of that child process to the tsconfig.json file's.
5. Any TSC process will stop evaluating a sub directory (= any directory different from -project) if it finds a tsconfig.json file in there.

The new command line parameter (e.g. -browseConfig) should require a directory information, similar to the -project command line parameter. This will allow the user to provide a dedicated sub directory as root for scanning.

#### Example

The following example depicts the original TSC process to start at -browseConfig "." ("Project root"):

TSC -browseConfig "."

Browsing the project directory tree it finds three tsconfig.json files. For each of the tsconfig.json files it spawns a new TSC process:

CD "scripts"
TSC

CD "scripts/sub directory n"
TSC

CD "scripts/sub directory n/sub directory n"
TSC

So there will be four TSC processes running in parallel (good for multi-core systems 😉), each evaluating its sub tree of the project. The first process won't perform any transpilation. It only browses the directory tree.

The following images depict the main TSC process and the three sub processes, highlighting the files they transpile:

To speed up browsing for tsconfig.json files, the first/main/original TSC process could/should better be started using: TSC -browseConfig "./scripts/"

[paulvanbrenk]

What you describe is something a build engine can/should provide and not something the compiler provides. I'm working on making sure MSBuild does what you describe, and you should see that in the 1.8 timeframe.

[lazdmx]

@paulvanbrenk I'm voting for this job to be done by compiler but not by an external tool like MSBuild, since:

• MSBuild tool must support all environments that tsc does (linux, osx etc)
• it is common to build/compile files during deployment process, and MSBuild (or any other tool) will complicate deployment at whole
• devs will have to juggle with another one tool and handle version compatibility and interoperability

[paulvanbrenk]

These tools (i.e. grunt/gulp/msbuild) are already in wide use, and used for deployment, minification, etc. They are perfectly situated for figuring out where the tsconfig.json files live, managing dependencies, invoking process across multiple cores, machines, handle incremental compilation, etc. Those are all solved problems in these tools.
To get any decent support the TypeScript compiler would need to duplicate all the great work, and we prefer to add value to the language... not re-invent the wheel.

[lazdmx]

I use grunt/gulp in my day-by-day work and I don't happy with them (especially when working with big projects with multiple submodules). I've opened #3645 not because these tools are perfect and I see problems that I would like to be handled by compiler.

@paulvanbrenk There is a real-world usecase when we want to deploy a codebase into platform like Heroku, so my question is: will MSbuild allows to build project in npm postinstall script when codebase is deployed into Heroku (please consider platform environment and restrictions)?

[paulvanbrenk]

As I said tsc.exe is not a general purpose build tool; other tools like Grunt/Gulp/MSBuild are designed for that purpose. Trying to bolt that functionality on the TypeScript compiler will result in duplicate work, and wasted effort which will not add value for most users.

If you're un-happy with Grunt/Gulp/MSBuild you should file bugs on those projects, so they can be improved for all users not just the TypeScript users.

[lazdmx]

In this issue tracker we discuss a feature that I and other people want to be a part of tsc. MSBuild like any other tool may evolve separately, and I would be happy to know that there are tools that provide some kind of solution. But since TS is getting better and more teams start to use it in their daily work support for subprojects becomes more indispensable.

It is ok to use Grunt/Gulp/MSBuild but it useless for many devs working in linux, osx environments. It like using notepad.exe for source editing and ignoring a needs in syntax highlighting, autocompletion and other stuff.

[SetTrend]

I agree with lazutkin:

Two examples:

• ".gitignore" is interpreted intrinsically by git, not by gulp/grunt/MSBuild.
• "web.config" is interpreted intrinsically by ASP.NET, not by gulp/grunt/MSBuild

Extracting the management of tsconfig.json files out of TSC adds unnecessary configuration overhead to the programmer's project, because while TSC is browsing the files/directories anyway and is able to interpret tsconfig.json files intrinsically while doing this, an external configuration file would be necessary to be maintained manually in adddition to maintaining a number of tsconfig.json files.

So, extracting this feature out of TSC would require the programmer to always edit things at two different places: When moving/creating/deleting a tsconfig.json file, he would always be required to maintain the corresponding gulp/grunt/MSBuild file, too, to keep both in sync.


If inheritance was true for the tsconfig.json file, things could be so easy: Whenever the transpiler reaches a directory for transpiling, it would first look for a tsconfig.json file. If it exists, the transpiler would push its current run-time settings on a stack, overwrite parts of its current settings with the data from the tsconfig.json file and transpile the directory's content. Nothing can be simpler.

[SetTrend]

May I add that this thread is being closed a bit too quickly for my liking.

We are not trolling around here but discussing a constructive improvement to the project. If I remember it right, the term "Open Source" implicates active participation. That's what's actually happening here.

[paulvanbrenk]

@lazutkin gulp/grunt etc work great in non-windows systems.

@SetTrend I guess we disagree on the scope of the project, but as clearly stated in bullet 4 of the https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals#non-goals:

Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.

This is not something we're interested in providing, since many tools exist that are either capable of exactly the functionality wanted, or can be more easily modified to have that capability.

[SetTrend]

I tend to agree on that we disagree on the scope on the project.

To me it looks like you seem to consider the TSC transpiler reacting to a single configuration file a compiler matter while reacting to multiple configuration being an external build configuration.

I cannot share this point of view. To me it's like a hen allowed to only lay one single egg during its life but not to lay many eggs.

Nonetheless, the non-goals section gives statement about an end-to-end pipeline, but it doesn't give any guideline about whether multiple configuration files are considered a non-goal. So I would believe your reason is rather interpretation than argument.

[paulvanbrenk]

What you're proposing is more like a hen laying another hen, in addition to an egg...

[SetTrend]

OK, I think arguments have been exchanged sufficiently now. I'm excited to see the outcome of v1.8. And we'll see then.

I don't believe it makes any more sense now to discuss unhatched eggs 😉

[SetTrend]

/cc @DovydasNavickas wrote that you have the impression that I'm trying to talk you into adding a file watcher to TSC. Is that correct?

I'm by no means suggesting this.

In case that's what you truely believe, please let me clarify ...

This is (in pseudo code) what TSC currently does:

void Main(string[] args)
{
  _config = getDefaultConfiguration();  // get (const) TSC default configuration
  if (File.Exists("tsconfig.json")) _config = readTsConfig(_config);   // if tsconfig.json file exists, read it and overwrite default configuration

  Directory.SetCurrentDirectory(_config.ProjectPath);   // set root for relative paths here
  CompileDirectory(_config.ProjectDirectory);   // start browsing the directory for .ts files ...
}


void CompileDirectory(string path)
{
  foreach (string directory in Directory.GetDirectories(path)) CompileDirectory(directory);   // If there are any sub directories, compile them first
  foreach (string file in Directory.GetFiles(path, "*.ts")) CompileFile(Path.Combine(path, file));  // compile all the .ts files in this directory
}


void CompileFile(string filePath)
{
  ...
}

All I suggest is to **move the interpretation of the tsconfig.file from the root procedure to the directory browsing procedure**:

void Main(string[] args)
{
  _config = getDefaultConfiguration();  // get (const) TSC default configuration

  Directory.SetCurrentDirectory(_config.ProjectPath);   // set root for relative paths here
  CompileDirectory(_config.ProjectDirectory);   // start browsing the directory for .ts files ...
}


void CompileDirectory(string path)
{
  bool configExists = File.Exists(Path.Combine(path, "tsconfig.json"));   // true, if tsconfig.json file exists

  if (configExists)
  {
    _configStack.Push(_config.Clone());   // push current configuration on stack
    _config = readTsConfig(_config, path);   // update current configuration with content of tsconfig.json file
  }

  foreach (string directory in Directory.GetDirectories(path)) CompileDirectory(directory);   // If there are any sub directories, compile them first
  foreach (string file in Directory.GetFiles(path, "*.ts")) CompileFile(Path.Combine(path, file));  // compile all the .ts files in this directory

  if (configExists) _config = _configStack.Pop();   // restore previous configuration
}


void CompileFile(string filePath)
{
  ...
}

You see? There is no file watching whatsoever.

Me and @lazutkin are only suggesting to move reading a tsconfig.json file from the root function up to the directory function. That's all. There is no build functionality involved whatsoever.

We are suggesting to use dynamic configuration in preference to static configuration.

[paulvanbrenk]

No, I understood what you meant. And still think MSBuild is the right tool for the job. i.e.

Invoke MSBuild mysolution.sln, MSBuild reads the solution file and invokes MSBuild on all the projects contained there in. Each MSBuild process reads the project file and invokes the appropriate compiler for the content, i.e. csc.exe for the C# files, Less/Sass for the css files, tsc.exe for each tsconfig.json file.

And because MSBuild has a bunch of context here, it can correctly schedule each build step.

[SetTrend]

If you want to create a separate build job (aka. build task) for each of the tsconfig.json files, you may as easily consider to omit support for tsconfig.json files at all as all the options to the compiler may be provided as command line parameters.

It would be preposterous to create/maintain a build job via any of the task runners in one file and additionally maintain the options for the compiler in a second file, i.e. a tsconfig.json file.

There'd be no reason in supporting tsconfig.json files then.

[SetTrend]

Although I admit that your way fairly is a solution. Many roads lead to rome. Yours does, too.

[paulvanbrenk]

Except that a tsconfig.json allows Sublime, VSCode, Atom, to open a single .ts. file, and after finding the tsconfig.json file in the root of the dir, setup the compiler options for IntelliSense etc...

But yes, if you're all in on MSBuild based solutions there is no specific need for tsconfig.json.