New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Relative subninja paths don't resolve #977
Comments
All paths are relative to the build directory, not to the file containing the subninja line. |
That makes no sense, though. That means generators are going to have to all implement some way to prepend a prefix to the files, which means changing the working directory commands run in (which may change the original intent of the build configuration depending on how the generator/commands run). What, then, are contrasting use cases for A subninja with paths relative to the build file would be useful in that depdencies that are configured to run within their own directory can do so without having to modify their paths, but can still contribute to the dependency graph of the parent ninja configuration.
|
The idea is that the generator generates all .ninja files, so they can write paths relative to the build directory. It's an interesting idea to combine ninja files generated by different generators (it sounds like that's what you want to do?), but that's not something that's supported at the moment. Correct, the difference between |
That makes sense, though I still don't see much of a benefit (other than scope).
Exactly. Being able to include Ninja itself as a submodule for my generator, and then another CMake project (which is configured to output Ninja build files), and then generating the Ninja config files for both and including them as If that makes sense. Immediate use-case that I see with my generator in particular is that it's borrowing a few concepts from Tup (which IIRC influenced some design decisions within Ninja itself) in that I can include N subprojects, all with their own I personally think that'd make Thoughts? |
How about: subninja path/to/build.ninja relative path/to and an absent Or, following in the steps of other Ninja constructs, maybe subninja path/to/build.ninja
relative = path/to |
Suppose you have a project at .../foo and it has a subdirectory bar, and that Ninja had the relative-path logic you suggest. If your build system wants to write all build outputs to /foo/obj, a subninja in /foo/bar that used directory-relative paths would need to know to write its output into ../obj/bar, as that's the path to the file from that subdirectory. So whatever is generating your build.ninja files must already be aware of the global path hierarchy, in which case making all paths relative is effectively the same problem as prepending a bar/ to the paths in the bar/ directory. Maybe there are enough people who write build output in their source directories that the above doesn't matter, though. I mostly hear from people who want even stronger separation -- like the people who sent patches to Ninja to make it so they can build Ninja with the build output in a totally unrelated directory. |
I'm not sure I understand. Subninjas, in the use-case I've described in particular, don't normally know about the parent ninja's structure (at least, don't need to know). I'm sure someone out there could find a case where they do, though. The problem I'm facing right now is dependencies (third party libraries, etc.) that don't use my generator (which is 100% of them) currently need to be built using a bootstrapper, and have to be bootstrapped every time they're updated or changed, etc. Most generators I work with have the options to output to a ninja configuration, but they're all configured for that directory alone (i.e. relative to that directory). Being able to perform selective rebuilds using their configurations and graphs would be huge. Currently, I cannot do this due to the fact |
Evan: The way I understood this is that you'd create a tree like this:
and since generators usually support putting the build dir in arbitrary places, building project 1 in subbuild1 and project 2 in subbuild 2 should work. Then there's a toplevel ninja file in builddir that Qix- wants to drive building the subprojects. Unrelated: this |
Paths in subninjas need to be relative to the build directory - ninja-build/ninja#977 Note: paths in commands (pre/postbuild, custom rule/build) in Premake is relative to project. so it mightbe problematic.
I'm hitting this problem now with Ninja and Premake. All of my libraries exist as their own projects in entirely unrelated directories which I pull into which ever application is being built at the time. So an example directory structure for my libraries looks something like this:
And a project which uses some of those libraries would have something like this:
So The build script The Premake Ninja generator correctly produces But none of them actually build because So how is this supposed to work? |
All relative paths in Ninja are interpreted as relative to the directory
you run Ninja from (which you can set with the -C flag).
…On Sun, Jul 9, 2023 at 7:46 AM James ***@***.***> wrote:
I'm hitting this problem now with Ninja and Premake. All of my libraries
exist as their own projects in entirely unrelated directories which I pull
into which ever application is being built at the time.
So an example directory structure for my libraries looks something like
this:
Core
Gwee
Build
gwee.lua
<gwee sources>
Carbon
Build
carbon.lua
<carbon sources>
etc...
And a project which uses some of those libraries would have something like
this:
Project
Build
premake5.lua
Engine
Build
engine.lua
<engine sources>
App
Build
app.lua
<app sources>
So premake5.lua includes engine.lua, app.lua, and whichever libraries it
needs, such as gwee.lua.
The Premake Ninja generator correctly produces .ninja files next to each
lua project file in their respective directories, and the build.ninja
file in Project/Build correctly references them with relative paths.
But none of them actually build because subninja doesn't change to the
directory containing the actual ninja file.
So how is this supposed to work?
—
Reply to this email directly, view it on GitHub
<#977 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAA6B3IGUTWHILWTEXMN3DXPK75PANCNFSM4BJAOQYA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
I'm running Ninja from the Build directory of the main project. But that ninja file just runs SubNinja for a bunch of other generated ninja files. The individual ninja files for each library are built with their internal paths relative to themselves. Hence the above compile error of not being able to find the source files being compiled. Each project that I have uses Premake and pulls in the libraries it needs as other Premake lua files so that the generated files for any given project are self contained in a single build script. (Or in the case of Xcode/Visual Studio a single workspace/solution that has all libraries needed for that project.) But crucially, the project files (and ninja files) for libraries are located within each library's Build directory. Basically, I end up with:
Where build.ninja just consists of:
|
Is running subninja from the directory that contains the referenced ninja file all that is needed? |
On Sun, Jul 9, 2023 at 10:50 AM James ***@***.***> wrote:
I'm running Ninja from the Build directory of the main project. But that
ninja file just runs SubNinja for a bunch of other generated ninja files.
The individual ninja files for each library are built with their internal
paths relative to themselves. Hence the above compile error of not being
able to find the source files being compiled.
Ninja doesn't support the idea of composing multiple different projects
with different notions of the source root. It assumes all your build.ninja
files are generated under the same global view of paths.
Message ID: ***@***.***>
… |
On Sun, Jul 9, 2023 at 11:12 AM Evan Martin ***@***.***> wrote:
On Sun, Jul 9, 2023 at 10:50 AM James ***@***.***> wrote:
> I'm running Ninja from the Build directory of the main project. But that
> ninja file just runs SubNinja for a bunch of other generated ninja files.
> The individual ninja files for each library are built with their internal
> paths relative to themselves. Hence the above compile error of not being
> able to find the source files being compiled.
>
Ninja doesn't support the idea of composing multiple different projects
with different notions of the source root. It assumes all your build.ninja
files are generated under the same global view of paths.
...er sorry, I misunderstood. It looks like your project does this
already. Yes, I think you just need to invoke Ninja from the right initial
directory (or use -C).
… Message ID: ***@***.***>
>
|
I'm running Ninja from the Build directory that contains the build.ninja file. Looking at the individual ninja files, it does look like their paths are correct, but things just aren't working. (And having looked inside the individual ninja files, them having paths relative to the current project isn't what I want or need anyway because that won't work for other projects.) So rewriting my entire build system from scratch so it works on Linux isn't something I'm prepared to do, so maybe Ninja isn't going to work for me. |
In general, the Ninja behavior here is fairly straightforward ("all paths
in all files must be relative to the same directory"), which means any
complexity/configuration here is managed by premake. You might have more
success asking a premake list.
…On Sun, Jul 9, 2023 at 12:03 PM James ***@***.***> wrote:
I'm running Ninja from the Build directory that contains the build.ninja
file. Looking at the individual ninja files, it does look like their paths
are correct, but things just aren't working. (And having looked inside the
individual ninja files, them having paths relative to the current project
isn't what I want or need anyway because that won't work for other
projects.)
So rewriting my entire build system from scratch so it works on Linux
isn't something I'm prepared to do, so maybe Ninja isn't going to work for
me.
—
Reply to this email directly, view it on GitHub
<#977 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAA6BYALRC7AQE3EOGWFFDXPL57VANCNFSM4BJAOQYA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Yeah, it's always tricky to know exactly where the responsibility falls when you're dealing with issues like these. :) |
I would attempt a PR for this, but I'm not entirely sure this is a bug or if it's intended.
When including a subninja, I get
although running ninja directly in that directory works just fine. Under the suspicion that relative paths may be to blame, I copied the project structure to a new directory, prepended each input/output with the absolute path, and ran ninja from the parent directory (containing the
build.ninja
thatsubninja
'd the module). It built just fine.Is this suggesting we should be generating our ninja configurations with absolute paths? This has a lot of problems, along with the fact not every generator I've seen does this. Further, the documentation doesn't make a distinction, and Ninja runs just fine otherwise.
I want to err on the side of it's a bug, but I feel like a PR that does runtime path canonicalization might introduce a performance hit (though I'm waving my hands here without any benchmarks to back up that suspicion).
The text was updated successfully, but these errors were encountered: