--artifact-dir (nee --out-dir) Tracking Issue
#6790
Comments
ehuss
added
the
C-tracking-issue
|
I find this feature very helpful, since I get output in a directory named like this However, it doesn't seem that OUT_DIR environment variable is set to the value of |
|
+1 on this issue. I have a program that uses a template directory for rust compilation and would like to now have to recompile everything. Yes I could symlink all the folders within target, but it would be nicer to be able to specify the binary output path. I'm not sure why this is such an overlooked issue. |
|
When will this feature be available in the stable release? It would greatly simplify my build/CI pipeline. |
|
+1 on this. This greatly improves the workflow on our CI, and we'd like to avoid using a nightly compiler just for this. What needs to happen for this to move forward? |
|
This issue makes nmattia/naersk unusable on stable rust. (another build system issue) |
|
This would streamline some of my Dockerfiles and build scripts quite nicely. |
|
Is there a reason this doesn't work with |
|
I just published a crate called cargo-out-dir to crates.io, which can ease some of the frustration while we wait for this to stabilize. All it does is print out the current' crate's out directory, as determined by running |
|
Another potential use case for out-dir is making in order for native APIs like (the moral equivalent of) But in this case it would be more helpful to be able to put |
|
|
I run into this regularly (multiple times a year), and I am not the only one. At the moment, people hard-code
Cargo's current file handling is unfortunate, and stabilizing this value would help a lot. Specifically, all temporary build files should be freely movable to an external directory, while the build outputs (i.e. the binaries or shared objects) should land in a fixed path for easy packaging. (In an ideal world, Related: #6100 (comment) |
|
@piegamesde exactly, I'm used to having |
|
Why is it still unstable? |
The key scenario for this is integrating with other build systems, especially where you are cross-compiling: since Cargo will place the build artifacts into a target-triple directory, you need to teach the outer build system about Rust's target-triples so that you know where to look to find the build artifacts. I'd mostly be happy if there was an option to suppress the target-triple directory (and the |
...? But in that case you need to have invoked them using |
Just from experience with various (non-embedded) CI and embedded build systems, it's often easier to determine a path for the binary artifacts early and up-front and then use that everywhere, than try to reverse engineer your build tool's scheme for placement. Especially if there's no agreement that the build tool's placement will remain stable. Consider also that with CI/CD you're configuring things in YAML, so no functions to call like There may be more concrete reasons too, that's just what came to mind. |
|
CI configuration languages generally accept variables, indeed this is virtually required for making a CI matrix work. And in a project I have recently been working on which requires constructing a custom build system, I made it so cargo is always invoked with a |
One problematic use case I ran into was with some application that was built with Meson. The problem is that it just copied |
You might know the target triple within the Rust project, but if you are wrapping calls to Cargo from another build system then there are likely other non-Rust projects in that build system that are unaware of Rust's target triples. It's much easier in a build system to assume that the built artifact is Things only get worse if a project can customize its target triple: it might be trivial to create some Make/CMake function to generate the target triple based on the target architecture of the build, but if an individual project can customize that triple (e.g., MUSL vs GNU, MSVC vs GNU, or using a |
|
As a practical example, this is a problem when one builds and packages mdcat - besides the binary $ git clone https://codeberg.org/flausch/mdcat
$ cargo fetch --locked
$ cargo build --frozen --release --target-dir=target
$ ls target/release/mdcat
target/release/mdcat
$ ls -d target/release/build/mdcat-*
target/release/build/mdcat-2a83e790d2b3f7a6 target/release/build/mdcat-7decb70a642f31fb
$ find target/release/build -type d -path "target/release/build/mdcat-*/out"
target/release/build/mdcat-7decb70a642f31fb
$ ls target/release/build/mdcat-7decb70a642f31fb/out
completions _mdcat mdcat.bash mdcat.fish _mdcat.ps1 |
|
I need this Not a fan of how the flag is teased in non-nightly cargo's help menu, only to bail when invoked. Don't bother me. Either let me use the flag, or at least drop it from the CLI documentation. Alternatively, I could use nightly, and require my users to use nightly. For DevOps users and other fans of pinned, immutable versions, nightly is not a valid option. Alternatively, I would have less need for this feature if cargo simply nested binary artifacts in a more archive amenable tree structure, like Alternatively, one could write a custom shell script to accomplish this. But that wouldn't work very well across all Rust projects, and it would needlessly restrict Windows devs, and proper robust shell programming is rather a dark art, and shell scripts substantially depart from the Rust ecosystem. Alternatively, one could configure archive commands to strip out the junk sibling files, but that's a maintenance nightmare, as cargo will surely (as any package manager reasonably would) introduce additional file patterns in the build tree internal directory over time. Alternatively, one could configure archive commands to look exclusively for file patterns matching I don't want to get too involved with low level details of cargo internal build files. I just want more reasonable defaults. Or failing that, landing |
|
I believe this comment addresses most of the concerns against stabilizing this issue (I don't know of any others), and I don't see anyone raising objections to it. This issue is still labeled as "waiting on feedback", so what further feedback is needed? This flag would simplify my Docker setup, so I'd love to see it stabilized. What's left to be done until it can be? |
|
i do hope the one discussing about this flag will not be dead until this gets enabled in stable :) |
|
Can someone PR a rename of the flag from |
where did you get the impression from that out-dir is not good enough, and what is the purpose to block it even longer? |
Confusion with the
Created #13809. Doesn't work properly right now, but hopefully it gets the ball rolling. |
|
To add to what was already said, cargo itself already has this inside its implementation: cargo/src/cargo/core/compiler/build_config.rs Lines 39 to 44 in eee4ea2 If the actual code inside cargo does not think |
|
What would help move this along is if someone could summarize the state of this thread and the feature in general with the goal of catching everyone up on this. Keep in mind, we have over 1300 issues in our backlog and it takes time to come up to speed on any specific thread. That then needs to happen for the whole team to be able to sign off on stabilization. While this is a blocker for some, there issues that are a blocker for others and we have to pick where we put our time. Yes, there was a comment earlier that covered some of the high level topics but was more focused on addressing them, rather than summarizing the conversation to help others come up to speed. As for use cases, I was recently thinking about this as I was a way to unblock moving the target dir into a central location after rust-lang/rfcs#3371 is stabilized. The vague thought I had was that we could have |
|
wrt naming to move forward #6790 without blocking that on any of what I just said, I was wondering if this is a popular enough of an unstable feature that we should offer a transition route for users by supporting both and warnings users that the old one is going to be removed. There is also the name itself. We use |
I think a lot of the discussion has been happening in #6100. As far as I can tell, it mostly consists of many Cargo users saying "please stabilize this flag" interspersed with some others wanting to hold off on stabilization until Cargo's entire output directory structure has been redesigned:
The frustrating part for me, and I believe many others as well, is seeing people make these proposals as reasons not to stabilize an existing and wholly implemented feature and then do no work on exploring the design space of those proposals. I don't expect them to champion such work on their own, but some help would be nice if they're arguing against a feature that's already implemented.
My implementation of renaming it to |
|
I'm also looking forward to seeing this option merged quickly. In that spirit: Is it really necessary to rename the option? I understand that the worry is that there could be confusion between In both cases you're asking the build tool to place it's output artifact in a certain directory, it just happens that in one case you're asking cargo, and in the other instance someone (cargo) is asking |
No. As I understand it, the |
|
Two people have asked "why rename the option?" with an air of "...when we could ship what's currently there, and get it faster?" I wonder if people imagine that, since we have been waiting so long, if someone renames the flag we will have to wait another long time for it to stabilize? That doesn't happen, in my experience. As the minimum wait from FCP to stable release is 6~12 weeks1, and the only prior users were necessarily using the nightly toolchain, I have seen teams rename something and then immediately start the stabilization clock. But I have also seen a team member that does not think something is "finished" decide to raise a blocking concern. Including on naming. And that by definition can delay the stabilization process indefinitely. I speak in generalities of course. "Should the name be Footnotes |
|
@valadaptive In #6790 (comment) you asked this:
Note: I am not a T-cargo member. But to answer to the best of my ability: T-cargo uses rfcbot to do FCPs like most of the Rust teams. That means any T-cargo member can call for a T-cargo FCP, and that is what you're looking to convince someone to start. There is T-cargo's unstable feature process that explains everything but... probably skip towards the end, as this has been baking quite a while. More of a background note, but might make things make more sense: like most teams this process is partially templated on the rustc stabilization guide. There's also interesting tidbits1 of related documentation in the Cargo contributor guide. Can't see anything about deprecation tags though! Footnotes |
|
#7493 also seems to be a blocker here. |
|
Here's a recap of the unresolved questions:
Over the years, many people in that issue thread have proposed their dissatisfaction with
Still relevant and will need to be solved.
See above for all the developers who want this to be added. An important use case is being able to directly output artifacts where you want them, without having to know where the target directory is or copy out of it as an extra step. Figuring out the former is complicated by the A couple pain points with the current situation seem to be knowing whether to reach into the
Solved by renaming to As I noted above, #7493 is also a blocker that will need to be resolved before we can stabilize this flag. |
Per discussion in rust-lang#6790. The --out-dir CLI option and out-dir config option are often confused with the OUT_DIR environment variable, when the two serve very different purposes (the former tells Cargo where to copy build artifacts to, whereas the OUT_DIR environment variable is set *by* Cargo to tell build scripts where to place their generated intermediate artifacts). Renaming the option to something less confusing is a prerequisite to stabilizing it.
Per discussion in rust-lang#6790. The --out-dir CLI option and out-dir config option are often confused with the OUT_DIR environment variable, when the two serve very different purposes (the former tells Cargo where to copy build artifacts to, whereas the OUT_DIR environment variable is set *by* Cargo to tell build scripts where to place their generated intermediate artifacts). Renaming the option to something less confusing is a prerequisite to stabilizing it.
Rename --out-dir to --artifact-dir Progress towards unblocking #6790. Renames the experimental `--out-dir` argument to `--artifact-dir`, both to reflect that it's where the final build *artifacts* will be copied to, and to avoid confusion with the `OUT_DIR` environment variable which serves an entirely different purpose. For transition purposes, `--out-dir` argument and `out-dir` config key will still work with a deprecation message encouraging the use of the new arg and config key. ### Rationale A lot of people seem to be confused by the naming of the `--out-dir` argument, and are misled into thinking it serves the same purpose as the `OUT_DIR` environment variable: > [However, it doesn't seem that OUT_DIR environment variable is set to the value of --out-dir when build.rs is executed.](#6790 (comment)) > [I understand that the worry is that there could be confusion between --out-dir for cargo and the environment variable OUT_DIR for build.rs, but doesn't it mean exactly the same in both cases?](#6790 (comment)) > [--out-dir: Things will be built into $PWD/target as normal, but copies some of the artifacts into the directory specified by out-dir (not a profile specific subdirectory). Unstable flag, added in March 2018. cargo build --out-dir #5203 Ability to specify output artifact name #4875. **Mimicks the behavior of OUT_DIR.**](#6100 (comment)) > [I recently had a couple of people express an interest in --out-dir being stabilized and from my initial digging it seems like what they may actually want is to switch to OUT_DIR, which is already stable.](#6100 (comment))
epage
changed the title
--artifact-dir (nee --out-dir) Tracking Issue
That is not sufficient reason to ignore those concerns. However, since then (besides team members rotating)
Looking at those, I agree with #6100 (comment) that having
What would be important for organizing all of the use cases is to understand what is in or out of scope (or if we need more, lower level knobs as mentioned). e.g. #6100 (comment)
A feature may sound like it works for one person or it may indeed work for one person but we need to consider the overall ecosystem and "where will this lead" both in terms of second order effects (not concerned about that here) and what will people find lacking and ask to have changed or extended. We don't have to solve everyone's problem but we need to have a rough idea of what the future is to make sure we are solving the right problems. For me, I'm less concerned with #6100 for this flag but whether we'll need several additional flags after this because we didn't quite solve the right problem. |
I think that to a large extent, the need being addressed can be split into two parts:
The "certain build artifacts" can also be split into two parts:
By way of example, I have recently been trying to implement fine-grained code coverage reporting in CI on a Rust project. It is necessary to tell the coverage tools where both the coverage instrumented binary artifacts are, and where the coverage data is. The latter can be controlled through environment variables, so perhaps we can ignore that. But the former is extremely hard right now.
So to simply generate the command lines for a coverage tool in CI, I need to implement this logic in CI or a build script. I can't just have I also can't just use the JSON output of Cargo because there is no part of that that gives me the base directory, only the full path to each individual artifact. The base directory is required eg. for most instrumentation analysis tools, for caching and storing artifacts consistently, and because the full list might actually be too long for many shells' limit on environment variables. So I need to implement a more complex mapping that duplicates Cargo's internal artifact directory construction logic. In shell script. Or the CI config syntax. Or another build script language. This is undesirable because:
This is not just an issue for code coverage. This applies to:
Now consider that if I were just able to predict (or extract from Cargo's output) where artifacts will end up, I can effectively meet "the need to control where certain build artifacts will be placed" by writing I'll also preempt the argument that there are tools that abstract this away for some of the use cases I mention eg. In the last four years of using Rust in earnest and trying to do novel things related to code quality, instrumentation and testing, I just keep running up against this. I don't mean this as a kind of pressure to just "get it done", because I respect (a) Cargo's commitment to quality of API and (b) the fact that (veeery much like the |
and others
Note: this was renamed from
--out-dirto--artifact-dirin #13809Original issue: #4875Nightly: --out-dir
Implementation PR: #5203
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#artifact-dir
Issues: Z-out-dir
Summary
Adds
--out-dir=PATHflag tocargo buildto specify a directory to place final artifacts.Tasks:
--out-dirUnresolved questions
--out-dirjustified or can we find another way of accomplishing it? #6100 — concern about similarity with--target-dir, proliferation of flags--out-dir(Rename --out-dir to --artifact-dir #13809)build.rssOUT_DIR(depends on ifbuild.rsis generating intermediate or final artifacts) (Rename --out-dir to --artifact-dir #13809)The text was updated successfully, but these errors were encountered: