New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Precise capturing #3617
base: master
Are you sure you want to change the base?
Precise capturing #3617
Conversation
f60865c
to
0231781
Compare
0231781
to
2a6cab8
Compare
To fully stabilize, in Rust 2024, the Lifetime Capture Rules 2024 that we accepted in RFC 3498, we need to stabilize some means of precise capturing. This RFC provides that means. We discussed this feature, the need for it, and the syntax for it in the T-lang planning meeting on 2024-04-03. The document here follows from that discussion.
2a6cab8
to
5ac2eb1
Compare
// ^ Captures `B`, `C`, and `D` but not `'a` or `A`. | ||
``` | ||
|
||
Here, the `..` means to include all in-scope generic parameters and `!` means to exclude a particular generic parameter even if previously included. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be nice to allow use<..>
from the start as an explicit way to capture all in-scope generic parameters. Using this short-hand together with explicit explicit captures or excluded captures should still remain future work.
Especially in code where precise captures would be frequently used, it might be preferable to always be explicit, even in cases where the implicit capture-all is what we want. Similar to how the syntax is used with struct initialisation, using use<..>
could mean that we know that we will always want to capture all generics, even if more are added, whereas manually enumerating each one might mean that we want to explicitly have to choose with every newly added generic.
Furthermore, since the stabilisation strategy suggests that at first only capturing all parameters may be supported, supporting use<..>
from the start would make this shorter to type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An alternative meaning for use<..>
is "capture all APITs".
|
||
## Argument position impl Trait | ||
|
||
Note that for a generic type parameter to be captured with `use<..>` it must have a name. Anonymous generic type parameters introduced with argument position `impl Trait` (APIT) syntax don't have names, and so cannot be captured with `use<..>`. E.g.: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cannot be captured with
use<..>
Are they captured implicitly now?
If they are, then what is the opt out of that?
Are unnamed lifetimes implicitly captured by RPIT?
What is the opt out in that case?
(If unnamed early bound lifetimes are possible at all.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope, we don't capture APITs or elided lifetimes with use<'a, T>
. The opt-in here is to promote your APITs to real type generics and give your unnamed lifetimes names.
The only caveat here is that you can name the elided lifetime in the output in the same cases you are allowed to normally in RPITs -- i.e. fn hello(x: &u8) -> impl use<'_> Sized { x }
works fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nope, we don't capture APITs or elided lifetimes with
use<'a, T>
.
My first question was whether they are captured if use
is not written.
The text says that all generic parameters are captured (on 2024 edition at least), but it's not clear whether APITs and elided lifetimes are included into this "all".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All type and const generic parameters (including APITs) are captured for all RPITs regardless if they're in edition 2021 or 2024.
The only change between 2021 and 2024 involves capturing all lifetimes parameters in scope (incl. elided lifetimes, since those become early- or late-bound lifetime params) -- previously we only captured lifetimes mentioned in the bounds of RPITs. The opt-out here is to use use<'a, T>
with the set of lifetimes that previously showed up in your opaque's bounds.
The only corner case is when you have an APIT and a lifetime you want to not capture -- you'll need to turn your APITs into real type parameters to make use of the use<'a, T>
syntax.
Regarding opt-out for capturing type or const generic parameters currently, it's not possible to do currently; we will likely support that eventually, but if you see https://github.com/rust-lang/rfcs/blob/TC/precise-capturing/text/3617-precise-capturing.md#stabilization-strategy, we're probably not going to stabilize that initially because it requires exercising new paths of the type system (namely, bivariant unconstrained type parameters), and it's not necessary to mitigate the fallout of the new edition 2024 lifetime capture rules.
One way to think about `use<..>` is that, in Rust `use` brings things into scope, and here we are bringing certain generic parameters into scope for the hidden type. Let's point this out.
We had included one `use<T>` in a pre-migration example when it should have only appeared in a post-migration example. Let's fix this error. (Thanks to @kennytm for pointing this out.)
@rfcbot fcp merge We've tried hard to avoid an explicit syntax like this but I think it's clear by now that it will be useful and it unblocks important Edition work. We had a reasonably thorough deep dive into the syntactic options and I believe the RFC lays out the options pretty well and the tradeoffs around them. Personally while I have some minor qualms about overloading Note that in this fcp I am explicitly wearing my @rust-lang/lang hat -- I think the @rust-lang/types team should (before stabilization) vet the overall semantics and our implementation thereof but that's not really a question to be answered in the RFC. |
Team member @nikomatsakis has proposed to merge this. The next step is review by the rest of the tagged team members: No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns. |
|
||
We considered a number of different possible syntaxes before landing on `impl use<..> Trait`. We'll discuss each considered. | ||
|
||
### `impl use<..> Trait` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect that it's going to get annoying pretty quick (compared to impl<..> Trait
), if precise captures become something that is specified often (e.g. by convention to minimize captures).
|
||
Picking an existing keyword allows for this syntax, including extensions to other positions, to be allowed in older editions. Because `use` is a full keyword, we're not limited in where it can be placed. | ||
|
||
By not putting the generic parameters on `impl<..>`, we reduce the risk of confusion that we are somehow introducing generic parameters here rather than using them. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't feel strongly about this syntax either way, but:
This would hardly be the only place where syntax for introducing generic parameters is similar to syntax for using them. fn foo<T>()
vs foo::<T>()
for example.
Also, there is a symmetry of a sort between the two kinds of impl
generics. In a trait implementation, they introduce generic parameters available for use by the type and trait; in RPIT, they would introduce the generics available for use by the hidden type.
} | ||
``` | ||
|
||
Here, the opaque type of the closure is capturing `T`. We may want a way to specify which outer generic parameters are captured by closure-like blocks. We could apply the `use<..>` syntax to closure-like blocks to solve this, e.g.: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why can't this "just work"? Shouldn't the compiler be able to figure out that the closure is not using T
? There should be no semver hazard, because to return the closure you need RPIT or TAIT , at which point you as API designer have the opportunity to specify the captures you commit to in public API.
🔔 This is now entering its final comment period, as per the review above. 🔔 |
For the formal syntax, we had used the existing `GenericParams` production. However, that production isn't exactly appropriate for this case. What's needed here is essentially a generic argument list that only accepts generic parameters as elements. Since none of the other existing productions provide this, we'll define our own. (Thanks to @kennytm for pointing this out.)
In the T-lang design meeting on 2024-04-24, a new syntax option was raised: `use<..> impl Trait`. While some people liked this, others did not, and no clear consensus formed to change the main proposal in this RFC. Nevertheless, let's discuss this as an alternative.
We had meant to say "parentheses" but had said "parenthesis" in two places. Let's fix that.
Since this From the caller's point of view, if a function's RPIT (in covariant position only) has a capture removed, the hidden type's potential shortest lifetime is lengthened, which is compatible with existing caller code. So I think semver should allow removing captures from covariant-position RPIT in a minor update. - fn callee1<'a, 'b>(aaa: &'a u8, bbb: &'b u8) -> &'a u8 { &0 }
+ fn callee1<'a, 'b>(aaa: &'a u8, bbb: &'b u8) -> &'static u8 { &0 }
- fn callee2<'a, 'b>(aaa: &'a u8, bbb: &'b u8) -> impl use<'a> Sized { &0 }
+ fn callee2<'a, 'b>(aaa: &'a u8, bbb: &'b u8) -> impl use<> Sized { &0 } Meanwhile for TAIT, ATPIT and RPITIT changing the capture list in either direction should be considered a major breaking change. (For APIT the capture list is irrelevant.) |
So, just procedurally... While I do think solving the underlying ergonomic and semantic issues here are quite important, I do have to note that this syntax and RFC seem to have moved quite fast. Even for me - who, while doesn't attend lang meetings, does try to keep up to date of active lang things - I could have almost missed this. To put some dates to this on typical milestones in our current processes:
I know we're on a time crunch because of the edition, but I worry about this all moving just a bit too fast. Of course, accepting this RFC doesn't necessarily mean that actually stabilizing this feature will also happen quickly, but I worry that the pace so far is a harbinger for that. I hate to sit here and be the one that say "wait, we're moving too fast", especially because not moving fast here puts edition work in jeopardy, but I'd rather this concern be voiced than ignored. I do want to be clear that my above comments have little reflection on my thoughts on the contents of this RFC or semantics of this feature. My concerns here apply to even the best of features (to put it into perspective: let-else, which imo is a very clear win all around took about a month from RFC open to RFC merge, with a fair amount of prior discussion; and this is what I consider a "fast" RFC process). |
As with other lists of generic arguments in Rust, the grammar for `use<..>` specifiers supports an optional trailing comma. This is already specified in the formal grammar, but let's also note this in the guide-level section. (Thanks to Josh Triplett for raising this point.)
As I explained in the lang team meeting, I prefer The reasons for preferring this syntax are:
The arguments in favor of use-after were later added to the RFC. Most of the discussion against revolved around the fact that it would require another migration to macro fragment specifiers (specifically // Before 2024:
my_unmigrated_macro! {
fn foo<'t, T>(_: &'t (), x: T) -> impl Sized { x }
}
// After 2024 (note the parentheses):
my_unmigrated_macro! {
fn foo<'t, T>(_: &'t (), x: T) -> (use<T> impl Sized) { x }
}
// After 2024, macro has been migrated to new matchers:
my_migrated_macro! {
fn foo<'t, T>(_: &'t (), x: T) -> use<T> impl Sized { x }
} After all the discussion I still find myself preferring use-before and wish we had more time to experiment with syntax. But this is the RFC, not stabilization, and I do not wish to block the RFC or the feature itself on this particular question. Footnotes
|
|
We had earlier written up a section on `use<..> impl Trait` syntax that mostly focused on why we had not adopted it. We didn't spend much text on why it's appealing, probably because we are in fact sympathetic to it and consider the reasons that it's appealing obvious. Still, we should write all those reasons down. Let's extend the section on this syntax with the best possible argument in favor. We also see more clearly now the fundamental intuitive tension behind this syntax and `impl use<..> Trait`, so let's write that down too. Finally, let's describe the historical and other factors that led to picking one syntax over the other. (Thanks to tmandry for suggesting the `use<..> impl Trait` syntax and many of the arguments in favor of it.)
02bbac9
to
f012228
Compare
Thanks @tmandry for writing that up. I've now greatly extended that section of the document to incorporate these1 and other points in favor of See in particular the discussion of the fundamental tension here. In short, the RFC lays out two intuitions for
These intuitions are both true, but they might suggest two different syntaxes, and this may be related to why the choice here is challenging. Footnotes
|
This RFC does not specify what an RPITIT/ATPIT in a trait impl is allowed to capture in order for it to be compatible with the trait definition. Here is an example of multiple trait implementations, the validity of which are not clear from the RFC. // RPITIT only allowed to capture Y.
trait Trait<X, Y> {
fn test() -> impl use<Y> Sized;
}
// Y = (A, B).
// Are we allowed to capture A?
// Is it considered a refinement to not capture B?
impl<A, B> Trait<(), (A, B)> for i8 {
fn test() -> impl use<A> Sized {}
}
// Y = <A as Iterator>::Item.
// Are we allowed to capture A now that it appears in a projection type?
// I don't think so since projections do not constrain their parameters.
impl<A: Iterator> Trait<A, A::Item> for u8 {
fn test() -> impl use<A> Sized {}
}
// Y = &'a str.
// The case for lifetimes and consts should be made clear.
impl<'a> Trait<(), &'a str> for i16 {
fn test() -> impl use<'a> Sized {}
}
// Y = &'a str.
// RPITIT can't capture `'b` even though `'a == 'b`.
// Lifetime comparison is syntactic.
impl<'a: 'b, 'b: 'a> Trait<&'b str, &'a str> for u16 {
fn test() -> impl use<'b> Sized {}
} |
Unlike the It certainly cannot return OTOH returning Basically you have to do refinement unless the capture list is relaxed to accept arbitrary GenericArgs. |
We have an example showing how to avoid capturing a higher ranked lifetime in a nested opaque type so that the code can be migrated to Rust 2024. However, because we didn't parameterize the trait in the example with a lifetime, the code could be migrated by just removing the `for<..>` binder. This makes the example weaker than it could be, so let's strengthen it by parameterizing the trait. (Thanks to aliemjay for raising this point.)
Hmm, do we have |
We do |
We had included a reference desugaring of `use<..>` in RPIT using ATPIT. Let's add a similar desugaring for `use<..>` in RPITIT. In doing this, we'll make some changes to the RPIT desugaring so that it better parallels the RPITIT one. In particular, we want to turbofish all generic parameters for clarity, and we want to eliminate all function arguments for conciseness. Doing this means that all of the lifetimes become early bound. This seems fine, since it's rather orthogonal to the semantics we're trying to demonstrate here. We also want to demonstrate using const generics in the hidden type. We could do this using arrays, e.g. `[(); N]`, but it seems more clear to just define a type constructor that uses all of the generics, so we'll sacrifice a bit of conciseness to do it that way.
We had added an RPITIT desugaring that was rather complicated. This complication was due to trying to explain what it would mean to not capture generic parameters that are part of the trait header. However, it's likely that there's no good way to express that semantic in the surface syntax. Let's instead simplify the desugaring and make a note of its limitations.
Would it be fair to say that For instance, say you have a type called use std::time::Duration;
struct Flux {
value: f64,
rate: f64,
}
struct Moment {
value: f64,
}
impl Flux {
fn at<'a>(&'a self, secs: Duration) -> use<'a> Moment {
Moment {
value: self.value + self.rate * secs.as_secs_f64(),
}
}
}
fn main() {
let mut flux = Flux {
value: 0.0,
rate: 1.0,
};
let moment = flux.at(Duration::from_secs(1));
flux.value = 2.0; // <- error: cannot assign to `flux.value` because it is borrowed
println!("{}", moment.value);
} I imagine Edit: Maybe it could just capture parameters like how associated types do, but idk if that works |
@Yokinman that's an interesting idea. It's definitely true that |
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
During the FCP, some questions came up related to how refinement and reparameterization in the impl are handled. This handling is implied by other text in the RFC and by the existing behavior of Rust, so let's go ahead and add clarifications to address these questions. The hardest of these questions relate to how things would behave if we were to allow `use<..>` in trait definitions to not capture the generic input parameters to the trait (including `Self`). It's unlikely this will be possible for the foreseeable future, and while we will not leave these as open questions, certainly much might be learned between now and the point at which that might become possible, so we'll make note of that. We'll also add a clarification to address a question that came up in the 2024-04-24 design meeting about what it means to capture a const generic parameter. (Thanks to aliemjay for raising many of these great questions.)
Thanks @aliemjay for those great questions. We've now added examples and discussion to clarify each of those. Note that the hardest subset of those questions relate to how things would work if we were able to capture less than all of the generic input parameters to the trait (including |
Great work! I think this feature is a rather advanced feature and maybe this syntax appearing on simple APIs may discourage beginners when using fundamental functionalities in a crate. However, I have designed the following API pattern multiple times: fn parse(path: impl Path) -> impl Iterator<Item = Foo> {} I wonder if I should add a I think this pattern is very common and basic in most crates. Should there be some special treatment in rustdoc for |
@Evian-Zhang That's a good question (though not I think one that needs to block progress on the RFC). That said, I think even better would be if we could avoid using (We've also discussed (and even done some exploration of) having the compiler recognize the pattern of a "mostly monomorphic" function that just does transforms in the beginning and avoid code duplication: that's worth doing as a first step, I believe, but it wouldn't have the borrow checker benefits.) |
@nikomatsakis Thank you for your response! I agree that the coercions should be in the caller side, since I have found many places where the following code pattern appears in the Rust std source code: fn foo(path: impl AsRef<Path>) {
fn inner_foo(path: &Path) { ... }
inner_foo(path.as_ref())
} It is more graceful if this pattern can be automatically done by the compiler. I am not familiar with the RFC discussion guidelines, and I think maybe I should put the rustdoc's |
@Evian-Zhang: It'd be better to open a thread on Zulip (or perhaps on IRLO) to discuss that further. |
@Evian-Zhang (...and no apologies required) |
To fully stabilize, in Rust 2024, the Lifetime Capture Rules 2024 that we accepted in RFC 3498, we need to stabilize some means of precise capturing. This RFC provides that means.
This RFC adds
use<..>
syntax for specifying which generic parameters should be captured in an opaque RPIT-likeimpl Trait
type, e.g.impl use<'t, T> Trait
. This solves the problem of overcapturing and will allow the Lifetime Capture Rules 2024 to be fully stabilized for RPIT in Rust 2024.One way to think about
use<..>
is that, in Rustuse
brings things into scope, and here we are bringing certain generic parameters into scope for the hidden type.For some history about the progress toward this feature predating this RFC, see this comment.
Rendered