--- title: "Design meeting 2024-04-11: for await / async gen etc." tags: ["WG-async", "design-meeting", "minutes"] date: 2024-04-11 discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Design.20meeting.202024-04-11 url: https://hackmd.io/bYaPiCdqR3WIyjKhFWzdtQ --- # Discussion ## Attendance - People: TC, Yosh, Daria, Vincenzo ## Meeting roles - Minutes, driver: TC ## Discussion with Oli yosh: Oli made an argument about what the low level desugaring needs to be and why various optimizations can't work in any other reasonable way in a reasonable timeframe, and this was persuasive to me. However, I'm still interested in the question of what the user-facing API is. TC: So if we accept the low level API, what do you think means in terms of the high level API? yosh: From talking with Oli it seems that we may not need to stabilize either the high-level or even low-level APIs to provide a desugaring of (async) gen blocks. In theory even the low-level API could remain unstable for the high-level feature. There may be reasons why we don't want to do that though; but it's important that it can even be detached from the high-level API. yosh: Still though; we should make very sure that this is true, not just a game of telephone. Also "could" is not "should". But it's good to separate hard requirements from soft ones. TC: Sure, it makes sense that we could stabilize `async gen` blocks that desugar to opaque types that implement traits that we don't stabilize and use that in the desugaring of `for await` loops. However, that could be rather limiting. We wouldn't want to live long in that world. Yosh: Definitely. ## Issues with pinned/not-pinned combinators yosh: It's tempting to think of `AsyncIterator::poll_next` as `PinnedAsynIterator`. However in practice none of the combinators are pinned - things like `map` aren't; only things like `for_each` can work with pinned iterators. They all have carry a `where Self: Unpin`. TC: If you pin a type it's `Unpin`, so it would work. Yosh: Oh yeah that's right - good times with the `Pin` API. TC: The challenge there, with respect to `Pin`, is whether that complexity is accidental or inherent. There's certainly some inherent complexity here. Yosh: It seems largely accidental - it seems increasingly clear immovable types should have been a language extension instead. A lot of the issues we're seeing with it now are because we cleverly encoded it as a library type. ## Forward compatibility with keyword generic async iterator Daria: In case of we get async generic keyword for traits, we should plan forward compatibility between `async Iterator` and `for await`, in other words make sure we would be able to switch to other (compatible with `async Iterator`) desugar. I consider conflicting trait implementation could be a problem if not addressed. Also perhaps consider integrating with hypothetical `async Into(Async)Iterator`? Yosh: Here's what Oli wrote the other day that we converged toward: ```rust impl<T: IntoAsyncIterator> IntoAsyncGen for T { type Item = T::Item; type AsyncGen = AsyncGenAdaptor<T::IntoAsyncIter>; fn into_async_gen(self) -> Self::AsyncGen { AsyncGenAdaptor { iter: self.into_async_iter(), next: None, } } } // Note: does not yet do self-referential lifetimes #[pin_project] struct AsyncGenAdaptor<T: AsyncIterator> { iter: T, #[project] next: Option<T::next::return>, // or whatever RTN syntax will be } impl<T: AsyncIterator> AsyncGen for AsyncGenAdaptor<T> { type Item = T::Item; fn poll_next(self: Pin<&mut Self>, context: &Context) -> Poll<Option<Self::Item>> { let mut this = self.project(); let next = this.next.get_or_insert_with(|| this.iter.next()); match next.poll(context) { Poll::Pending => return Poll::Pending, Poll::Ready(val) => { self.project().next = None; return Poll::Ready(Some(val)); } } } } ``` TC: That self-referential type there needs the `unsafe<'a>` binders we've been discussing. TC: The approach above is the same as what I took in the prototypes we discussed here: - [Design meeting 2024-02-15: AsyncIterator prototype](https://hackmd.io/EplPcmBCTCSQ9LPOU6IZDw) - [ITE design meeting 2024-02-22: AsyncIterator](https://hackmd.io/Gk630R_SRYKPfmoYOOwFcw) (E.g., search for `next_item`; that's what people seeking a high-level interface would implement.) --- yosh: You can substitute `AsyncIterator` with the effect-generic `async Iterator` type here. Daria: If possible we could keep a sort of placeholder trait for `async Iterator` to figure out forward compatibility. ## Next steps yosh: Write out all the currently primary traits in their high-level form, then show a mapping to the `Gen` forms and `for..in` desugarings. Fallible, lending, pinned, async, bare I think are the main ones. The combinations of these should fall out of that. yosh: The current primary traits are: - `Iterator` - `AsyncIterator` with `async fn next` - `FallibleIterator` - `PinnedIterator` - `LendingIterator` So go through the desugarings, mappings, and bridgings for all of these. ## How to implement `LendingIterator` Daria: This code works (checked the different version of this): ```rust trait LendingIterator { type Item<'a> where Self: 'a; fn next(&mut self) -> Option<Self::Item<'_>>; } impl<L, T> Iterator for L where L: LendingIterator, for<'a> L: LendingIterator<Item<'a> = T> + 'a, { type Item = T; fn next(&mut self) -> Option<Self::Item> { <Self as LendingIterator>::next(self) } } ``` TC: The `+ 'a` bound there means that the type must outlive *any possible lifetime*, so this probably only works in much more constrained situations than what you intend. Yosh: I think this does ask a good question - what is the desugaring for `LendingIterator` when used in `for..in` loops? What's the lowest-level trait we desugar into? yosh: I think this might have been the issue? ```rust trait MaybeLendingIterator { type Item<'a = 'owned> where Self: 'a; // missing lang feature: 'owned lifetime fn next(&mut self) -> Option<Self::Item<'_>>; } ``` TC: If you mean that borrowck ignores that lifetime, that's what we have in mind for the `unsafe<'a>` binders. Yosh: I don't know if this is the same thing. CE would probably know the answer there. What I'm thinking might be more about optional lifetimes? Maybe-lifetimes? Not sure if that's the same. Daria: Checked my code sample from above with non-static lifetimes. Trait solver ~~dissapointed me once again~~ did not like this change. ## Stabilizing the low level components TC: So if we did the low level implementation described above and we could show that the effect generics work could be mapped to these, then do you have any concerns about stabilizing the low level components at that time? yosh: That is my main concern. I'd want to be really sure that those mapped, but that is the main concern. ## Coroutines Yosh: There seem to be different perspectives from different parties about how we get to full coroutines. I'd like to have some path sketched. Oli has expressed particular constraints on what can be implemented that we should discuss. Yosh: This ties back to language evolution; which is the point of effect-generics. (The meeting ended here.)