Lifetime of Future returned by async function

Question

Trying to write recursive Stream, where each stream takes poll from his parent and when Poll is ready with value it makes own asynchronous fetcher to process a parent response, but I have no idea how to store the current future given by fetcher, compiler complains on lifetime until I trying to use a free async function (that not bound to struct). There incomplete example, but it illustrates the compiler error (two lines are commented in Stream implementation):

struct AsyncFetcher {}

impl AsyncFetcher {
    async fn fetch(&self, request: String) -> String {
        format!("Response({request})")
    }
}

enum State {
    PendingParent,
    ToProcess(Option<String>),
    Processing(Pin<Box<dyn Future<Output = String>>>)
}

struct RecStream {
    parent: Option<Pin<Box<dyn Stream<Item = String>>>>,
    state: State,
    fetcher: AsyncFetcher,
}

impl RecStream {
    fn new(parent: Pin<Box<dyn Stream<Item = String>>>, fetcher: AsyncFetcher) -> Self {
        Self {
            parent: Some(parent),
            state: State::PendingParent,
            fetcher: fetcher,
        }
    }

    fn with_result(result: String, fetcher: AsyncFetcher) -> Self {
        Self {
            parent: None,
            state: State::ToProcess(Some(result)),
            fetcher: fetcher,
        }
    }
}

impl Stream for RecStream {
    type Item = String;

    fn poll_next(
        self: Pin<&mut Self>,
        cx: &mut std::task::Context<'_>,
    ) -> Poll<Option<Self::Item>> {
        let ref_mut = self.get_mut();
        let future = ref_mut.fetcher.fetch("Some str".to_string()).boxed();
        
        // let future = free_fut().boxed(); // - THIS WORKS
        ref_mut.state = State::Processing(future); // Getting error 'lifetime may not live long enough'
        
        return Poll::Pending;
    }
}

async fn free_fut() -> String {
    "Free string".to_string()
}

Answer 1

impl AsyncFetcher {
    async fn fetch(&self, request: String) -> String {
        format!("Response({request})")
    }
}

In this function, Rust is inferring that the returned future captures the lifetime of &self even though you never actually use it. This makes the future non-'static but you are attempting to cast it to dyn Future which is implicitly 'static (in the absence of an explicit lifetime annotation).

You can work around this by changing the function from async fn to fn and returning impl Future instead, returning an async move block. Since the block won't capture self, the returned future will be 'static:

impl AsyncFetcher {
    fn fetch(&self, request: String) -> impl Future<Output = String> {
        async move { format!("Response({request})") }
    }
}

You could also just drop the &self parameter entirely.

Note that if you do intend to use self in the future, the problem will resurface. In that case you would need your AsyncFetcher owned by something else, or having shared ownership (Arc<AsyncFetcher>) and the future can own its own Arc. Otherwise you are effectively trying to store a reference to a value in the same struct that owns it, which isn't something you can (easily) do in Rust.