How to `join_all` `Vec<Result<Vec<Foo>, anyhow::Error>>` into `Result<Vec<Foo>, anyhow::Error>`

Question

I have a working Rust example using a (Cloud) API generated by OpenAPI Generator.

The API represents a hierarchy of resources (Apps->Services->Deployments) so the implementation of the functions is necessarily nested; the code needs to iterate over each App to get the App's Services and then iterate over the Services ...

I'm unable to merge API calls into a async fn's for, at least, 2 reasons:

1. Joining futures
2. Collecting errors

I was hoping to define 3 functions with a common return type; each function has different arguments:

async fn fetch_and_transform_apps() -> Result<Vec<Foo>, anyhow::Error> {...}
async fn fetch_and_transform_services() -> Result<Vec<Foo>, anyhow::Error> {...}
async fn fetch_and_transform_deployments() -> Result<Vec<Foo>, anyhow::Error> {...}

I cannot work out how to implement #1 & #2 above (#3 is working). Here's what I have:

async fn fetch_and_transform_services(...) -> Result<vec<Foo>, anyhow::Error> {
    let list_services_reply = list_services(...).await?;
    let services = list_services_reply
        .services.ok_or_else(|| anyhow::anyhow!("No services found"))?;

    let items_futures = services
        .iter()
        .map(|service| async move {
            let items: Result<Vec<Foo>, Error> = fetch_and_transform_deployments(...).await;
            items
        });
    let items = join_all(items_futures)
        .await
        // The type here is: Vec<Result<Vec<Foo>, anyhow::Error>>
        .into_iter()
        // The collect type results in the error
        .collect::<Result<Vec<koyeb::Koyeb>, anyhow::Error>>();

    items
}

The error:

a value of type std::vec::Vec<Foo> cannot be built from an iterator over elements of type std::vec::Vec<Foo> the trait FromIterator<std::vec::Vec<Foo>> is not implemented for std::vec::Vec<Foo>, which is required by Result<std::vec::Vec<Foo>, anyhow::Error>: FromIterator<Result<std::vec::Vec<Foo>, anyhow::Error>> the trait FromIterator<Foo> is implemented for std::vec::Vec<Foo> for that trait implementation, expected Foo, found std::vec::Vec<Foo> required for Result<std::vec::Vec<Foo>, anyhow::Error> to implement FromIterator<Result<std::vec::Vec<Foo>, anyhow::Error>>

Answer 1

I don't think collect is capable of this, but you can use try_fold instead.

fn collect_result_vecs<Iter, Item, E>(iter: Iter) -> Result<Vec<Item>, E>
where
    Iter: IntoIterator<Item = Result<Vec<Item>, E>>,
{
    iter.into_iter().try_fold(Vec::new(), |mut v, item| {
        v.extend(item?);
        Ok(v)
    })
}

You would use this function here:

let items = collect_result_vecs(join_all(items_futures).await);

If the outer Vec has many items (>1000), you may want to build the Vec in between awaits, so that you don't block for an extended period of time.

use futures::stream::FuturesOrdered;
let items_futures = FuturesOrdered::from_iter(items_futures);
let mut items = Vec::new();
while let Some(next_vec) = items_futures.next().await {
    items.extend(next_vec?);
}
Ok(items)

This also will cancel unfinished futures as soon as one fails instead of running every one to completion, but you could finish consuming items_futures instead of returning with next_vec? if you need to complete them all.