Python with for list of things

Question

If I have a list of things I want to use in a with block, how can I do that? If there are a fixed number of things, you can write them out like so:

with (
    items[0] as inner_0,
    items[1] as inner_1,
    ...
):
    ...

In my case, the number of things I need to __enter__ is unknown, so I would like to have something like this:

with items as inners:
    # inners[n] is whatever items[n].__enter__() returned

The documentation of "The with Statement" does not support this and I can't find any utilities to do this in contextlib.