Proper exception handling when interleaving context managers

Question

I need to lock the __enter__ of a context manager but release the lock right after the context has been entered. How can this be done cleanly? Note that both contexts are provided in libraries, so I cannot move the first context into the enter of the second. (In my case the first is an async lock and the second is an async botocore session).

In simple terms, what I'm trying to achieve is:

ctx1.__enter__()
ctx2.__enter__()
ctx1.__exit__()
...
ctx2.__exit__()

Using a lock makes this easier, because it doesn't have to be a context:

lock.acquire()
with context():
    lock.release()
    ...

But how can I add proper exception handling? A try-except around the with statement would only release the lock after the context. Is the only way to call enter and exit for the second context manually and use a flag?

entered = False
try:
    with lock:
        context.__enter__()
        entered = True

    ...
finally:
    if entered:
        context.__exit__()