This post comes from my recent attempt at solving https://github.com/ropensci/targets/issues/1172 (more background: https://books.ropensci.org/targets/cloud-storage.html). In a reproducible pipeline, the targets R package needs to check the existence of a specific version of each AWS S3 object in its data store. The version of a given object is the version ID recorded in the metadata, which may or may not be the most current version in the bucket. Currently, targets accomplishes this by sending a HEAD request for each relevant object-version pair.
Is there a more efficient/batched way to do this for each version/object pair? list_object_versions() returns every version of every object of interest, which is way too many versions to download efficiently, and neither list_objects() nor list_objects_v2() return any version IDs at all.