ExportDicomData request of Google Cloud Healthcare API on GitHub tutorials never finish

Question

I'm trying AutoML Vision of ML Codelabs on Cloud Healthcare API GitHub tutorials. https://github.com/GoogleCloudPlatform/healthcare/blob/master/imaging/ml_codelab/breast_density_auto_ml.ipynb

I run the Export DICOM data cell code of Convert DICOM to JPEG section and the request as well as all the premise cell code succeeded.

But waiting for operation completion is timed out and never finish. (ExportDicomData request status on Dataset page stays "Running" over the day. I did many times but all the requests were stacked staying "Running". A few times I tried to do from scratch and the results were same.)

I did so far:

1) Remove "output_config" since INVALID ARGUMENT error occurs. https://github.com/GoogleCloudPlatform/healthcare/issues/133

2) Enable Cloud Resource Manager API since it is needed.

This is the cell code.

# Path to export DICOM data.
dicom_store_url = os.path.join(HEALTHCARE_API_URL, 'projects', project_id, 'locations', location, 'datasets', dataset_id, 'dicomStores', dicom_store_id)
path = dicom_store_url + ":export"

# Headers (send request in JSON format).
headers = {'Content-Type': 'application/json'}

# Body (encoded in JSON format).
# output_config = {'output_config': {'gcs_destination': {'uri_prefix': jpeg_folder, 'mime_type': 'image/jpeg; transfer-syntax=1.2.840.10008.1.2.4.50'}}}
output_config = {'gcs_destination': {'uri_prefix': jpeg_folder, 'mime_type': 'image/jpeg; transfer-syntax=1.2.840.10008.1.2.4.50'}}
body = json.dumps(output_config)

resp, content = http.request(path, method='POST', headers=headers, body=body)
assert resp.status == 200, 'error exporting to JPEG, code: {0}, response: {1}'.format(resp.status, content)
print('Full response:\n{0}'.format(content))

# Record operation_name so we can poll for it later.
response = json.loads(content)
operation_name = response['name']

This is the result of waiting.

Waiting for operation completion...
Full response:
{
  "name": "projects/my-datalab-tutorials/locations/us-central1/datasets/sample-dataset/operations/18300485449992372225",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.healthcare.v1beta1.OperationMetadata",
    "apiMethodName": "google.cloud.healthcare.v1beta1.dicom.DicomService.ExportDicomData",
    "createTime": "2019-08-18T10:37:49.809136Z"
  }
}


AssertionErrorTraceback (most recent call last)
<ipython-input-18-1a57fd38ea96> in <module>()
     21 timeout = time.time() + 10*60 # Wait up to 10 minutes.
     22 path = os.path.join(HEALTHCARE_API_URL, operation_name)
---> 23 _ = wait_for_operation_completion(path, timeout)

<ipython-input-18-1a57fd38ea96> in wait_for_operation_completion(path, timeout)
     15 
     16   print('Full response:\n{0}'.format(content))
---> 17   assert success, "operation did not complete successfully in time limit"
     18   print('Success!')
     19   return response

AssertionError: operation did not complete successfully in time limit

API Version is v1beta1.

I was wondering if somebody has any suggestion.

Thank you.

Answer 1

After several times kept trying and stayed running one night, it finally succeeded. I don't know why.

Answer 2

There was a recent update to the codelabs. The error message is due to the timeout in the codelab and not the actual operation. This has been addressed in the update. Please let me know if you are still running into any issues!