I'm using the python3 API google-cloud-language 1.3.0
in composer-1.8.3-airflow-1.10.3
.
It used to be well but from last two weeks it reported ImportError: cannot import name 'aio' from 'grpc.experimental'
I've checked the doc of package, it looks like google-cloud-language
now requires grpcio>=1.29.0
. However according to Cloud Composer version list, even for the lastest version of composer its PyPI package grpcio stays on 1.23.0
I tried to force its version up to 1.29.0
but failed.
Does anyone can check this issus?
I was able to successfully upgrade the
grpcio
library version to1.31.0
. As a first step I would suggest you to upgrade your Cloud Composer instance to the most current version(composer-1.11.3-airflow-1.10.9
), since it should be more stable and solved some issues.To install the newer version of PyPi package in Cloud Composer, you can follow these steps or official documentation:
requirements.txt
file and pass the package name with version:The steps that I have taken to confirm that
grpcio
version has changed from1.23.0
to1.31.0
are following:GKE_CLUSTER
value within your environment inENVIRONMENT CONFIGURATION
->GKE cluster
variable.GKE_LOCATION
replace with the zone name:composer-<version>-...
, and POD_NAME with:airflow-worker-...
:grpcio==1.31.0
package.Update:
When you try to install the new packages Composer tries to create a new build, but it fails, this can be seen in the logs for Cloud Build by using the following advanced filter in Stackdriver Logging:
The problem is conflicting PyPi dependencies, each update operation triggers a Cloud Build operation that rebuilds images. In the version of Composer that you are using Cloud Build failed the update operation when saw any conflict. In the newest version
1.11.3
you will be able to choose if you want to allow conflicts or not.Based on this, I would share two recommendations to avoid the issue you are experiencing:
19.0.2
instead of9.0.3
. Therefore pip itself may be finding dependencies better.