module 'six.moves' has no attribute 'collections_abc'

Question

I have a script that connects to YouTube API version 3 and retrieves public data. This script is deployed in airflow, it was working fine for a month, today it failed with this message for the following line:

def youtube_search(term,region):
        youtube = build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION, developerKey=DEVELOPER_KEY,cache_discovery=False)

 File "/usr/local/airflow/.local/lib/python3.6/site-packages/googleapiclient/discovery.py", line 455, in build_from_document
    if isinstance(client_options, six.moves.collections_abc.Mapping):
AttributeError: module 'six.moves' has no attribute 'collections_abc'

I went to check the 455th line on discovery.py

if isinstance(client_options, six.moves.collections_abc.Mapping):
        client_options = google.api_core.client_options.from_dict(client_options)

The six module has not changed for a long time, nor has my script deployed in airflow, which config has not changed either.

As Google module imports six, which has not changed, I can't figure out why would I get this error at random?

Answer 1

Are you explicitly specifying the version of google-api-python-client that you're using, or just using the latest version available?

There was a new version (v1.12.0) released on 9/14/20, followed by a bug fix version v.1.12.1 that fixes a bug related to the dependency on the six module. (release notes from GitHub: require six>=1.13.0 (#1030) (4acecc3))

You may be running into the bug in v1.12.0.

Answer 2

Six is one of the libraries installed by default. According to this Github issue:

At the moment tests always install the latest version of dependencies. We're going to move to having explicit requirements.txt for tests runs so we catch lower bound issues like this automatically in the future.

For now, you can use 1.11.0 or upgrade six. A new release 1.12.1 should be released later today with the adjusted pin on six. https://github.com/googleapis/google-api-python-client/issues/1029

Answer 3

simply run pip install --upgrade six The cause of this issue is usually that the module 'six' in your local environment is out of date with the version that is required by the google-api-python-client library.