ModuleNotFoundError: No module named 'pyathena' when running AWS Glue Job

Question

Despite setting the parameter for my Python AWS Glue Job like this:

--additional-python-modules pyathena 

I still get the following error when I try and run the job:

ModuleNotFoundError: No module named 'pyathena' 

I have also tried the following parameters:

--additional-python-modules pyathena
--pip-install pyathena
--pip-install pyathena==2.23.0

Answer 1

In case someone finds themselves with a similar issue this is what worked for me.

I solved this issue by removing the '--additional-python-modules pyathena' argument and upgrading the AWS Glue Job Python version to 3.9 (where pyathena is automatically installed as part of the default analytics package).

As I understand it, the --additional-python-modules argument does not work with earlier versions of AWS Glue Python.