How to choose python version 3 while deploying AWS glue Job with glue version 1.0 using YAML(serverless)

1.8k Views Asked by At

How to choose python version 3 while deploying AWS glue Job with glue version 1.0 using YAML(serverless)? I'm deploying AWS glue using serverless YAML code. AWS has provided GlueVersion parameter to choose the version of glue to use which I'm setting to '1.0'. I want to use Python 3 but it is deploying to python 2 in glue job. Does any have an idea how to set it to use python3 using YAML deployment code without manual changes in the glue job?

1

There are 1 best solutions below

0
On

Confirmed with AWS, their document is confusing where they mentioned PythonVersion parameter is for python shell. They are working on correcting the wording of the document. Below could formation template work for me.

Type: AWS::Glue::Job
Properties:
  Name: <job name>
  Description: ETL Job
  GlueVersion: '1.0'
  Command:
    Name: glueetl
    PythonVersion: 3
    ScriptLocation: <script location>
  ExecutionProperty:
    MaxConcurrentRuns: 1
  AllocatedCapacity: <number for workers>
  DefaultArguments:
   <any default arguments for glue job>
  Role:
    Ref: <glue roles>
  Connections:
    Connections:
      - Ref: <any JDBC connections>