I launched an on-demand spark cluster using spark 1.4 and spark-ec2. I then logged into the cluster and find that the aws client is ancient.
aws --version
aws-cli/0.8.2 Python/2.6.9 Linux/3.4.37-40.44.amzn1.x86_64
On my local, the aws client is
aws --version
aws-cli/1.3.24 Python/2.7.7 Linux/2.6.32-431.el6.x86_64
The aws version installed by spark-ec2 is so old that it uses a completely different syntax. I would like to be able to use notation:
aws s3 cp src target
but I get this error instead:
$ aws s3 cp src tgt
usage: aws s3 operation
aws s3: error: argument operation: invalid choice: 'cp' (choose from u'list- object-versions', u'put-bucket-policy', u'list-objects', u'put-bucket-website', u'put-bucket-notification', u'put-bucket-logging', u'upload-part', u'put-object', u'delete-bucket-cors', u'put-bucket-versioning', u'get-bucket-cors', u'put-bucket-lifecycle', u'get-bucket-acl', u'get-bucket-logging', u'head-bucket', u'put-bucket-acl', u'delete-bucket-website', u'delete-bucket-policy', u'delete-objects', u'get-object', u'copy-object', u'list-buckets', u'put-bucket-request-payment', u'head-object', u'delete-bucket-tagging', u'get-object-torrent', u'get-bucket-lifecycle', u'create-bucket', u'complete-multipart-upload', u'get-bucket-website', u'create-multipart-upload', u'delete-bucket', u'get-bucket-policy', u'get-bucket-versioning', u'list-multipart-uploads', u'get-bucket-request-payment', u'put-bucket-tagging', u'get-bucket-tagging', u'abort-multipart-upload', u'put-object-acl', u'get-bucket-location', u'put-bucket-cors', u'delete-bucket-lifecycle', u'get-bucket-notification', u'list-parts', u'get-object-acl', u'upload-part-copy', u'delete-object', u'restore-object', 'help')
Is there an updated AMI that installs a recent version of AWS CLI?