How do you load the aws credentials and config file in PBS scripts(.pbs) file? I want to submit a job that includes file transfer from a remote server to s3 buckets, but getting endpoint url error. Also, the s3 bucket is not public. Any insights will be helpful, thank you :)
I tried using 'export' to export the credentials and doing 'aws s3 --profile [profile-name] --no-verify-ssl sync . s3://[bucket-name]' inside the PBS file, but it is not working.
If you're setting the credentials in env variables, you don't need to specify the profile via --profile [profile-name].
More info here.