cloudshell rename only specific files that has date timestamp

64 Views Asked by At

i am trying to rename 1000s of csv files that are in s3 bucket which has date part in it and struggling with code.

s3://inventoryfiles/warehouse/2020/jan/inv12_2020_03_01_22_01_45.csv 
s3://inventoryfiles/warehouse/2020/jan/inv22_2020_03_02_22_01_45.csv 
s3://inventoryfiles/warehouse/2020/jan/inv23_2020-03-03_22-01-45.csv

they should be renamed to

s3://inventoryfiles/warehouse/2020/jan/inv12_2020-03-01_22-01-45.csv 
s3://inventoryfiles/warehouse/2020/jan/inv22_2020-03-02_22-01-45.csv 
s3://inventoryfiles/warehouse/2020/jan/inv23_2020-03-03_22-01-45.csv

below is the code i wrote in shell script and stored it in cloudshell. tHE CODE is working for AWS MV command but its doing MOVE command for all files. i would like the IF statement to work ONLY when files have YYYY_MM_DD_HHMMSS (2020_03_02) and change it to YYYY-MM-DD_HH-MM-SS can someone help correct the code.

Code

FILES=$(aws  s3api list-objects --bucket s3://inventoryfiles --prefix warehouse/2020/jan/  | jq -r '.Contents[] | select(.Size > 0) | .Key')
for src in $FILES; 
        do 
                regex='([0-9][0-9][0-9][0-9]_[0-9][0-9]_[0-9][0-9]_[0-9][0-9]_[0-9][0-9]_[0-9][0-9]/)'
                        if [[ ${src} =~ $regex ]]; then
                                dst=$(echo "$src" | sed -E 's/(_[0-9]{4})_([0-9]{2})_([0-9]{2})(_[0-9]{2})_([0-9]{2})_([0-9]{2}\.csv)$/\1-\2-\3-\-4\5-\6/');
                                SOURCE="s3://${S3_BUCKET}/${src}"
                                DESTINATION="s3://${S3_BUCKET}/${dst}"
                                echo ""
                                echo "old location ${SOURCE}"
                                echo "new location ${DESTINATION}"; 
                fi
                fi
done````
can you please recommend the correct code ? thank you
0

There are 0 best solutions below