rename files with dates timestamp in aws cloudshell

96 Views Asked by At

i am trying to rename 100s of csv files that are in s3 bucket which has date part in it and struggling with code.

s3://inventoryfiles/warehouse/2020/jan/inv12_20200301_220145.csv s3://inventoryfiles/warehouse/2020/jan/inv22_20200302_220145.csv s3://inventoryfiles/warehouse/2020/jan/inv23_20200303_220145.csv

they should be renamed to s3://inventoryfiles/warehouse/2020/jan/inv12_2020-03-01_22-01-45.csv s3://inventoryfiles/warehouse/2020/jan/inv22_2020-03-02_22-01-45.csv s3://inventoryfiles/warehouse/2020/jan/inv23_2020-03-03_22-01-45.csv

below is the code i wrote in shell script and stored it in cloudshell, however it is not working (i would like to echo values first so i can know if it works, so actual move command will be written later)


FILES=$(aws  s3api list-objects --bucket s3://inventoryfiles --prefix warehouse/2020/jan/  | jq -r '.Contents[] | select(.Size > 0) | .Key')


for i in $FILES
do
            regex='([0-9][0-9]_[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]_[0-9][0-9][0-9][0-9][0-9][0-9])'
                if [[ ${i} =~ $regex ]]; then 
                                date="${BASH_REMATCH[1]}"
                                        regex='date'
                if [[ ! ${i} =~ $regex ]]; 
then 
                i=$(sed "s/\r$//" <<< ${i})
                    DESTINATION=${i%/*}${date}${i##*/}
                    echo "new location"      ${DESTINATION}
                fi
                fi
done

can you please recommend the correct code ? thank you

1

There are 1 best solutions below

0
On
for src in $files; do dst=$(echo "$src" | sed -E 's/(_[0-9]{4})([0-9]{2})([0-9]{2})(_[0-9]{2})([0-9]{2})([0-9]{2}\.csv)$/\1-\2-\3\4-\5-\6/'); echo "new location ${dst}"; done

this code worked thank you @jhnc