Spaces aren't being removed from first variable

71 Views Asked by At

This script is supposed to remove leading and trailing spaces from variables in a file, and insert them into an SQL statement. This is working for every variable except for the first one. It will not remove trailing spaces from the user ID. I have attached the sample file and code being used to gather the variable data.

Sample File:

"User ID"|First|Last|Password|"E-Mail Address"|CreateDate|RequestDate|Requestor|Title|"Site Location"|"Client Name"|"Requestor Email"|Roles|Tasks|"Correspondance Packages"
"A145078 "|"James "|Joyce|" DO NOT USE"|[email protected]|" 2015/06/22 16:30:00"|"2015/06/20 17:00:00"|"John Edwards"|CSR|Aetna|Aetna|[email protected]|"CTM Data Analyst, Medical Appeals Status/Status Reasons, RCA Status and Status Reasons, Grievance Status/Status Reasons, CTM Status/Status Reasons"||

Output Being Produced (see space after user_id:

INSERT INTO users(user_id, first, last, password, email, createdate, requestdate, requestby)VALUES('A145078 ','James','Joyce','DO NOT USE','[email protected]','2015/06/22 16:30:00','2015/06/20 17:00:00','John Edwards');

Code Utilized:

    #!/bin/bash
> user_adds.sql

while IFS="|" read -r uid first last password email createdate requestdate requestor title site client requestemail roles tasks packages; do
   #printf '%s %s\n' "$uid" "$first" "$roles"
   uid=$(echo "$uid" | sed 's/\"//g' | sed 's/^[ \t]*//' | sed 's/[ \t]*$//')
   first=$(echo "$first" | sed 's/\"//g' | sed "s/'/ /g" | sed 's/^[ \t]*//;s/[ \t]*$//')
   last=$(echo "$last" | sed 's/\"//g' | sed "s/'/ /g" | sed 's/^[ \t]*//;s/[ \t]*$//')
   echo $uid - "$uid"
   echo $roles

   echo "INSERT INTO users(user_id, first, last, password, email, createdate, requestdate, requestby)VALUES('$uid','$first','$last','DO NOT USE','$email','$createdate','$requestdate','$requestor');" >> user_adds.sql

   #echo "INSERT INTO user_demographics(user_id, title, site_location, comment, client_name, requester_email,last_password_date_changed)select id,'$title','$site_location','$title','$client','$request_email',NULL from users where user_id in('$user_id');" >> demo_adds.sql


done < 'new_users.csv'
2

There are 2 best solutions below

4
On
$ cat tst.awk        
BEGIN{ FS="[[:space:]\"]*[|][[:space:]\"]*" }
NR>1 {
    gsub(/^[[:space:]"]+|[[:space:]"]+$/,"")
    printf "INSERT INTO users(user_id, first, last, password, email, createdate, requestdate, requestby)VALUES('%s','%s','%s','%s','%s','%s','%s','%s');\n", $1,$2,$3,$4,$5,$6,$7,$8
}

$ awk -f tst.awk file
INSERT INTO users(user_id, first, last, password, email, createdate, requestdate, requestby)VALUES('A145078','James','Joyce','DO NOT USE','[email protected]','2015/06/22 16:30:00','2015/06/20 17:00:00','John Edwards');
0
On

Here is the basic template you can use:

#!/bin/bash

typeset -a  parts

cat <<'EOF'  |
"A145078 "|"James "|Joyce|" DO NOT USE"|[email protected]|" 2015/06/22 16:30:00"|"2015/06/20 17:00:00"|"John Edwards"|CSR|Aetna|Aetna|[email protected]|"CTM Data Analyst, Medical Appeals Status/Status Reasons, RCA Status and Status Reasons, Grievance Status/Status Reasons, CTM Status/Status Reasons"||
EOF
while read line; do
    line=$( echo "${line}" | sed -e 's/" /"/g' -e 's/ "/"/g' -e 's/"//g')
    typeset -a parts=( ${line//|/ } )
    echo ${parts[@]}
    uid="${parts[0]}"
    first="${parts[1]}"
    last="${parts[2]}"
    echo "INSERT INTO users(user_id, first, last) VALUES ('$uid','$first','$last');"
done