I have a txt document with many short urls.Each url is seperated by a single line.I want to resolve the URLs to get the final link.Also some URLs are redirected twice.How to automate this to get the final urls with output format of one url per line? Update: Input text file:
http://www.example.com/go/post-page-1
http://www.example.com/go/post-page-2
http://www.example.com/go/post-page-3
Output format needed in txt file:
http://www.example.org/post-page-name
http://www.example.org/post-page-name
http://www.example.org/post-page-name
Here is how the links are redirected:
Initial URL:http://www.example.com/go/post-page
==>301 Permanent Redirect
Intermediate url:http://click.affiliate.com/tracking?url=http://www.example.org/post-page-name
==>302 Temporary Redirect
Final URL: http://www.example.org/post-page-name
Here is the code i tried but it doesn't resolve URLs to the final link but rather to the intermediate link.
#!/bin/bash
rm resolved_urls.txt
for url in $(cat url.txt); do
wget -S "$url" 2>&1 | grep ^Location >> resolved_urls.txt
done
So, it's not 100% clear on what you're asking for. But what I'm seeing, and what I'm guessing, I think this'll do it for you:
This could probably be a lot faster without all the redirects, but I think this satisfies what you're looking for.