Gettting Error 500 when trying Get from website using lwp::useragent

126 Views Asked by At

Someone help me with this using lwp::Useragent please

my $mech = WWW::Mechanize->new(autocheck => 0);
$mech->get($url);
my $content=$mech->content;

but getting Error 500 when trying to get https://camelcamelcamel.com/

1

There are 1 best solutions below

2
On

It seems that the site blocks requests from "bad" (undesired) user agents. You may make WWW::Mechanize (LWP::UserAgent) present itself as another user agent using agent parameter for new or by calling agent method. Full IE8 identification string fixed the problem.

I have tested it using Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1.
[As listed by "User Agent Switcher" plugin for Firefox]

Short list of user agents (xml file)
Long list of user agents

WARNING
The site(s) may use or choose to use other means to block unwanted requests.