How do I get the output of curl into a variable in Perl if I invoke it using backtics?

25.7k Views Asked by At

I'm trying to get the response of a curl call into a variable in perl.

my $foo = `curl yadd yadda`;

print $foo;

does not work. When I run this at the command line the curl call prints all its output correctly in the terminal, but the variable is not filled with that data.

Is there a way to do this without installing and calling the Perl curl lib?

9

There are 9 best solutions below

4
On BEST ANSWER

It probably sends its stuff to stderr. Try

my $foo = `curl yadd yadda 2>&1`;
0
On

Very old post, but the real way of using curl in backticks is using the appropriate switch of curl.

This switch is -o which defines where to send the output.

More from the curl man page:

Specifying the output as '-' (a single dash) will force the output to be done to stdout.

This also prevents having possible errors in $foo, which would happen if you redirect the complete STDERR to STDOUT on errors:

my $foo = `curl -o - yadd yadda`;
0
On

You can open a pipe as if it were a file.

$url = "\"http://download.finance.yahoo.com/d/quotes.csv?s=" . 
"$symbol&f=sl1d1t1c1ohgvper&e=.csv\"";

open CURL, "curl -s $url |" or die "single_stock_quote: Can't open curl $!\n";
$line = <CURL>;
close CURL;
0
On

You also might consider looking at LWP::UserAgent or even LWP::Simple.

0
On

Try this:

$var = `curl "http://localhost" 2>/dev/null`; 
print length($var)

curl displays progress information on stderr, redirecting that to /dev/null makes it easier to see what's going on.

1
On

It might be that some of the output you want to capture is in standard err, not standard out. Try this:

my $foo = system "curl http://www.stackoverflow.com";
print $foo;
4
On

This works on my system:

#!/usr/bin/perl

use strict;
use warnings;

my $output = `curl www.unur.com`;

print $output;

__END__

C:\> z1

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd"><html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

etc.

1
On

In the shell 2> means redirect fileno 2. Fileno 2 is always what a program sees as stderr. Similarly, fileno 0 is stdin and fileno 1 is stdout. So, when you say 2>&1 you are telling the shell to redirect stderr (fileno 2) into stdout (fileno 1). Since the backticks operator uses the the shell to run the command you specify, you can use shell redirection, so

my $foo = `curl yadda yadda 2>&1`;

is telling curl to redirect its output into stdout, and since the backtick operator catches stdout, you get what you were looking for.

0
On

What do you really want to do? Use curl at all costs, or grab the contents of a web page?

A more perlish way of doing this (which relies on no external programs that may or may not be installed on the next machine where you need to do this) would be:

use LWP::Simple;

my $content = get("http://stackoverflow.com/questions/1015438/")
   or die "no such luck\n";

If you want to see why the GET failed, or grab multiple pages from the same site, you'll need to use a bit more machinery. perldoc lwpcook will get you started.