Running AnyEvent under Dancer application

Question

I would like to do some non-blocking SSH to a couple of thousand machines that i'm tracking (my own machines), I have a Dancer application up and running, and I'm willing to use AnyEvent::timer to execute SSH commands asynchronously (each machine has its own polling interval, and I don't want one machine to wait for another to complete with its SSH work).

I'm wondering, what is the best way to act asynchronously in a synchronous environment?

Answer 1

It is not very good idea to run any external commands from within your web scripts. For one, should your external call block or crash for any reason, it will create bad experience for the user (even it that user is just you). Then, running external commands as web user may have a lot of security implications - I would think your web user most likely has passwordless ssh set up, doesn't it? What if someone figures out some security hole in your script and manages to use it to ssh into your servers?

Instead, you should create separate service or process which will regularly poll your servers status using ssh (or what else) and save results of that scan into database - Postgres or MySQL.

Then, change your Dancer app to display collected results from database, rather than doing live ssh request. This way it will be very fast and secure.

Answer 2

I could not be a good idea but it is possible. I have a big Dancer application to execute scripts remotely and I'm doing it with fork and Net::SSH2. I tried with thread but there are some modules that are not thread-safe so I recommend to use fork.

I have some comments in my blog http://perlondancer.blogspot.mx/2014/04/executing-remote-commands-from-dancer.html and in this gist is the code example below: https://gist.github.com/johandry/11197516

#!/usr/bin/env perl

use strict;
use warnings;
use Dancer;
use Net::SSH2;

sub execCommand ($$) {
  my ( $ssh2, $cmd ) = @_;

  my %args=(
    timeout => 1_000,  # polling timeout
    bufsize => 10_240, # read buffer size when polling
  );

  $ssh2->blocking(1); #needed for ssh->channel
  my $chan=$ssh2->channel(); # create SSH2 channel
  if ($ssh2->error()) {
    return (undef, undef, 100);
  }

  # exec $cmd (caveat: only one command line can be executed over this channel. No "ls -l;whoami" combo. Use ssh->shell instead.
  unless ($chan->exec($cmd)) {
    return (undef, undef, 500);
  }

  # defin polling context: will poll stdout (in) and stderr (ext)
  my @poll = ( { handle => $chan, events => ['in','ext'] } );

  my %std=();     # hash of strings. store stdout/stderr results
  $ssh2->blocking( 0 ); # needed for channel->poll
  while(!$chan->eof) { # there still something to read from channel
      $ssh2->poll( $args{'timeout'}, [ @poll ] ); # if any event, it will be store into $poll;

      my( $n, $buf ); # number of bytes read (n) into buffer (buf)
      foreach my $poll ( @poll ) { # for each event
          foreach my $ev ( qw( in ext ) ) { #for each stdout/stderr
              next unless $poll->{revents}{$ev};

              #there are something to read here, into $std{$ev} hash
              if( $n = $chan->read( $buf, $args{'bufsize'}, $ev eq 'ext' ) ) { #got n byte into buf for stdout ($ev='in') or stderr ($ev='ext')
                  $std{$ev}.=$buf;
              }
          } #done foreach
      }
  }
  $chan->wait_closed(); #not really needed but cleaner

  my $exit_code=$chan->exit_status();
  $chan->close(); #not really needed but cleaner

  $ssh2->blocking(1); # set it back for sanity (future calls)

  return ($std{'in'},$std{'ext'},$exit_code);
}   

sub execute ($$$$) {
  my ($ip, $username, $password, $cmd) = @_;
  my $pid = fork();

  if ($pid) {
    # This is the parent (DANCER)
    debug "Process started with PID $pid\n";
  } elsif ( $pid == 0 ) {
    # This is the child 
    my $ssh2 = Net::SSH2->new();
    $ssh2->connect( $ip ) or debug("Cannot connect to $ip");
    my $publicKeyFile  = './id_rsa.pub'; # path(setting('appdir'), 'db', 'id_rsa.pub'); # I prefer to copy the public key in your app dir due to permissions issues
    my $privateKeyFile = './id_rsa';     # path(setting('appdir'), 'db', 'id_rsa'); # I prefer to copy the private key in your app dir due to permissions issues
    if ( $ssh2->auth_publickey( $username, $publicKeyFile, $privateKeyFile, $password ) ) {
      my ($stdout, $stderr, $exitcode) = execCommand($ssh2, $cmd);
    } else {
      debug "Could not authenticate to $ip with $username";
    }
    $ssh2->disconnect();
  } else {
    debug "Could not fork: $!\n";
  }
}

set logger => "console";
set log    => "core";
set show_errors => 1;

get '/uptime/:ip' => sub {

  my $username = "the username";
  my $password = "the password";

  execute(param('ip'), $username, $password, "uptime > /tmp/dancer_example.txt");

  return 'uptime is running';
};

dance;

true;

Answer 3

Net::SSH2 can be used asynchronously, but it is quite buggy and crashes often. Forget about using it for running thousands (or just hundreds) of connections in parallel on the same process. It may be ok if you use it wrapped in new processes as recomended by @Johandry, but then you can just run the ssh command using AnyEvent::Util::run_cmd.

Net::OpenSSH is another Perl module that can be used asynchronously. It shouldn't be too difficult to integrate it inside AnyEvent.