Message size varies TCPServer Ruby

Question

I'm working with an AVL (Skypatrol TT8750+) and the messages that it sends (using TCP) are supposed to be 59bytes long but it always sends a first message (the message has some information about the AVL, so the user can identify it) of 33bytes.

So the question is, How can I handle those different size messages on ruby?

require 'socket'

portnumber = 12050
socketServer = TCPServer.open(portnumber)

while true
  Thread.new(socketServer.accept) do |connection|
  puts "Accepting connection from: #{connection.peeraddr[2]}"
  t = Time.now.strftime("%d-%m-%Y %H%M")
  file_name = t + '.txt'
  out_file = File.new(file_name, "w+")
  begin
    while connection
      incomingData = connection.gets()
      if incomingData != nil
        incomingData = incomingData
      end
      hex_line = incomingData.unpack('H*')[0]
      out_file.puts(hex_line)
      puts "Incoming: #{hex_line}"
    end
    rescue Exception => e
      # Displays Error Message
      puts "#{ e } (#{ e.class })"
    ensure
      connection.close
      puts "ensure: Closing"
    end
  end
end

This is the experimental code that I'm using.

Answer 1

I'm posting this answer to explain a comment I made to Anderson's answer. Most of the code isn't mine.

---

moving the if out of the loop

When the if statement is within a loop, it will be evaluated each and every time the loop runs, increasing the number of CPU instructions and the complexity of each loop.

You could improve performance by moving the conditional statement out of the loop like so:

require 'socket'
require 'celluloid/io'

portnumber = 12050
socketServer = TCPServer.open(portnumber)
incomingData = nil

while true
  Thread.new(socketServer.accept) do |connection|
  puts "Accepting connection from: #{connection.peeraddr[2]}"
  # this should probably be changed,
  # it ignores the possibility of two connections arriving at the same timestamp.
  t = Time.now.strftime("%d-%m-%Y %H%M")
  file_name = t + '.txt'
  out_file = File.new(file_name, "w+")

  begin
    if connection
      incomingData = conection.recv(33)
      if incomingData != nil
        incomingData = incomingData.unpack('H*')[0]
        out_file.puts(incomingData)
        puts "Incoming: #{incomingData}"
      end
    end
    while connection
      incomingData = connection.recv(59)
      if incomingData != nil
        incomingData = incomingData.unpack('H*')[0]
        out_file.puts(incomingData)
        puts "Incoming: #{incomingData}"
      end
    end
    rescue Exception => e
      # Displays Error Message
      puts "#{ e } (#{ e.class })"
    ensure
      connection.close
      out_file.close
      puts "ensure: Closing"
    end
  end
end

Optimizing the recv method

Another optimization I should probably mention (but won't implement here) would be the recv method call.

This is both an optimization and a possible source for errors that should be addressed.

recv is a system call and as network messages might be combined (or fragmented) across TCP/IP packets, it might become more expensive to call recv than to handle an internal buffer of data that resolved fragmentation and overflow states.

Reconsidering the thread-per-client design

I would also recommend avoiding the thread-per client design.

In general, for a small number of clients it probably doesn't matter much.

However, as clients multiply and threads become busier, you might find the system spends more resources on context switches than actual tasks.

Another concern might be the allocated stack each thread requires (1Mb or 2Mb for Ruby threads, if I remember correctly)... In a best case scenario, 1,000 clients will require more than a GigaByte of memory allocation just for the stack (I'm ignoring kernel structure data table and other resources).

I would consider using EventMachine or Iodine (I'm iodine's author, so I'm biased).

An evented design could save you many resources.

For example (untested):

require 'iodine'
# define the protocol for our service
class ExampleProtocol
  @timeout = 10
  def on_open
    puts "New Connection Accepted."
    # this should probably be changed,
    # it ignores the possibility of two connections arriving at the same timestamp.
    t = Time.now.strftime("%d-%m-%Y %H%M")
    file_name = t + '.txt'
    @out_file = File.new(file_name, "w+")
    # a rolling buffer for fragmented messages
    @expecting = 33
    @msg = ""
  end

  def on_message buffer
    length = buffer.length
    pos = 0
    while length >= @expecting
        @msg << (buffer[pos, @expecting])
        out_file.puts(msg.unpack('H*')[0])
        length -= @expecting
        pos += @expecting
        @expecting = 59
        @msg.clear
    end
    if(length > 0)
        @msg << (buffer[pos, length])
        @expecting = 59-length
    end
  end

  def on_close
    @out_file.close
  end
end
# create the service instance
Iodine.listen 12050, ExampleProtocol
# start the service
Iodine.start

Answer 2

The solution was quite simple

require 'socket'
require 'celluloid/io'

portnumber = 12050
socketServer = TCPServer.open(portnumber)

while true
  Thread.new(socketServer.accept) do |connection|
  puts "Accepting connection from: #{connection.peeraddr[2]}"
  t = Time.now.strftime("%d-%m-%Y %H%M")
  file_name = t + '.txt'
  out_file = File.new(file_name, "w+")
  messagecounter = 1

  begin
    while connection
      if messagecounter == 1
        incomingData = conection.recv(33)
        messagecounter += 1
      else
        incomingData = connection.recv(59)
      end
      if incomingData != nil
        incomingData = incomingData.unpack('H*')[0]
      end
      out_file.puts(incomingData)
      puts "Incoming: #{incomingData}"
    end
    rescue Exception => e
      # Displays Error Message
      puts "#{ e } (#{ e.class })"
    ensure
      connection.close
      puts "ensure: Closing"
    end
  end
end

I just needed an extra variable and an if to auto increment the variable, and that's it.