How to add timeout while sending logs to remote syslog server?

Question

I have python code sending logs to remote syslog server which works fine but In case of remote syslog server goes down, below code keeps on waiting indefinitely.

Code:

def writeSyslog (mtype, msg):
    """
    Send messages/log to Syslog server
    """
    try:
        global loggers
        if loggers.get('SplunkLogger'):
            splunk_logger = loggers.get('SplunkLogger')
        else:
            handler = logging.handlers.SysLogHandler(address = (SyslogServer,SyslogPort), socktype=socket.SOCK_STREAM)
            handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s %(message)s'))
            splunk_logger = logging.getLogger('SplunkLogger')
            splunk_logger.addHandler(handler)
            loggers['SplunkLogger'] = splunk_logger
        if "emerg" in mtype:
                splunk_logger.emergency(msg)
        elif "alert" in mtype:
                splunk_logger.alert(msg)
        elif "crit" in mtype:
                splunk_logger.critical(msg)
        elif "err" in mtype:
                splunk_logger.error(msg)
        elif "warn" in mtype:
                splunk_logger.warning(msg)
        elif "notice" in mtype:
                splunk_logger.notice(msg)
        elif "info" in mtype:
                splunk_logger.info(msg)
        else:
                splunk_logger.debug(msg)
    except:
        sys.stdout.write("\t\tSyslog failed sending to %s:%d\n" % (SyslogServer,SyslogPort))

python: v2.7

I tried to add timeout but its not expecting any timeout parameter.

I want it to give-up in case remote syslog server is not responding.

Answer 1

Most of the logging modules handlers can be configured by subclassing. The SysLogHandler has a createSocket method for this purpose. What you need to do is write an implementation of this method that adds a timeout. It depends a little on your exact setup/configuration but your code should in the end look somewhat similar to this:

import logging.handlers

class SysLogHandlerCustomTimeout(logging.handlers.SysLogHandler):
    def createSocket(self):
        super().createSocket()
        self.socket.settimeout(10)

# other code ...

handler = SysLogHandlerCustomTimeout(...)

Answer 2

What working for me is overriding the constructor of SysLogHandler to add timeout before socket connect.

Below code is tested for python2.7 and python 3.8. It has everything same as in-built module with extra timeout feature. Handle timeout exception as per the requirement.

class SysLogHandlerWithTimeout(logging.handlers.SysLogHandler):
def __init__(self, address=('localhost', 514),
             facility=1, socktype=None, timeout = 1):
    """
    Initialize a handler.

    If address is specified as a string, a UNIX socket is used. To log to a
    local syslogd, "SysLogHandler(address="/dev/log")" can be used.
    If socktype is specified as socket.SOCK_DGRAM or socket.SOCK_STREAM, that
    specific socket type will be used. For Unix sockets, you can also specify a
    socktype of None, in which case socket.SOCK_DGRAM will be used, falling
    back to socket.SOCK_STREAM. Default timeout is 1 second.
    """
    logging.Handler.__init__(self)

    self.address = address
    self.facility = facility
    self.socktype = socktype
    # Added below variable on purpose
    self.isTimeout = False

    if isinstance(address, str):
        self.unixsocket = True
        # Syslog server may be unavailable during handler initialisation.
        # C's openlog() function also ignores connection errors.
        # Moreover, we ignore these errors while logging, so it not worse
        # to ignore it also here.
        try:
            self._connect_unixsocket(address)
        except OSError:
            pass
    else:
        self.unixsocket = False
        if socktype is None:
            socktype = socket.SOCK_DGRAM
        host, port = address
        ress = socket.getaddrinfo(host, port, 0, socktype)
        if not ress:
            raise OSError("getaddrinfo returns an empty list")
        for res in ress:
            af, socktype, proto, _, sa = res
            err = sock = None
            try:
                sock = socket.socket(af, socktype, proto)
                # Set Connection Timeout
                sock.settimeout(timeout)
                if socktype == socket.SOCK_STREAM:
                    sock.connect(sa)
                break
            except socket.timeout:
                # Catch connection timout
                # Code here to handle exception as per the requirement
                self.isTimeout = True
                break
            except OSError as exc:
                err = exc
                if sock is not None:
                    sock.close()
        if err is not None:
            raise err
        self.socket = sock
        self.socktype = socktype

NOTE: Timeout here is for connection establishment with Syslog server not for sending events.