I am using subprocess.Popen class with threading and it is not working for me

Question

I have following files:

ping.py:

from utilities.env import PACKET_COUT, PING_TIME, MAX_WORKERS_COUNT
from utilities.cmds import create_ping_cmd
from subprocess import Popen, PIPE, DEVNULL, TimeoutExpired
from re import findall
from concurrent.futures import ThreadPoolExecutor


MAX_PING_TIME = float(PACKET_COUT) * (float(PING_TIME))
ping_cmd = create_ping_cmd()

def check_ping_process(ping_process):
    ping_result = {}
    try:
        out = ping_process.communicate(timeout=MAX_PING_TIME)[0]
        is_successes = findall("TTL", out)
        if ping_process.returncode == 0 and is_successes:
            return True
        else:
            return False
    except TimeoutExpired:
        ping_process.kill()
        return False


def ping_ip(ip):
    ping_cmd.append(ip)
    ping_process = Popen(
        ping_cmd,
        stdout=PIPE,
        stderr=DEVNULL,
        text=True
    )
    result = check_ping_process(ping_process)
    return result


def ping_ip_list(ip_list):
    ping_results = []
    with ThreadPoolExecutor(max_workers=MAX_WORKERS_COUNT) as executor:
        results = executor.map(ping_ip, ip_list)
        for ip, result in zip(ip_list, results):
            print({ip, result})
            ping_results.append({ip, result})
    return ping_results

main.py:

from ping import ping_ip_list


ip_list = [
    '192.168.0.100',
    '192.168.0.1',
    '192.168.0.104',
    '192.168.0.124',
    '192.168.0.103'
]
def monitor_network_devices(ip_list):
    results = ping_ip_list(ip_list)

monitor_network_devices(ip_list)

The goal of the program is find out if network device is pingable or not. I am running main.py file, which is giving wrong results. I tried debug, it is appeared the problem with threads, but I cannot resolve it.

Any help would be greatly appreciated

Answer 1

A few observations

• As mentioned, you keep appending more to ping_cmd, your command will have more and more IP addresses. It is better to build a new command every time instead of appending to the ping_cmd list

• Consider this line

    is_successes = findall("TTL", out)
  

  I look at the output from Linux, it looks like this:

    PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
    64 bytes from 192.168.1.1: icmp_seq=1 ttl=63 time=11.9 ms
    64 bytes from 192.168.1.1: icmp_seq=2 ttl=63 time=5.18 ms
  
    --- 192.168.1.1 ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 1024ms
    rtt min/avg/max/mdev = 5.182/8.541/11.901/3.359 ms
  

  In your code, you search for TTL (upper case), which will always result in False. Also, to search for text, you don't need to use regular expression:

    is_success = "ttl" in out
  

• Since you did not post the contents of your utilities package, I have no idea what the values such as PACKET_COUT are

• When dealing with multi-threading, it is better to use logging instead of print because the former is thread-safe and not the later.

With that, here is my proposed solution

# ping.py
import logging
import subprocess
from concurrent.futures import ThreadPoolExecutor

logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s | %(levelname)s | %(threadName)-15s | %(funcName)-18s | %(message)s",
)


def ping_ip(ip):
    ping_command = ["ping", "-c1", ip]
    logging.debug("Execute command %r", ping_command)
    completed_process = subprocess.run(
        ping_command,
        text=True,
        capture_output=True,
        check=False,
    )
    success = completed_process.returncode == 0
    logging.debug("%s -> %r", ip, success)
    return success


def ping_ip_list(ip_list):
    with ThreadPoolExecutor() as executor:
        out = dict(zip(ip_list, executor.map(ping_ip, ip_list)))
    return out