Linux tool to capture process level metrics in real time?

Question

I want to monitor metrics consumed by a specific process in real time, like cpu, memory etc.

I have evaluated various tools but found none suiting my needs.

1. pidstat captures all i want, it gives me good memory usage per process but it reports more cpu usage(even greater than the total cpu consumed by the machine)

2. sar is more like pidstat but just dump all the data to file.

3. ps is more like snapshot tool, doesn't gives me real time stats

I am looking for a tool which can give me cpu usage for a process in real time.

Answer 1

These are some of the top command line process monitoring tools for linux.

1. Top – Linux Process Monitoring
2. VmStat – Virtual Memory Statistics
3. Lsof – List Open Files
4. Tcpdump – Network Packet Analyzer
5. Netstat – Network Statistics
6. Htop – Linux Process Monitoring
7. Iotop – Monitor Linux Disk I/O
8. Iostat – Input/Output Statistics
9. IPTraf – Real Time IP LAN Monitoring
10. Psacct or Acct – Monitor User Activity

If you need a GUI interface I suggest

1. Gnome System Monitor
2. GKrellM
3. Nagios
4. Monitorix
5. System Load Indicator

Answer 2

I do not know of a tool that does specific process monitoring on an interval; but, something like that wouldn't be hard to write; here are some facts:

• /proc/<pid>/stat will provide a lot of information on process stats.
• /proc/<pid>/cmdline will provide the processes command line.
• /proc/<pid>/environ will provide the environment variables that the process knows of.
• /proc/<pid>/io for IO stats.
• /proc/<pid>/statm for memory usage.
• /proc/<pid>/status which provides stat and statm in human readable format.
• /proc/<pid>/sched will give you various CPU and load based stats of the process.

You can read more with man proc; you could easily write a daemon that pulls this information on an interval, and then stores it somewhere centrally, such as with graphite or Elastic stack.

Answer 3

Everything you need to know about the available tools at each level:

http://www.brendangregg.com/linuxperf.html