How to use NYTProf when there are multiple processes

Question

In my application, I have multiple processes. Initially I start one Perl process by running .pl file, it in turn calls two more .pl scripts creating 2 new processes. How to use Profile such a code. I did NYTProf, merged the results and opened with html but all the profiling information relating to the functions is missing. Information to profile this kind of application will help?

Answer 1

The PROFILING section of the documentation gives an example:

PERL5OPT=-d:NYTProf

That's also very handy when you can't alter the perl command line being used to run the script you want to profile. Usually you'll want to enable the "addpid=1" option to ensure any nested invocations of perl don't overwrite the profile.

The docs for the addpid option explain that:

Append the current process id to the end of the filename. This avoids concurrent, or consecutive, processes from overwriting the same file. If a fork is detected during profiling then the child process will automatically add the process id to the filename.

You say that you "merged the results". I presume you mean using nytprofmerge. That should only be used to merge multiple result files made by exactly the same source code. E.g., by a process that forked to create child processes, or multiple runs of an unchanged script. In your case you're generating profiles from different scripts so each profile output should be used to generate a separate report without merging.

You say that "all the profiling information relating to the functions is missing". I'll need more information about exactly how you're profiling the code and generating the reports before I can help you there.

Answer 2

Adding on to the answer by @TimBunce, here's the step-by-step of how I used addpid to generate a report (tested in bash, with Devel::NYTProf 6.06).

In my case, I was doing multiple runs of the same code. Since this is the question that comes up at the top of the Google results for that situation, I am adding this answer in case it helps future readers.

# Clean any old output
rm -rf nytprof*

# Run the program with addpid=1.
#   `{1..10}` => ten runs
#   `foo.pl` => whatever your program name and other Perl options are
# This makes files `nytprof.out.####`, where `####` is the PID.
(export NYTPROF=addpid=1 ; for i in {1..10} ; do perl -d:NYTProf foo.pl ; done ; )    

# Merge the result files.  This produces `nytprof-merged.out`.
nytprofmerge -v nytprof.out.*

# Make the HTML report into `nytprof/`
nytprofhtml -f nytprof-merged.out